Virtual Machines and Containers

For more information, please check the cloud computing page and the Singularity page in Compute Canada’s technical wiki.

“Hiding large numbers of files in container overlays”

Webinar (2023-Jan-17) by Alex Razoumov

Many unoptimized HPC cluster workflows result in writing large numbers of files to distributed filesystems which can create significant problems for the performance of these shared filesystems. One of the ways to alleviate this is to organize write operations inside a persistent overlay directory attached to an immutable read-only container with your scientific software. These output files will be stored separately from the base container image, and to the host filesystem an overlay appears as a single large file. In this presentation, we demo running parallel OpenFOAM simulations where all output goes into overlay images, and the total number of files on the host filesystem is reduced from several million to several dozen or less. The same approach can be used in post-processing and visualization, where you can read simulation data from multiple overlays both in serial and in parallel. In this webinar we walk you through all stages of creating and using overlays. We assume no prior knowledge of the container technology.

“Storage options for cloud computing”

Webinar (2022-Nov-09) by Sarah Huber

As opposed to a cluster computing environment, projects in Infrastructure-as-a-Service clouds like those in the Alliance are responsible for selecting and administering their project’s storage systems. Several types of cloud storage are available, enabling users to choose the storage type(s) best suited for their use case. In this webinar (not recorded, only slides available), we discuss the storage options available:

  • volume storage,
  • ephemeral disc storage,
  • object storage, and

  • shared filesystem storage.

  • PDF slides

“Best practices for securing your own OpenStack VM / Amazon instances / Containers”

Webinar (2022-Mar-30) by Ken Bigelow

“Using Singularity containers in VMs and HPC”

Webinar (2021-May-26) by Venkat Mahadevan

“Singularity in HPC”

Webinar (2020-Feb-05) by Grigory Shamov

“Getting started on the cloud”

Webinar (2018-Nov-28) by John Simpson

We edited out the first few minutes of this video (lots of background noise), jumping straight into the section in which John explains cloud computing and its difference from traditional HPC.

“How to deploy HTTP and HTTPS sites on a virtual machine”

Webinar (2018-Dec-12) by John Simpson