Beginner's guides
Getting Started Resources
High-performance computing
Click on each webinar for its recording and materials. Most current Digital Research Alliance of Canada systems, as well as many local advanced research computing (ARC) systems and clusters support high-performance computing (HPC) and high-throughput computing (HTC) workflows. HPC workflows tend to focus on tightly coupled parallel jobs and, as such, they must execute within a particular site with low-latency interconnects. Conversely, HTC workflows consist of independent, mostly sequential jobs that can be individually scheduled on many different computing resources. In this webinar, we guide you through the best practices for deploying your HPC or HTC computations on our clusters: Presentation on dealing with hardware and software instabilities on Cedar from a user's perspective: causes, workarounds, bad workflows, and best practices. It was part of a larger WestGrid's March 2019 Town Hall -- the embedded video below starts at the relevant section. This is an introduction to working on our HPC systems for new users. Best practices in HPC/HTC environments (2024-Oct-15)
Managing Cluster Instability for Users (March 2019)
How to submit and run jobs on Compute Canada HPC systems (2018-Sep-19)
Collapse all webinars in this section
Linux command line
For a basic introduction to Bash command line – as well as programming in Python or R, and version control with Git – please check out a Software Carpentry workshop closest to you. For an online tutorial, we highly recommend their Introduction to using the shell in a HPC context.
Click on each webinar for its recording and materials. In recent years, a number of open-source utilities for the Unix shell have emerged. Some are meant as replacements for classic tools with improved performance, better defaults, or nicer-looking outputs; others add novel functionality. Several of them were recently installed on the Alliance clusters. In this webinar I cover a selection of tools that are very popular, well-maintained, and that have served me well in my daily workflows: I also talk about three useful Zsh plugins: For each tool/plugin, I talk about installation on a personal computer and on the Alliance clusters and give live demos. Knowing basic Linux commands is essential for researchers using remote systems such as HPC clusters. Bash is the most commonly used Linux shell, which you will use by default on most Alliance hardware. Although we teach Bash basics in various online schools and in-person workshops many times a year, there are some useful Bash features and tricks that we never get to teach, due to our usual time constraints. Finally we can share some of them with you in this webinar! In this presentation, we talk about running commands in a subshell, subsetting string variables, Bash arrays, modifying separators with IFS, running Python code from inside self-contained Bash functions, editing your command history, running unaliased versions of commands, handy use of bracket expansion, and a few other topics. Have you experienced issues with unstable SSH connections that prevent you from being productive on your favourite cluster? Do you sometimes wish you had an extra terminal open so that you could run additional programs while waiting for tasks to complete? tmux is a terminal multiplexer (installed on the clusters) that can provide solutions to both connectivity and productivity sore points. It keeps your work session alive (even under the worst network conditions) while also providing you with the power to turn your terminal into a tiled window interface. This talk walks through some of the best features that will help you get work done on the cluster. In this webinar we talk about the following command-line tools: In the presentation, we also use a few other neat command-line tools such as hyperfine - for sophisticated benchmarking - and diff-so-fancy - which makes your diffs a lot more readable. For the Emacs users, we finish the workshop with two Emacs utilities: Modern shell utilities (2025-Jan-28)
Lesser known but very useful Bash features (2022-12-07)
Turbo boost your interactive experience on the cluster with tmux (2022-11-23)
A few of our favourite tools (2022-03-16)
Fun tools to simplify your life in the command line (2020-02-19)
Collapse all webinars in this section
Git version control
Click on each webinar for its recording and materials. While it is important to know how to use Git from the command line, this makes for an austere experience: a series of commands are required to gather information on the state of the working tree, see changes to files, or get a schematic of the commit history. Committing sections of files interactively and other operations are just awkward affairs. On the other hand, the many graphic interfaces for Git are often buggy, slow, and limiting. One option is to write exciting functions with tools such as fzf to make things more friendly and visual. A simpler and more polished option is to use an already built user interface for Git that runs directly in the command line. Lazygit is one such open source tool. After years of development, it is a mature, beautiful tool that allows to perform any Git operation in the command line in a convenient, fast, and visual fashion. In this webinar, I demo how I use lazygit in my daily workflow to run routine as well as more complex Git commands. Lazygit (2024-Oct-29)
Building software with EasyBuild
Click on each webinar for its recording and materials. Installing software using EasyBuild in your cluster home directory (2021-12-08)
Building software on Compute Canada clusters using EasyBuild (2020-04-15)
Collapse all webinars in this section
Authoring scientific documents and static site generators
The new R Markdown: authoring dynamic scientific documents with Quarto (2023-04-11)
This webinar shows how you can easily create beautiful publications -- webpages, PDFs, websites, presentations, books, etc, complete with formatted text, dynamic code and figures -- with Quarto. Quarto is the successor to R Markdown. By combining the powers of Jupyter or knitr with Pandoc, it works with R, but also with Python and Julia code blocks, adding new functionality to the old tool.
- Speaker: Marie-Hélène Burle
- Online slides
- Quarto documentation
Emacs
Click on each webinar for its recording and materials. At the core of Emacs functioning are modes: major modes set the appearance and behaviour of Emacs for various types of texts (e.g. a Python script and a Markdown document display different syntax highlighting and have different functions activated), while minor modes provide additional features that can be turned on or off (e.g. spell checking). Understanding how modes work is key to customizing Emacs and exploiting its strengths. In this webinar, I explain the functioning of Emacs modes and show how to manage and even customize them. Finally, I demo how the package Polymode allows to embed sections of a type of text in another type (e.g. snippets of code in a Markdown document). Emacs might have been created in the 70s, but its development is alive and well. 10 years ago version 24 brought huge speedups with lexical binding. In 2022, version 28 added -- among other things -- just-in-time native compilation for elisp code for improved performance. Version 29 last year brought countless exciting new additions such as official tree-sitter support and built-in Eglot and use-package. In addition to Emacs itself, a profusion of modern packages have emerged over the past few years (e.g. the vertico/consult/orderless/marginalia/embark completion system; corfu and cape for at point completion) bringing great speed and sleekness to the user experience. Once upon a time (not that long ago), powerful text editors such as Vim and Emacs were the only nice interfaces to work with code. Nowadays, there are countless sleek and more GUI-oriented tools such as VS Code, RStudio, or JupyterLab that provide amazing IDEs, without the learning curve. So why would one still use Emacs as a programming IDE? What does that even look like? In this webinar, I show some of the many reasons why I can't let go of Emacs, then show how it can be used as a programming IDE for Python, R, and Julia. Understanding Emacs modes (2024-Sep-17)
Modern Emacs: new tools that make Emacs better and faster (2024-Mar-05)
Emacs as a programming IDE for Python, R, and Julia (2023-Nov-14)