Beginner's guides
Getting Started Resources
Table of Contents: Tips & tricks on getting programming help • Introduction to HPC • Linux command line • Git version control • Building software with EasyBuild • Managing Cluster Instability for Users • Scientific documents and static site generators • Emacs talks
“Tips & tricks on getting programming help”
Webinar (2020-11-13) by Marie-Hélène Burle
Introduction to HPC
- Supercomputing in Canada: An Introduction (PDF slides)
- High Performance Computing: Short Introduction - Univ. of Manitoba perspective (PDF slides)
“How to submit and run jobs on Compute Canada HPC systems”
Webinar (2018-Sep-19) - Introduction to working on our HPC systems for new users.
Other materials
Read WestGrid’s QuickStart Guide for Getting Started on CC National Systems.
Watch Compute Canada’s Getting Started with CC National Systems mini-webinars – a series of short video tutorials that demonstrate how to access the national clusters, submit jobs, access software, avoid common mistakes, and get help from the Compute Canada support team.
Browse the latest materials from WestGrid’s Introductory HPC course (ZIP file).
Linux command line
For a basic introduction to Bash command line – as well as programming in Python or R, and version control with Git – please check out a Software Carpentry workshop closest to you. For an online tutorial, we highly recommend their Introduction to using the shell in a HPC context.
“Fun tools to simplify your life in the command line”
Webinar (2020-02-19) by Marie-Hélène Burle
“A few of our favourite tools”
Webinar (2022-03-16) by Marie-Hélène Burle and Alex Razoumov
In this webinar we talk about the following command-line tools:
- lazygit a wonderful terminal UI for Git,
- bat: a great syntax highlighter,
- ripgrep: a fast alternative to grep,
- fd: a really fast alternative to find, and
- pass: a command line password manager.
In the presentation, we also use a few other neat command-line tools such as hyperfine - for sophisticated benchmarking - and diff-so-fancy - which makes your diffs a lot more readable. For the Emacs users, we finish the workshop with two Emacs utilities:
- TRAMP: a remote file access system,
- Helm: a “framework for incremental completions and narrowing selections”.
“Turbo boost your interactive experience on the cluster with tmux”
Webinar (2022-11-23) by Chris Want
Have you experienced issues with unstable SSH connections that prevent you from being productive on your favourite cluster? Do you sometimes wish you had an extra terminal open so that you could run additional programs while waiting for tasks to complete? tmux is a terminal multiplexer (installed on the clusters) that can provide solutions to both connectivity and productivity sore points. It keeps your work session alive (even under the worst network conditions) while also providing you with the power to turn your terminal into a tiled window interface. This talk walks through some of the best features that will help you get work done on the cluster.
“Lesser known but very useful Bash features”
Webinar (2022-12-07) by Alex Razoumov and Marie-Hélène Burle
Knowing basic Linux commands is essential for researchers using remote systems such as HPC clusters. Bash is the most commonly used Linux shell, which you will use by default on most Alliance hardware. Although we teach Bash basics in various online schools and in-person workshops many times a year, there are some useful Bash features and tricks that we never get to teach, due to our usual time constraints. Finally we can share some of them with you in this webinar!
In this presentation, we talk about running commands in a subshell, subsetting string variables, Bash arrays, modifying separators with IFS, running Python code from inside self-contained Bash functions, editing your command history, running unaliased versions of commands, handy use of bracket expansion, and a few other topics.
Git version control
Building software with EasyBuild
“Installing software using EasyBuild in your cluster home directory”
Webinar (2021-12-08) by Ata Roudgar
“Building software on Compute Canada clusters using EasyBuild”
Webinar (2020-04-15) by Ali Kerrache
Managing Cluster Instability for Users
Presentation on dealing with hardware and software instabilities on Cedar from a user’s perspective: causes, workarounds, bad workflows, and best practices. It was part of a larger WestGrid’s March 2019 Town Hall – the embedded video below starts at the relevant section.
- Combined PDF slides from all presenters from WestGrid’s March 2019 Town Hall
Authoring scientific documents and static site generators
“The new R Markdown: authoring dynamic scientific documents with Quarto”
Webinar (2023-04-11) by Marie-Hélène Burle
This webinar shows how you can easily create beautiful publications – webpages, PDFs, websites, presentations, books, etc, complete with formatted text, dynamic code and figures – with Quarto. Quarto is the successor to R Markdown. By combining the powers of Jupyter or knitr with Pandoc, it works with R, but also with Python and Julia code blocks, adding new functionality to the old tool.
Emacs
“Emacs as a programming IDE for Python, R, and Julia”
Webinar (2023-Nov-14) by Marie-Hélène Burle
Once upon a time (not that long ago), powerful text editors such as Vim and Emacs were the only nice interfaces to work with code. Nowadays, there are countless sleek and more GUI-oriented tools such as VS Code, RStudio, or JupyterLab that provide amazing IDEs, without the learning curve. So why would one still use Emacs as a programming IDE? What does that even look like? In this webinar, I show some of the many reasons why I can’t let go of Emacs, then show how it can be used as a programming IDE for Python, R, and Julia.
“Modern Emacs: all those new tools that make Emacs better and faster”
Webinar (2024-Mar-05) by Marie-Hélène Burle
Emacs might have been created in the 70s, but its development is alive and well. 10 years ago version 24 brought huge speedups with lexical binding. In 2022, version 28 added – among other things – just-in-time native compilation for elisp code for improved performance. Version 29 last year brought countless exciting new additions such as official tree-sitter support and built-in Eglot and use-package. In addition to Emacs itself, a profusion of modern packages have emerged over the past few years (e.g. the vertico/consult/orderless/marginalia/embark completion system; corfu and cape for at point completion) bringing great speed and sleekness to the user experience.