Expanse: Singularity Containers

Singularity is a container platform that allows you, as an HPC user, to pack the applications and libraries into a single container (as a Singularity image). With the Singularity image, you can share, move and reproduce the application you build in the container. You can also build the image outside the supercomputer cluster with your laptop, which will give you complete control over choosing any major Linux distributions and system libraries and can decide how to install the application you need. What is more important is, unlike other container platforms such as Docker, Singularity is dedicated to running on clusters! Therefore, once the Singularity image is created, you can quickly run it on supercomputer clusters, including TSCC and Expanse at SDSC. You do not have to worry about how to install all the software you need in each different cluster.

In this webinar, we will first show you how to build Singularity images on your local resource, such as your laptop. Then, we will introduce some commonly used repositories, e.g., Docker Hub, so you will learn where to pull other containers to create the Singularity images. For more complicated software requests, we will introduce the proper workflow to build a Singularity sandbox for test purposes and how to record them into a Singularity recipe so a final custom container can be created accordingly. Once a Singularity image is created, it can be uploaded and run on an SDSC cluster. We will show three ways to run the applications within the container. Based on the cluster file system structure, we will introduce the binding point technique from Singularity to locate the proper storage and how to add binding points into the recipe. Also, we will let you know how to make a short command to replace the long Singularity run command.

Instructor
Yuwu Chen
Computational Data Science Researcher, SDSC

Yuwu Chen received his PhD from Louisiana State University in chemical engineering with an emphasis on molecular dynamics simulation of proteins. Yuwu’s key responsibility is to provide a high degree of HPC customer support and professionalism to the research communities of UCSD and other institutions. Yuwu is a Singularity user and has been providing Singularity user support for HPC users since 2019.