Using Containers in Supercomputing Environment

Sometimes, container images may not be readily available in registries for our purpose. In that case, we have to either modify an existing docker image or build a new one. Unfortunately, the docker-related operations can only be done on our local machines or any host machine where we have privileged root access. This tutorial explains how to save docker image locally and eventually use it with Apptainer in HPC environment.

Part of this tutorial requires a workstation with docker client installed and thus can’t be done in CSC HPC environment. Use PWD terminal instead.

Expected outcome of this tutorial:

After this tutorial, you will learn to:

Save a docker image locally ( In PWD environment)
Launch a Appainer container from a local docker image (In Puhti environment)

Converting a local docker image to Apptainer

Let’s use the [trimmomatic software] as an example software as available in Red hat Quay Registry. In PWD terminal, run the following command to pull an image:
```
 docker pull quay.io/biocontainers/trimmomatic:0.32--hdfd78af_4
```
In a real-world use case, you may want to modify the image pulled from registries or even build new one on your local machine to meet your needs.
Once an image is pulled or built on your local machine, the image is stored in local registry (usually at /var/lib/docker) on host machines. In order to find the stored docker images, use docker images command.
```
docker images
```
From the above command, you need to find an image ID of trimmomatic image to save it locally.

Create a tarball of the Docker image (with image id which in my case is cc8b303fee58) using the docker save command as below:

docker save cc8b303fee58 -o trimmomatic_image.tar  # in this case image_id is : cc8b303fee58
ls -l  trimmomatic_image.tar  # to see the local copy of image tarball

Copy the image tarball from PWD to your scratch folder on Puhti
```
scp trimmomatic_image.tar yourcscusername@puhti.csc.fi:/scratch/project_xxx/yourcscusername
```
‼️ This step requires CSC password. if you don’t want to share your password here, you can skip this step and download the tar file directly from Puhti instead

Build Apptainer image from the tarball.

 cd /scratch/project_2xxxx/yourcscusername  # replace `project_2xxxx` with a valid project number
 # uncomment the following command if you have not copied the tar file from PWD
 # wget https://a3s.fi/biocontainers2024/trimmomatic_image.tar
 apptainer build local_trimmomatic_image.sif docker-archive://trimmomatic_image.tar

Launch Apptainer container and check if you can get a command-line help for trimmomatic software
```
apptainer exec local_trimmomatic_image.sif trimmomatic --help
```

This site is open source. Improve this page.