This tutorial explains how to build a singularity/Apptainer image on HPC systems from a local docker image. In reality, existing docker images may not be suitable for our purpose. And hence, we have to either modify an existing docker image or build a new one. Unfortunately, the docker-related operations can only be done on our local machines or any host machine where we have privileged root access.
After this tutorial, you will learn to:
Let’s use trimmomatic software, which is a flexible read trimming tool for Illumina NGS data, as available in Quay Registry. Visit the webpage of Quay registry and search for the trimmomatic image (using keyword: trimmomatic) on the top right hand corner of the page. You can find the trimmomatic images from different repositories/accounts. Pick the one under biocontainer repository (i.e., biocontainers/trimmomatic). And also search for different tags available for the image (hint: on the left side menu, click on tags icon) and pick one tag from the list. Once you managed to find a fully qualified URI (i.e., hostname/repository/imagename:tag) for docker image, you are ready for pulling the image. Go to PWD terminal and run the following command (with your own tag) to pull trimmomatic image (here example is with tag: 0.32–hdfd78af_4):
docker pull quay.io/biocontainers/trimmomatic:0.32--hdfd78af_4
In a real-world use case, you may have to modify the image pulled from registries or even build new one on your local machine to meet your needs.
Once an image is pulled or built on your local machine, the image is stored in local registry (usually at /var/lib/docker) on host machine. In order to find the stored docker images, use docker images command.
docker images
From the above command, you need to find an image ID (this is unique to you) of trimmomatic image to save it locally.
Create a tarball of the Docker image (with image id as cc8b303fee58 which would be different for you) using the docker save command as below:
docker save cc8b303fee58 -o trimmomatic_image.tar # in this case image_id is : cc8b303fee58
ls -l trimmomatic_image.tar # to see the local copy of image tarball
Copy the image tarball from PWD to your *scratch folder on Puhti
scp trimmomatic_image.tar YOURCSCUSERNAME@puhti.csc.fi:/scratch/project_xxx/YOURCSCUSERNAME # replace `project_xxxx` with a valid project number
Build Apptainer image from the tarball.
cd /scratch/project_xxxx/YOURCSCUSERNAME # replace `project_xxxx` with a valid project number
apptainer build local_trimmomatic_image.sif docker-archive://trimmomatic_image.tar
Launch Apptainer container and check if you can get a command-line help for trimmomatic software
apptainer exec local_trimmomatic_image.sif trimmomatic --help