Reproducible Computational Environments Using Containers: Introduction to Docker and Singularity

This course aims to introduce the use of containers with the goal of using them to effect reproducible computational environments. Such environments are useful for ensuring reproducible research outputs and for simplifying the setup of complex software dependencies across different systems. The course will introduce the use of Docker and Singularity containers but the material will be of use for whatever container technology you plan to, or end up, using.

After completing this session you should:

  • Have an understanding of what Docker/Singularity containers are, why they are useful and the common terminology used
  • Have a working Docker installation on your local system to allow you to use containers
  • Understand how to use existing Docker containers for common tasks
  • Be able to build your own Docker/Singularity containers by understanding both the role of a Dockerfile/Singularity recipe in building containers, and the syntax used in Dockerfiles/Singularity recipes
  • Understand how to manage Docker/Singularity containers on your local system
  • Appreciate decisions that need to be made around containerising research workflows
  • Understand the differences between Docker and Singularity containers and why Singularity is more suitable for multi-user systems (e.g. HPC)
  • Appreciate issues around reproducibility in software, understand how containers can address some of these issues and what the limits to reproducibility using containers are

The material on this site corresponds to part 1 of the course - covering Docker and running on 30 and 31 May 2023. Part 2 of the course will look at Singularity on 6 and 8 of Jun 2023 and the part 2 material is available on a separate site.

Schedule

Setup Download files required for the lesson
00:00 1. Introducing Containers What are containers, and why might they be useful to me?
00:20 2. Introducing the Docker Command Line How do I know Docker is installed and running?
How do I interact with Docker?
00:30 3. Exploring and Running Containers How do I interact with Docker containers and container images on my computer?
01:00 4. Cleaning Up Containers How do I interact with a Docker container on my computer?
How do I manage my containers and container images?
01:10 5. Finding Containers on Docker Hub What is the Docker Hub, and why is it useful?
01:30 6. Creating Your Own Container Images How can I make my own Docker container images?
How do I document the ‘recipe’ for a Docker container image?
02:05 7. Creating More Complex Container Images How can I make more complex container images?
03:05 8. Examples of Using Container Images in Practice How can I use Docker for my own work?
03:25 9. Containers in Research Workflows: Reproducibility and Granularity How can I use container images to make my research more reproducible?
How do I incorporate containers into my research workflow?
03:45 Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.