• DockerExample

This page describes a very simple project with a Dockerfile and a Makefile as required for all projects and theses supervised in our group. The basic idea is really easy: if you were afraid of Docker before, you won't be anymore after reading this page.

We also provide a more extensive reproducibility example on Github. This could be overwhelming at first, so we recommend that you read this page first, if you are new to the Docker-world.

Example Project

Let's consider a tiny example project, which consists of two files. The first file is a Makefile with the following contents (the first character in the second line must be TAB, this is standard Makefile syntax).

help:
        @echo "Help yourself before you help others :-)"

The second file is a simple bash startup script, traditionally called bashrc (the rc stands for run commands) with the following contents. This is used to turn bash completion on (a must-have feature) and to print a welcome message when the container has started:

if [ -f /etc/bash_completion ]; then source /etc/bash_completion; fi
echo
echo "Welcome to this Docker container, type \"make help\" to get some help"
echo

Dockerfile

In the same folder as your Makefile, create a file Dockerfile with the following contents:

FROM ubuntu:16.04
LABEL maintainer="Firstname Lastname <your@ma.il>"
RUN apt-get update && apt-get install -y make vim && rm -rf /var/lib/apt/lists/*
COPY Makefile Makefile
COPY bashrc bashrc
CMD ["/bin/bash", "--rcfile", "bashrc"]

# docker build -t <firstname>-<lastname>-[project|thesis] .
# docker run -it -v $(pwd)/path/to/input:/input:ro -v $(pwd)/path/to/output:/output:rw --name <firstname>-<lastname>-[project|thesis] <firstname>-<lastname>-[project|thesis]

The first line (FROM ...) says that we want to build an image based on Ubuntu 16.04. The second line (LABEL ...) sets the Author field of the image, you should of course use your own name and email address here. The third line (RUN ...) installs make and vim in the image (it's a naked Ubuntu 16.04, with almost nothing pre-installed); the rm -rf clears the apt cache, which helps to keep the image size small. The fourth line (COPY ...) copies the Makefile to the image (you can also copy whole directories here). The fifth line (CMD) specifies the command to be run when you run the container (in this case, we will be in an interactive bash shell, which executes the commands in the bashrc file at the beginning).

The last two lines are comments and their purpose is purely for documentation. They should always be there, however, so that it's clear with which command-line arguments the image should be built and the container should be run. See the next section for an explanation of the options.

You find more tips in this list of Dockerfile best practices.

Building the image and running it in a container

Now we can build the image (conceptually, a whole Ubuntu 16.04 with our Makefile from above in it, in reality, Docker will reuse as much as possible from the machine on which you are building the image) and run it inside a container:

docker build -t <firstname>-<lastname>-[project|thesis] .
docker run -it -v $(pwd)/path/to/input:/input:ro -v $(pwd)/path/to/output:/output:rw --name <firstname>-<lastname>-[project|thesis] <firstname>-<lastname>-[project|thesis]

The -t option in the first line specifies a name and the dot says that docker should look for everything in the current directory.

The -it option in the second line says that we want to run in interactive mode. That is, output will be written to the console, and input can be entered via the console.

The -v option is important to keep the image size small and should be used whenever your container reads or writes large amounts of data. The option specifies directories that should be accessible from within the container for reading or writing or for both reading and writing. You can have as many of these as you need. Usually, there is one for input and one for output. The part of the argument before the first colon specifies the full path to the directory outside the docker container, the part of the argument after the first colon specifies the full path of the director from within the docker container. The (optional) part after the second colon specifies if the directory is read-only ("ro") or writable ("rw") in your docker container.

The --name options specifies the name of the container. It is useful to identify the container among the (often many) container running on a machine. It's okay to take the same name as for the image (although, technically, an image and a container are two different things). Without the --name option, docker creates an artificial name for the container.

After the commands above, you should see a prompt of the following kind

root@4b3dbcdd8fff:/#

The root says that you are the root user (but only inside of this container). The part after the @ is the name of the container. You can now do 'ls -l' and you will see the typical directory structure of Ubuntu, as well your Makefile. You can edit it with vim if you like. Or you can just run make. Note that we explicitly specified in the Dockerfile to install vim and make, otherwise the two commands would not be available (but we could install them ourselves in the container with apt-get install ..., but we would have to do that every time we run the container).

Testing the Dockerfile on our machines

On our machines you cannot run sudo docker, because of security reasons. Instead, on most machines, we have made available wharfer, which can be used just like docker, but without the security risks. On those machines, you can run the following commands, with the same effect as described above. Please inform your supervisor, if wharfer is not installed on the machine you are working on.

wharfer build -t <firstname>-<lastname>-[project|thesis] .
wharfer run -it -v $(pwd)/path/to/input:/input:ro -v $(pwd)/path/to/output:/output:rw --name <firstname>-<lastname>-[project|thesis] <firstname>-<lastname>-[project|thesis]

Useful docker commands

docker ps | grep qlever      # show all containers with a matching name
docker images | grep qlever  # show all images with a matching name
docker stats                 # show the resource consumptions of the running containers
docker exec -it <name> bash  # run bash in running container (useful for debugging)
docker rm <name>             # remove container (rm --force if it is still running)
docker pause <name>          # pause the execution of the container (continue with docker unpause <name>)