GPU Support With Docker Compose

I use Docker quite a lot as it allows me to install, test, remove libraries or components without any risk to break my main system. VM is another option but even if a linux install is now just a few clicks in virt-manager GUI, it takes time and I don’t need the full isolation provided for this use-case. One thing that is quite difficult to setup in docker is the host GPU sharing. Same issue with the VM, unless having 2 cards installed in the host, giving access to the guest system either have an heavy performance impact, either require unlinking the card from the host to bind it to the guest system (and the host is now headless in the meantime, so hope the switch works without fail or you’ll be blind-fixing the issue in front of a dark screen).

It is usually not a problem as most of the containers are running services, and my emacs can easily be displayed on the host Xserver even when running inside a container. A Jupyter notebook with Tensorflow and GPU support or an instance of Leela to review a Go game is another story…

nvidia-docker, nvidia-docker2 or nvidia-container provided somewhat working solutions but replacing the default docker runtime broke some of my other containers, some were ok with nvidia-docker while others only with nvidia-container, and docker-compose was not usable with them at all. Of course, switching between the different approach required to uninstall/install the correct set of packages with matching versions. I had to rely to wrapper scripts to run the correct docker commands, which is basically the job of docker-compose… Somewhat working, but quite a messy solution.

GPU support was discussed for a long time and it seems we’re near fixing all of this docker-compose 1.28.0-rc1 is announcing “Support for Nvidia GPUs via device requests”.

I’ll update this with a more detailled setup when 1.28 is released, as things may change. The first part is the setup described here, moved to a container. The rest is a few lines to add to the compose yaml

My docker daemon config only move docker files out of /var and define a local registry:

> cat /etc/docker/daemon.json
{
  "data-root": "/home/docker",
  "registry-mirrors": [ "https://192.168.1.178:5000" ]
}

On the host the buster-backports nvidia-driver and nvidia-container-runtime are installed. In the Dockerfile, install nvidia-cuda-dev from buster-backports (thus we match the host driver version).

Tensorflow require cuDNN, so install the matching libcudnn8 too. I have a copy of this in a local debian repository: adding a source.list file to the image then apt-get install is faster than pushing it from the docker context (these libraries are huge).

In the docker-compose yaml, add:

jupyter:
  ...
  deploy:
    resources:
      reservations:
        devices:
        - 'driver': 'nvidia'
          'count': 1
          'capabilities': ['gpu', 'utility']
  devices:
    - /dev/nvidia-uvm:/dev/nvidia-uvm
    - /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools
  ...

The deploy parameters will act as the –gpus option. The nvidia-uvm module is not always loaded, running modprobe beforehand (or force loading the module at boottime) may be required. Cuda will fail to initialize if these devices are not present. Once the devices are available on the host, nvidia-container doesn’t map them into the container, so we manually add them in the compose yaml.