---
description: Run containerized workloads on a Slinky cluster with Apptainer/Singularity inside Slurm steps.
revision_date: 27.07.2026
---

# Containers

On this Slinky (Slurm-on-Kubernetes) cluster, the supported way to run containers is **[Apptainer](https://apptainer.org/)** (formerly Singularity) from inside a Slurm step. Every `srun`/`sbatch` step runs in a per-job [shared jail](https://docs.verda.com/clusters/instant-clusters/slinky/shared-jail/), and Apptainer works inside that jail.

!!! info
    `singularity` is an alias of `apptainer` (the `singularity` binary is a symlink to `apptainer`), so either name works. [`enroot`](https://github.com/NVIDIA/enroot) is also installed for manual use.

## Run a container in a Slurm step

Pull and run a public image directly. Apptainer converts the OCI image to a SIF on the fly:

```console
$ srun apptainer exec docker://alpine:3 cat /etc/os-release
NAME="Alpine Linux"
PRETTY_NAME="Alpine Linux v3.24"
```

See the [Slurm documentation](https://slurm.schedmd.com/man_index.html) for the full `srun`/`sbatch` reference, including `--time`, `--mem` and partition selection.

## GPU containers

Request GPUs with `--gpus=N` on the step and pass `--nv` to Apptainer so the host NVIDIA driver userspace is exposed inside the container. Exactly the requested number of GPUs appears:

```console
$ srun --gpus=1 apptainer exec --nv docker://nvidia/cuda:13.0.0-base-ubuntu24.04 nvidia-smi -L
GPU 0: NVIDIA B300 SXM6 AC (UUID: GPU-...)
```

!!! warning
    The step is GPU-cgroup fenced. A step launched with **no** `--gpus` sees no GPUs — `nvidia-smi -L` prints `No devices found.` from inside the container. Always request `--gpus=N` for GPU work.

!!! tip
    Pick an image whose CUDA / framework build matches the GPU architecture. These are Blackwell **B300** GPUs (`compute_cap` 10.3), so use CUDA 13.x and a recent framework build (e.g. a current PyTorch/NGC release). CUDA 12.x images predate B300 support and may fail to run kernels even though `nvidia-smi` lists the device.

## Pull once, reuse many times

Converting an OCI image on every step is wasteful for large images. Pull to a SIF in the shared `/home` filesystem once, then point every step at that file:

```bash
# Pull once, inside a job
srun --time=00:10:00 bash -c '
  export APPTAINER_CACHEDIR=/local/apptainer-cache APPTAINER_TMPDIR=/tmp/apptainer-tmp
  mkdir -p "$APPTAINER_CACHEDIR" "$APPTAINER_TMPDIR"
  apptainer pull /home/ubuntu/cuda.sif docker://nvidia/cuda:13.0.0-base-ubuntu24.04'

# In any step — reuse the local SIF (no re-download, no conversion)
srun --gpus=1 apptainer exec --nv /home/ubuntu/cuda.sif nvidia-smi -L
```

Because `/home` is shared (and bind-mounted into the jail), the same SIF is usable from any worker, including multi-node jobs.

!!! tip "Cache and temp directories"
    Set `APPTAINER_CACHEDIR` and `APPTAINER_TMPDIR` to node-local paths as above. The default cache lives under `$HOME`, which puts multi-gigabyte layers on the shared filesystem. The cache is not reclaimed automatically — clear it with `apptainer cache clean` when you are done.

## What is not available

### pyxis / `srun --container-image`

NVIDIA [pyxis](https://github.com/NVIDIA/pyxis) is **not** enabled on this cluster, so `srun --container-image=...` does not work:

```console
$ srun --container-image=alpine:3 true
srun: unrecognized option '--container-image=alpine:3'
```

The shared jail owns the step's mount namespace, and pyxis would need to own the same namespace — the two are mutually exclusive. Use Apptainer instead. See [Shared jail](https://docs.verda.com/clusters/instant-clusters/slinky/shared-jail/) for details on how the jail works.

### `docker run`

The `docker` client binary exists in the jail, but there is **no in-job Docker daemon**, so `docker run` (and any other daemon-backed command) fails inside a step:

```console
$ srun docker run --rm alpine:3 true
failed to connect to the docker API at unix:///var/run/docker.sock ...
```

To run a Docker image, reference it with Apptainer (`docker://...`) as shown above, or pre-pull it to a SIF.