---
description: Worked examples for Slinky clusters, starting with a containerized GPU job using Apptainer and sbatch.
revision_date: 09.07.2026
---

# Tutorials

## Run a containerized PyTorch job

Goal: run a GPU workload from an NGC container image as a normal Slurm batch job. The pattern is pull once, reuse everywhere: convert the image to a SIF file on the shared `/home`, then point every job at it.

### 1. Pull the image to a SIF (once)

Run the pull inside a job so the conversion happens on a worker. NGC PyTorch images are big: give the job memory and CPUs (the SIF compression is parallel) and point the Apptainer cache at node-local `/local` and its temp directory at the per-job `/tmp`, instead of the slower shared `/home`. The cache on `/local` survives the job, so a repeat pull on the same node skips the download. Pick a current [NGC PyTorch tag](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch); Blackwell GPUs need a recent build.

```bash
srun --mem=64G --cpus-per-task=16 --time=01:00:00 bash -c '
  export APPTAINER_CACHEDIR=/local/apptainer-cache APPTAINER_TMPDIR=/tmp/apptainer-tmp
  mkdir -p "$APPTAINER_CACHEDIR" "$APPTAINER_TMPDIR"
  apptainer pull /home/'"$USER"'/pytorch.sif docker://nvcr.io/nvidia/pytorch:25.06-py3
'
```

The pull takes about 10 minutes and the resulting SIF is around 13 GB. Because `/home` is shared and bind-mounted into the [jail](https://docs.verda.com/clusters/instant-clusters/slinky/shared-jail/), the same SIF is now usable from any worker, so this is a one-time step.

### 2. Submit the job

Write the test script and the batch file, then submit. The batch file references absolute paths on `/home` (they are filled in when you create the file, so the job itself does not depend on any runtime environment):

```bash
cat > /home/$USER/torch-test.py <<'EOF'
import torch
n = torch.cuda.device_count()
print(f"visible GPUs: {n}")
for i in range(n):
    print(" ", torch.cuda.get_device_name(i))
x = torch.randn(8192, 8192, device="cuda", dtype=torch.bfloat16)
y = x @ x
torch.cuda.synchronize()
print("matmul OK:", y.shape)
EOF

cat > /home/$USER/torch-test.sbatch <<EOF
#!/bin/bash
#SBATCH --job-name=torch-test
#SBATCH --nodes=1
#SBATCH --gres=gpu:8
#SBATCH --mem=128G
#SBATCH --time=00:10:00

apptainer exec --nv /home/$USER/pytorch.sif python3 /home/$USER/torch-test.py
EOF

sbatch /home/$USER/torch-test.sbatch
```

### 3. Check the result

The job takes under a minute. Watch it with `squeue --me`, then read its output:

```console
$ cat slurm-<jobid>.out
visible GPUs: 8
  NVIDIA B300 SXM6 AC
  NVIDIA B300 SXM6 AC
  ...
matmul OK: torch.Size([8192, 8192])
```

Key points:

* `--nv` exposes the host NVIDIA driver inside the container; `--gres=gpu:8` decides how many GPUs the step (and therefore the container) sees.
* Request memory (`--mem`) for both the pull and the job; image conversion and PyTorch both need it.
* For interactive experimentation, the same works under `srun --pty`:

    ```bash
    srun --gres=gpu:1 --mem=64G --time=00:30:00 --pty \
      apptainer exec --nv /home/$USER/pytorch.sif python3
    ```

More container details (GPU passthrough, what is not available) are on the [Containers](https://docs.verda.com/clusters/instant-clusters/slinky/containers/) page. For multi-user setups with per-user limits, see the [workshop use case](https://docs.verda.com/clusters/instant-clusters/local-users/use-case-workshop/).