---
description: What Slinky (Slurm on Kubernetes) is, how a Slinky instant cluster is put together, and what differs from a traditional Slurm cluster.
revision_date: 03.11.2026
---

# Slinky (Slurm on Kubernetes)

**Slinky** is [SchedMD's](https://github.com/SlinkyProject) way of running Slurm on top of Kubernetes: the Slurm control plane runs as Kubernetes pods, and each worker node runs a `slurmd` pod. You get the normal Slurm experience (`sbatch`, `srun`, `squeue`, `sacct`) on a Kubernetes-managed cluster.

Slinky is its own orchestrator option when you deploy an instant cluster, alongside plain Kubernetes. If you selected it, this is what you are running. It differs from a conventional Slurm installation in a few user-visible ways, documented in this section.

## Architecture

```mermaid
graph TD
    User([User])
    User -->|"SSH :22"| Pod
    User -->|"SSH :2222 admin"| Login

    subgraph cluster [Cluster private network]
        Login["Login node<br/>admin shell · kubectl"]
        subgraph service [Service node]
            Op["Slinky operator"]
            Ctl["slurm-controller<br/>(slurmctld)"]
            Acct["accounting<br/>(slurmdbd + DB)"]
            Rest["slurm-restapi"]
            Pod["login pod"]
        end
        subgraph workers [GPU workers]
            W1["slurmd pod · 8 GPUs<br/>jobs run here"]
            Wn["slurmd pod · 8 GPUs<br/>jobs run here"]
        end
        Login --- service
        Ctl --- W1
        Ctl --- Wn
    end
```

* SSH on port `22` lands in the **login pod**, where every user (including `ubuntu`) is themselves and runs the Slurm CLI directly. The **login node** behind it is the admin shell on port `2222` (`sudo`, `kubectl`, user management); see [User management](https://docs.verda.com/clusters/slinky/users/#how-users-log-in).
* The whole **Slurm management plane** (operator, controller, accounting, REST API and login pod) runs on the **service node**, so workers do nothing but run jobs.
* Each **worker** runs a `slurmd` pod that owns all 8 GPUs. Job steps run inside that pod as the submitting user; see [Job environment](https://docs.verda.com/clusters/slinky/job-environment/).
* `/home` is shared across the login node, the login pod and all workers: build environments once, use them from every job.
* Storage inside jobs: `/shared` is a common dataset area on the shared filesystem, and `/local` and `/tmp` are on the worker's NVMe. `/local` persists across jobs on that node; `/tmp` is private to each job and deleted when it ends. Extra shared filesystems you attach are mounted in the login pod and the worker pods as well, at `/<SFS_NAME>`.
* The operator owns `slurm.conf`. Configuration changes go through the controller resource, not through files; see [Changing the Slurm configuration](#changing-the-slurm-configuration) below.

## What is different on Slinky

* **[User management](https://docs.verda.com/clusters/slinky/users/)**: add users with one command, `slinky-user-add`, instead of driving Kanidm by hand. Users SSH straight into the Slurm login pod on port `22`, and identities can survive cluster re-provisioning via the identity ledger (replay).
* **[Job environment](https://docs.verda.com/clusters/slinky/job-environment/)**: steps run in the worker's `slurmd` pod (Ubuntu 24.04), not on the host. GPUs appear only when you request `--gpus`, and HPC-X/Lmod modules are not available inside jobs.
* **[Containers](https://docs.verda.com/clusters/slinky/containers/)**: Pyxis (`srun --container-image`) with Enroot, and Apptainer. There is no in-job Docker daemon.
* **[Observability](https://docs.verda.com/clusters/slinky/observability/)**: Slinky-specific Grafana dashboards, centralized metrics, and custom prolog/epilog hooks.

## What is the same

* The Slurm CLI and job model, see the [Slurm documentation](https://slurm.schedmd.com/man_index.html) and [Getting started](https://docs.verda.com/clusters/slinky/quickstart/).
* Python: run Python workloads in a container with Pyxis for now, not from a virtualenv on `/home`. See [Python environments](https://docs.verda.com/clusters/slinky/job-environment/#python-environments).
* [Monitoring](https://docs.verda.com/clusters/cluster-basics/monitoring/) access and [validation](https://docs.verda.com/clusters/cluster-basics/validation/) work as on any instant cluster; the dashboards and checks are Slinky-aware.

## Changing the Slurm configuration

On Slinky clusters there is **no `/etc/slurm/slurm.conf` to edit**. The operator generates `slurm.conf` and distributes it to the controller and all workers, so a hand-edited file is overwritten on the next reconcile. Extra settings go into the controller's `extraConf` field, whose lines are appended to the generated `slurm.conf`.

To add or change a setting (example: enable accounting enforcement):

1. Open the controller resource in an editor:

    ```bash
    kubectl -n slurm edit controller slurm
    ```

2. Find `spec.extraConf` (a multi-line text block) and append your line at the end, keeping the indentation:

    ```yaml
    extraConf: |
      # ...existing lines...
      AccountingStorageEnforce=associations,limits,qos
    ```

3. Save and exit. The operator regenerates `slurm.conf` and applies it live (equivalent to `scontrol reconfigure`), with no control-plane downtime and no pod restart.

4. Verify from the login node:

    ```bash
    scontrol show config | grep AccountingStorageEnforce
    ```

Two footnotes:

* A small set of Slurm parameters only take effect on a full `slurmctld` restart (noted in the [slurm.conf reference](https://slurm.schedmd.com/slurm.conf.html)). For those, delete the controller pod and let the operator recreate it: `kubectl -n slurm delete pod slurm-controller-0`.
* If you manage the `slurm` Helm release yourself, set `controller.extraConf` in the release values (`helm upgrade slurm <slurm-chart> -n slurm --reuse-values -f values.yaml`) instead: a live `kubectl edit` is overwritten by the next `helm upgrade` of the release.

## In this section

[Getting started](https://docs.verda.com/clusters/slinky/quickstart/)

[User management](https://docs.verda.com/clusters/slinky/users/)

[Job environment](https://docs.verda.com/clusters/slinky/job-environment/)

[Containers (Pyxis, Apptainer)](https://docs.verda.com/clusters/slinky/containers/)

[Observability](https://docs.verda.com/clusters/slinky/observability/)

[Tutorials](https://docs.verda.com/clusters/slinky/tutorials/)
