Architecture¶

A standard cluster has three node roles: a single login node (jumphost / bastion), a single service node, and N worker nodes (up to 16, each with 8 GPUs). The login node is the only node reachable from the internet — it is the SSH entry point and the NAT gateway for everything behind it. Worker nodes are interconnected by a high-speed InfiniBand fabric (exact topology differs between locations) and all nodes share a /home filesystem.

graph TD
    Internet([Internet / User])
    Internet -->|"SSH :22 · HTTPS :443"| Login

    subgraph cluster [Cluster private network]
        Login["Login node<br/>jumphost / bastion<br/>NAT gateway · Nginx → Grafana"]
        Service["Service node<br/>auth.cluster.verda.internal<br/>Slurm controller·<br/>k8s control plane · monitoring"]
        W1["Worker 1<br/>8 GPUs · local NVMe"]
        W2["Worker 2<br/>8 GPUs · local NVMe"]
        Wn["Worker N<br/>(up to 16)<br/>8 GPUs · local NVMe"]
        IB["InfiniBand fabric<br/>leaf / spine switches<br/>(topology varies by location)"]

        Login --- Service
        Login --- W1
        Login --- W2
        Login --- Wn
        W1 -. InfiniBand .- IB
        W2 -. InfiniBand .- IB
        Wn -. InfiniBand .- IB
    end

The Slurm controller, the Kubernetes control plane and the monitoring/observability stack run on the service node, not on the login host. Worker nodes use the login node as their default gateway and NAT firewall. The Grafana UI is reachable at https://<login-ip>:443 (see Monitoring).

Node naming¶

Cluster node names are based on the Hostname you specify when creating the cluster:

• Login / jump host: hostname-login (still labeled as jumphost in the Console and API)
• Service node: hostname-service, also reachable as auth.cluster.verda.internal
• Worker nodes: hostname-1, hostname-2, etc.

Storage¶

• A shared network filesystem is mounted at /home on every node of the cluster. It is created fresh with the cluster; anything that must outlive the cluster belongs on a shared filesystem you keep and re-attach.
• Each worker node has a local NVMe drive for fast scratch I/O, mounted at /mnt/local_disk on the node. On Kubernetes clusters it backs the local-disk and local-path (default) StorageClasses, alongside the RWX-capable shared-path class on the shared filesystem — see Storage. On Slinky clusters, job steps run inside the shared jail where the same drive backs /tmp, so use /tmp for node-local scratch inside jobs.

Preinstalled software¶

CUDA, doca-ofed and the NVIDIA drivers are installed on each server. HPC-X lives in /opt/hpcx and provides MPI (/opt/hpcx/ompi/bin/mpirun). A PyTorch environment helper (/usr/local/bin/pytorch.setup.sh) is available on all clusters; it installs uv on first run.