Skip to content

Storage

Storage classes

Three storage classes are pre-configured:

StorageClass Access modes Backed by Use for
local-path (default) RWO Node-local NVMe Scratch data, per-node caches, checkpoints — fastest I/O, tied to one node
shared-path RWX / ROX / RWO Shared filesystem (visible on every node) Datasets, shared model-weight caches, outputs that multiple pods on different nodes mount simultaneously
local-disk RWO Raw node-local NVMe PVs (/mnt/local_disk) Manually managed node-local volumes 
$ kubectl get storageclass
NAME                   PROVISIONER                             VOLUMEBINDINGMODE
local-disk             kubernetes.io/no-provisioner            WaitForFirstConsumer
local-path (default)   rancher.io/local-path                   WaitForFirstConsumer
shared-path            cluster.local/shared-path-provisioner   Immediate

Node-local scratch (local-path, default)

Dynamic PVCs against the default class are provisioned on the node where the consuming pod is scheduled — the fastest option for I/O-heavy scratch:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: scratch
spec:
  # storageClassName omitted → default (local-path)
  accessModes: [ReadWriteOnce]
  resources:
    requests:
      storage: 200Gi

The volume lives and dies with that node: it cannot follow a pod that gets rescheduled elsewhere, and a node replacement loses its contents. Use it for data you can regenerate (caches, temporary checkpoints, preprocessing scratch).

Shared, multi-node volumes (shared-path)

shared-path provisions volumes as directories on the cluster's shared filesystem, so the resulting PersistentVolumes have no node affinity and support ReadWriteMany: any pod on any node can mount the same volume simultaneously. This is the right class for the download-once-use-everywhere pattern:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: model-weights
spec:
  storageClassName: shared-path
  accessModes: [ReadWriteMany]
  resources:
    requests:
      storage: 500Gi

Typical uses:

  • Model weights — one download job fills the PVC, every inference/training pod mounts it read-only (see the Dynamo tutorial for a worked example)
  • Datasets shared by all ranks of a multi-node training job
  • Outputs that must survive any single node

Performance note: shared-path I/O goes over the network to the shared filesystem. For bandwidth-critical inner-loop I/O (e.g. streaming dataloader shards), stage data onto local-path scratch first and keep shared-path for the shared source of truth.

The shared filesystem itself

The same shared filesystem backs /home on every node (jumphost and workers), sized when you deploy the cluster — see Shared Filesystem (SFS). Anything under /home is also reachable from pods via hostPath volumes if you prefer path-based access over PVCs, e.g. for quick experiments with data you staged over SSH:

volumes:
  - name: home-data
    hostPath:
      path: /home/ubuntu/datasets
      type: Directory

For production workloads, prefer the shared-path StorageClass — PVCs are quota-friendly, self-documenting and survive refactors better than hard-coded host paths.