---
description: "Everything covered in each Verda product's Reference section, for users past the basics."
revision_date: 25.10.2026
---

# Advanced Guide

Each topic below is a short summary of a reference page: what it is for and the key steps. Follow the link at the end of each topic for the full guide, with every option and edge case.

## Instances

### Using Jupyter

JupyterLab runs on your instance on port `8888`. Never expose that port to the internet; reach it over an SSH tunnel instead.

1. Deploy an instance with **JupyterLab** as the Operating System, and [secure it](https://docs.verda.com/instances/secure-your-instance/).
2. Open a tunnel from your own machine and keep it open while you work:

    ```bash
    ssh -L 8888:localhost:8888 root@IP_OF_YOUR_INSTANCE
    ```

3. Open `http://127.0.0.1:8888/` and enter the token. Copy it from the `token=...` part of the URL behind **Open JupyterLab** on the instance card.

To work from VS Code instead, keep the tunnel open, install the **Jupyter** extension, choose **Existing Jupyter Server**, and enter `http://127.0.0.1:8888` with the same token.

See [Using Jupyter](https://docs.verda.com/instances/using-jupyter/) for the full guide.


### Confidential Computing

Confidential VMs (CVMs) keep your data encrypted while it is being processed. They run inside a hardware-enforced Trusted Execution Environment that encrypts CPU and GPU memory and isolates your workload from the hypervisor and from Verda. Available on RTX PRO 6000 (single GPU), B200, and B300.

- **Verify before you trust it.** Check CPU memory encryption with `sudo dmesg | grep -i sev-snp`, attest the CPU with `snpguest`, and attest the GPU with `nvattest attest --device gpu --verifier remote`. A helper script verifies the full measured boot chain.
- **Encrypt data at rest** with an encrypted home folder (`adduser --encrypt-home`), or boot your own LUKS2-encrypted OS that only unlocks after attestation.

!!! note "CVMs cannot reboot from inside the guest"
    An in-guest `reboot` will not bring a CVM back. To restart, use **Shutdown**, then **Start** in the console or API.

See [Confidential Computing](https://docs.verda.com/instances/confidential-computing/) for the full guide.


### Startup environment and variables

To set environment variables that persist across logins, add a **startup script** that appends them to `/etc/environment`:

```bash
#!/bin/bash
cat << 'EOF' >> /etc/environment
SECRET1='SECRET1VALUE'
EOF
```

See [Startup environment and variables](https://docs.verda.com/instances/startup-environment-and-variables/) for the full guide.


### Copy files between block devices

Find each volume's device path in the instance's **Storage** tab, then pick one of two tools:

- **`dd`** makes a full copy of a volume, for example `dd if=/dev/vda of=/dev/vdb bs=96M status=progress`. Double-check `if=` (input) and `of=` (output): a mistake destroys data.
- **`rclone`** copies folders between [mounted volumes](https://docs.verda.com/storage/block-volumes/attach-a-block-volume/), for example `rclone copy --progress SRC DST`.

See [Copy files between block devices](https://docs.verda.com/instances/copy-files-between-block-devices/) for the full guide.


### Profile with Nsight

Current Verda images come with NVIDIA Nsight Systems (`nsys`) and Nsight Compute (`ncu`) preinstalled, and neither needs `sudo`. Use `nsys` to find where your application spends time, then `ncu` to inspect a single kernel:

```bash
nsys profile --trace=cuda -o timeline ./your_app
ncu --kernel-name 'regex:KERNEL_NAME' --launch-count 1 -o kernel ./your_app
```

If you're not signed in as root, run `export PATH=/usr/local/cuda/bin:$PATH` first.

See [Profile with Nsight](https://docs.verda.com/instances/profile-with-nsight/) for the full guide.


## Private Networks

!!! warning "Pilot feature"
    Private Networks is in a limited pilot.

### Inside an instance

- **Only the private IP is on the interface.** The public IP is applied as 1:1 NAT outside the instance, so bind services to the private IP or `0.0.0.0`.
- **Read your own addresses** from the metadata service: `curl http://169.254.169.254/latest/meta-data/`. It has no authentication, so block TCP port 80 to that address if you run untrusted code.
- **No public IP means no outbound internet** until you set up [internet access](https://docs.verda.com/private-networks/internet-access/).

See [Inside an instance](https://docs.verda.com/private-networks/inside-an-instance/) for the full guide.


### Security model

Each private network is isolated from other customers, even when they use the same address ranges. Networks belong to one project; changing them requires the **developer** role or above.

See [Security model](https://docs.verda.com/private-networks/security-model/) for the full guide.


## Clusters

### Architecture

A cluster has one **login node**, one **service node**, and up to 16 **worker nodes** with 8 GPUs each.

- **Login node** (`hostname-login`): the only node reachable from the internet, and the SSH entry point.
- **Service node** (`hostname-service`): runs Slurm, Kubernetes, and monitoring. Grafana is at `https://<login-ip>`.
- **Workers** (`hostname-1`, `hostname-2`, ...): connected by InfiniBand.
- **Storage:** `/home` is shared by all nodes and deleted with the cluster. Each worker has fast local scratch at `/mnt/local_disk`.

See [Architecture](https://docs.verda.com/clusters/cluster-basics/architecture/) for the full guide.


### Networking and ports

- **Open ports on the login node:** `22` (SSH), `80` and `443` (Grafana over HTTPS), and `2222` (seamless SSH, Slinky only). Everything else is blocked.
- **Workers aren't reachable from the internet.** SSH to the login node, then to a worker by name (`ssh hostname-1`).
- **NCCL in containers:** on H200 and B200 clusters, run containers with `-e NCCL_IB_PKEY=1`. B300 clusters need no extra setting.

See [Networking and ports](https://docs.verda.com/clusters/cluster-basics/networking/) for the full guide.


### Tutorial: deploying vLLM inference on Instant Cluster using Ray

Serve a model across more than 8 GPUs by running vLLM on Ray across several nodes. This runs directly on the nodes, outside Slurm or Kubernetes.

1. Install vLLM and Ray on each node, and download the model.
2. Start Ray on the first node with `ray start --head`, then join each worker with `ray start --address="FIRST_NODE_IP:6379"`.
3. Start vLLM on the first node with `--tensor-parallel-size` set to GPUs per node and `--pipeline-parallel-size` to the number of nodes.

See [Deploying vLLM inference on Instant Cluster using Ray](https://docs.verda.com/clusters/tutorials/deploying-vllm-inference-on-instant-cluster-using-ray/) for the full guide.


### Gang-scheduled Multi-node Training with SkyPilot + Kueue

Launch multi-node training on a Kubernetes cluster with a single `sky launch`. Kueue starts all pods of a job together or not at all, and queues the rest.

1. Install SkyPilot locally (`pip install 'skypilot[kubernetes]'`). Recent cluster images already include Kueue.
2. Apply a queue manifest with your cluster's GPU, CPU, and memory quota.
3. Set your kubeconfig namespace to the queue's namespace. This is the most commonly missed step.
4. Launch the job with `sky launch`, and release its quota afterwards with `sky down`.

See [Gang-scheduled Multi-node Training with SkyPilot + Kueue](https://docs.verda.com/clusters/tutorials/gang-scheduled-multi-node-training-with-skypilot-kueue/) for the full guide.


### Deploying NVIDIA Dynamo on a Kubernetes Instant Cluster

NVIDIA Dynamo serves LLMs at scale behind an OpenAI-compatible endpoint, splitting prefill and decode across pods. You need an NGC API key, a Hugging Face token, `helm`, and the `shared-path` StorageClass set as default.

1. Run Dynamo's pre-deployment check, and install the GPU Operator if it is missing.
2. Install the Dynamo CRDs and platform with Helm.
3. Create secrets for your NGC key and Hugging Face token.
4. Apply a deployment request for your model and GPU type, then test the endpoint with `curl`.

To clean up, delete the generated deployment (DGD) first; deleting the request (DGDR) alone leaves the pods running.

See [Deploying NVIDIA Dynamo on a Kubernetes Instant Cluster](https://docs.verda.com/clusters/tutorials/deploying-nvidia-dynamo-on-a-kubernetes-instant-cluster/) for the full guide.


## Storage

### Tag immutability rules

Immutability rules stop images with matching tags from being overwritten or deleted, so a tag always points to the same image. Each rule matches repositories and tags using the [tag rules syntax](#tag-rules-syntax). For example, repositories **matching** `**` and tags **matching** `**` makes every tag immutable.

See [Tag immutability rules](https://docs.verda.com/storage/container-registry/tag-immutability-rules/) for the full guide.


### Tag retention rules and runs

Retention rules decide which images to keep, by age, tag, and repository. A **retention run** deletes every image that no rule keeps. Immutable tags are never deleted. Run a **dry run** first to preview what would be removed.

See [Tag retention rules and runs](https://docs.verda.com/storage/container-registry/tag-retention-rules-and-runs/) for the full guide.


### Tag rules syntax

Tag rules use glob patterns, not regex:

| Pattern | Matches |
|---------|---------|
| `*` | Any characters except `/` |
| `**` | Any characters including `/`, only as a whole path segment (`**/redis`) |
| `?` | One character |
| `[ab]` | One character from the set |
| `{beta,rc}` | Any one of the alternatives |

!!! warning
    `prefix-**` does not match nested paths. Write `prefix-*/**` instead.

See [Tag rules syntax](https://docs.verda.com/storage/container-registry/tag-rules-syntax/) for the full guide.


## Serverless Containers

### Quickstart: deploy with vLLM

Deploy `Qwen/Qwen2.5-1.5B-Instruct` with the official vLLM image. You need a Hugging Face token with `READ` permission.

1. Go to **Containers → New deployment**, name it, and pick a General Compute (24 GB VRAM) GPU.
2. Set the image to `docker.io/vllm/vllm-openai:v0.29.0` (**Public**), the port to `8000`, and the health check path to `/health`.
3. Turn on **Start Command** with this CMD, add `HF_TOKEN` as an environment variable, and deploy:

    ```bash
    --model Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.9 --model-loader-extra-config '{"enable_multithread_load": true}'
    ```

4. Create a key under **Credentials → Inference API Keys** and send a test request:

    ```bash
    curl -X POST <YOUR_CONTAINERS_API_URL>/v1/chat/completions \
    --header 'Authorization: Bearer <YOUR_INFERENCE_API_KEY>' \
    --header 'Content-Type: application/json' \
    --data '{"model": "Qwen/Qwen2.5-1.5B-Instruct", "messages": [{"role": "user", "content": "What is deep learning?"}]}'
    ```

See [Quickstart: deploy with vLLM](https://docs.verda.com/serverless-containers/quickstart/) for the full guide.


### Quickstart: migrate from Runpod

Serverless Containers serve a plain HTTP API, so replace the `runpod` handler with a small web server.

1. Wrap your existing handler in FastAPI with a `/health` route and a route for requests (for example `/query`), served by Uvicorn on port `8000`.
2. In your Dockerfile, replace the `runpod` package with `fastapi` and `uvicorn`, then build and push the image with a version tag.
3. Deploy it with port `8000`, health check path `/health`, and `HF_TOKEN` if the model needs it. Set `HF_HOME` to a path under `/data` so weights aren't downloaded again on every start.

See [Quickstart: migrate from Runpod](https://docs.verda.com/serverless-containers/tutorials/quickstart-migrate-from-runpod/) for the full guide.


### Quickstart: GPT-OSS 120B with Ollama

Serve OpenAI's GPT-OSS 120B from your own Ollama image.

1. [Publish your Ollama image](#publish-your-first-docker-image) with a version tag; `:latest` is not allowed.
2. Deploy it on a GPU with at least **80 GB of VRAM**, with port `8000` and health check path `/api/tags`. An incorrect health check is the most common reason for a failed deployment.
3. Wait for the first start: the weights download once to the persistent `/data` storage, so later restarts are much faster.
4. Send requests to `/v1/chat/completions` with `"model": "gpt-oss:120b"`.

See [Quickstart: GPT-OSS 120B with Ollama](https://docs.verda.com/serverless-containers/tutorials/quickstart-gpt-oss-120b-with-ollama/) for the full guide.


### Async Whisper inference

Transcribe long audio without keeping the client waiting. Add the `Prefer: respond-async` header to a request:

```bash
curl -X POST https://containers.verda.com/<deployment-name>/generate \
--header 'Authorization: Bearer <YOUR_INFERENCE_API_KEY>' \
--header 'Content-Type: application/json' \
--header 'Prefer: respond-async' \
--data '{"url": "<mp3 url>"}'
```

The response returns an `Id` straight away. Poll `https://containers.verda.com/status/<deployment-name>` with the header `X-Inference-Id: <Id>` until `Status` is `3`, then fetch the transcript from `https://containers.verda.com/result/<deployment-name>` with the same header.

See [Async Whisper inference](https://docs.verda.com/serverless-containers/tutorials/async-whisper-inference/) for the full guide, and [Async inference](https://docs.verda.com/serverless-containers/async-inference/) for every request mode.


### Publish your first Docker image

Deployments pull your image from a container registry. To publish one to a private Docker Hub repository:

1. Create a private repository in Docker Hub, and a personal access token with `Read, Write, Delete` permissions.
2. Build, log in with the token as the password, then tag and push with a version tag:

    ```bash
    docker build -t my-app .
    docker login -u <YOUR USERNAME>
    docker tag my-app your-username/my-app:1.0
    docker push your-username/my-app:1.0
    ```

Always use a specific version tag such as `:1.0`; `:latest` can be overwritten and is not allowed in deployments. GHCR, Google Artifact Registry, and Amazon ECR also work; see [Container registries](https://docs.verda.com/serverless-containers/container-registries/).

See [Publish your first Docker image](https://docs.verda.com/serverless-containers/tutorials/publish-your-first-docker-image/) for the full guide.


## More guides

Each product has more guides in its own section of the docs:

- **Instances:** see the [Instances overview](https://docs.verda.com/instances/).
- **Private Networks:** see the [Private Networks overview](https://docs.verda.com/private-networks/).
- **Clusters:** see the [Clusters overview](https://docs.verda.com/clusters/).
- **Storage:** see the [Storage overview](https://docs.verda.com/storage/) and the [Container Registry overview](https://docs.verda.com/storage/container-registry/).
- **Serverless Containers:** see the [Serverless Containers overview](https://docs.verda.com/serverless-containers/). Its reference section also has in-depth guides for [TGI](https://docs.verda.com/serverless-containers/tutorials/in-depth-deploy-with-tgi/), [SGLang](https://docs.verda.com/serverless-containers/tutorials/in-depth-deploy-with-sglang/), [vLLM](https://docs.verda.com/serverless-containers/tutorials/in-depth-deploy-with-vllm/), and [Replicate Cog](https://docs.verda.com/serverless-containers/tutorials/in-depth-deploy-with-replicate-cog/), and application guides for a [vector database](https://docs.verda.com/serverless-containers/tutorials/guide-vector-database/), a [RAG application](https://docs.verda.com/serverless-containers/tutorials/guide-rag-application/), and an [LLM API gateway](https://docs.verda.com/serverless-containers/tutorials/guide-llm-api-gateway/).
