---
description: Open ports, external SSH to workers, and InfiniBand partitioning on Verda instant clusters.
revision_date: 30.10.2026
---

# Networking and ports

## Default open ports (login node)

The login node firewall (configured from `/etc/default/verda_iptables`) drops all inbound traffic on the public interface by default, except for the ports below. Traffic from the internal cluster network and ICMP (ping) are always allowed.

| Port | Purpose |
|------|---------|
| `22` (TCP)  | SSH - login node shell. On Slinky clusters, the Slurm login pod instead (see [How users log in](https://docs.verda.com/clusters/slinky/users/#how-users-log-in)). |
| `80` (TCP)  | HTTP - Let's Encrypt ACME (HTTP-01) challenge and redirect to `443`. |
| `443` (TCP) | HTTPS - [Grafana](https://docs.verda.com/clusters/cluster-basics/monitoring/) via Nginx. |
| `2222` (TCP) | SSH - on Slinky clusters, the login node admin shell. Not used on other clusters. |

Worker and service nodes are **not** reachable from the internet by default; reach them by SSHing to the login node first (or see [external SSH to worker nodes](#external-ssh-to-worker-nodes-optional)).

## Limit access by source IP

On the login VM, edit `/etc/default/verda_iptables`. These settings need cluster-k8s 1.1.20 or later. Check that they exist in your configuration first.

For example, to allow SSH from one IP and an office subnet, and Grafana from the office subnet only:

```bash
SSH_ALLOW="203.0.113.10 198.51.100.0/24"
HTTPS_ALLOW="198.51.100.0/24"
HTTP_ALLOW="0.0.0.0/0"
```

Replace the example addresses with your public IP or VPN exit range. `SSH_ALLOW` covers ports `22`, `2222`, and enabled worker SSH ports. `HTTPS_ALLOW` covers Grafana on port `443`. Use spaces or commas between IPv4 addresses or CIDR ranges. `0.0.0.0/0` allows any source; an empty value blocks new public connections.

!!! warning
    Keep HTTP port `80` open for Let's Encrypt certificate renewal. Restricting `HTTP_ALLOW` can prevent renewal.

## Apply allowlist changes with automatic rollback

A timer can restore the old configuration if you lose SSH access. Keep your current session open. Do not reboot during these steps: the timer and backup are temporary.

1. Connect to the login VM and start a root shell. On Slinky, use port `2222`:

    ```bash
    ssh -p 2222 ubuntu@<cluster-ip>
    sudo -i
    ```

    On a Kubernetes-only cluster, omit `-p 2222`. Run the admin commands below in this root shell.

2. Back up the configuration and start a five-minute rollback timer:

    ```bash
    backup_dir=$(mktemp -d /run/verda-firewall.XXXXXX)
    rollback_unit="verda-firewall-rollback-$(date +%s)"
    cp -a /etc/default/verda_iptables "$backup_dir/verda_iptables" && \
    systemd-run --unit="$rollback_unit" --on-active=5m \
      --timer-property=AccuracySec=1s \
      /bin/sh -c 'cp -a "$1" /etc/default/verda_iptables && systemctl restart iptables-custom' \
      sh "$backup_dir/verda_iptables"

    systemctl is-active "$rollback_unit.timer"
    ```

    The last command must print `active`. If any command fails, stop before you edit the configuration.

3. Edit the allowlists, check the shell syntax, and apply:

    ```bash
    nano /etc/default/verda_iptables
    bash -n /etc/default/verda_iptables && systemctl restart iptables-custom
    ```

4. Before the timer runs, test a new SSH connection from another terminal. On Slinky, use `ssh -S none -p 2222 ubuntu@<cluster-ip>` and also check port `22`. Check Grafana if you changed its allowlist. `-S none` prevents reuse of an existing connection. An existing session can stay open even if the new rules block your IP.

    If a new connection fails, leave the timer running. Wait for it to restore the old configuration, then try again.

5. If the checks pass within five minutes, cancel the timer from the original root shell:

    ```bash
    systemctl stop "$rollback_unit.timer"
    ```

    If five minutes have passed, check the configuration first. The rollback may already have restored the backup.

## External SSH to worker nodes (optional)

Workers are not reachable from the internet by default. The usual path is to SSH to the login node and then to a worker by name (e.g. `ssh hostname-1`).

If you want to reach worker SSH directly from outside the cluster, enable DNAT on the login node:

1. On the login node, edit `/etc/default/verda_iptables` and set `WORKER_SSH_DNAT=1`.
2. Apply: `systemctl restart iptables-custom`

The login node will then forward `<login-ip>:1000N` to `hostname-N:22`. For example, `<login-ip>:10001` → `hostname-1`, `<login-ip>:10002` → `hostname-2`, and so on.

!!! warning
    Enabling DNAT exposes worker SSH to the public internet. Make sure each worker's `sshd` only accepts key-based authentication.

## Infiniband partitioning

Worker nodes are interconnected using a partitioned 400 Gb/s Infiniband fabric with `M_KEY`. For this reason commands like `ibhosts` will not work, while distributed workloads like MPI work correctly.

On B300 clusters the partition key is assigned at index 0, so Infiniband and NCCL work from inside a Docker container without any extra configuration.

On H200 and B200 clusters, to use Infiniband and NCCL from inside a Docker container make sure to set environment variable `NCCL_IB_PKEY=1`.

For example:

```bash
docker run -e NCCL_IB_PKEY=1
```
