---
description: What Serverless Containers are and when to use them.
revision_date: 11.11.2026
---

# Serverless Containers overview

Serverless Containers is our managed service for running your own container images on Verda GPU and CPU compute, without provisioning or maintaining any servers.

You supply the image, with the model, inference engine and dependencies it needs, pulled from any public or private registry, including your own [Container Registry](https://docs.verda.com/storage/container-registry/) (see [container examples](https://github.com/verda-cloud/container-examples) for reference Dockerfiles). We run it on the compute you choose, put an HTTPS endpoint in front of it, and add or remove copies of it as traffic changes. When no requests are arriving, no copies run, and an idle endpoint costs nothing.

## The problem it solves

Requests rarely arrive at a steady rate. Some hours are busy and others are quiet. A service you run yourself on an instance needs enough capacity for your busiest hour, and you pay for that capacity all day, including the quiet hours. You also build and maintain the layer around it: a request queue, an autoscaler, health checks, restarts when a server process dies, and a load balancer.

Serverless Containers provides that layer for you. We queue your incoming requests, start more copies of your container as the queue grows, remove them as it drains, and scale to zero when the queue is empty. Anything that runs in a container and serves HTTP can be deployed, whether that is a model behind an inference engine or another service that runs alongside one.

## How it works

A **deployment** is the unit that runs a container on Verda. It holds the details of what to run: which container image to use, which compute to run it on, and how many copies of it should be running at any moment. It also holds the details of how we reach the container: the HTTP port the application listens on, and a path we can call to check the container is healthy. A deployment is created, changed and deleted as one thing, and we give it an **endpoint**, an authenticated URL that accepts requests.

Deployments come in two kinds:

* **Continuous deployment**: serves requests as they arrive and scales with traffic.
* **Batch job deployment**: runs one job per replica and destroys that replica when the job exits, suited to work that takes too long to answer in a single request.

A **replica** is one running copy of the container, on the compute the deployment specifies. Every replica of a deployment runs the same image with the same settings and the same compute, and a deployment can have many of them at once, or none at all while it is idle. Requests go into an internal queue and replicas pick them up.

Every deployment has a 500 GiB **scratch volume** mounted at `/data`. It is shared by all replicas of the deployment and persists when the deployment scales to zero, which makes it the usual place to cache model weights. You can also mount a [Shared Filesystem](https://docs.verda.com/serverless-containers/storage/) when several deployments, or a deployment and an instance, need the same files.

## Autoscaling and health checks

**Autoscaling** works as follows:

* The default metric is queue load, calculated as queue length divided by replica count. CPU or GPU utilisation can be added as extra metrics.
* You set the minimum and maximum number of replicas, and how quickly scaling reacts in each direction.
* A replica selected for scale-down is signalled to shut down, with a short grace period to finish requests it's already handling.

The **health check** tells us when a replica is ready to serve. Without one, a replica can receive a request before the application has finished starting, and that request fails.

See [Scaling and health checks](https://docs.verda.com/serverless-containers/scaling-and-health-check/) for how to configure autoscaling and health checks in detail.

## Managing deployments

You can create and manage deployments from the console, or script them against the [RESTful API](https://api.verda.com/v1/docs#tag/serverless-containers) and [Python SDK](https://github.com/verda-cloud/sdk-python). The console also shows logs and metrics for each deployment.

## GPU drivers and CUDA compatibility

Serverless Containers uses **NVIDIA driver version 580.x.x (R580)** across all customer-serving GPU pools. Exact patch versions can vary between nodes. If your image requires a specific patch version, [contact support](mailto:support@verda.com) to confirm it for your selected GPU and location.

Verda manages the host NVIDIA driver. Your container image supplies the CUDA runtime and application libraries, so its CUDA version does not need to match the node image's CUDA toolkit version exactly. Choose an image whose driver requirements and supported GPU architectures match your deployment.

[NVIDIA's CUDA compatibility guide](https://docs.nvidia.com/deploy/cuda-compatibility/minor-version-compatibility.html) lists R580 as the minimum driver branch for CUDA 13.x minor-version compatibility. Newer drivers also support applications built with older CUDA toolkits, but compatibility within a CUDA major version has feature and PTX limitations. Check the image release's requirements as well as the CUDA compatibility table.

## Billing

You're billed per replica, for every minute in which that replica processed any usage, including the time it spent starting up or shutting down. Usage appears on your invoice in 10-minute intervals. A deployment at zero replicas is not charged. Spot pricing is available at a lower rate than on-demand, and replicas restart automatically, so an interrupted spot replica is less disruptive here than on an instance.

For current rates, see [Serverless Containers pricing](https://verda.com/pricing#serverless-containers).

## Cold starts

A deployment with no replicas running cannot answer a request until a replica has started up. That start-up delay is called a cold start. The request waits in the queue while the replica starts, and is forwarded once the replica reports healthy. Requests that follow, while that replica keeps running, are served straight away.

A cold start is made up of three stages, and the delay is how long they take together:

1. **Compute allocation.** The time to place the replica on compute of the type the deployment asks for.
2. **Image pull.** The time to fetch the container image from its registry. Larger images take longer.
3. **Container start-up.** The time the container takes to boot and load its model into GPU memory.

Loading the model (3) is usually the longest of the three for a large model, because a multi-gigabyte weight file has to be read and moved onto the GPU before the container can answer its first request. In total a cold start takes tens of seconds to a few minutes, depending mostly on model size.

How much of the wait you can remove depends on how your image is built, where your weights are stored, and whether you keep a replica warm. Caching model weights on `/data`, where they survive scale-to-zero, is the usual first step. Setting a minimum replica count of 1 or more removes the cold start entirely, at the cost of paying for that replica whether or not requests arrive.

## Long-running work

Autoscaling adds and removes replicas based on how many requests are waiting in the queue. A request that a replica has already picked up is no longer counted there, so a replica that is busy for a long time looks the same as one doing nothing.

That is fine for work measured in seconds. Once a single request takes more than about three minutes, the scale-down delay becomes hard to set. Set it high enough to let the work finish and you pay for idle replicas; set it low and a replica can be stopped while it is still working.

A [batch job](https://docs.verda.com/serverless-containers/batch-jobs/) deployment is built for that case. Each job gets its own replica, calls to it are always asynchronous, and the replica is destroyed when the container exits. You set the maximum number of concurrent jobs and a deadline after which a replica is stopped.

[Async inference](https://docs.verda.com/serverless-containers/async-inference/) covers the related case where the work is shorter, but you do not want a client holding an open connection while it runs.

## How this differs from Instances and Clusters

All three are ways to get compute from us. They differ in how much of the machine you take on, and in what shape the workload has to be.

|  | Serverless Containers | Instances | Clusters |
| --- | --- | --- | --- |
| **You provide** | A container image | Nothing; you build on the machine | Your jobs, submitted to the orchestrator |
| **You manage** | Your image and its behaviour | The OS, drivers, server process and uptime | Your workloads on the cluster |
| **You reach it through** | An HTTPS endpoint | SSH, with root on the machine | Kubernetes or Slinky, pre-installed |
| **Scaling** | Automatic, including to zero | Manual: you start and stop it | Fixed cluster size, from 16 to 128 GPUs |
| **You pay** | Per replica, per active minute | For as long as the instance runs | For the cluster, Pay As You Go |
| **Suits** | Request-shaped work with uneven traffic | Development, fine-tuning, interactive or stateful work | Distributed training and multi-node inference |

**Instances** give you a whole machine and full control of it. Use one for anything interactive, anything stateful, or any service that does not break down into discrete requests. Everything below your application, from the operating system to uptime, is yours to manage.

**Clusters** are for workloads that span several machines and need the InfiniBand interconnect between them, which is a different problem from serving traffic.

**Serverless Containers** manages the machine for you. There is no host to log into, no OS to patch and no autoscaler to write. The workload does have to be request-shaped, and a request that arrives after the deployment has scaled to zero waits for a replica to start.

As a rule: use Serverless Containers to serve a container behind an endpoint, Instances when you need a machine of your own to work on, and Clusters when a single job has to run across several machines.

## Quickstart

- [Deploy with vLLM](https://docs.verda.com/serverless-containers/quickstart/): the fastest path to a working LLM endpoint.
- [Publish your first Docker image](https://docs.verda.com/serverless-containers/tutorials/publish-your-first-docker-image/): if you don't have a container image yet.
- [Migrate from Runpod](https://docs.verda.com/serverless-containers/tutorials/quickstart-migrate-from-runpod/): if you're moving an existing serverless deployment.
- [Pricing](https://verda.com/pricing#serverless-containers): current rates for each GPU and CPU tier.

## Configure your deployment

- [Scaling and health checks](https://docs.verda.com/serverless-containers/scaling-and-health-check/): tune replica counts and scaling sensitivity.
- [Container registries](https://docs.verda.com/serverless-containers/container-registries/): pull from public and private registries.
- [Storage](https://docs.verda.com/serverless-containers/storage/): the scratch volume and Shared Filesystem mounts.
- [Batching and streaming](https://docs.verda.com/serverless-containers/batching-and-streaming/): concurrent requests per replica, and streaming responses.
- [Async inference](https://docs.verda.com/serverless-containers/async-inference/): submit a request and poll for the result.
- [Batch jobs](https://docs.verda.com/serverless-containers/batch-jobs/): one replica per job, for work that runs longer than a few minutes.
