Serverless Container Tutorials¶
End-to-end examples for Verda Serverless Containers: quick deployments, in-depth framework guides, and complete application builds.
Quickstarts¶
-
Deploy with vLLM
The fastest path to a working LLM inference endpoint.
-
Migrate from Runpod
Move an existing Runpod-style container workflow to Verda.
-
GPT-OSS 120B with Ollama
Deploy GPT-OSS 120B with Ollama on Serverless Containers.
In depth: deploy with a framework¶
-
Deploy with vLLM
A thorough guide to serving language models with vLLM.
-
Deploy with TGI
Serve models with Text Generation Inference.
-
Deploy with SGLang
Serve language models with SGLang.
-
Deploy with Replicate Cog
Package and deploy a Cog model.
Build an application¶
-
Semantic search with pgvector
Deploy an embedding model and query a vector database by meaning.
-
Retrieval-Augmented Generation (RAG)
Answer questions from your own documents, with a citation on every claim.
-
LLM API gateway with LiteLLM
One API in front of several endpoints, with per-app keys, rate limits and failover.
-
Secrets management with OpenBao
Deliver short-lived credentials to a Serverless Container with OpenBao.
-
Agentic AI assistant
An LLM that decides which tools to call, over pgvector and OpenBao.
Operate¶
-
Async Whisper inference
Queue long audio transcription jobs with asynchronous inference.
-
Publish your first Docker image
Build and publish a Docker image, then deploy it on Serverless Containers.