---
description: "Build semantic search on Verda: deploy an embedding model on a Serverless Container, store vectors in Postgres with pgvector on a CPU instance, and query by meaning instead of keywords."
revision_date: 20.09.2026
---
# Tutorial: Semantic Search with a Vector Database (pgvector)
## What you'll build
In this guide we will build a search index that finds things by **meaning** instead of
keywords, running on a Verda CPU instance with a Block Volume, with the embedding model on a
Serverless Container.
- Someone searches for **"cat"** and finds entries about **kittens, tabbies and felines**,
none of which contain the word "cat".
- It ranks by closeness in meaning, so given a cat, a **lion** comes back before a
**bicycle**. That is what powers "show me similar items" and near-duplicate detection.
Several databases can do this. The common self-hostable options are **Qdrant**,
**Weaviate**, **Milvus** and **pgvector**. **This guide uses pgvector**, an extension inside
Postgres, so vectors live in the same database as your other tables, under the backups,
access control and monitoring you already have.
**Nothing here generates text.** The one model you deploy is an embedding model: it turns
text into numbers so they can be compared. A search returns your own rows, ordered by
closeness in meaning, each with a similarity score.
Vector search is not a replacement for the full-text search Postgres already has. They fail
in opposite directions:
| Full-text search wins | Vector search wins |
|---|
| Exact terms: error codes, product codes, file names | Paraphrases and synonyms |
| Boolean logic and precise operators | "Find me something like this" |
| No embedding model to run | Cross-language matching |
If people cannot find things because they phrase queries differently from your content, that
is the problem this guide solves. Both kinds of search can live in the same Postgres, so you
can also run them together and merge the results.
## Architecture
The system has two flows. They are separate processes, started at different times, but they
share the same database and embedding model.
**Indexing** prepares your content for search, and runs whenever that content changes:
+--------------------+
| Your content | object storage, a database, an API...
+---------+----------+
| 1. read
v
+--------------------+ 2. embed +----------------------+
| | ------------> | |
| index.py | | embedding model |
| | <------------ | |
+---------+----------+ vectors +----------------------+
| 3. store GPU, Serverless Container
v
+----------------------------------+
| Postgres + pgvector | CPU instance + Block Volume
+----------------------------------+
**Search** runs once per query, and reads what indexing produced:
query text
|
v
+--------------------+ 1. embed +----------------------+
| | ------------> | |
| | | embedding model |
| | <------------ | |
| | vector +----------------------+
| search.py | GPU, Serverless Container
| | 2. search +----------------------+
| | ------------> | |
| | | Postgres + pgvector |
| | <------------ | |
+---------+----------+ ranked rows +----------------------+
| CPU instance + Block Volume
|
v
ranked results + scores
`index.py` and `search.py` are the two scripts you write, and both run on the CPU instance.
Postgres never calls the container: the script embeds the text, then passes the numbers to
Postgres as a query parameter.
| Component | Tier | Runs when |
|---|
| Database | CPU.4V.16G instance | Continuously, while the instance is on |
| Data volume | NVMe Block Volume | Always, whether or not the instance runs |
| Embeddings | Serverless Container, L40S | On demand, scales to zero when idle |
**Both flows use the same deployment.** Indexing and searching must use the same model,
version and settings, because vectors from different models cannot be compared and nothing
reports the mismatch. That is why both scripts import one embedding function.
## Prerequisites
- An [Inference API Key](https://docs.verda.com/welcome-to-verda/api-credentials/#create-inference-api-keys), which
your code uses to call the embedding endpoint
- A Hugging Face account and token, to fetch the model weights
Everything here is done in the Verda console and over SSH.
## Step 1: Create the instance and volume
Log in to the [Verda cloud console](https://console.verda.com/signin) and create a CPU instance. The volume is part
of the same form, so both are created together:
| Setting | Value | Notes |
|---|---|---|
| Instance type | `CPU.4V.16G` | No GPU needed. [Sizing](#sizing-cpu-and-gpu) has larger options |
| Image | Ubuntu 24.04 | |
| SSH key | Your registered key | |
| Block Volume | 50 GiB NVMe | At the storage step, use **Add volume**. The database goes here, and 50 GiB is ample for this guide. **Add existing storage** is how you reattach a volume you already have |
**Why a separate volume instead of the instance's own OS disk?** A Block Volume can be resized
as the collection grows, detached on demand, and attached to a new instance as a data disk.
Postgres runs on the OS volume without difficulty, but it does none of the three, and RAM
eventually forces a move to a larger instance.
## Step 2: Install Postgres and pgvector
SSH into the instance. Everything in this step runs there.
### Prepare the volume
See what is attached:
```bash
lsblk -f
```
In the output, the OS volume is the one mounted on `/`. Your Block Volume is the line with
the `FSTYPE`, `UUID` and `MOUNTPOINT` columns all empty. Names run `/dev/vda`, `/dev/vdb`,
`/dev/vdc` and so on, lettered in the order the volumes were attached, so the OS volume is
usually `vda` and your Block Volume `vdb`, as this guide assumes you attached only one. If
yours carries a different letter, use that name in the commands below.
An empty `FSTYPE` means the volume has no filesystem, so nothing can be written to it yet.
We create one using `mkfs.ext4`. **It erases the volume**, so run it only on a blank one:
```bash
# If lsblk -f showed ext4, the volume already holds data.
# Skip this command and go straight to mounting.
sudo mkfs.ext4 /dev/vdb
```
Mount it where Postgres keeps its data:
```bash
sudo mkdir -p /var/lib/postgresql
sudo mount /dev/vdb /var/lib/postgresql
```
That mount lasts only until the instance restarts. Add an `/etc/fstab` entry so the system
repeats it at every boot:
```bash
VOL_UUID=$(sudo blkid -s UUID -o value /dev/vdb)
echo "UUID=$VOL_UUID /var/lib/postgresql ext4 defaults,nofail 0 2" | sudo tee -a /etc/fstab
```
The entry uses the volume's **UUID rather than `/dev/vdb`**, because device letters can
change. **`nofail`** lets the instance boot normally when the volume is absent, instead of
dropping into emergency mode.
### Install from the PostgreSQL project repository
Ubuntu 24.04 ships pgvector **0.6.0**, which is too old: it predates the iterative index
scans that filtered search needs (0.8.0). Use the PGDG repository instead:
```bash
sudo apt-get update
sudo apt-get install -y postgresql-common
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
sudo apt-get install -y postgresql-16 postgresql-16-pgvector
```
Confirm the installed version before continuing:
```bash
apt-cache policy postgresql-16-pgvector
```
The rest of this guide assumes **0.8.0 or newer**. If you see 0.6.x, the PGDG repository was
not applied.
### Tune Postgres for vector work
Connect as the `postgres` superuser:
```bash
sudo -u postgres psql
```
```sql
ALTER SYSTEM SET shared_buffers = '4GB'; -- about 25% of RAM
ALTER SYSTEM SET maintenance_work_mem = '2GB'; -- index build speed
ALTER SYSTEM SET max_parallel_maintenance_workers = 3;
```
`ALTER SYSTEM` only writes the values to config. `shared_buffers` needs a restart, so leave
psql with `\q` and then:
```bash
sudo systemctl restart postgresql
sudo -u postgres psql -c "SHOW shared_buffers;"
```
These values suit `CPU.4V.16G`: a quarter of RAM, an eighth of RAM, and one less than the
vCPU count. Scale them with the instance if you pick a larger one from
[Sizing](#sizing-cpu-and-gpu).
`maintenance_work_mem` is easy to overlook. If the HNSW graph does not fit in it while being
built, the build spills to disk and becomes significantly slower. Set it generously before
building an index, and lower it afterwards.
## Step 3: Create the schema
Create a database and open a session on it:
```bash
sudo -u postgres createdb vectordb
sudo -u postgres psql -d vectordb
```
The prompt becomes `vectordb=#`. Run the following there:
```sql
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE items (
item_id TEXT PRIMARY KEY,
title TEXT NOT NULL,
body TEXT NOT NULL,
category TEXT,
source_url TEXT,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
embedding vector(1024) NOT NULL
);
```
`vector(1024)` matches `Qwen/Qwen3-Embedding-0.6B`, the model deployed in
[Step 4](#step-4-deploy-the-embedding-model), which returns 1024 numbers per text. The column
must match your model exactly, or inserts fail with `expected N dimensions`. Another model's
count is listed on its Hugging Face page, and the curl test in Step 4 confirms it: the
response contains one number per dimension.
## Step 4: Deploy the embedding model
The vectors come from an embedding model. In this guide, we deploy it as a Serverless
Container, which scales to zero when it is not in use and saves cost. Create a deployment
with these settings:
| Field | Value |
|---|---|
| Deployment name | `embedder` |
| GPU type | L40S 48GB |
| Container image | `docker.io/vllm/vllm-openai:v0.27.1` |
| Exposed HTTP port and healthcheck port | `8000`, the port vLLM listens on |
| Healthcheck path | `/health`, the path vLLM exposes |
| Start command | Toggle on, which reveals the Entrypoint and CMD fields |
| Entrypoint | Leave empty, so the image's own entrypoint is used |
| CMD | `Qwen/Qwen3-Embedding-0.6B --runner pooling --served-model-name embedder` |
| Environment variables | Add one with the name `HF_TOKEN` and `` as the value. Leave `HF_HOME` as the form sets it |
The first start takes a few minutes: the image is pulled, then the model weights are loaded
into GPU memory. Until that finishes, the deployment reports itself unavailable.
The system log shows the normal sequence: `Scheduled`, `Pulling`, `Pulled`, `Created`,
`Started`, then several `Startup probe failed: connection refused` entries while the weights
load. Those failures are expected, because nothing is listening on the port yet. What is not
expected is the container starting repeatedly, which means it is crashing and restarting, and
the reason is in the container log rather than the system log.
Once it reports healthy, the deployment page shows an **Endpoint Address**, which is the host
plus your deployment name. Use it for `` below.
Test it from anywhere with your Inference API Key:
```bash
curl -sL -X POST /v1/embeddings \
--header "Authorization: Bearer " \
--header 'Content-Type: application/json' \
--data '{"model": "embedder", "input": "how do I cancel my subscription"}' \
| head -c 200
```
A working response starts like this:
```json
{"id":"embd-a1b2c3","object":"list","created":1757090000,"model":"embedder",
"data":[{"index":0,"object":"embedding","embedding":[0.0123,-0.0456,0.0031,
```
`head -c 200` keeps the output readable: the full response is around 20,000 characters, because
the embedding is **1024 numbers**, one per dimension. That is what the `vector(1024)` column in
[Step 3](#step-3-create-the-schema) is sized for.
The `"model"` value stays `embedder` because it comes from `--served-model-name`, not from
the URL, so naming the deployment something else does not change it.
**Check the vLLM version.** Qwen3-Embedding requires vLLM **0.8.5 or newer**, and the flags
differ between versions. Pin a tag rather than `latest`, so that a later change cannot break
your deployment without warning.
!!! warning
Changing the image on an existing deployment may not re-pull it. If you edit a deployment and it keeps failing with the previous image's error, create a new deployment instead.
## Step 5: Index your content
Run everything in this step on the CPU instance over SSH, since it connects to Postgres
locally.
**Install the Python packages** in a virtual environment:
```bash
sudo apt-get install -y python3-venv
python3 -m venv ~/vecenv
source ~/vecenv/bin/activate
pip install requests "psycopg[binary]" numpy
```
!!! info
Ubuntu 24.04 does not let pip modify the system Python, so the packages go in a virtual environment. `python3-venv` is not in the cloud image, and without it `python3 -m venv` fails with `ensurepip is not available`.
**Create two database roles**, so the indexer can write while the search side only reads. Pick
a password for each and substitute it for the placeholders, brackets included.
Connect to the `vectordb` database, not to `postgres`:
```bash
sudo -u postgres psql -d vectordb
```
```sql
CREATE ROLE vec_write LOGIN PASSWORD '';
GRANT INSERT, UPDATE, DELETE, SELECT ON items TO vec_write;
CREATE ROLE vec_read LOGIN PASSWORD '';
GRANT SELECT ON items TO vec_read;
```
Leave psql with `\q`, then **set the environment variables**, using the same two passwords.
If a password contains `@`, `:` or `/`, percent-encode it here or the DSN will be misparsed:
```bash
export VERDA_INFERENCE_KEY=''
export PG_WRITE_DSN='postgresql://vec_write:@localhost/vectordb'
export PG_READ_DSN='postgresql://vec_read:@localhost/vectordb'
```
**Check the connection before going further:**
```bash
psql "$PG_WRITE_DSN" -c "select 1"
```
Getting a `1` means the role and password match. If you see it, continue to
[The indexer](#the-indexer).
Only if you get `password authentication failed`, the DSN password differs from the one you
set. Reset it in psql as `postgres` on `vectordb` with the following SQL, then run the check
again:
```sql
ALTER ROLE vec_write PASSWORD ''; -- only if the check failed
```
### The indexer
With the venv still active, save this as `index.py` in your home directory, and replace
`` on the `ENDPOINT` line with the Endpoint Address from your deployment
page:
```python
# index.py
import os, requests, numpy as np, psycopg
ENDPOINT = "/v1/embeddings"
KEY = "".join(os.environ["VERDA_INFERENCE_KEY"].split())
BATCH = 64
def embed(texts: list[str]) -> np.ndarray:
"""The single source of truth for turning text into vectors."""
resp = requests.post(
ENDPOINT,
headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"},
json={"model": "embedder", "input": texts},
timeout=300, # generous: covers cold start
)
resp.raise_for_status()
rows = [d["embedding"] for d in sorted(resp.json()["data"], key=lambda d: d["index"])]
vecs = np.asarray(rows, dtype=np.float32)
vecs /= np.linalg.norm(vecs, axis=1, keepdims=True) # unit length, for cosine
return vecs
def index_items(items):
"""items: iterable of dicts with item_id, title, body, category, source_url."""
conn = psycopg.connect(os.environ["PG_WRITE_DSN"])
with conn.cursor() as cur:
for i in range(0, len(items), BATCH):
batch = items[i:i + BATCH]
# Embed title and body together
vecs = embed([f"{it['title']}\n\n{it['body']}" for it in batch])
cur.executemany(
"""INSERT INTO items
(item_id, title, body, category, source_url, embedding)
VALUES (%s, %s, %s, %s, %s, %s)
ON CONFLICT (item_id) DO UPDATE SET
title = EXCLUDED.title, body = EXCLUDED.body,
category = EXCLUDED.category, source_url = EXCLUDED.source_url,
embedding = EXCLUDED.embedding, updated_at = now()""",
[
(it["item_id"], it["title"], it["body"],
it.get("category"), it.get("source_url"), v.tolist())
for it, v in zip(batch, vecs)
],
)
conn.commit()
print(f"indexed {i + len(batch)}/{len(items)}")
if __name__ == "__main__":
from sample_data import SAMPLE
index_items(SAMPLE)
```
Two functions do the work. `embed()` turns text into vectors, and `search.py` imports the
same function later, so both use the same model. `index_items()` embeds your rows and writes
them to `items`, updating a row if it already exists rather than duplicating it.
Save those sample data as `sample_data.py`, next to `index.py`:
```python
# sample_data.py
SAMPLE = [
{"item_id": "1", "title": "Ending your subscription",
"body": "Stop your plan at any time from the billing page. Access continues to the "
"end of the current period.",
"category": "billing", "source_url": "/help/ending-your-subscription"},
{"item_id": "2", "title": "Updating your payment card",
"body": "Add or replace a card under billing details. The new card is used for the "
"next charge.",
"category": "billing", "source_url": "/help/payment-card"},
{"item_id": "3", "title": "Resetting a forgotten password",
"body": "Use the forgot password link on the sign-in screen to get a reset email.",
"category": "account", "source_url": "/help/password-reset"},
{"item_id": "4", "title": "Inviting teammates",
"body": "Owners can invite people from team settings and choose their role.",
"category": "team", "source_url": "/help/invite-teammates"},
{"item_id": "5", "title": "Exporting your data",
"body": "Request a full export from the account page. You get an email when the "
"archive is ready.",
"category": "account", "source_url": "/help/export-data"},
]
```
Run it:
```bash
python3 index.py
```
You should see `indexed 5/5`, which means all five rows were embedded and written to the
`items` table.
### Build the index
With the rows loaded, create the index once. Back in psql as the `postgres` superuser, since
`vec_write` cannot create indexes:
```bash
sudo -u postgres psql -d vectordb
```
```sql
SET maintenance_work_mem = '2GB';
CREATE INDEX items_embedding_idx
ON items USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);
```
The index is created here, not in Step 3 with the table, because an existing index has to be
updated by every insert. Building it once at the end is much faster for a large load.
`vector_cosine_ops` tells the index which distance measure to organise the vectors by.
pgvector offers several, each with its own operator, and this guide uses cosine distance,
written `<=>` in the search query in Step 6. **If you query with a different operator, build
the index with the matching operator class**, or the index is silently ignored and every
search becomes a sequential scan.
## Step 6: Search
Save this as `search.py` **in the same directory as `index.py`**, since it imports `embed()`
from it. Keep the venv active and the environment variables from Step 5 set:
```python
# search.py
import os, psycopg
from index import embed
conn = psycopg.connect(os.environ["PG_READ_DSN"])
SQL = """
SELECT item_id, title, category, source_url,
1 - (embedding <=> %(q)s::vector) AS similarity
FROM items
ORDER BY embedding <=> %(q)s::vector
LIMIT %(k)s
"""
def search(query: str, k: int = 10):
qvec = embed([query])[0].tolist()
with conn.cursor() as cur:
cur.execute("SET LOCAL hnsw.ef_search = 100")
cur.execute(SQL, {"q": qvec, "k": k})
cols = [d.name for d in cur.description]
return [dict(zip(cols, r)) for r in cur.fetchall()]
if __name__ == "__main__":
import sys
query = " ".join(sys.argv[1:]) or "how do I cancel"
for row in search(query, k=3):
print(f"{row['similarity']:.3f} {row['title']}")
```
Run it with a query:
```bash
python3 search.py "how do I cancel"
```
On the sample data that returns:
```
0.568 Ending your subscription
0.506 Updating your payment card
0.502 Exporting your data
```
**"Ending your subscription" comes first despite sharing no words with the query.** That is
the point of vector search: the match is on meaning rather than on keywords.
Each query makes one call to the embedding container, for the query text only. The stored
items were embedded once, at indexing time, and are never re-embedded unless you change
models. Cosine **distance** (`<=>`) runs from 0 (identical) to 2 (opposite), so
`1 - distance` gives the **similarity** score you would show a user: 1.0 is effectively the
same text, and 0 means unrelated. [Thresholds](#thresholds) covers how to pick a cutoff.
Congratulations! You now have a working semantic search. The rest of this page covers what
to adjust once it is running.
## After it works
### Thresholds
Vector search **always** returns *k* results, no matter how irrelevant. Query the sample
index for "how do I bake sourdough" and you still get rows back, ranked as if they were
relevant.
If your UI should not show unrelated results, add a floor to the query. This takes three
edits in `search.py`.
**First, the `SQL` constant.** Add one `WHERE` line between `FROM` and `ORDER BY`. Only the
marked line changes:
```sql
SELECT item_id, title, category, source_url,
1 - (embedding <=> %(q)s::vector) AS similarity
FROM items
WHERE 1 - (embedding <=> %(q)s::vector) > %(floor)s -- add this line
ORDER BY embedding <=> %(q)s::vector
LIMIT %(k)s
```
**Second, `search()` itself.** That new `%(floor)s` is a parameter, so the function must
supply a value, or psycopg raises an error about the missing key. Only the marked lines
change:
```python
def search(query: str, k: int = 10, floor: float = 0.0): # add this line
qvec = embed([query])[0].tolist()
with conn.cursor() as cur:
cur.execute("SET LOCAL hnsw.ef_search = 100")
cur.execute(SQL, {"q": qvec, "k": k, "floor": floor}) # and this line
cols = [d.name for d in cur.description]
return [dict(zip(cols, r)) for r in cur.fetchall()]
```
**Third, the runner.** Pass the floor you want when it calls `search()`. Only the marked
line changes:
```python
if __name__ == "__main__":
import sys
query = " ".join(sys.argv[1:]) or "how do I cancel"
for row in search(query, k=3, floor=0.55): # add this line
print(f"{row['similarity']:.3f} {row['title']}")
```
Defaulting to `0.0` filters nothing, so any other call to `search()` behaves as before.
Run the same command to see the effect:
```bash
python3 search.py "how do I bake sourdough"
```
With a floor in place, an unrelated query returns nothing instead of three weak matches.
**There is no universal cutoff, and the useful range is narrower than you would expect.**
Measure your own: run a handful of real queries plus a deliberately nonsense one, compare the
scores, and set the floor between them. Re-check it whenever you change embedding models,
because the scale shifts.
### What else the index can do
Similarity is now an expression you can use in SQL, so one index does more than a search
box. A few examples:
- **Filter by metadata.** A normal `WHERE` clause on `category` narrows a search to one
section or one customer's documents.
- **Related items.** Order by distance from a stored vector instead of a query string and you
have "more like this", with no embedding call at all.
- **Near-duplicate detection.** The same query with a high floor, around 0.95, surfaces items
that are almost the same text.
You can also merge these results with Postgres full-text search for hybrid ranking. It is
all ordinary SQL against a table you already have, so look up the syntax when you need it.
### Sizing, CPU and GPU
Creating embeddings runs on the GPU, in the Serverless Container. Storing and searching
vectors needs CPU and RAM, since pgvector has no GPU support, and that is the instance you
size here.
**RAM is the constraint, not disk.** The HNSW index is what should fit in memory, and it is
larger than the vectors it indexes. Once your own content is loaded, check its size. On the
instance:
```bash
sudo -u postgres psql -d vectordb
```
```sql
SELECT pg_size_pretty(pg_relation_size('items_embedding_idx'));
```
That figure, plus what Postgres and the operating system need, is the RAM to size for.
Query latency rising while CPU stays low usually means the index no longer fits in memory. Raise
`maintenance_work_mem` if index builds are also slow. To move to a larger instance, detach the
Block Volume and attach it there.
### Backups
**Vectors are derived data.** The source text and the embedding model produce them, so with
both in hand you can regenerate them by re-running the indexer, and losing the database costs
GPU time rather than information. Back up those two inputs first. Keep the source text
somewhere durable, such as Object Storage, and record the model and its settings in version
control, because a different model produces different numbers and vectors from two models
cannot be compared.
Back up the database anyway, because re-embedding a large collection is slow.
**Logical backup.** `pg_dump` writes the rows out in a form any Postgres can reload, including
a different version or machine. It does not copy the index, so the restore rebuilds it. On the
instance:
```bash
sudo -u postgres pg_dump -Fc -d vectordb > vectordb.dump
```
Copy the dump off the instance afterwards. A backup on the same volume as the database is not
a backup.
To restore, you replay the dump into a database. Let's try it on an example database. On the
instance:
```bash
sudo -u postgres createdb vectordb_restore # create the example database
sudo -u postgres pg_restore -d vectordb_restore < vectordb.dump
```
**Volume clone.** This copies the whole database directory, index included. It needs the volume
**detached, or the instance shut down**, so it is not an online backup, and it is a manual
action. To recover, you attach the clone to an instance and start Postgres, with no index to
rebuild. See
[Cloning a block volume](https://docs.verda.com/storage/block-volumes/cloning-a-block-volume/)
for the steps.
## Conclusion
You now have Postgres with pgvector storing your content as vectors, an embedding model on a
Serverless Container, and search that ranks your rows by closeness in meaning.
**To clean up.** The instance and the volume keep billing while they exist. When you are
finished, [discontinue](https://docs.verda.com/cpu-and-gpu-instances/shutdown-hibernate-and-delete/)
the instance, [permanently delete](https://docs.verda.com/storage/deleting-storage/) the
volume, and delete the container deployment.
## Related guides
- [Storage](https://docs.verda.com/storage/): attaching, resizing, cloning
and deleting volumes.
- [Serverless Containers](https://docs.verda.com/containers/overview/): how deployments work.
- [Scaling and health checks](https://docs.verda.com/containers/scaling/): replica behaviour,
including scale to zero.