Tutorial: Semantic Search with a Vector Database (pgvector)¶

What you'll build¶

In this guide we will build a search index that finds things by meaning instead of keywords, running on a Verda CPU instance with a Block Volume, with the embedding model on a Serverless Container.

• Someone searches for "cat" and finds entries about kittens, tabbies and felines, none of which contain the word "cat".
• It ranks by closeness in meaning, so given a cat, a lion comes back before a bicycle. That is what powers "show me similar items" and near-duplicate detection.

Several databases can do this. The common self-hostable options are Qdrant, Weaviate, Milvus and pgvector. This guide uses pgvector, an extension inside Postgres, so vectors live in the same database as your other tables, under the backups, access control and monitoring you already have.

Nothing here generates text. The one model you deploy is an embedding model: it turns text into numbers so they can be compared. A search returns your own rows, ordered by closeness in meaning, each with a similarity score.

Vector search is not a replacement for the full-text search Postgres already has. They fail in opposite directions:

If people cannot find things because they phrase queries differently from your content, that is the problem this guide solves. Both kinds of search can live in the same Postgres, so you can also run them together and merge the results.

Architecture¶

The system has two flows. They are separate processes, started at different times, but they share the same database and embedding model.

Indexing prepares your content for search, and runs whenever that content changes:

  +--------------------+
  |   Your content     |  object storage, a database, an API...
  +---------+----------+
            | 1. read
            v
  +--------------------+   2. embed    +----------------------+
  |                    | ------------> |                      |
  |      index.py      |               |   embedding model    |
  |                    | <------------ |                      |
  +---------+----------+    vectors    +----------------------+
            | 3. store                   GPU, Serverless Container
            v
  +----------------------------------+
  |      Postgres + pgvector         |  CPU instance + Block Volume
  +----------------------------------+

Search runs once per query, and reads what indexing produced:

       query text
            |
            v
  +--------------------+   1. embed    +----------------------+
  |                    | ------------> |                      |
  |                    |               |   embedding model    |
  |                    | <------------ |                      |
  |                    |    vector     +----------------------+
  |     search.py      |                 GPU, Serverless Container
  |                    |   2. search   +----------------------+
  |                    | ------------> |                      |
  |                    |               | Postgres + pgvector  |
  |                    | <------------ |                      |
  +---------+----------+  ranked rows  +----------------------+
            |                            CPU instance + Block Volume
            |
            v
   ranked results + scores

index.py and search.py are the two scripts you write, and both run on the CPU instance. Postgres never calls the container: the script embeds the text, then passes the numbers to Postgres as a query parameter.

Both flows use the same deployment. Indexing and searching must use the same model, version and settings, because vectors from different models cannot be compared and nothing reports the mismatch. That is why both scripts import one embedding function.

Prerequisites¶

• An Inference API Key, which your code uses to call the embedding endpoint
• A Hugging Face account and token, to fetch the model weights

Everything here is done in the Verda console and over SSH.

Step 1: Create the instance and volume¶

Log in to the Verda cloud console and create a CPU instance. The volume is part of the same form, so both are created together:

Why a separate volume instead of the instance's own OS disk? A Block Volume can be resized as the collection grows, detached on demand, and attached to a new instance as a data disk. Postgres runs on the OS volume without difficulty, but it does none of the three, and RAM eventually forces a move to a larger instance.

Step 2: Install Postgres and pgvector¶

SSH into the instance. Everything in this step runs there.

Prepare the volume¶

See what is attached:

lsblk -f

In the output, the OS volume is the one mounted on /. Your Block Volume is the line with the FSTYPE, UUID and MOUNTPOINT columns all empty. Names run /dev/vda, /dev/vdb, /dev/vdc and so on, lettered in the order the volumes were attached, so the OS volume is usually vda and your Block Volume vdb, as this guide assumes you attached only one. If yours carries a different letter, use that name in the commands below.

An empty FSTYPE means the volume has no filesystem, so nothing can be written to it yet. We create one using mkfs.ext4. It erases the volume, so run it only on a blank one:

# If lsblk -f showed ext4, the volume already holds data.
# Skip this command and go straight to mounting.
sudo mkfs.ext4 /dev/vdb

Mount it where Postgres keeps its data:

sudo mkdir -p /var/lib/postgresql
sudo mount /dev/vdb /var/lib/postgresql

That mount lasts only until the instance restarts. Add an /etc/fstab entry so the system repeats it at every boot:

VOL_UUID=$(sudo blkid -s UUID -o value /dev/vdb)
echo "UUID=$VOL_UUID /var/lib/postgresql ext4 defaults,nofail 0 2" | sudo tee -a /etc/fstab

The entry uses the volume's UUID rather than /dev/vdb, because device letters can change. nofail lets the instance boot normally when the volume is absent, instead of dropping into emergency mode.

Install from the PostgreSQL project repository¶

Ubuntu 24.04 ships pgvector 0.6.0, which is too old: it predates the iterative index scans that filtered search needs (0.8.0). Use the PGDG repository instead:

sudo apt-get update
sudo apt-get install -y postgresql-common
sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
sudo apt-get install -y postgresql-16 postgresql-16-pgvector

Confirm the installed version before continuing:

apt-cache policy postgresql-16-pgvector

The rest of this guide assumes 0.8.0 or newer. If you see 0.6.x, the PGDG repository was not applied.

Tune Postgres for vector work¶

Connect as the postgres superuser:

sudo -u postgres psql

ALTER SYSTEM SET shared_buffers = '4GB';              -- about 25% of RAM
ALTER SYSTEM SET maintenance_work_mem = '2GB';        -- index build speed
ALTER SYSTEM SET max_parallel_maintenance_workers = 3;

ALTER SYSTEM only writes the values to config. shared_buffers needs a restart, so leave psql with \q and then:

sudo systemctl restart postgresql
sudo -u postgres psql -c "SHOW shared_buffers;"

These values suit CPU.4V.16G: a quarter of RAM, an eighth of RAM, and one less than the vCPU count. Scale them with the instance if you pick a larger one from Sizing.

maintenance_work_mem is easy to overlook. If the HNSW graph does not fit in it while being built, the build spills to disk and becomes significantly slower. Set it generously before building an index, and lower it afterwards.

Step 3: Create the schema¶

Create a database and open a session on it:

sudo -u postgres createdb vectordb
sudo -u postgres psql -d vectordb

The prompt becomes vectordb=#. Run the following there:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE items (
    item_id      TEXT PRIMARY KEY,
    title        TEXT NOT NULL,
    body         TEXT NOT NULL,
    category     TEXT,
    source_url   TEXT,
    updated_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
    embedding    vector(1024) NOT NULL
);

vector(1024) matches Qwen/Qwen3-Embedding-0.6B, the model deployed in Step 4, which returns 1024 numbers per text. The column must match your model exactly, or inserts fail with expected N dimensions. Another model's count is listed on its Hugging Face page, and the curl test in Step 4 confirms it: the response contains one number per dimension.

Step 4: Deploy the embedding model¶

The vectors come from an embedding model. In this guide, we deploy it as a Serverless Container, which scales to zero when it is not in use and saves cost. Create a deployment with these settings:

The first start takes a few minutes: the image is pulled, then the model weights are loaded into GPU memory. Until that finishes, the deployment reports itself unavailable.

The system log shows the normal sequence: Scheduled, Pulling, Pulled, Created, Started, then several Startup probe failed: connection refused entries while the weights load. Those failures are expected, because nothing is listening on the port yet. What is not expected is the container starting repeatedly, which means it is crashing and restarting, and the reason is in the container log rather than the system log.

Once it reports healthy, the deployment page shows an Endpoint Address, which is the host plus your deployment name. Use it for <endpoint-address> below.

Test it from anywhere with your Inference API Key:

curl -sL -X POST <endpoint-address>/v1/embeddings \
  --header "Authorization: Bearer <your-inference-api-key>" \
  --header 'Content-Type: application/json' \
  --data '{"model": "embedder", "input": "how do I cancel my subscription"}' \
  | head -c 200

A working response starts like this:

{"id":"embd-a1b2c3","object":"list","created":1757090000,"model":"embedder",
 "data":[{"index":0,"object":"embedding","embedding":[0.0123,-0.0456,0.0031,

head -c 200 keeps the output readable: the full response is around 20,000 characters, because the embedding is 1024 numbers, one per dimension. That is what the vector(1024) column in Step 3 is sized for.

The "model" value stays embedder because it comes from --served-model-name, not from the URL, so naming the deployment something else does not change it.

Check the vLLM version. Qwen3-Embedding requires vLLM 0.8.5 or newer, and the flags differ between versions. Pin a tag rather than latest, so that a later change cannot break your deployment without warning.

Step 5: Index your content¶

Run everything in this step on the CPU instance over SSH, since it connects to Postgres locally.

Install the Python packages in a virtual environment:

sudo apt-get install -y python3-venv
python3 -m venv ~/vecenv
source ~/vecenv/bin/activate
pip install requests "psycopg[binary]" numpy

Create two database roles, so the indexer can write while the search side only reads. Pick a password for each and substitute it for the placeholders, brackets included.

Connect to the vectordb database, not to postgres:

sudo -u postgres psql -d vectordb

CREATE ROLE vec_write LOGIN PASSWORD '<write-password>';
GRANT INSERT, UPDATE, DELETE, SELECT ON items TO vec_write;

CREATE ROLE vec_read LOGIN PASSWORD '<read-password>';
GRANT SELECT ON items TO vec_read;

Leave psql with \q, then set the environment variables, using the same two passwords. If a password contains @, : or /, percent-encode it here or the DSN will be misparsed:

export VERDA_INFERENCE_KEY='<your-inference-api-key>'
export PG_WRITE_DSN='postgresql://vec_write:<write-password>@localhost/vectordb'
export PG_READ_DSN='postgresql://vec_read:<read-password>@localhost/vectordb'

Check the connection before going further:

psql "$PG_WRITE_DSN" -c "select 1"

Getting a 1 means the role and password match. If you see it, continue to The indexer.

Only if you get password authentication failed, the DSN password differs from the one you set. Reset it in psql as postgres on vectordb with the following SQL, then run the check again:

ALTER ROLE vec_write PASSWORD '<write-password>';   -- only if the check failed

The indexer¶

With the venv still active, save this as index.py in your home directory, and replace <endpoint-address> on the ENDPOINT line with the Endpoint Address from your deployment page:

# index.py
import os, requests, numpy as np, psycopg

ENDPOINT = "<endpoint-address>/v1/embeddings"
KEY = "".join(os.environ["VERDA_INFERENCE_KEY"].split())
BATCH = 64

def embed(texts: list[str]) -> np.ndarray:
    """The single source of truth for turning text into vectors."""
    resp = requests.post(
        ENDPOINT,
        headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"},
        json={"model": "embedder", "input": texts},
        timeout=300,               # generous: covers cold start
    )
    resp.raise_for_status()
    rows = [d["embedding"] for d in sorted(resp.json()["data"], key=lambda d: d["index"])]
    vecs = np.asarray(rows, dtype=np.float32)
    vecs /= np.linalg.norm(vecs, axis=1, keepdims=True)   # unit length, for cosine
    return vecs

def index_items(items):
    """items: iterable of dicts with item_id, title, body, category, source_url."""
    conn = psycopg.connect(os.environ["PG_WRITE_DSN"])
    with conn.cursor() as cur:
        for i in range(0, len(items), BATCH):
            batch = items[i:i + BATCH]
            # Embed title and body together
            vecs = embed([f"{it['title']}\n\n{it['body']}" for it in batch])
            cur.executemany(
                """INSERT INTO items
                     (item_id, title, body, category, source_url, embedding)
                   VALUES (%s, %s, %s, %s, %s, %s)
                   ON CONFLICT (item_id) DO UPDATE SET
                     title = EXCLUDED.title, body = EXCLUDED.body,
                     category = EXCLUDED.category, source_url = EXCLUDED.source_url,
                     embedding = EXCLUDED.embedding, updated_at = now()""",
                [
                    (it["item_id"], it["title"], it["body"],
                     it.get("category"), it.get("source_url"), v.tolist())
                    for it, v in zip(batch, vecs)
                ],
            )
            conn.commit()
            print(f"indexed {i + len(batch)}/{len(items)}")

if __name__ == "__main__":
    from sample_data import SAMPLE
    index_items(SAMPLE)

Two functions do the work. embed() turns text into vectors, and search.py imports the same function later, so both use the same model. index_items() embeds your rows and writes them to items, updating a row if it already exists rather than duplicating it.

Save those sample data as sample_data.py, next to index.py:

# sample_data.py
SAMPLE = [
    {"item_id": "1", "title": "Ending your subscription",
     "body": "Stop your plan at any time from the billing page. Access continues to the "
             "end of the current period.",
     "category": "billing", "source_url": "/help/ending-your-subscription"},
    {"item_id": "2", "title": "Updating your payment card",
     "body": "Add or replace a card under billing details. The new card is used for the "
             "next charge.",
     "category": "billing", "source_url": "/help/payment-card"},
    {"item_id": "3", "title": "Resetting a forgotten password",
     "body": "Use the forgot password link on the sign-in screen to get a reset email.",
     "category": "account", "source_url": "/help/password-reset"},
    {"item_id": "4", "title": "Inviting teammates",
     "body": "Owners can invite people from team settings and choose their role.",
     "category": "team", "source_url": "/help/invite-teammates"},
    {"item_id": "5", "title": "Exporting your data",
     "body": "Request a full export from the account page. You get an email when the "
             "archive is ready.",
     "category": "account", "source_url": "/help/export-data"},
]

Run it:

python3 index.py

You should see indexed 5/5, which means all five rows were embedded and written to the items table.

Build the index¶

With the rows loaded, create the index once. Back in psql as the postgres superuser, since vec_write cannot create indexes:

sudo -u postgres psql -d vectordb

SET maintenance_work_mem = '2GB';

CREATE INDEX items_embedding_idx
    ON items USING hnsw (embedding vector_cosine_ops)
    WITH (m = 16, ef_construction = 64);

The index is created here, not in Step 3 with the table, because an existing index has to be updated by every insert. Building it once at the end is much faster for a large load.

vector_cosine_ops tells the index which distance measure to organise the vectors by. pgvector offers several, each with its own operator, and this guide uses cosine distance, written <=> in the search query in Step 6. If you query with a different operator, build the index with the matching operator class, or the index is silently ignored and every search becomes a sequential scan.

Step 6: Search¶

Save this as search.py in the same directory as index.py, since it imports embed() from it. Keep the venv active and the environment variables from Step 5 set:

# search.py
import os, psycopg
from index import embed

conn = psycopg.connect(os.environ["PG_READ_DSN"])

SQL = """
SELECT item_id, title, category, source_url,
       1 - (embedding <=> %(q)s::vector) AS similarity
FROM items
ORDER BY embedding <=> %(q)s::vector
LIMIT %(k)s
"""

def search(query: str, k: int = 10):
    qvec = embed([query])[0].tolist()
    with conn.cursor() as cur:
        cur.execute("SET LOCAL hnsw.ef_search = 100")
        cur.execute(SQL, {"q": qvec, "k": k})
        cols = [d.name for d in cur.description]
        return [dict(zip(cols, r)) for r in cur.fetchall()]

if __name__ == "__main__":
    import sys
    query = " ".join(sys.argv[1:]) or "how do I cancel"
    for row in search(query, k=3):
        print(f"{row['similarity']:.3f}  {row['title']}")

Run it with a query:

python3 search.py "how do I cancel"

On the sample data that returns:

0.568  Ending your subscription
0.506  Updating your payment card
0.502  Exporting your data

"Ending your subscription" comes first despite sharing no words with the query. That is the point of vector search: the match is on meaning rather than on keywords.

Each query makes one call to the embedding container, for the query text only. The stored items were embedded once, at indexing time, and are never re-embedded unless you change models. Cosine distance (<=>) runs from 0 (identical) to 2 (opposite), so 1 - distance gives the similarity score you would show a user: 1.0 is effectively the same text, and 0 means unrelated. Thresholds covers how to pick a cutoff.

Congratulations! You now have a working semantic search. The rest of this page covers what to adjust once it is running.

After it works¶

Thresholds¶

Vector search always returns k results, no matter how irrelevant. Query the sample index for "how do I bake sourdough" and you still get rows back, ranked as if they were relevant.

If your UI should not show unrelated results, add a floor to the query. This takes three edits in search.py.

First, the SQL constant. Add one WHERE line between FROM and ORDER BY. Only the marked line changes:

SELECT item_id, title, category, source_url,
       1 - (embedding <=> %(q)s::vector) AS similarity
FROM items
WHERE 1 - (embedding <=> %(q)s::vector) > %(floor)s      -- add this line
ORDER BY embedding <=> %(q)s::vector
LIMIT %(k)s

Second, search() itself. That new %(floor)s is a parameter, so the function must supply a value, or psycopg raises an error about the missing key. Only the marked lines change:

def search(query: str, k: int = 10, floor: float = 0.0):          # add this line
    qvec = embed([query])[0].tolist()
    with conn.cursor() as cur:
        cur.execute("SET LOCAL hnsw.ef_search = 100")
        cur.execute(SQL, {"q": qvec, "k": k, "floor": floor})     # and this line
        cols = [d.name for d in cur.description]
        return [dict(zip(cols, r)) for r in cur.fetchall()]

Third, the runner. Pass the floor you want when it calls search(). Only the marked line changes:

if __name__ == "__main__":
    import sys
    query = " ".join(sys.argv[1:]) or "how do I cancel"
    for row in search(query, k=3, floor=0.55):          # add this line
        print(f"{row['similarity']:.3f}  {row['title']}")

Defaulting to 0.0 filters nothing, so any other call to search() behaves as before.

Run the same command to see the effect:

python3 search.py "how do I bake sourdough"

With a floor in place, an unrelated query returns nothing instead of three weak matches.

There is no universal cutoff, and the useful range is narrower than you would expect. Measure your own: run a handful of real queries plus a deliberately nonsense one, compare the scores, and set the floor between them. Re-check it whenever you change embedding models, because the scale shifts.

What else the index can do¶

Similarity is now an expression you can use in SQL, so one index does more than a search box. A few examples:

• Filter by metadata. A normal WHERE clause on category narrows a search to one section or one customer's documents.
• Related items. Order by distance from a stored vector instead of a query string and you have "more like this", with no embedding call at all.
• Near-duplicate detection. The same query with a high floor, around 0.95, surfaces items that are almost the same text.

You can also merge these results with Postgres full-text search for hybrid ranking. It is all ordinary SQL against a table you already have, so look up the syntax when you need it.

Sizing, CPU and GPU¶

Creating embeddings runs on the GPU, in the Serverless Container. Storing and searching vectors needs CPU and RAM, since pgvector has no GPU support, and that is the instance you size here.

RAM is the constraint, not disk. The HNSW index is what should fit in memory, and it is larger than the vectors it indexes. Once your own content is loaded, check its size. On the instance:

sudo -u postgres psql -d vectordb

SELECT pg_size_pretty(pg_relation_size('items_embedding_idx'));

That figure, plus what Postgres and the operating system need, is the RAM to size for.

Query latency rising while CPU stays low usually means the index no longer fits in memory. Raise maintenance_work_mem if index builds are also slow. To move to a larger instance, detach the Block Volume and attach it there.

Backups¶

Vectors are derived data. The source text and the embedding model produce them, so with both in hand you can regenerate them by re-running the indexer, and losing the database costs GPU time rather than information. Back up those two inputs first. Keep the source text somewhere durable, such as Object Storage, and record the model and its settings in version control, because a different model produces different numbers and vectors from two models cannot be compared.

Back up the database anyway, because re-embedding a large collection is slow.

Logical backup. pg_dump writes the rows out in a form any Postgres can reload, including a different version or machine. It does not copy the index, so the restore rebuilds it. On the instance:

sudo -u postgres pg_dump -Fc -d vectordb > vectordb.dump

Copy the dump off the instance afterwards. A backup on the same volume as the database is not a backup.

To restore, you replay the dump into a database. Let's try it on an example database. On the instance:

sudo -u postgres createdb vectordb_restore        # create the example database
sudo -u postgres pg_restore -d vectordb_restore < vectordb.dump

Volume clone. This copies the whole database directory, index included. It needs the volume detached, or the instance shut down, so it is not an online backup, and it is a manual action. To recover, you attach the clone to an instance and start Postgres, with no index to rebuild. See Cloning a block volume for the steps.

Conclusion¶

You now have Postgres with pgvector storing your content as vectors, an embedding model on a Serverless Container, and search that ranks your rows by closeness in meaning.

To clean up. The instance and the volume keep billing while they exist. When you are finished, discontinue the instance, permanently delete the volume, and delete the container deployment.

Related guides¶

• Storage: attaching, resizing, cloning and deleting volumes.
• Serverless Containers: how deployments work.
• Scaling and health checks: replica behaviour, including scale to zero.