Tuning recall and latency with three parameters in pgvector 0.8 HNSW indexes

When operating vector search in production, you eventually encounter a situation where complaints about low recall and slow queries arrive at the same time. I once had a rough time on my team when a legal document search service returned results with recall 0.85 — digging into the cause revealed the index had been created with default parameter values. That's when I started taking this topic seriously.

pgvector's HNSW index determines the balance point between recall and latency through three parameters (m, ef_construction, ef_search). And hnsw.iterative_scan, added in pgvector 0.8.0 (released November 2024, the stable version as of 2026), structurally resolves the result-missing problem that occurred when using filters together with vector search. This article examines what each parameter actually controls, how they can be configured differently by workload type, and how to measure recall on your own data, with code examples.

What HNSW Actually Does, and What the Parameters Touch

An Intuitive Understanding of Multi-Layer Graph Traversal

HNSW (Hierarchical Navigable Small World) is a multi-layer graph with vectors as nodes and edges connecting similar vectors. When a query arrives, it starts at the top layer and narrows the search range as it descends to lower layers. This allows approximate nearest neighbors (ANN) to be found quickly without traversing all the data.

Looking at the role each parameter plays in this structure:

m — Maximum number of connections per node (default 16, range 2~100)

Determines how many neighbors each node connects to. Larger values make the graph denser, improving recall, but increase index size and build time. And it cannot be changed after the index is created. Changing m requires a complete index rebuild, so it is important to set it carefully during initial design.

ef_construction — Number of candidate searches during build (default 64)

Controls how many neighbors are explored when deciding connections for each node during index construction. Higher values improve index quality but increase build time. pgvector internally enforces the condition ef_construction >= 2 * m, so if m=32, ef_construction must be at least 64. This also cannot be changed without rebuilding the index.

hnsw.ef_search — Number of search candidates at query time (default 40)

The number of candidates explored when executing a query. This is the only parameter that can be freely adjusted at runtime without rebuilding the index. Increase it to raise recall; decrease it to reduce latency.

The Recall-Latency Curve Is Not Linear

Honestly, at first I assumed "raising ef_search would linearly raise recall too," but actual measurements show otherwise. Observations reported from a specific dataset, hardware, and parameter combination in the Crunchy Data benchmark are as follows (based on that benchmark; the slope of the curve may differ in other environments):

• In the recall 0.80 → 0.95 range, the latency increase is relatively gradual
• In the recall 0.95 → 0.99 range, latency rises sharply

Summarizing the approximate shape of the curve in a table (these are conceptual examples only):

The exact shape of the curve varies greatly depending on data distribution, embedding dimensions, hardware, and cache state, so you must measure it on your own data. If recall above 0.99 is absolutely required, pgvector HNSW alone may not be sufficient, and it is time to consider an IVFFlat + re-ranking strategy or an external dedicated vector DB.

How to Set Parameters by Workload

Four Representative Scenarios

These are starting points by workload, compiled based on the pgvector official repository documentation and community cases. They are not absolute answers but baselines for initial settings; you should measure on your own data and then adjust.

For domains where accuracy is absolutely critical, it is better to start with m=32~64 from the beginning. Since changing m later requires a rebuild, the approach of "start with defaults and increase later" becomes a heavy burden as data scale grows.

Index Creation Code

For latency-first cases like recommendation systems:

CREATE INDEX ON items
USING hnsw (embedding vector_cosine_ops)
WITH (m = 10, ef_construction = 40);

For recall-first cases like legal document search:

CREATE INDEX ON legal_documents
USING hnsw (embedding vector_cosine_ops)
WITH (m = 48, ef_construction = 192);

When adjusting ef_search at runtime (no index rebuild required):

-- Apply to the entire session
SET hnsw.ef_search = 120;
 
-- Override only within a specific transaction scope
BEGIN;
SET LOCAL hnsw.ef_search = 100;
SELECT id, content FROM legal_documents
ORDER BY embedding <=> $1
LIMIT 10;
COMMIT;

Check Memory Settings Before Building

The memory required for HNSW index building has a rough lower bound of approximately N × D × 4 bytes for raw vector storage, plus graph edge overhead. Edge overhead scales linearly with m (up to m neighbor pointers per node, with up to 2m additional pointers in upper layers), and a working set for candidate queues and temporary structures during the build is also needed.

In practice, N × D × 4 × 2 is used as a rough lower bound (a loose coefficient lumping together raw vectors plus graph and working overhead). However, when m is large (e.g., 48~64), this lower bound is significantly exceeded, so more headroom should be added depending on the m value. For 5 million 1536-dimensional vectors, this lower bound alone is around 60GB, and with m=48 it is safe to assume a considerable amount more.

If maintenance_work_mem is not raised sufficiently, the build will spill to disk and slow down significantly.

SET maintenance_work_mem = '8GB';
 
CREATE INDEX ON documents
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

Using Filters with Vector Search: iterative_scan

The Overfiltering Problem

In cases like multi-tenant RAG services that combine filters like WHERE tenant_id = 42 with vector search, the old pgvector fetched a fixed-size batch from the index and then applied the filter. When filter selectivity was high (the search target is a very small portion of the total), almost no results in the batch satisfied the condition, causing an overfiltering problem where LIMIT 10 could not be filled.

iterative_scan, added in pgvector 0.8.0, structurally solves this problem. If results are insufficient, it continues searching the index further to secure enough results.

Choosing Between Two Modes

-- strict_order: maintains exact distance ordering (accuracy first)
SET hnsw.iterative_scan = strict_order;
 
SELECT id, content
FROM documents
WHERE tenant_id = 42
ORDER BY embedding <=> $1::vector
LIMIT 10;
 
-- relaxed_order: approximate ordering, lower latency (speed first)
SET hnsw.iterative_scan = relaxed_order;

The tradeoff is that latency increases as filter selectivity increases (narrower range), but result accuracy is guaranteed. It is a good idea to always create a separate B-tree index on the filter column.

CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON documents (tenant_id);

Memory Savings for Large Corpora: halfvec

The Neon blog (the URL contains a typo halvec, but the actual type name is halfvec) even recommends starting with halfvec from the beginning. halfvec, which uses float16 instead of float32, can reduce memory by roughly half with very little recall loss under the condition of using cosine similarity with normalized embeddings.

The caveat is that this low-loss characteristic applies to normalized embeddings + cosine distance workloads. With non-normalized embeddings or Euclidean (L2) distance-based search, the relative error may be larger, so you should actually measure the recall loss on your own data before adopting it.

When creating a new column:

ALTER TABLE documents ADD COLUMN embedding halfvec(1536);
 
CREATE INDEX ON documents
USING hnsw (embedding halfvec_cosine_ops)
WITH (m = 16, ef_construction = 64);

If an existing vector column already exists, you must actually perform the data migration. Just adding a column leaves only an empty column.

-- 1) Add new halfvec column
ALTER TABLE documents ADD COLUMN embedding_h halfvec(1536);
 
-- 2) Copy existing float32 data by casting to halfvec
UPDATE documents SET embedding_h = embedding::halfvec(1536);
 
-- 3) Create index on new column
CREATE INDEX ON documents
USING hnsw (embedding_h halfvec_cosine_ops)
WITH (m = 16, ef_construction = 64);
 
-- 4) Switch application code to new column, then clean up old column
-- ALTER TABLE documents DROP COLUMN embedding;
-- ALTER TABLE documents RENAME COLUMN embedding_h TO embedding;

Or to change the column type in one step:

ALTER TABLE documents
    ALTER COLUMN embedding TYPE halfvec(1536)
    USING embedding::halfvec(1536);

For large tables, this approach can hold a long lock, so the 4-step approach above is safer for zero-downtime deployments.

How to Actually Measure Recall

Many people just trust a number like "recall 0.95," but measuring it directly on your own data is far more reliable.

Getting an Exact kNN Baseline

Setting only enable_indexscan = off still allows PostgreSQL to choose a bitmap index scan path. To force a pure sequential scan, enable_bitmapscan = off must also be disabled. And you should always verify with EXPLAIN which plan is actually chosen.

SET enable_indexscan = off;
SET enable_bitmapscan = off;
 
EXPLAIN (ANALYZE, BUFFERS)
SELECT id FROM documents
ORDER BY embedding <=> $1::vector
LIMIT 10;

If the plan shows Seq Scan on documents, the baseline calculation can be trusted.

Calculating Recall in a Single Query

You can get exact kNN results and ANN results each in a CTE and immediately calculate the intersection ratio. Just use the query vector $1::vector twice.

-- Session settings example
-- SET enable_indexscan = on;
-- SET enable_bitmapscan = on;
-- SET hnsw.ef_search = 60;
 
WITH truth AS (
    -- Exact kNN: force seq scan path
    SELECT id
    FROM documents
    ORDER BY embedding <=> $1::vector
    LIMIT 10
),
approx AS (
    -- ANN results: use HNSW index
    SELECT id
    FROM documents
    ORDER BY embedding <=> $1::vector
    LIMIT 10
)
SELECT
    COUNT(*) FILTER (WHERE approx.id IS NOT NULL)::float / 10 AS recall_at_10
FROM truth
LEFT JOIN approx USING (id);

The caveat is that session GUCs (like enable_indexscan) cannot be applied differently to each CTE within the same query. In practice, it is safer to measure in the following order:

The skeleton of a Python measurement script reflecting this is as follows (conceptual example; actual connection and error handling omitted):

import psycopg
import numpy as np
 
def fetch_ids(cur, vec, use_index):
    if use_index:
        cur.execute("SET enable_indexscan = on")
        cur.execute("SET enable_bitmapscan = on")
    else:
        cur.execute("SET enable_indexscan = off")
        cur.execute("SET enable_bitmapscan = off")
    cur.execute(
        "SELECT id FROM documents "
        "ORDER BY embedding <=> %s::vector LIMIT 10",
        (vec,),
    )
    return {row[0] for row in cur.fetchall()}
 
def measure_recall(conn, query_vectors, ef_search=60):
    recalls = []
    with conn.cursor() as cur:
        cur.execute(f"SET hnsw.ef_search = {ef_search}")
        for vec in query_vectors:
            truth = fetch_ids(cur, vec, use_index=False)
            approx = fetch_ids(cur, vec, use_index=True)
            recalls.append(len(truth & approx) / len(truth))
    return float(np.mean(recalls))

It is best to use sample query vectors logged from actual traffic. Measuring with random vectors can result in recall characteristics that differ greatly from real workloads.

Tradeoff Summary and Common Mistakes

Three common mistakes:

1. Thinking m can be changed later. In practice, the entire index must be rebuilt, which can take hours for large data sizes. It is important to set it from the beginning with workload characteristics in mind.

2. Not managing the cache when the corpus size exceeds RAM. pgvector HNSW still has random page access as a bottleneck. If data doesn't fit in RAM, query performance degrades non-linearly. Along with the shared_buffers size, to eliminate cold starts, it is good to preload the index into shared buffers as follows:

   sql

   SELECT pg_prewarm('documents_embedding_idx');

3. Operating with the default ef_search value (40) without measuring recall. Acceptable recall levels differ by workload. Recommendation systems and medical search have completely different requirements — running both with the same settings means neither is properly served.

Parameter Selection Decision Flow

Conclusion

The key to HNSW parameter tuning is ultimately defining what you can afford to sacrifice first. Raising recall increases latency, and reducing latency lowers recall. This curve is not linear, and the cost rises sharply once recall exceeds 0.95.

So the entry point must be back-calculated from workload requirements. For domains like recommendation systems where recall 0.90 is sufficient, it is better to start with a low m and low ef_search from the beginning. For domains like legal or medical search where missing results is itself an incident, you must start heavy with m=32~64 from the beginning. Only for cases like general RAG where requirements are ambiguous is the approach of using the defaults (m=16, ef_construction=64, ef_search=40) as a baseline and varying ef_search while measuring recall valid.

Regardless of which entry point you choose, without numbers from measuring recall on your own data, tuning becomes guesswork. ef_search is a good candidate for experimentation since it can be adjusted without a rebuild, while m and ef_construction should be treated as initial design decisions. For 5M+ scale, consider adopting halfvec together (confirming the prerequisite of normalized embeddings and cosine distance is essential), and if WHERE filters are involved, iterative_scan can practically be considered a default setting as of 2026.

References

• pgvector GitHub Official Repository
• pgvector 0.8.0 Released — PostgreSQL Official News
• Announcing pgvector 0.8.0 on Nile — Detailed explanation of iterative scan
• HNSW Indexes with Postgres and pgvector — Crunchy Data (benchmark source)
• Accelerate HNSW indexing with pgvector on Amazon Aurora — AWS Official Blog
• Don't use vector. Use halfvec instead — Neon Blog (halvec in the URL slug is a typo in the original; the actual type name is halfvec)
• The Complete Guide to pgvector Tuning — HNSW/IVFFlat/Quantization comprehensive guide
• Scaling pgvector: Memory, Quantization, and Index Build Strategies