Vector index parameters reference#

How to tune Relata's HNSW + DiskANN + IVF vector indexes. Covers index creation, parameter selection, distance metrics, quantization, and search-time knobs.

Index lifecycle#

A vector index lives on a typed table for every column whose name matches _emb_* (the convention for embedding slots). The index is keyed on (object_type, modality, model_tag, tenant_id) — multi-tenant by default.

By default, the index is lazy — the first query that touches an _emb_* field triggers build. Set RELATA_ANN_EAGER=true to build at insert time (higher ingest cost; lower first-query latency).

HNSW parameters (index-time)#

These are set at index creation (first insert) and cannot be changed without a rebuild.

Memory cost: ~3 KB × vectors at f32 / 1536-d (OpenAI). ~1 KB × vectors at int8 / 384-d (MiniLM). For 1 M vectors at f32/1536-d: ~3 GB.

Choosing M and ef_construction#

HNSW parameters (search-time)#

Tuning ef_search#

-- Tighten ef_search for fast low-recall queries:
SELECT * FROM Vec WHERE SIMILAR TO '[...]' FIELD=_emb_text LIMIT 10 EF_SEARCH=20
 
-- Loosen for high recall:
SELECT * FROM Vec WHERE SIMILAR TO '[...]' FIELD=_emb_text LIMIT 10 EF_SEARCH=200

The default max(k, 10) is fine for most workloads. For high-recall applications (face recognition, dedup), set ef_search = 5 * k.

Distance metrics#

Today, cosine is the default and most-optimised metric.

SELECT * FROM Face WHERE SIMILAR TO '[...]' FIELD=_emb_face METRIC=L2 LIMIT 10

Quantization#

The HNSW hot tier defaults to full (raw f32 — RELATA_VECTOR_QUANT=full). Set RELATA_VECTOR_QUANT=int8 for a 4× memory reduction at ±0.4% recall. The IVF cold tier uses int8 with optional PQ (32–64× compression).

Cold tier (DiskANN + IVF)#

When the index exceeds RELATA_DISKANN_MAX_RESIDENT, the cold tier engages:

# Bound RAM-resident vectors in the cold tier staging area
RELATA_VECTOR_COLD_RESIDENT_MAX=200000   # default 100 000
 
# Bound total HNSW-resident vectors (warns past this)
RELATA_DISKANN_MAX_RESIDENT=1000000      # 0 / unset = unbounded

Search-time knobs#

/search body#

{
  "query": "alice",
  "vector": [0.1, 0.2, ...],
  "vector_field": "_emb_text",
  "metric": "cosine",
  "limit": 10,
  "ef_search": 100,
  "filter": {"tenant_id": "org-acme"}
}

SQL SIMILAR TO#

SELECT * FROM Vec
  WHERE SIMILAR TO '[0.1, 0.2, ...]'
  FIELD=_emb_text
  METRIC=COSINE
  LIMIT 10
  EF_SEARCH=100

Pre-filter vs post-filter#

For selective predicates (e.g. "only vectors where tenant_id = 'org-acme'"), Relata chooses automatically based on selectivity:

HYBRID_SEARCH WHERE pushdown#

HYBRID_SEARCH's WHERE clause always applies correctly as a post-fusion filter — every result is guaranteed to match it. Separately (an optimization, not a correctness requirement), Relata also tries to push cheap predicates into the ANN leg itself, so a selective filter doesn't burn the vector search's top_k budget on rows that get dropped afterward anyway (the filtered-ANN "recall cliff" failure mode).

Which predicates push down:

When pushdown applies, the candidate row-id allowlist is intersected with any ACL read-allowlist before reaching the ANN leg — a row must both be readable by the caller and match the pushable predicate to be rankable. Pushdown is skipped when the allowlist comes back empty, or is larger than top_k × 10 (the scan cost outweighs the recall benefit at that selectivity — the post-fusion filter handles it correctly regardless).

Adaptive over-fetch: whenever the filter (or part of it) is not covered by an applied pushdown, the ANN leg's requested top_k is widened up to 4× (bounded by RELATA_MAX_VECTOR_K), giving the post-fusion filter more raw candidates to work with.

Reading EXPLAIN:

EXPLAIN HYBRID_SEARCH FROM Case QUERY 'wire fraud' LIMIT 10 WHERE status = 'open'

step | stage    | detail
-----|----------|----------------------------------------------------------
0    | Scan     | Case access=ANN (HYBRID_SEARCH) est_rows=12345
1    | Pushdown | pushable_predicates=1 allowlist_size=42 budget=500 \
                  applied=true effective_k=50 over_fetch=false \
                  reason="applied: 1 pushable predicate(s), allowlist=42"
2    | Limit    | n=10

• applied=true — the allowlist was handed to the ANN leg.
• applied=false with over_fetch=true — pushdown was skipped but the ANN fetch was widened to compensate.
• applied=false with over_fetch=false — no WHERE clause, or the allowlist came back empty.

EXPLAIN ANALYZE is not yet supported for HYBRID_SEARCH; use plain EXPLAIN above for the pushdown plan.

Performance characteristics#

Common pitfalls#

See also#

• Search and retrieval
• Performance tuning