Search and retrieval#

RelataDB combines search with governance, time, identity, and provenance.

Search surfaces#

Ask in plain English (no LLM required)#

The nl_query MCP tool / REST endpoint translates a plain-English question to governed SQL and executes it. With no RELATA_LLM_URL configured a deterministic in-tree translator handles a documented set of intents — so analysts get structured answers with zero external dependencies. Set RELATA_LLM_URL (Ollama / vLLM / LM Studio) for richer free-form parsing.

Supported deterministic phrasings:

A recognised intent whose entities can't be extracted returns a clear "couldn't translate; try …" error — it never guesses and runs the wrong query. The active translator (local vs LLM) is reported in the startup posture block.

World-class search UX#

RelataDB's search engine includes features inspired by Meilisearch and Typesense. The /search REST endpoint accepts JSON parameters for fine-grained control.

Subword tokenization#

camelCase, PascalCase, and letter↔digit transitions are split during tokenization so "iPhone14Max" produces tokens ["i", "phone", "14", "max"]. This means "phoneNumber" is findable by searching "phone". snake_case and kebab-case already split via the non-alphanumeric pass. Applied symmetrically at index and query time.

Matching strategy#

Controls which query terms are required vs optional when scoring documents. All strategies still BM25-rank results; the difference is which documents are eligible.

curl -s -X POST http://127.0.0.1:9090/search \
  -H 'Content-Type: application/json' \
  -d '{"query":"alice wonderland","type":"Person","limit":10,"matching_strategy":"all"}'

Boolean operators — with "matching_strategy": "boolean", uppercase AND / OR / NOT in the query text combine per-term BM25 matches as posting-list set operations (left-associative; a bare space means OR):

curl -s -X POST http://127.0.0.1:9090/search \
  -H 'Content-Type: application/json' \
  -d '{"query":"sanction OR ofac NOT expired","type":"Alert","limit":10,"matching_strategy":"boolean"}'

The same grammar is reachable from SQL as MATCH(col, 'q', BOOLEAN) and from the typed /search door as rank_by: ["boolean", "<column>", "<query>"].

Per-query typo tolerance#

Override the global RELATA_SEARCH_PRESET per query. Gate fuzzy expansion by word length, disable specific words, or turn typos off entirely.

{
  "query": "john smith",
  "type": "Person",
  "limit": 10,
  "typo_tolerance": {
    "enabled": true,
    "min_word_size": 5,
    "disable_on_words": ["smith"],
    "disable_on_attributes": ["email"]
  }
}

Typo tolerance in SQL MATCH — POST /query#

The same typo_tolerance object is accepted by POST /query alongside the SQL text. When set, SQL MATCH / PHRASE / SUFFIX / INFIX conditions apply per-token levenshtein fuzzy matching: 1 edit for words ≥ min_word_size (default: any length), 2 edits for words ≥ 9 chars.

POST /query
{
  "sql": "SELECT * FROM Person WHERE MATCH(name, 'jon')",
  "typo_tolerance": {
    "enabled": true,
    "min_word_size": 3
  }
}

This will also return rows where name contains "john" (1-edit match).

BM25F per-field weights#

Boost results where a query term appears in high-priority fields (e.g. title outweighs body). Pass fieldWeights — a map of field names to f32 multipliers. The BM25 score is scaled by the highest weight of any named field that contains a query term. Fields absent from the map default to 1.0.

{
  "query": "database",
  "type": "Article",
  "limit": 10,
  "fieldWeights": {"title": 3.0, "body": 1.0, "tags": 5.0}
}

Rows whose query terms appear only in tags are scaled ×5; rows where the term appears in title are scaled ×3; rows with the term only in body are unaffected. Omitting fieldWeights (or passing {}) leaves all scores unchanged.

Faceted search#

The FACETS SQL clause returns per-attribute value counts. Facets are computed over the full matching set (up to 100 000 rows), not just the top-k returned rows — so browse-UI facet counts are accurate even when LIMIT is small.

For numeric facet columns, the response also includes facetStats with min, max, sum, avg, and count:

{
  "facets": {"status": {"active": 42, "closed": 7}},
  "facetStats": {"amount": {"min": 100, "max": 50000, "sum": 125000, "avg": 2314, "count": 54}}
}

Estimated total hits#

Both /query and /search responses include estimatedTotalHits — the full matching-set size (accurate up to the 100k facet cap; a lower bound otherwise) — alongside count/rows (the actual returned rows). Use this for pagination UX.

Multi-search / federated#

POST /multi-search runs N independent queries in one round-trip and returns combined results in input order — useful for dashboard widgets and federated type searches:

curl -s -X POST http://127.0.0.1:9090/multi-search \
  -H 'Content-Type: application/json' \
  -d '{"queries":[{"query":"alice","type":"Person","limit":5},{"query":"transfer","type":"Transaction","limit":10}]}'

Typed query door — namespace().query() (no SQL)#

For the search / RAG persona, /search accepts a second body shape — a typed JSON query that compiles to a governed SQL plan server-side, so you never hand-build SQL. Governance (purpose, ACL, cell masking, audit) is identical to a hand-written query. The typed shape is detected by the absence of a query key (its presence keeps the legacy Meilisearch shape above on the same route).

curl -X POST http://127.0.0.1:9090/search \
  -H 'Content-Type: application/json' \
  -d '{"from":"Document",
       "rank_by":["bm25","title","graph retrieval"],
       "filters":[{"field":"status","op":"eq","value":"published"}],
       "include_attributes":["id","title"],"limit":10}'
# returns the governed /query envelope: {rows, columns, ...}

Python SDK — flagship namespace() surface#

from relata import RelataClient
 
with RelataClient("http://localhost:9090", purpose="rag") as client:
    docs = client.namespace("Document")
    docs.write([{"id": "d1", "title": "Knowledge graphs 101", "body": "..."}])  # schemaless (POST /ingest/auto)
    res = docs.query(
        text="graph retrieval",
        match_column="title",
        filters=[{"field": "status", "op": "eq", "value": "published"}],
        limit=10,
    )
    for row in res:
        print(row["title"])

.write() is schemaless: POST /ingest/auto auto-creates the type on first write with field types inferred from the rows — no DDL. Async mirrors (AsyncNamespace, await docs.query(...)) and one shared connection pool across every namespace.

Composable ranking + COMPUTE side outputs (SQL)#

In SQL, ranking is a full expression tree and per-hit signals are a COMPUTE clause (the typed door's rank_by/compute_attributes lower to these):

-- Blend BM25 with a saturated popularity signal, a time decay, and geo distance.
SELECT * FROM Article WHERE MATCH(content, 'ai')
RANK BY SUM(BM25(), PRODUCT(0.5, SATURATE(popularity, 100, 2)),
            DECAY(published_at, 86400, 1), DIST(location, 1.0, 2.0))
LIMIT 10;
 
-- Attach per-hit signals without changing matching or order.
SELECT * FROM Doc WHERE MATCH(content, 'ai')
RANK BY VECTOR()
COMPUTE bm25_score AS BM25(), snippet AS HIGHLIGHT(content)
LIMIT 10;

Ranking primitives: SUM / MAX / PRODUCT / SATURATE / DECAY / ATTRIBUTE(name) / DIST(attr, x, y, …) / BM25() / VECTOR(). HIGHLIGHT(field) is compute-only (renders a snippet).

Namespace branching + pinning#

• Branch — constant-time COW: BRANCH dev FROM main (SQL) or POST /v1/namespaces/dev/branch with {"branch_from":"main"}. See Branching & Namespaces for the full surface (including how namespace branches differ from schema branches and sub-tenant namespaces).
• Pin — RELATA_PINNED_NAMESPACES=acme,contoso reserves a dedicated NVMe cache slice per hot namespace so its index is evict-immune under pressure.

Multi-query batch + reciprocal-rank fusion (RRF)#

One round-trip, up to 16 typed subqueries, fused with rank-based RRF. This is the "build a search dashboard in one call" feature — multiple ranking strategies against one snapshot, merged into a single ranked list.

POST /search with {"queries": [...], "rerank_by": ["RRF", {"rank_constant": 60}], "limit": 20}:

curl -X POST http://127.0.0.1:9090/search \
  -H 'Content-Type: application/json' \
  -d '{
    "queries": [
      {"from":"Document", "rank_by":["bm25","title","graph retrieval"], "limit":50},
      {"from":"Document", "rank_by":["field_weight",{"title":3.0,"body":1.0}, "graph retrieval"], "limit":50},
      {"from":"Document", "rank_by":["vector","ann","graph retrieval"], "limit":50}
    ],
    "rerank_by": ["RRF", {"rank_constant": 60}],
    "limit": 20
  }'

What's happening:

• All subqueries are pinned to a single AS OF SYSTEM TIME snapshot, sampled once before fan-out — concurrent writes cannot split results.
• Each subquery runs through the same governed /query path (PURPOSE, ACL, tenant, cluster fan-out identical to N independent calls).
• rerank_by: ["RRF", {rank_constant: K}] fuses the per-subquery ranked lists using reciprocal-rank fusion (score = Σ 1/(K + rank_i)). Default K = 60 (the standard RRF constant). Fused rows carry _rrf_score.
• Vector/ANN rank_by is rejected in batch mode (the AS OF clause isn't in the HYBRID_SEARCH grammar today) — use BM25 / field-weight subqueries for the batch path.

Response shape:

{
  "results": [ {/* per-subquery result set */}, {/* ... */} ],
  "as_of_system": 1735490000000000000,
  "processing_time_ms": 18,
  "fused": {
    "rank_constant": 60,
    "rows": [ /* merged, ranked, with _rrf_score */ ],
    "data": [ /* ... */ ]
  }
}

Tips & takeaways#

• RRF needs ≥ 2 subqueries. A single subquery with rerank_by: ["RRF"] is rejected — there's nothing to fuse.
• Different rank_by strategies make RRF sing. Combine BM25 (lexical) + field_weight (weighted BM25F) + filter variants; each contributes a different ranking signal, and RRF is robust to score-scale differences (it uses ranks, not scores).
• One snapshot = consistent dashboarding. Because all subqueries share one AS OF, the facets/counts/hits in each pane are mutually consistent — no flicker from writes landing mid-search.
• rank_constant tuning: lower K (e.g. 1) weights top ranks more heavily (good when you trust the top of each list); higher K (e.g. 100) smooths toward a popularity vote. 60 is the literature default and a safe start.
• Governance is per-subquery, not bypassed. PURPOSE / ACL / cell masking apply to every subquery independently — a row you can't see in one subquery won't appear in the fused result either.

Cross-ref: Hybrid search (concepts) · Vector params · SQL RANK BY expression · Search & Retrieval reference top

Governance rules#

Search is not a policy bypass. Search and retrieval paths must preserve:

• optional purpose (recorded for audit),
• ACL pushdown,
• cell masking,
• provenance handling,
• temporal semantics,
• audit/cost accounting.

Example topics#

• Search documents by text.
• Search as of a system time.
• Retrieve with provenance.
• Combine identity lookup with document search.
• Explain why a restricted result is masked or absent.

Validation rule#

Any published latency/recall claim must cite a benchmark, conformance row, or reproducible command output.