AI in RelataDB#

RelataDB is built both for AI and with AI — and those are two different things. This page separates them, shows the SmartIngest pipeline that turns raw data into governed knowledge, walks through one complete RAG example, and explains why this beats the usual RAG stack.

The one-line distinction. Agents and apps call into RelataDB to get governed memory, tools, and retrieval (for AI). RelataDB calls out to models to embed, search, and interpret (with AI). The trust path — identity, access control, provenance, audit — is deterministic and never depends on a model.

The mental model: knowledge first, AI second#

Most "AI databases" bolt a vector index onto a pile of JSON and call it memory. RelataDB inverts that: the knowledge is the product; the AI sits on top of it and underneath it, but never in the trust path.

   you preprocess               RelataDB does this automatically            agents/apps query
   (chunk, enrich, embed)                                                  with AI
        │                                                                 │
        ▼                                                                 │
  ┌─────────────┐   ingest door   ┌───────────────────────────┐   recall / RAG / nl_query
  │ raw records │  ─────────────► │  SmartIngest (deterministic)│  ──────────────────────►
  │ docs, CDRs, │                 │  • validate + canonicalize  │     hybrid BM25 + HNSW
  │ posts, PDFs │                 │  • identity detect + fuse   │     identity-graph recall
  └─────────────┘                 │  • provenance stamp         │     provenance citations
                                  │  • bi-temporal rows         │     governed (ACL + PURPOSE)
                                  │  • embeddings (lazy MV)     │
                                  └───────────────────────────┘
                                                  │
                                                  ▼
                                  one governed knowledge graph ── the substrate for RAG & memory

The payoff: retrieval isn't just "find text that looks similar." Because SmartIngest canonicalizes identities and links records across sources, a query about Alice finds the whole identity cluster (her email, phone, accounts, co-occurrences) and every fact attached to it — not just chunks that mention the word "Alice."

Built for AI — what agents and LLMs consume#

Every surface below runs through the same governed path as a human-issued query — ACL, cell masking, PURPOSE, tenant isolation, audit. Governed AI: the agent gets tools, not a back door.

Built with AI — what runs under the hood#

Each piece below is model-agnostic and opt-in — point it at the provider you trust. Nothing is hard-wired to a vendor.

All model-touching config lives on one page: LLM & Embedding Configuration — the LLM endpoint (RELATA_LLM_URL) for NL query, and the embedder sidecar (RELATA_EMBED_URL, formerly RELATA_ACCEL_ENDPOINT) for vectors.

The SmartIngest pipeline — preprocess once, retrieve forever#

SmartIngest is the deterministic engine that turns raw input into the governed substrate RAG and memory draw from. You can preprocess (chunk, enrich, pre-embed) before you ingest; SmartIngest then does the rest, automatically, on every write.

What SmartIngest does (deterministic, checksum-gated)#

raw text / cell value
   │
   ▼
tokenize  ──►  per-token shape gate (regex)
                   │ pass / fail ──► skip
                   ▼
              format + checksum validate     ◄── IBAN mod-97, IMEI Luhn,
                   │ pass / fail ──► skip         Aadhaar Verhoeff, VIN…
                   ▼
              DetectionHit (CanonicalKind + Identity)
                   │
                   ▼
              IdentityIndex: (kind, bytes) → all observations
                   │
                   ▼
              fuse records that share a validated identifier → one cluster

This runs lazily as materialized views off the WAL, so it never blocks the write hot path and it's re-runnable — upgrade a detector and the MV refreshes, no source backfill.

Why this supercharges RAG and agent memory#

The honest boundary: RelataDB does identifier extraction, not general named-entity recognition. There is no in-tree transformer guessing names out of prose. For fuzzy NER, sentiment, or intent, register an external scorer — RelataDB writes its output back as governed, provenance-stamped typed assertions and fuses it into the same graph. (SmartIngest deep dive)

The end-to-end flow for an AI app#

1. PREPROCESS (optional, you)     2. INGEST (one door)         3. QUERY WITH AI (agent/LLM)
   chunk long docs                   client.ingest(...)          recall / SEARCH HYBRID
   pre-compute _emb_text             POST /ingest/document       nl_query("…")
   enrich with a scorer              PUT via S3 / Mongo /        similar_multimodal(...)
   tag with purpose + class          Redis / pgwire door         → governed, cited, replayable
            │                                  │                            ▲
            └──────────────► SmartIngest ──────┘                            │
                               canonicalize · detect · fuse · stamp         │
                               embed (async MV) ── into the knowledge graph ┘

Build RAG on RelataDB — one complete example#

A governed retrieval-augmented agent over an AML document corpus. Four steps; Python here, the same shapes exist in TypeScript and Go.

0. Declare the ontology once#

You don't CREATE TABLE — you declare types. The schema shapes what SmartIngest validates and what the SDKs expose. (Ontology & schema)

# ObjectType: IntelChunk — a retrievable text chunk with a vector + a source.
curl -X POST http://localhost:9090/types \
  -H "Authorization: Bearer $RELATA_BEARER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "IntelChunk",
    "properties": [
      {"name": "text",    "type": "string", "indexed": "bm25"},
      {"name": "_emb_text","type": "[1536]f32"},
      {"name": "source",  "type": "string"},
      {"name": "mentions","type": "identity"},
      {"name": "tags",    "type": "[]string"}
    ]
  }'

text is BM25-indexed for keyword recall; _emb_text carries the vector (caller-supplied or populated by the embedder sidecar); mentions is an Identity column that SmartIngest will populate from identifiers in the text. Schema-as-code, versioned in git, evolvable without downtime.

1. Ingest — SmartIngest does the heavy lifting#

Drop documents in; they're auto-chunked, identity-detected, provenance-stamped, and embedded asynchronously. Identifiers in the text (IBANs, emails, phones) are canonicalized and linked into the identity graph.

# Bulk ingest chunks — or use POST /ingest/document with auto_chunk:true for whole PDFs
client.ingest("IntelChunk", [
    {"_pk": "c1", "text": "Alice wired $1.2M to Shell Co Ltd (IBAN GB29 NWBK 6016 1331 9268 19).",
     "source": "sar-2024-07.pdf", "tags": ["wire", "shell-co"]},
    {"_pk": "c2", "text": "Shell Co Ltd is 100% owned by Atlas Holdings, BVI.",
     "source": "registry-bvi.json", "tags": ["ownership"]},
    {"_pk": "c3", "text": "Alice's known account alice@corp.io; primary +1 415 555 0111.",
     "source": "kyc.csv", "tags": ["kyc"]},
])
# SmartIngest: GB29…IBAN, alice@corp.io, +1 415… → validated, canonicalized,
# fused into one identity cluster for "Alice" — across all three sources.

2. Retrieve — hybrid + provenance, ACL-safe#

# Simple path — the SDK helper
hits = client.search("shell company laundering", "IntelChunk", limit=10, highlight=True)
for h in hits.hits:
    print(h.score, h.fields["source"], h.highlights)
 
# Full path — SQL with hybrid + provenance citations
rows = client.query("""
    SELECT id, text, source, score
    FROM IntelChunk
    WHERE SEARCH HYBRID('shell company laundering', top_k = 10)
    WITH PROVENANCE
""")
# Every returned chunk carries where it came from — no hallucinated citations.

3. Ground the agent — memory + RAG in one plane#

Because IntelChunk rows and MemoryItem rows live in the same governed store, your agent recalls documents and its own notes through one governed surface.

from relata import Memory
 
mem = Memory("http://localhost:9090", purpose="aml", bearer_token=TOKEN)
 
# Agent remembers a working hypothesis…
mem.add("Alice → Shell Co → Atlas Holdings looks like layering", confidence=0.8)
 
# …and recalls both its notes and the corpus, ranked together.
for m in mem.search("alice shell company", top_k=5):
    print(m["content"], "←", m.get("source", "agent-note"))

// TypeScript — same verbs, same governance
import { createClient } from "@zysec-ai/relata-sdk";
const relata = createClient("http://localhost:9090", { defaultPurpose: "aml", bearerToken: TOKEN });
await relata.remember("Alice → Shell Co → Atlas Holdings looks like layering");
const hits = await relata.recall("alice shell company", { topK: 5 });

Same thing, over MCP — for Claude / Cursor / Cline#

// Point your MCP client at Relata, then call tools directly:
{ "name": "recall",  "arguments": { "q": "alice shell company", "top_k": 5, "purpose": "aml" } }
{ "name": "nl_query","arguments": { "query": "who owns Shell Co Ltd and how does Alice connect?",
                                    "purpose": "aml", "interpret": true } }

nl_query returns governed rows plus generated_sql, model_id, and llm_used (so you always know whether an LLM produced the SQL). interpret: true adds a natural-language summary with full model provenance.

That's the whole loop: declare once → ingest → retrieve with citations → ground the agent. No separate vector DB, no graph DB, no audit store, no ETL between them.

Wire it to an agent#

Drop-in adapters wrap the memory verbs — purpose tracking and ACL stay on. None of the Python adapters imports its framework, so they're safe to install without it.

# LangChain example — governed long-term memory on any chain
from langchain_openai import ChatOpenAI
from langchain.chains import ConversationChain
from relata_adapters.langchain import RelataMemory
 
chain = ConversationChain(
    llm=ChatOpenAI(),
    memory=RelataMemory(
        base_url="http://localhost:9090",
        bearer_token=TOKEN,
        purpose="support-agent",
        session_id="cust-42",
        top_k=5,
    ),
)

LangGraph works the same way via RelataCheckpointer (governed checkpoint persistence over the A2A door) — see the Agent Memory adapters table.

Why it's better than the usual RAG stack#

The typical stack is Postgres + Pinecone + Neo4j + a memory store + an audit log, glued together with ETL. RelataDB replaces all of it with one binary — and the differences show up exactly where RAG fails in production.

The net: where others give you retrieval, RelataDB gives you retrieval you can defend. Same SDK ergonomics, none of the seams.

The honest trade-off#

RelataDB optimizes for accountability; convenience-memory tools optimize for fast onboarding. If your RAG just needs "push JSON, get semantic hits" with no governance, a vector DB is simpler. If the answer must be right, sourced, replayable, and access-controlled — regulated, intel, LEA, FININT, court-grade, or enterprise — that's where RelataDB wins.

The deterministic boundary — why you can trust it#

Models are great at guessing; they're bad at proof. So RelataDB keeps the proof path deterministic:

• Identity resolution is checksum-gated, not learned. An IBAN's mod-97 must pass before records link. You trust the merge because you trust the math. (SmartIngest)
• Access control is compiled into scans. ACL decisions are bitmap predicates, not model outputs — identical on every call.
• Provenance and audit are tamper-evident. Every row carries a PROV-O link; commits form a SHA-256 hash chain. No model is in that loop.

The AI is modular and replaceable; the guarantees are not. Swap the embedder, the LLM, or the scorer without touching governance, history, or audit.

Choose your posture#

Every model-touching piece is opt-in, so you pick where on the spectrum you run:

Whatever you pick, the governed path does not change.

See also#

• Agent Memory — the 10 cognitive verbs, recall ranking, and every adapter
• SmartIngest — the deterministic identity pipeline in depth
• Hybrid Search — BM25 + HNSW + RRF
• Governed RAG use case — a vertical walkthrough
• LLM & Embedding Configuration — every model-touching env var
• Governance — why every AI call rides the same policy path
• MCP Tools · Agent Memory reference