How It Works#

This page is the conceptual entry point for engineers. It explains what RelataDB does to your data and why it is built that way, without dumping you straight into file paths and ADR numbers. Read this once and the rest of the architecture set makes sense.

For the deeper, reference-oriented version (planes, invariants, ADRs), see Architecture Overview.

The 90-second mental model#

RelataDB is a single Rust binary that replaces a stack of databases — relational, graph, vector, full-text search, and an audit ledger — with one engine. The trade-off is deliberate: the engineering cost of building every plane in-process is accepted in exchange for eliminating the consistency, governance, and operational seams that fragment a polyglot stack.

Whatever door data enters, three things happen to every row, automatically:

1. Recognize — identifiers (emails, phones, IBANs, VINs, IMEIs, MMSIs, hashes, …) are detected, validated, and stored as compact canonical types, then shaped to your declared ontology.
2. Govern — every row is stamped with principal/agency/classification, carries provenance (where it came from), and is written to a tamper-evident audit hash chain.
3. Remember in time — every row is bi-temporal. Updates supersede; nothing is overwritten. You can replay exactly what the database believed at any moment.

Reads go through the same lens: access control compiles into the scan itself (not a post-filter), cells can be masked, and every read is attributed and logged.

One binary, three profiles#

The same binary (relata) runs in three profiles, selected at startup with RELATA_PROFILE=free|server|cluster. The wire protocol, on-disk format, SQL dialect, and ACL semantics are identical across all three — only capacity knobs and which subsystems boot change.

This is why the path from a laptop prototype to a cluster deployment is a config change, not a rewrite.

The five planes#

Every feature belongs to exactly one of five planes. Knowing which plane a feature lives in predicts its operational surface — its latency, its failure mode, and its audit story.

Cross-plane work goes through typed interfaces. There are no shared mutable globals between planes — a property that keeps the blast radius of any change bounded.

The journey of a row#

The clearest way to understand RelataDB is to follow one row from the outside world to a query result.

You connect                Recognize             Govern                Remember in time
───────────                ─────────             ──────                ────────────────
psql · S3 · Mongo ·  ──►   canonicalize    ──►   ACL + tenant     ──►  bi-temporal rows
Redis · ClickHouse ·       validate identities   provenance stamp       WAL (hash-chained)
Neo4j · Bolt ·             shape to ontology     audit entry            Parquet segment
Arrow · HTTP · MCP                                                        IdentityIndex delta

On the way in#

1. A door receives the payload. The door is the source of truth — an S3 object, a Mongo document, or a Redis key is a governed row. Each door is opt-in with one env var and a port.
2. The gateway authenticates the caller into a principal (OIDC JWT / mTLS cert), resolves the tenant, and records any declared PURPOSE.
3. The eager phase validates and canonicalizes declared columns, appends the write-ahead log (fsync), and writes the bi-temporal row. It does only declared-column work and stays in single-digit-millisecond p99.
4. The lazy phase (asynchronous, driven by the indexer) refreshes materialized views, generates embeddings, runs SmartIngest identity extraction, and infers relationship edges. Every lazy step is re-runnable from the last committed WAL offset.

See Data Flow for the full eager/lazy split and the query path.

On the way out#

1. A request arrives over any door and is authenticated into a principal.
2. The coordinator parses, checks the optional PURPOSE, and compiles the principal's ACL into per-type bitmap predicates.
3. The cost-based optimizer picks join order, scan strategy, and applies materialized-view rewrite.
4. Execution is streaming — columnar scan, graph traversal, vector ANN, and BM25 full-text run fused, with no full intermediate materialization.
5. Results ship as Arrow batches (optionally with PROV-O columns), and an audit entry is written.

The key idea: policy lives in the query path, not as a post-filter. ACL decisions are evaluated at the vectorized loop level alongside the filter, which is why governance does not mean a slow database.

SmartIngest, in one paragraph#

SmartIngest is the deterministic identity pipeline that runs on ingest. It detects identifiers using per-type shape gates plus checksum validators (so an IBAN's mod-97 or an IMEI's Luhn must pass), indexes them into the universal IdentityIndex, and fuses records that share a validated identifier into one cluster. This is what lets the same human — appearing as a phone number in one feed, an email in another, and an account ID in a third — resolve to one entity, automatically.

The honest boundary: RelataDB does identifier extraction, not general named-entity recognition. There is no in-tree transformer. For names-in-prose, intent, or sentiment, you register an external scorer; RelataDB writes its output as governed, provenance-stamped assertions.

See SmartIngest.

Jobs and what triggers them#

A core invariant: every background work item is a typed Job. No opaque daemons, no cron, no shell scripts. Indexing, materialized-view refresh, compaction, and pattern detection all surface in system.jobs with state, owner, and progress.

Triggers include the WAL-delta-driven lazy indexer, ON COMMIT and INCREMENTAL materialized-view refresh, scheduled compaction, and the continuous pattern-detection engine. See Jobs & Triggers.

Design invariants that shape everything#

These are non-negotiable. A change that breaks any of them produces a different database:

• Bi-temporal everywhere — every row carries four timestamps. No opt-out.
• The ontology is the schema — you declare types, you never CREATE TABLE.
• Cell-level ACL is pushed into scans — bounded under ~2.5× raw scan p50.
• Object-store native — open formats (Parquet, Arrow IPC, content-addressed blobs); no proprietary on-disk format. A file written by RelataDB is readable by any Parquet reader.
• Hash-chained, tamper-evident commit manifests — tampering with any historical file breaks the chain, detectable offline.

Where to go next#

• Data Flow — ingest (eager/lazy) and the query path, in depth.
• SmartIngest — identity detection and its honest boundary.
• Jobs & Triggers — the typed-Job engine and what fires it.
• Architecture Overview — the reference version: planes, ADRs, invariants.
• Data Model · Storage Engine · Query Engine