SIEM / SOC: detection and investigation

Relata DB 3.0
Single machine, one tenant. Relata DB does not collect or parse logs: connectors, Sigma conversion and event normalization belong to OpenETL, which loads mapped rows through POST /v1/ingest/bulk. Sigma import is no longer part of core; native rules at /v1/rules stay. See SIEM/SOC migration.

The problem

A SOC wants the alert, the events behind it, the access policy that applied to them and the investigator's pivots to live in one place, so that an alert can be traced to its source rows and the export can be checked later.

The shape

  1. Map and load (OpenETL). Each raw event becomes a row of the shape you chose and is sent to POST /v1/ingest/bulk.
  2. Gate (Relata). PUT /v1/ingest/contracts/{dataset} installs a versioned data contract. Rows that violate it are quarantined with the reason and never block the valid rows. GET /v1/ingest/quarantine and GET /v1/ingest/contracts/{dataset}/quality show the quarantine and schema drift.
  3. Detect. A window rule fires one alert per group per window. The supported semantics are threshold and correlation over tumbling event-time windows with a durable cursor; anything else is refused with 400 unsupported-rule-semantics, never approximated.
curl -s -X POST "$RELATA/v1/rules?purpose=audit" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -d '{
  "name": "brute force", "target_type": "AuthEvent", "severity": "high",
  "window": {"kind": "threshold", "event_type": "login_failed", "min_count": 3,
             "window_ns": 300000000000, "allowed_lateness_ns": 60000000000,
             "time_field": "ts", "group_field": "user", "event_type_field": "kind"}}'
  1. Triage. PATCH /v1/alerts/{id} moves an alert through open, acknowledged, resolved and false_positive. A forbidden transition is 409 invalid-alert-transition, and every accepted change is appended to the alert's status history with its actor.
  2. Evidence. GET /v1/alerts/{id}/evidence returns the alert, the contributing events under the caller's read policy, the rule version and window, a completeness report and a SHA-256 over the events. GET /v1/alerts/{id}/timeline orders the same material.
  3. Pivot and hunt. POST /v1/alerts/{id}/pivot follows one entity into the other records the caller may read. Host relationships are links (POST /v1/links) that GET /v1/graph/traverse and GET /v1/graph/shortest-path walk. Rule tuning (snooze, suppression, exceptions) is durable.
  4. React to changes. GET /v1/changes is a resumable change feed (off unless RELATA_CHANGE_FEED_ENABLE is set).

Governance is in the path: a principal without read access to the event type gets the alert but every event as not_authorized, and the export is marked partial.

Telemetry from your own services can arrive over OTLP at POST /v1/traces, POST /v1/logs and POST /v1/metrics (off unless enabled; see Observability).

Honest limits

  • Writes are accepted, not yet durable by default. Loading rows with X-Relata-Durability: sync waits for stable storage; otherwise poll GET /v1/ingest/tasks/{id}.
  • Pivots are exact-equality lookups, not graph traversal.
  • Evidence lists at most 1000 event ids per alert.
  • Single node, local storage. Hard kill was tested; host power loss, failover and an external SIEM were not.
  • The migration from an existing SIEM has been rehearsed only on a synthetic fixture. See SIEM/SOC migration.

Exercised by

Scenario V3-14 (brute force to alert, triage, evidence, pivot, graph hunt, kill -9 recovery) and V3-04/V3-05 (OTLP) in the relata-test 3.0 battery, run against a debug build of the integration branch on 2026-11-03. The SIEM run passed its defining assertions and recorded one defect: evidence.reproduce.sql omits the rule's event_type filter, so the reproduction query returns one extra row (the user's successful login) compared with the alert's evidence. Treat the SHA-256 export as the authoritative record and the reproduction query as a starting point until that is fixed.