Platform Data Architecture

Companion to Platform Solution Design Specification. This document covers data that is not ontology content: lifecycle ledgers, registries, job state, and generated artifacts. Ontology and RDF modelling stays normative in the layer READMEs (ontology/foundation/, ontology/surface/, ontology/applied/*) and in ontology-architecture.md. This document exists because the platform now needs a second data model, alongside the ontology, for its own operation.

1. Three Data Realms

The platform has three realms with different authorities. No realm may hold another realm’s authoritative content.

A GraphReference (tenantId, projectId, graphIri, revisionHash) is the only pointer that crosses realms. Every table in PostgreSQL that needs to refer to graph content stores a GraphReference, never a copy of the content. See semantic-platform.md for the cross-runtime contract this pointer serves.

2. Conceptual Data Model

2.1 Operational entities (PostgreSQL realm)

2.2 Artifact entities (artifact realm)

2.3 Semantic graph families (Fuseki realm)

Restated from surface-workflow.md and surface-projection-mork.md because the platform’s operational tables reference these families by name:

3. System of Record Matrix

This is the direct answer to “which component is the system of record for which data.” Every other component holding a copy of this data holds a cache or a reference, not authority.

4. Data Flows

flowchart LR
  subgraph Browser
    Studio[Surface Contract Studio]
    Bench[MORK Review Workbench]
  end
  subgraph ControlPlane[Control Plane HTTP API]
    SurfaceAPI[Surface Workflow API]
    ReleaseAPI[Release API]
    MorkAPI[MORK Review and Governance API]
  end
  subgraph Postgres[(PostgreSQL)]
    Ledger[(surface_revision_ledger)]
    Registry[(surface_graph_artifact)]
    ReleaseLedger[(release_ledger_*)]
    ReviewLedger[(mork_review_*)]
    GovLedger[(governance_ledger_entry)]
  end
  subgraph Broker[RabbitMQ]
    SurfaceQ[[surface job queues]]
    ProjQ[[projection lowering queues]]
    MorkQ[[mork analysis queues]]
  end
  subgraph Workers[Python Worker Tier]
    SurfaceWorker[Surface job worker]
    ProjWorker[Projection lowering worker]
    MorkWorker[MORK analysis worker]
  end
  Fuseki[(Fuseki RDF dataset)]
  Artifacts[(Artifact store / OCI registry)]

  Studio -->|HTTPS JSON| SurfaceAPI
  Studio -->|HTTPS JSON| ReleaseAPI
  Bench -->|HTTPS JSON| MorkAPI
  SurfaceAPI --> Ledger
  SurfaceAPI --> Registry
  SurfaceAPI -->|outbox row, same transaction| SurfaceQ
  ReleaseAPI --> ReleaseLedger
  MorkAPI --> ReviewLedger
  MorkAPI --> GovLedger
  SurfaceQ --> SurfaceWorker
  ProjQ --> ProjWorker
  MorkQ --> MorkWorker
  SurfaceWorker <-->|authorized read/write by GraphReference| Fuseki
  ProjWorker <-->|authorized read/write by GraphReference| Fuseki
  MorkWorker -->|read only, role-projected| Fuseki
  SurfaceWorker -->|generated output bytes| Artifacts
  ReleaseAPI -->|packages digests referenced by the ledger| Artifacts
  SurfaceWorker -->|result event| SurfaceAPI
  ProjWorker -->|result event| SurfaceAPI
  MorkWorker -->|result event| MorkAPI

Each arrow from a worker back to the Control Plane is a result event consumed on a dedicated result queue, not a direct database write from Python into PostgreSQL. The Control Plane is the only writer of operational state. See solution-design-specification.md §2 for the full process maps this diagram summarizes.

5. Data Sharing and Concurrent Access Rules

1. Optimistic concurrency on lifecycle state. Every SurfaceRevision read returns a version. Every transition command carries expectedVersion. A mismatch is a 409, never a silent overwrite. This is the only concurrency control on lifecycle rows, there is no row locking held across a user think-time.
2. Immutable graph-family conflict rule. (tenantId, projectId, graphIri) may be registered once. Registering the same key with a different hash, family, or owner revision is rejected outright, it is not resolved by “last write wins.”
3. Idempotent job replay. (jobId, requestDigest) is the deduplication key for Surface jobs. A retried delivery with an identical request replays the cached result without recompiling. A reused jobId with different content is rejected.
4. Snapshot staleness rule. A MORK review decision must supply the exact snapshotHash of the snapshot it decides against. A newer registered snapshot makes the older hash stale, and a decision against a stale hash is rejected rather than merged.
5. Tenant and project scoping. Every graph reference, job, snapshot, and ledger row carries tenantId and projectId. Cross-scope reads are rejected at the policy layer (GraphAccessPolicy) before they reach a data store, not filtered out afterward in application code.
6. No cross-realm foreign keys. PostgreSQL never stores a foreign key into Fuseki, and Fuseki never stores a foreign key into PostgreSQL. The GraphReference is the only join, resolved at read time, so the two stores can be backed up, restored, or migrated independently.
7. Single writer per data class. Exactly one component writes each table in §3. Readers other than that writer access the data only through the Control Plane’s read APIs, never through a direct database connection from a browser or from another component’s worker process.

6. Retention, Immutability, and Lifecycle

• Graph content, generated output, and OCI artifacts are retained by content digest and are never mutated in place. Deletion is a retention-policy operation, not a routine one, and remains explicitly out of LATTICE’s scope for the release stack (ADR-A31).
• Ledger tables (release_ledger_event, mork_review_decision, governance_ledger_entry) are append-only audit trails. They are the mechanism by which “who inspected this evidence and what did they do with it” stays answerable, per the Robustness and Component Design sections of the master specification.
• surface_revision_ledger and mork_review_snapshot currently overwrite the row’s current-state columns rather than retaining a full state-transition history table. This is flagged as a gap in §7, because it limits point-in-time reconstruction of “what did the revision look like when approval X was granted” to what the ledger event trail (once implemented per §7) can reconstruct.

7. Open Gaps

These are implementation gaps, not design ambiguity. Each has a resolution direction so the next implementation slice has a concrete target.