# trusty-postgres -- Design Memo Status: DRAFT. Pre-implementation. Author: design pass over the LCG architecture-antipatterns lessons, the Store trait surface, and the trusty-sqlite reference implementation. Reader: Eric. This memo defines the contract, schema, and architectural decisions for `trusty-postgres` BEFORE any code is written. The point of writing it now is the meta-principle in `insight-LCG-ARCH-01`: features that "we will add in v2" become 5-10x more expensive to retrofit than to build in from day one. Every section below names the lesson it draws from so the design's lineage is auditable. The reference implementation to mirror is `crates/adapters/trusty-sqlite/src/store.rs:1-955` and `crates/adapters/trusty-sqlite/src/schema.rs:1-49`. The trait being implemented lives at `crates/core/trusty-store/src/lib.rs:22-48`. --- ## 1. Goal and non-goals ### Goal `trusty-postgres` is a sibling adapter crate (likely separate Gitea repo `Modo-Melius/trusty-postgres`) that implements `trusty_store::Store` and the higher-level `Trusty` operations against PostgreSQL 14+ with optional TimescaleDB extension. It targets the deployment tier named in `s14_insight_store_trait_multi_backend_landscape.toml`: "mid-scale cloud, time-series query acceleration on received_at via hypertable." The product framing comes from `s14_insight_oid_anchored_chain_identity.toml`: trusty-postgres is the "OID-anchored TrustyDB" surface for enterprise buyers who already run Postgres + PKI infrastructure. The pitch is "your existing Postgres, plus cryptographic proof of stream completeness/authenticity," not "a new database." This framing forecloses any temptation to bundle application logic, distributed coordination, or marketplace primitives into the crate. It is a Store adapter and the higher-level `Trusty` methods, nothing more. ### Non-goals - Not a replacement for `trusty-sqlite`. SQLite remains the embedded / edge / single-process reference. Postgres targets multi-writer, fleet-scale, or organizationally-required-Postgres deployments. - Not a distributed-coordination layer. Multi-region, multi-master, or consensus-flavored deployments belong in a hypothetical `trusty-scylladb` or above. Postgres logical replication is fine if the operator wants it; we do not invent any cross-cluster chain reconciliation primitive here. - Not an analytics surface. DuckDB / columnar read paths are a separate adapter per the four-tier landscape tag. - Not a migration of the locked v1 byte-shape. The point of the Store trait is that `trusty-core` primitives are untouched. Any change that would shift v1 bytes is out of scope by definition; see `s5_invariant_v1_byte_shape_preserved` and `CLAUDE.md` "Don't Touch." - Not a TCB-redefining crate. The trusted computing base added by this adapter is: postgres + libpq + the chosen Rust driver. Document this on page one of the README (per `learn-LCG-ARCH-13` / TCB manifest discipline) and let it sit there as a deliberate cost. ### Relationship to Distant Star (DS) trusty-postgres is orthogonal to the DS network protocol (dspm, dsnl, dsutils, ds-cli -- the Monorepo-scratch stack). DS is a UDP gossip transport with X-Wing KEM, exponential-decay hop probability, and tag-based knowledge-network routing; it transports data, it does not store it. trusty-postgres stores data behind the Store trait. The two compose, they do not overlap. Concrete composition points (relevant so this crate's API does not accidentally foreclose them): - **trusty-ds-fs (#10)** and **trusty-ds-network (#12)** may use trusty-postgres as their persistence layer when a DS node is deployed in the mid-scale cloud tier rather than at the edge. The Postgres adapter must therefore expose a `stream_anchors_since` and a ProofBundle export surface that DS handlers can call without coupling to any internal detail. - **Advertise-don't-copy pattern from the DS README.** DS nodes advertise file existence (tag + hash) and serve the file only on demand. The same pattern applies to chain proofs: a DS-integrated trusty-postgres node should advertise `(chain_id, tip_hash)` and serve the ProofBundle on pull, not push full chains through the gossip layer. This shapes the read API: prefer iterator-based proof export over "materialize the whole bundle in memory." - **Key distribution is NOT DS's job** per the S14 SSH-paradigm decision (`s14_decision_decouple_key_distribution_from_ds_handshake`). DS session keys (X-Wing) and trusty verification keys (SipHash + Ed25519 via #17 trusty-keydist) live on separate tracks. trusty-postgres stores the SipHash keys in the `producer_identity` table (section 8); it does not participate in any DS handshake. Out of scope for this memo but noted: a future `trusty-ds-postgres` integration crate (separate from trusty-postgres) would implement the `CapabilityHandler` trait from dspm and wire the Store trait to DS marketplace messages. That crate is not trusty-postgres. --- ## 2. Scope: trait methods + higher-level Trusty methods ### Store trait surface The 7 methods at `crates/core/trusty-store/src/lib.rs:22-48`: - `last_row(entity_id)` - `insert_row(&RowInsert)` - `iter_chain(entity_id) -> Iterator>` - `get_tip(chain_id)` - `get_tip_for_entity(entity_id)` - `set_tip(&TipUpsert)` - `kv_cross_check()` - `with_txn(&mut dyn FnMut(&mut dyn Store) -> StoreResult<()>)` All 7 must be implemented. `iter_chain` is the one with non-trivial ergonomics in async-Postgres land -- see section 7 (with_txn) and section 9 (concurrency). ### Higher-level Trusty methods (NOT in the Store trait today) Per `s14_insight_store_trait_multi_backend_landscape.toml`, four higher-level methods live in `trusty-sqlite::Trusty` rather than behind the trait. They must be re-implemented in the Postgres adapter: - `append` / `append_signed` / `append_with_method` / `append_with_method_signed` -- the orchestration around `compute_record` + `write_envelope` + `append_hash_index` + `set_tip`. Source: `store.rs:152-283`. - `append_batch(&[RowSpec]) -> Vec` -- single-transaction amortization. Source: `store.rs:528-553`. - `verify(entity_id) -> VerifyReport` -- iterate `provenance_envelope` in id-asc order, reconstruct `ChainLink`s, hand to `trusty_core::verify_links`. Source: `store.rs:467-500`. - `breaks_since`, `liveness_check`, `anchors_since`, `export_bundle`, `history`, `integrity`, `anchor` -- the read/observability surface. Source: `store.rs:502-836`. Estimated 300-400 lines of mechanical work after the trait is implemented, per the multi-backend tag. ### Out of scope for v1 of trusty-postgres - `tk_*` binaries / APERC tooling -- no adapter-side hooks. - Schema migrations between trusty-postgres versions. Document the v1 schema as locked by analogy to the locked v1 byte-shape; any v2 schema work is a future memo. (See section 11 cross-ref.) - Connection pooling configuration as a public API. Use the driver's pool with sane defaults; expose the pool object only if a downstream user demands it. --- ## 3. Schema design ### Three-table layout, Postgres dialect Mirror `crates/adapters/trusty-sqlite/src/schema.rs:1-49` one-to-one in structure. The three-table layout is locked per CLAUDE.md "Don't Touch": `provenance_envelope`, `hash_index`, `chain_tips`. The KV cross-check (`integrity` method) depends on both `chain_tips` and `hash_index` agreeing on `(entity_id, chain_id, current_tip_hash)`. Postgres dialect translation: - `INTEGER PRIMARY KEY AUTOINCREMENT` -> `BIGSERIAL PRIMARY KEY` or `BIGINT GENERATED BY DEFAULT AS IDENTITY`. Use the latter; identity columns are SQL-standard and easier to reset for tests. - `TEXT` -> `TEXT` (Postgres TEXT is unbounded VARCHAR; same semantics). - `INTEGER` for sequence_int -> `BIGINT`. Sequence numbers are i64 in the Rust types. - Indexes: same set as SQLite. Add a partial index `CREATE INDEX idx_pe_break ON provenance_envelope(entity_id, detected_at) WHERE event_type = 'BREAK'` to accelerate `breaks_since`. The SQLite version scans; Postgres can short-circuit with the partial index. - `raw_payload TEXT` -> `raw_payload TEXT`. Do NOT use `JSONB`. The payload is a canonical byte-stable JSON string that the verifier reads back and recomputes hashes over. JSONB normalizes whitespace and key order, which would silently break v1 byte-shape verification. This is a v1-byte-shape-preservation invariant; document it inline in the migration SQL. ### TimescaleDB hypertable on `received_at` Decision: support TimescaleDB as an optional Cargo feature (`features = ["timescale"]`), default off. When enabled, the migration runs `SELECT create_hypertable('provenance_envelope', 'received_at', chunk_time_interval => INTERVAL '7 days')`. Reasoning: - The IoT/sensor tier (per `project_trusty_role.md` in memory) is the primary trusty-postgres adopter. Time-series query acceleration on `received_at` is the right enterprise pitch. - Hypertables do not change row identity, primary keys, or read/write semantics from the chain's perspective. The hash chain is independent of physical chunking. - Default-off avoids forcing the TimescaleDB dependency on operators who do not have it. The adapter must work on stock Postgres 14+. - One constraint: `chain_tips` and `hash_index` are NOT hypertables. They are small lookup tables; chunking by time would hurt, not help. OPEN QUESTION: do we want a separate `received_at` column type (`TIMESTAMPTZ`) versus the SQLite TEXT-ISO8601 string used today? Recommendation: yes, store as `TIMESTAMPTZ` for query performance, but expose as RFC3339 string at the Store trait boundary so the trait is dialect-neutral. This is purely an internal storage choice and does not affect chain bytes. Confirm with Eric. --- ## 4. Tri-state null (apply `learn-LCG-ARCH-03`) The tag is unambiguous: "wherever absence of a value might be legitimate, encode three states (PRESENT, ABSENT, UNKNOWN) explicitly." SQL NULL alone conflates ABSENT and UNKNOWN. Audit of nullable columns in the trusty schema: | Column | NULL means today | Tri-state interpretation | |---------------------------------|------------------------------------------------|---------------------------------------------------------------------| | `provenance_envelope.row_hash` | "BREAK row, no hash applies" | ABSENT (BREAK rows structurally have no hash). Two-state is fine. | | `provenance_envelope.anchor_hash` | same as row_hash | ABSENT. Two-state is fine. | | `provenance_envelope.raw_payload` | same as row_hash | ABSENT. Two-state is fine. | | `provenance_envelope.field_name`/`old_value`/`new_value` | "non-BREAK row, fields don't apply" | ABSENT. Two-state is fine. | | `chain_tips.current_tip_hash` | TODAY ambiguous | NEEDS TRI-STATE. See below. | | `chain_tips.chain_id` | "no chain bound to this entity yet" | UNKNOWN/ABSENT ambiguity. See below. | The cold-start case is the one that bites: > An OID-bound producer is provisioned (its OID is registered, its > Ed25519 pubkey is in `producer_identity`) but it has not yet > appended a single row. In trusty-sqlite, this state is unrepresentable: `chain_tips` rows are created on first append, so an unappended producer simply has no row. That conflates "we know this OID exists and has appended nothing" with "we have never heard of this OID." For an enterprise PKI integration that pre-registers devices, this matters. Decision: introduce explicit cold-start representation in the trusty-postgres schema: - `chain_tips.current_tip_hash`: encoded as `tip_hash TEXT` paired with `tip_state TEXT NOT NULL CHECK (tip_state IN ('UNKNOWN', 'COLD', 'WARM'))`. `UNKNOWN` = pre-provisioned, never appended. `COLD` = appended once but the chain was sealed by a break and a new chain is pending. `WARM` = chain is live and tip_hash is the current valid head. - `chain_tips.chain_id`: NULL only when `tip_state = 'UNKNOWN'`. Otherwise NOT NULL. - New optional column `producer_oid TEXT` (see section 8). Per `learn-LCG-ARCH-03`, prefer enum/check-constrained columns over sentinel-value tricks. The pair `(tip_hash, tip_state)` makes every `get_tip` result explicit about which of the three states it is in. This forces a trait-level decision: should `TipRow` (defined at `crates/core/trusty-store/src/lib.rs:60-66`) carry a tip_state field? Recommendation: YES, additively. Add `tip_state: TipState` to `TipRow` and `TipUpsert` with a default `WARM` for backward compatibility with trusty-sqlite. trusty-sqlite would always return `WARM` (its existing semantics) and trusty-postgres would return the true state. Verify this does not break any existing trusty-sqlite test before merging. OPEN QUESTION: does this trait-level addition rise to the level of needing v2 of the Store trait, or can it be additive on v1? Eric to decide. Recommendation: additive with a default; this matches the v2-payload-additive precedent in `project_trusty_v2_signing_design.md`. --- ## 5. Compaction strategy (apply `learn-LCG-ARCH-02`) The tag is explicit: "chain format supports compaction snapshots FROM v1. Archive boundaries are themselves attested. No retrofit possible without breaking historical verifiability." The append-only chain has no semantic notion of compaction today. trusty-sqlite grows unbounded; the IoT-fleet pitch implies hundreds of devices each appending hundreds of rows per day, which crosses "too large to verify in a reasonable time" within months. ### What compaction means for an append-only hash chain Three architectural options: **Option A: snapshot tables.** Periodically materialize `(chain_id, sequence_int_high_water, snapshot_root_hash, snapshot_payload)` into a `chain_snapshot` table. The `snapshot_root_hash` is computed canonically from the slice it covers (a Merkle root of all row_hashes in the slice, or simpler: a SipHash over the concatenation, with domain separation). The snapshot row is itself a valid chain envelope of `event_type = 'SNAPSHOT'`. After a snapshot is committed, archived slices can be moved to a cold-storage table or dropped, and verifiers can resume mid-chain by trusting the snapshot. **Option B: TCB manifest checkpoint.** Maintain a `chain_checkpoint` table with `(chain_id, sequence_int, row_hash, checkpoint_attestation)`. Verifiers can prune their verification window to "from the most recent checkpoint forward" and trust earlier state by checkpoint signature. Lighter weight than snapshots; does not enable archival, only verification acceleration. **Option C: Anchor checkpoints.** Repurpose existing per-row `anchor_hash` as the compaction primitive. Periodically aggregate anchors over a window into a single `aggregated_anchor` and publish it externally (S3, public ledger, whatever). Verifiers spot-check against the aggregated anchor. This is the lowest-cost option and mirrors Certificate Transparency Merkle-tree consistency proofs. ### Decision Ship Option B (`CHECKPOINT` envelope, verification-layer only) as the v1 primitive in trusty-postgres. Reasoning: - Issue #13 explicitly says "MUST NOT change v1 byte-shape. Checkpoints are a verification-layer construct, not a payload change." Option A (SNAPSHOT) adds a new event_type which, while additive, is a semantic change to the chain contract and requires a new trusty-core aggregation primitive -- that is a larger design pass than v1 of trusty-postgres should attempt. - Option B keeps the existing chain contract untouched. A checkpoint is a signed (`chain_id, sequence_int, row_hash, attestation)` tuple written to a separate `chain_checkpoint` table. Verifiers can prune their verify window to "from the most recent valid checkpoint forward." No new envelope type, no new canonical bytes, no trusty-core change beyond exposing the checkpointing API in a higher-level module. - Archival (#5) and row-aggregation compaction (Option A) are deferred to v2. Both build on top of the checkpoint primitive once it is proven in production. - Issues #5 and #13 resolve cleanly: #13 is the v1 feature; #5 is the v2 feature that uses checkpoints as archive boundaries. - Terminology lock: the word `compaction` is retired from this crate's vocabulary in favor of `checkpoint` (for v1) and `archive` (for v2). No ambiguity. Schema: add a `chain_checkpoint` table: ``` chain_checkpoint ( id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, chain_id TEXT NOT NULL, sequence_int BIGINT NOT NULL, row_hash TEXT NOT NULL, checkpoint_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), attested_by TEXT NOT NULL, -- OID or pubkey fingerprint attestation_sig BYTEA NOT NULL, -- Ed25519 over (chain_id, sequence_int, row_hash) UNIQUE (chain_id, sequence_int) ) ``` API on `Trusty`: - `checkpoint(chain_id, attestation_key) -> CheckpointRow` -- walks the chain to head, computes a fresh checkpoint at head, writes the row. - `verify_since_checkpoint(chain_id) -> VerifyReport` -- reads the most recent valid checkpoint, iterates forward from there, same report shape as `verify` today. This is all adapter-level; no trusty-core change required for v1. Archival and row-aggregation compaction become separate design memos when the archival product case (#5) is ready to ship. --- ## 6. Revocation semantics (apply `learn-LCG-ARCH-06`) The tag is blunt: "Every system that said 'we will figure revocation out in v2' paid for it in v2 with interest." Existing trusty memory treats revocation as application-layer. Re-examine for OID-anchored deployment: ### The case under OID anchoring A producer device has OID `1.3.6.1.4.1..` and an Ed25519 keypair. The chain is signed (v2 payload) per `project_trusty_v2_signing_design.md`. Mid-stream, the device's key is revoked -- compromised hardware, retired device, or operator policy. Three sub-cases: **Sub-case (a): retroactive revocation of past rows.** Should rows appended before the revocation timestamp still verify? If yes, the chain remains valid up to the revocation point and is sealed there. If no, the entire chain is poisoned by retroactive distrust. **Sub-case (b): forward block on new appends.** After revocation, any append signed by the revoked key must fail. This is structurally the chain-break primitive: a revoked-key append is a break, sealed chain ends, no new genesis under the same OID. **Sub-case (c): key rotation.** Revoke old key, install new key, continue chain under new key. Requires a chain-of-custody attestation linking old-key-tip to new-key-genesis. ### Recommendation **Adopt the Cert-Transparency stance: chains are append-only and revocation is forward-only.** Sub-case (a) is rejected: past rows remain verifiable forever. Sub-case (b) is the default behavior (the existing chain-break primitive handles it for free if the condition library checks signing-key validity; that condition needs to exist). Sub-case (c) is a v2 feature involving a `KEY_ROTATION` envelope and an attestation chain to the new key. Schema: add a `producer_revocation` table: ``` producer_revocation ( producer_oid TEXT NOT NULL, revoked_at TIMESTAMPTZ NOT NULL, revoked_pubkey BYTEA NOT NULL, reason TEXT NOT NULL, attested_by_pubkey BYTEA, -- who authorized the revocation revocation_envelope_id BIGINT -- FK to a provenance_envelope row ) ``` The revocation event is itself a chain envelope of `event_type = 'REVOCATION'` so the act of revoking is part of the auditable chain. This satisfies both the append-only invariant and the auditability requirement. Per the LCG-ARCH-06 runtime invariant: "revocation of an upstream capability invalidates derived attestations within N blocks. Testable via assert_invariants.py: insert edge, attest, revoke edge, verify attestation fails on re-check." Translate to a trusty-postgres integration test: append signed rows, write revocation envelope, attempt new append signed by revoked key, assert it produces a chain-break. OPEN QUESTION: where does the condition that checks "is this signing key currently revoked?" live? Options: (i) a new core-level Condition in `trusty-core`, (ii) a Postgres-adapter-only Condition. Recommendation: (i), so the semantics are uniform across backends. Schedule as part of the trusty-core SNAPSHOT-primitive memo (section 5) -- same memo, same set of additive trusty-core changes. --- ## 7. with_txn decision Source of friction: `s14_learn_with_txn_async_friction.toml`. Sync trait `fn with_txn(&mut self, f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>) -> StoreResult<()>` does not compose with async Postgres drivers (sqlx, tokio-postgres) because closures cannot be async without boxing a Future and dealing with lifetime/pinning. Three options enumerated in the tag: - **(a) `block_on` in adapter.** Keep the sync trait. The Postgres adapter wraps a `tokio` runtime internally and `block_on`s every call. Pros: zero changes to trusty-sqlite, zero changes to the trait. Cons: the adapter's caller cannot be in an async context without nested runtime issues; or we use the `synchronous-postgres` driver (libpq directly via `postgres` crate) and skip async entirely. - **(b) Boxed async closure.** Add an async variant of with_txn to the trait: `async fn with_txn_async(&mut self, f: BoxFuture<...>) -> StoreResult<()>`. Pros: native async. Cons: trait object methods cannot be generic AND cannot be async-fn-in-trait without macro support; ergonomics are awful for callers. - **(c) Split into begin / commit / rollback.** Drop the closure pattern entirely. Trait gains `begin_txn`, `commit_txn`, `rollback_txn`. Pros: composes with both sync and async. Cons: callers lose RAII safety (forget to commit and the txn leaks); trusty-sqlite must be updated too. ### Recommendation **Option (a) -- block_on with the synchronous `postgres` crate (not sqlx/tokio-postgres) -- for v1 of trusty-postgres.** Reasoning: 1. The synchronous `postgres` crate (built on `tokio-postgres` but exposing a sync API) is mature, widely used, and avoids the nested-runtime problem entirely. No `block_on` needed -- the driver is sync from the user's perspective. 2. The trait stays unchanged. trusty-sqlite stays unchanged. Other adapters (DuckDB, ScyllaDB) can make their own sync-vs-async choice without us locking the trait shape. 3. Async-Postgres consumers who want native `tokio` integration can wait for a v2 trait or implement Option (c) themselves; this is not a v1 requirement for the IoT/compliance buyer pitch. **However**, document the trait friction explicitly in trusty-postgres README so the next person who tries to wire it into an async axum/tonic server is not surprised. Add a worked example of using `tokio::task::spawn_blocking` to bridge. **Defer Option (c) to a future trait-v2 pass.** When we get serious about ScyllaDB or another genuinely-async backend, revisit. At that point all adapters get updated together. Eric to decide whether to formally schedule that pass; not blocking trusty-postgres v1. OPEN QUESTION: confirm the `postgres` (sync wrapper) crate is the right choice over `rust-postgres` direct or `diesel`. Recommendation: `postgres` crate; it is the de facto standard sync Postgres client for Rust and has the same maintainer as `tokio-postgres`. --- ## 8. OID + chain_id binding (apply `s14_insight_oid_anchored_chain_identity`) The OID-anchored model is the entire commercial reason this crate exists. Get this right at v1. ### The binding Per the insight tag, the model is: - An OID (e.g. `1.3.6.1.4.1..`) identifies a producer (device, sensor, agent). - That OID is the canonical chain identity. Either chain_id IS the OID, or chain_id is derivable from the OID + a generation counter. - An Ed25519 pubkey is bound to the OID via a known_producers TOFU registry (per the SSH-paradigm tag). The trusty-sqlite chain_id format today is `chain__<6-digit-gen>` (see `make_chain_id`, used at `store.rs:299` and parsed at `store.rs:965-975`). Eric: this format is fine for SQLite where entity_id is opaque, but for OID-anchored Postgres deployments we want OID legibility. ### Schema Add a `producer_identity` table, separate from the three locked tables: ``` producer_identity ( producer_oid TEXT PRIMARY KEY, ed25519_pubkey BYTEA NOT NULL, siphash_key BYTEA NOT NULL, -- derived per SSH-paradigm tag registered_at TIMESTAMPTZ NOT NULL, display_name TEXT, metadata JSONB -- ok to use JSONB here: -- not part of chain bytes ) ``` And add a `producer_oid TEXT` column to `chain_tips`, foreign-keyed to `producer_identity.producer_oid` when present (nullable for backward compat with non-OID deployments). Decision: do NOT change the chain_id format string. The format `chain__<6-digit-gen>` is locked by trusty-core's `make_chain_id` (at `crates/core/trusty-core/src/chain.rs:100`). Note: `parse_generation` currently lives in the SQLite adapter at `crates/adapters/trusty-sqlite/src/store.rs:965`; it should be promoted to trusty-core as a pre-requisite of this work so trusty-postgres does not copy-paste. Additive change; no v1 byte-shape impact. Instead: - For OID deployments, set `entity_id` to the OID string. `make_chain_id` will produce `chain_1.3.6.1.4.1.._000001`, which is human-readable and OID-legible. - The `producer_oid` column on `chain_tips` denormalizes the OID for indexing/joins to `producer_identity`. ### Reconciliation with Gitea #14 Issue #14 proposes `chain_id = H(domain_sep || pubkey || nonce)`. Taken literally this contradicts the decision above. Reading the full issue body resolves the conflict: #14 is explicitly additive -- "Existing v1 chains without attribution remain verifiable." The two models coexist: - **chain_id string** stays as `make_chain_id(entity_id, gen)` for human legibility, log grep-ability, and OID integration. - **Cryptographic binding** moves into (a) the v2/v3 genesis-row signature (the Ed25519 signature over the signing-view commits to chain_id via the row_hash), plus (b) the `producer_identity(producer_oid, ed25519_pubkey, ...)` row that proves which key is allowed to sign for this OID. A rival chain with the same chain_id string cannot produce a valid genesis signature without the private key, so the "malicious party presents a rival chain" threat in #14 is defended against by the signature chain, not by the id format. #14 becomes "require genesis sig + producer_identity lookup before trusting any append," not "change the chain_id derivation." No trusty-core change to `make_chain_id`. The `producer_identity` table is the local cache equivalent of an SSH `known_hosts` file (per the SSH-paradigm tag) -- not a CA, just a TOFU registry. Whether OIDs cross-reference to X.509 cert chains is application-layer and out of scope here. OPEN QUESTION: should the `producer_identity` table be optional (feature-gated) or always created? Recommendation: always created, but rows are optional. Adapters that do not use OID-anchoring simply leave it empty. Cost is one empty table; benefit is one schema for all deployments. OPEN QUESTION: OID validation -- enforce the dotted-numeric format at the column level (CHECK constraint) or accept arbitrary strings? Recommendation: accept arbitrary strings. trusty-postgres should not become an OID validator; that is the producer's responsibility. We can add a CHECK later if a misuse pattern emerges. --- ## 9. Concurrency model ### The core difference from SQLite trusty-sqlite is single-writer by SQLite's nature (WAL mode plus `BEGIN IMMEDIATE`, see `store.rs:535`). The chain primitive's correctness depends on one fundamental invariant: between reading the tip and writing the next row, no other writer mutates the tip. SQLite gives us this trivially. Postgres allows multi-writer. Two clients calling `append()` for the same `entity_id` concurrently can both: 1. Read the same tip. 2. Compute next sequence = tip.sequence + 1. 3. Insert two rows with the same sequence and the same prev_hash. Result: chain integrity violation. Both rows verify in isolation (both have correct prev_hash) but the chain has forked. Verification will eventually catch it (sequence collision in iter_chain) but silent damage is done. ### Decision: per-entity row-level lock at append time Use Postgres advisory locks keyed on entity_id: ``` SELECT pg_advisory_xact_lock(hashtext($1)); -- $1 = entity_id ``` Inside the transaction, take an advisory lock derived from `entity_id` (via `hashtext` or a dedicated 64-bit hash). The lock is held for the duration of the transaction and released on commit or rollback. Two appenders for the same entity serialize; appenders for different entities run concurrently. Rationale: - Advisory locks are cheap (no row materialization), session-scoped or transaction-scoped, and Postgres-native. - They serialize exactly the operations that need serialization: the read-tip / compute / write-row / update-tip cycle for a single entity. - Different entities are independent; no global lock, so multi-entity fleets scale. Alternative considered: `SELECT ... FOR UPDATE` on `chain_tips` rows. This works but materializes a row lock on every append even when no contention exists. Advisory locks are more direct. Alternative considered: optimistic concurrency with a retry loop on sequence-collision unique-violation. Possible but complicates the adapter (retry loops, error mapping). Defer. ### Test invariants Add a concurrent-append test: spawn N threads each appending K rows to the same entity_id. Assert: total rows = N*K, sequences are 0..N*K-1 with no gaps, `verify` passes, `kv_cross_check` passes. OPEN QUESTION: should the advisory lock be transaction-scoped (`pg_advisory_xact_lock`) or session-scoped (`pg_advisory_lock` + explicit unlock)? Recommendation: transaction-scoped. Releases automatically on commit/rollback; no leak risk. --- ## 10. Migration story: trusty-sqlite -> trusty-postgres The product use case: a customer runs trusty-sqlite at the edge for a year, then promotes their archive to trusty-postgres for long-term audit. This must work without recomputing chain hashes. ### Byte-shape preservation The locked v1 byte-shape (see CLAUDE.md "Don't Touch") guarantees that `raw_payload` strings are byte-identical regardless of backend. Migration is a row-by-row copy with no recomputation: 1. SELECT all rows from sqlite `provenance_envelope` ordered by `id`. 2. INSERT into postgres `provenance_envelope` preserving every column verbatim. The `id` column re-issues from the postgres identity sequence; that is fine because chain semantics depend on sequence_int + prev_hash + chain_id, not on the surrogate id. 3. SELECT all rows from sqlite `hash_index`, INSERT into postgres `hash_index` verbatim. 4. SELECT all rows from sqlite `chain_tips`, INSERT into postgres `chain_tips` verbatim. Set `tip_state = 'WARM'` for all migrated rows; the migration tool cannot reconstruct cold/unknown state. 5. Run `verify` for every entity_id on the postgres side; assert reports match the sqlite side row-for-row. 6. Run `kv_cross_check` on the postgres side; assert OK. ### Anchor hashes Anchor hashes are NOT recomputed -- they are byte-identical between backends because they are derived from the locked v1 primitives. Copy them verbatim. If the post-migration verify pass fails on anchor mismatch, the bug is in the migration tool, not in the chain. ### Implementation Ship a `trusty-migrate` binary in the trusty-postgres crate: ``` trusty-migrate \ --from sqlite://path/to/file.db \ --to postgres://user:pass@host/db \ --verify ``` `--verify` runs the full verify pass on both sides post-migration and asserts byte-identity. Exit non-zero on any mismatch. OPEN QUESTION: do we want bidirectional migration (postgres -> sqlite for offline audit)? Recommendation: yes, but defer to a v2 of the migration tool. v1 ships sqlite -> postgres only. The inverse direction has the same byte-shape guarantee so it is mechanical to add later. --- ## 11. Testing strategy The bar to merge trusty-postgres v1: ### Required test suites 1. **All trusty-store testing-feature tests pass.** The Store trait ships a conformance suite under `crates/core/trusty-store/src/testing.rs` (per the existing `#[cfg(feature = "testing")]` module, see `crates/core/trusty-store/src/lib.rs:15-16`). The v1 conformance suite is 6 tests, enumerated explicitly: - `insert_and_read_back` -- insert a row; `last_row` returns it. - `iter_chain_ordered` -- insert 3 rows; iterator yields them in sequence order with no gaps. - `tip_roundtrip` -- `set_tip` + `get_tip_for_entity` returns same row; `get_tip(chain_id)` agrees. - `kv_cross_check_clean` -- fresh insert produces OK report. - `with_txn_commit` -- inserts inside `with_txn` closure persist. - `with_txn_rollback` -- `Err` return from closure rolls back. trusty-postgres must pass all 6 identically to trusty-sqlite and trusty-memory. 2. **v1 byte-shape conformance.** Add a test that appends N rows via trusty-postgres, exports a `ProofBundle`, and asserts every row_hash and anchor_hash matches the same sequence appended via trusty-sqlite with the same SecretKey, same payload, same timestamps. This is the runtime version of `s5_invariant_v1_byte_shape_preserved` for the Postgres adapter. 3. **Multi-writer concurrency.** Per section 9: N threads, K rows each, single entity. Assert no sequence gaps, no duplicate sequences, verify passes. 4. **Cold-start tri-state.** Provision a producer_identity row with no chain_tips row; assert `get_tip_for_entity` returns `tip_state = UNKNOWN`. After first append, assert `tip_state = WARM`. 5. **Revocation flow.** Append signed rows, write a REVOCATION envelope, attempt append signed by revoked key, assert chain-break. 6. **Compaction round-trip.** Append N rows, run `compact()` at high_water = N/2, assert verifier can still verify the full chain (using the snapshot envelope to skip archived rows). 7. **Migration tool round-trip.** Append M rows to sqlite, migrate to postgres, run verify on both; assert reports identical. 8. **TimescaleDB feature gate.** With `--features timescale`, the migration runs `create_hypertable` and inserts/queries succeed. Without the feature, the same tests pass on stock Postgres. ### Test infrastructure Use `testcontainers-rs` or equivalent to spin a Postgres container per test run. Do not depend on a developer-local Postgres install; CI must be hermetic. OPEN QUESTION: should the conformance suite in `crates/core/trusty-store/src/testing.rs` be expanded with the tri-state, concurrency, and OID tests so all backends run them? Recommendation: yes, but in a follow-up trusty-store PR. Initially add them inline in trusty-postgres; promote to the conformance suite once stable and once trusty-sqlite is updated to support tri-state TipState (default WARM). --- ## 11b. Cross-cutting policy decisions (added post-research pass) Decisions on four policies that applied across multiple sections and were surfaced in the S14-addendum research pass. These are the memo's current position; any of them can be re-opened but all four are in scope as implementation-ready defaults. ### Payload version accept policy trusty-postgres accepts reads of v1, v2, and v3 payloads (dispatch already lives in trusty-core via `row_hash_for_version` and `extract_payload_version`, confirmed at `crates/core/trusty-core/src/chain.rs:52,211`). For new appends the adapter writes v3 only. This guarantees: - The `trusty-migrate sqlite -> postgres` tool copies v1/v2 rows byte-verbatim; no recomputation, no upgrade of existing bytes. - All new rows benefit from domain separation. - `compute_record_signed` is v2-only today (handoff acknowledged gap). trusty-postgres signed-append uses v2 bytes until `compute_record_signed_multi` or a v3-signed path lands in trusty-core -- tracked as a pre-req of #16 (patch-policy multi-sig). ### Schema versioning discipline Schema migrations are planned, not forbidden. A `schema_meta` table ships in v1 carrying `(key TEXT PRIMARY KEY, value TEXT)` with a single row `('schema_version', '1')`. Any future schema change ships with: - A new version number written to `schema_meta`. - An idempotent forward migration tested end-to-end. - A design memo at `docs/trusty-postgres-schema-vN.md` documenting the change. Same rigor as payload v1/v2/v3 work. Do not pretend the schema is locked; pretending is how drift happens. ### External anchor publish App-level, not adapter-level. trusty-postgres exposes two read APIs that downstream crates use to build publish pipelines: - `stream_anchors_since(chain_id, sequence) -> impl Iterator` - `export_bundle_since(chain_id, sequence) -> ProofBundle` No S3, Git, CT-log, or DS-transport dependency in the adapter. Retention/transport pipeline (#5) is a separate crate (likely `trusty-archive` or the DS integration crate) that consumes these iterators. Keeps trusty-postgres's TCB and dependency graph thin. ### Observability `tracing` spans only. Every public method on the adapter emits a span with `chain_id`, `entity_id`, and operation name. No prometheus/opentelemetry crate dependency -- consumers plug in any `tracing_subscriber` exporter they want (`tracing-opentelemetry`, `tracing-prometheus-exporter`, etc.) at their layer. Zero-cost when no subscriber is attached. Span namespacing: `trusty_postgres::append`, `trusty_postgres::verify`, `trusty_postgres::with_txn`, `trusty_postgres::checkpoint`. Include advisory-lock wait time as a span event so lock contention is visible to the subscriber. --- ## 11c. Multi-tenant federated deployment (real target) The original memo framed trusty-postgres as a generic mid-scale-cloud Store adapter. The actual product goal is tighter: a **federated shared knowledge base** spanning up to ~50 users, each with many agents writing concurrently, glued between user nodes by Distant Star, with the existing kb-mesh content (pins, synthesis artifacts, architectural decisions) materialized as trusty chains. This section captures the deployment shape implied by that goal; it supersedes no prior decision but adds concrete structure where the original memo was generic. ### Deployment shape - **Per-user Postgres schema.** Each user gets a Postgres schema (namespace) named for their OID arc leaf, e.g. `usr_eric`, `usr_partner`, `usr_agent03`. Inside that schema the three-table layout (`provenance_envelope`, `hash_index`, `chain_tips`) is replicated. Schema-per-user gives hard isolation (an agent cannot accidentally write to another user's rows), per-schema index locality, and straightforward per-user backup/archive. - **Cross-user schema in `public`.** One schema-wide table set: `public.producer_identity` (OID -> Ed25519 pubkey + SipHash key + metadata, the known_producers registry), `public.schema_meta` (global schema version), `public.received_bundles` (inbound DS-delivered bundles awaiting or completed verification). - **Cross-user read view.** `public.all_provenance_envelope AS SELECT ... FROM usr_eric.provenance_envelope UNION ALL ... FROM usr_partner.provenance_envelope ...` regenerated whenever a user is provisioned or deprovisioned. Read-side queries spanning all users (e.g. "every pin tagged trusty across the mesh") go through this view; write-side never touches it. - **One chain per (user, agent, topic) tuple.** `entity_id` encodes `//`. `make_chain_id` on top of that produces `chain_//_000001`. Topic is caller-chosen (session id, KB pin topic, sensor stream name, etc.). ### Ownership and authorization - Every agent has its own Ed25519 keypair and an OID leaf under its user's arc. `producer_identity` rows bind (OID -> pubkey). - Append is rejected at the adapter if the signing key does not match the `producer_identity` row for the target chain's OID. This turns the two-party "one writer per chain" rule into a general N-party invariant enforced at write time. - Agents provisioned/deprovisioned by a user-level operator flow (out of scope for this crate; in scope for a future `trusty-admin` tool). ### Concurrency at scale - Advisory lock per `entity_id` inside each user's schema. `hashtext(entity_id)` already makes different entities contention- free; per-entity serialization matters only when two of one user's agents race the same chain. - Connection pool (pgbouncer or equivalent) required in any deployment exceeding ~100 concurrent agents. Memo-level recommendation: transaction-pooling mode with a pool of `max_connections / N_users` per user's app tier. - Postgres's own MVCC handles the 50-user + multi-agent load comfortably. The bottleneck is per-entity advisory locks, not the database. Measured scaling will come from real deployment; design-time assumption is that the chain-break rate stays O(errors) not O(normal appends). ### Federation via Distant Star - Each user runs their own Postgres + DS node. Writes land locally under the user's schema. - DS gossips ProofBundles peer-to-peer. Advertise-don't-copy: a DS advertisement carries `(chain_id, tip_hash, size_hint)`; bundles are pulled on demand. - Inbound bundles land in `public.received_bundles` with: `bundle_id`, `source_producer_oid`, `chain_id`, `tip_hash`, `received_at`, `verify_status`, `verify_detail`, `bundle_blob` (bytea). Verification runs against `producer_identity` (did the claimed OID sign this?); failed bundles stay with `verify_status = 'REJECTED'` and a reason -- not silently dropped, not replayed. - A user's view of "what partner has" is a query against `received_bundles WHERE source_producer_oid IN (partner's OIDs)`. Historical verification remains local: even if the partner goes offline or leaves the federation, bundles already ingested are still cryptographically verifiable. ### kb-mesh migration The existing kb-mesh primitive (ephemeral pins + promoted patterns) maps cleanly: - Each pin becomes a chain row with `event_type = 'PIN'` and payload data carrying `topic`, `turns`, `source_session`, `synthesis_level`, pin body. - `action = 'create' | 'update' | 'overwrite' | 'close' | 'promoted'` from INDEX.jsonl becomes `extraction_method` or a structured field inside `data`. - Per-repo kb-mesh pins roll into a per-user chain named something like `usr_/agent_/kb-mesh-pins`. INDEX.jsonl becomes redundant (`SELECT ... FROM ...provenance_envelope WHERE event_type = 'PIN'` replaces it). - Promoted patterns (ephemeral pin -> durable `.kb/patterns/` file) become CHECKPOINT envelopes over the pin chain + a supplementary write to a pattern chain. A dedicated `trusty-kb-mesh` (or `trusty-kb`) crate owns the mapping; trusty-postgres is the storage. Gitea issue to follow. ### Changes to earlier sections this implies - §1 add non-goal: trusty-postgres is not a tenant manager; user provisioning and schema creation live in a future `trusty-admin` crate or operator script. - §4 (tri-state null): `tip_state` is per-user; cold-start state is per-agent per-chain, not global. - §8 (OID + chain_id): chain_id convention tightened -- `//` inside the template. `entity_id` encodes the same. Unchanged at the byte level; just a convention on the strings. - §9 (concurrency): advisory lock is per-entity within a schema, not global. Hash collision risk on `hashtext(entity_id)` across schemas is irrelevant because different schemas are different critical sections anyway. - §10 (migration): `trusty-migrate sqlite -> postgres` must now specify a target schema (or default to `public` for single-user deployments). ### Open questions specific to the multi-tenant shape - How do we test 50-user + many-agent concurrency before deployment? Probably a load-gen harness that spins N schemas + K async appenders per schema. Out of scope for trusty-postgres v1; in scope for a `trusty-load` test binary later. - How do schema migrations roll out across N user schemas without blocking active writers? Partial answer: ALTER runs per-schema in sequence, advisory-locked against that schema's writers. Full answer needs a real migration tool. - Bundle ingestion rate limits per source producer? Probably yes, to prevent a rogue peer from flooding; app-level concern, not adapter. --- ## 12. Open questions and explicit deferrals Items needing Eric's input before implementation. Items marked RESOLVED were decided in the post-research pass; strike-through anything that later changes. 1. **RESOLVED: TipState additive trait change (S16).** Approved as v1-compatible additive change with default WARM. Precedent: v2 signing (`project_trusty_v2_signing_design`) added fields to `PayloadFields` and a new top-level `compute_record_signed` without breaking v1 callers; the same additive-default pattern applies to trait fields. `TipState` is store metadata, not chain-payload bytes, so v1 byte-shape is unaffected by construction. 2. **RESOLVED: Compaction primitive scope** (section 5). Decision: ship `CHECKPOINT` (verification-layer only) for v1; archival and row-aggregation compaction are v2. No trusty-core change. Retired the word "compaction" from this crate's vocabulary. 3. **RESOLVED: Revocation Condition placement (S16).** trusty-core. Uniform across backends prevents two adapters from diverging on what counts as revoked, which would silently fork chain verification semantics. The cost (a small hash-set lookup in the core verify path) is negligible vs. the consistency win. 4. **RESOLVED: Sync vs async driver (S16).** Sync `postgres` crate for v1 confirmed. Trait stays unchanged; `with_txn` is already validated. Defer Option (c) trait-v2 split until ScyllaDB or similar makes async unavoidable. 5. **RESOLVED: OID validation (S16).** Enforce dotted-numeric format at column level via `CHECK` constraint (`chain_id ~ '^[0-9]+(\\.[0-9]+)*$'` or equivalent on the producer_oid column). OID format is the trust spine of federation; arbitrary strings invite ad-hoc IDs that defeat the cryptographic identity model. Fail fast at the boundary. 6. **RESOLVED: TimescaleDB column type (S16).** Use `TIMESTAMPTZ` internally; trait surface stays RFC3339 strings. Indexing on `TEXT` timestamps is bad; conversion is transparent at the adapter boundary; conformance test enforces the RFC3339 round-trip. No byte-shape impact (timestamps live in payload data, not envelope columns subject to v1 lock). 7. **RESOLVED: Bidirectional migration (S16).** v1 ships sqlite -> postgres only. postgres -> sqlite is deferred to v2 (or never). Federated multi-tenant is the production target; nobody migrates back from federation to a single-file edge backend. 8. **RESOLVED: Conformance suite expansion (S16).** Promote new tests into `trusty-store::testing` AFTER the first stable trusty-postgres release. Keeps churn local during development; promotion gated on proven stability. 9. **RESOLVED: Repo location (S16).** Separate Gitea repo `mm-trusty/trusty-postgres` (already pushed via subtree split, commit 427c7a7), depending on `trusty-core` and `trusty-store` via path or git dep. Matches the workspace shape already in flight; keeps trusty-core/sqlite TCB minimal; users pick path/git/crates.io per deployment. 10. **RESOLVED: TCB manifest (S16).** trusty-postgres README page-one section. Matches `learn-LCG-ARCH-13` recommendation; one place, explicit, no separate doc to drift. 11. **RESOLVED: chain_id derivation** (section 8). Decision: keep `make_chain_id` string template; move cryptographic binding into genesis-row signature + `producer_identity` table. #14 becomes "require genesis sig + identity lookup," additive only, no trusty-core change to `make_chain_id`. 12. **RESOLVED: payload version accept policy** (section 11b). Decision: accept v1/v2/v3 reads; new appends are v3. 13. **RESOLVED: schema versioning discipline** (section 11b). Decision: plan migrations; ship `schema_meta` table in v1; same rigor as payload-version work. 14. **RESOLVED: external anchor publish** (section 11b). Decision: app-level. Adapter exposes `stream_anchors_since` and `export_bundle_since` read APIs; publishing lives in a downstream crate. 15. **RESOLVED: observability** (section 11b). Decision: `tracing` spans only, no prometheus/otel dependency. Consumers plug in any subscriber layer. 16. **RESOLVED: `parse_generation` location (S16).** Promote to `trusty-core` as part of the Tier 1 refactor (`s14_insight_general_library_improvements`). Pure string parser, no byte-shape involvement. Every future backend would re-copy-paste otherwise. 17. **RESOLVED: `compute_record_signed` v3 gap (S16).** `compute_record_signed` now version-dispatches by `fields.payload_version`: v1/v2 callers stay byte-identical (v1 upgrades to v2 per S5 behavior), v3 callers get the domain-tagged signed path via `row_hash_for_version` / `anchor_hash_for_version`. Two new tests: `v3_signed_record_verifies` and `v3_tampered_signature_fails_verify`. 50 trusty-core tests pass (was 48). HARD BLOCKER lifted; §13 P1/P4/P7 unblocked. `compute_record_signed_multi` (tied to #16) remains a separate follow-on for N-signer multi-sig. 18. **First-class edges (Graphiti P6, deferred from §13).** Cross- user / cross-agent provenance today rides on the chain_id string template (`//`). Graphiti models edges as their own chained relation with endpoint refs + source-episode pointer, which would make federation traces queryable. Conflicts mildly with the §8 chain-id-as-template legibility decision and adds a parallel structure. Needs its own scoping memo before any code; not adopted in §13. --- ## 13. Patterns lifted from Cozo + Graphiti Cross-reference of design patterns from CozoDB (embedded Datalog graph DB; pure Rust; HNSW + per-relation time-travel) and Graphiti (Zep's temporal context graph; episode-based provenance) against the trusty-postgres KB. Each pattern is mapped to its existing anchor in the design memo or APERC tag substrate. **All eight patterns are additive to v1 byte-shape -- no change to `compute_record`, `build_payload`, or the three locked tables' chained columns.** ### 13.1 Substrate **P1 -- Stored-relation registry (Cozo).** A `chained_relations` meta table where each app-level relation opts in to chaining via a row + trigger calling `trusty_append`. Existing anchor: `s15_insight_kb_surface_taxonomy_for_trusty_kb_adapter` mandates "four clean mapping functions" (policy / decisions / tags / pins) and "one entry point per concern." A registry formalizes that mandate -- the four kb-mesh surfaces become registry rows instead of bespoke crate-level glue. Slots into §3 schema and §11c kb-mesh chain mapping. **P5 -- Episode as canonical append unit (Graphiti).** `provenance_envelope` already is the episode; `event_type` already names the kind (PIN / DECISION / POLICY_ROW per s15_insight). Vocabulary alignment only -- glossary edit in §1 and docstring edits referring to rows as "episodes." No schema delta. **P6 -- First-class edges (Graphiti). DEFERRED.** See §12 OQ #18. Heavy add; conflicts with chain-id-as-string-template in §8; needs its own scoping memo. ### 13.2 Temporality **P3 -- Per-relation time-travel (Cozo bitemporal).** Today trusty has system-time only (chain_timestamp); supersession is a payload-level concept (`supersedes: [prior_row_hash]` per s15_insight). Add a companion table: ```sql CREATE TABLE bitemporal_index ( chain_id TEXT NOT NULL, row_hash TEXT NOT NULL, valid_range tstzrange NOT NULL, PRIMARY KEY (chain_id, row_hash), FOREIGN KEY (chain_id, row_hash) REFERENCES provenance_envelope (chain_id, row_hash) ); CREATE INDEX bitemporal_index_range ON bitemporal_index USING GIST (chain_id, valid_range); ``` **Critical invariant: `provenance_envelope` columns and the `build_payload` byte-shape are NEVER modified by P3.** The bitemporal index is a sidecar projection keyed by (chain_id, row_hash). Conformance suite vs. trusty-sqlite stays green. **P8 -- Bi-temporal supersession projection (Graphiti).** Ships with P3. A Postgres view `current_facts` materializes "facts whose `valid_range` contains `now()`," excluding superseded rows. The supersession event itself remains a chained envelope per s15_insight (never mutate prior rows); P8 just makes the projection queryable. ### 13.3 Retrieval (unblocks the trusty-monitor-daemon pin) **P4 -- HNSW vector indices on chained relations (Cozo).** Adopt pgvector. Per chained relation that opts in (via the registry from P1), an `embedding vector(N)` column on a sidecar table keyed by `(chain_id, row_hash)` plus an HNSW index. Existing anchor: pin `../.kb-mesh/pins/trusty/trusty-monitor-daemon-chat-capture.md` asks for "captures chat as unstructured data, auto-tags" -- auto-tag-over-chain is exactly pgvector + HNSW. Currently no path; P4 is the path. **P7 -- Hybrid retrieval contract (Graphiti).** One query API combining vector similarity + tsvector keyword + recursive graph traversal, with chain-verify result as a returned column on every row. Existing anchor: `s15_insight` specifies pre-write federation search as `kb_search(keywords) -> Vec` across the user's schema + `received_bundles`. P7 extends the signature to `kb_search(keywords?, embedding?, graph_filter?) -> Vec`. Single API surface for both the kb-mesh adapter (s15_insight) and the trusty-monitor-daemon pin. ### 13.4 Query API **P2 -- Datalog-shaped chain queries via recursive CTEs.** Existing anchor: `s14_insight_general_library_improvements_from_postgres_impl` item (1) -- promote `history`, `breaks_since`, `liveness_check`, `integrity` to `trusty-store` as a `TrustyExt` trait. P2 reframes that refactor: the trusty-store side ships a named-relation query library; each backend provides the templates (recursive CTEs on Postgres, WITH RECURSIVE on SQLite, Datalog rules on a future Cozo adapter). Same surface, backend-specific implementation. ### 13.5 Adoption summary | Pattern | KB anchor | Verdict | |---------|-----------------------------------------------------------------|--------------------------------------| | P1 | `s15_insight_kb_surface_taxonomy` | Adopt -- completes four-surface mandate | | P2 | `s14_insight_general_library_improvements_from_postgres_impl` | Adopt -- reframes promote-to-store refactor | | P3 | `s15_insight_kb_surface_taxonomy` (supersession), §11c | Adopt -- companion table only | | P4 | pin `trusty-monitor-daemon-chat-capture`, §11c federation | Adopt -- direct unblocker | | P5 | `provenance_envelope` already exists | Adopt -- vocabulary only | | P6 | §8 chain-id-as-template, §11c federation | Defer -- §12 OQ #18 | | P7 | `s15_insight_kb_surface_taxonomy` (kb_search spec), pin | Adopt -- direct extension | | P8 | `s15_insight_kb_surface_taxonomy` (supersession decision) | Adopt -- bundles with P3 | ### 13.6 Upstream blockers and locked-rule statement - **v3 signed-path gap (§12 OQ #17, escalated to HARD BLOCKER in `s15_decision_multitenant_shared_kb_shape`)** is upstream of P1 / P4 / P7. Federated hybrid retrieval over `received_bundles` is meaningless without verifiable v3 signed rows. Resolve OQ #17 before implementing §13. - **Locked-rule check.** P1 adds a meta table. P2 ships pure query library code. P3 adds a sidecar table keyed by (chain_id, row_hash); never modifies `provenance_envelope`. P4 adds a sidecar table per opted-in relation; never modifies `provenance_envelope`. P5 is documentation. P7 is API composition over P1 / P3 / P4. P8 is a view. **None touches `compute_record`, `build_payload`, or the three core locked columns.** Conformance suite vs. trusty-sqlite remains the gate. --- ## Appendix A. Lessons applied -- index | Lesson | Section | Where in design | |---------------------------------|--------------------|-------------------------------------------------------------------------------| | `insight-LCG-ARCH-01` meta | every section | "build it in v1 because retrofit is 5-10x more expensive" | | `learn-LCG-ARCH-02` compaction | 5 | snapshot tables in v1; SNAPSHOT envelope; chain_compaction_state | | `learn-LCG-ARCH-03` tri-state | 4 | `tip_state` column; `TipState` enum on the trait; cold-start representation | | `learn-LCG-ARCH-06` revocation | 6 | `producer_revocation` table; REVOCATION envelope; runtime invariant test | | `learn-LCG-ARCH-13` TCB | 1, 12 | TCB statement in README page one; postgres + libpq + driver named explicitly | | Trusty insight | Section | Where | |--------------------------------------------------------|---------|------------------------------------------------------------------| | `s14_insight_oid_anchored_chain_identity` | 8 | `producer_identity` table; OID-as-entity_id pattern | | `s14_insight_store_trait_multi_backend_landscape` | 2 | scope of higher-level Trusty methods to re-implement | | `s14_learn_with_txn_async_friction` | 7 | Option (a) sync `postgres` crate; trait unchanged | | `s5_invariant_v1_byte_shape_preserved` | 3, 11 | TEXT not JSONB for raw_payload; conformance test vs sqlite | | `project_trusty_v2_signing_design` | 6, 4 | additive-only precedent for trait additions | --- ## Appendix B. Files referenced - `crates/core/trusty-store/src/lib.rs:22-48` -- Store trait definition. - `crates/core/trusty-store/src/lib.rs:54-115` -- LastRow, TipRow, StoredRow, RowInsert, TipUpsert, KvCrossCheckReport. - `crates/adapters/trusty-sqlite/src/store.rs:1-955` -- reference adapter implementation including all higher-level Trusty methods. - `crates/adapters/trusty-sqlite/src/schema.rs:1-49` -- reference schema (three tables, locked). - `APERC/tags/s14_insight_oid_anchored_chain_identity.toml` -- OID-anchored model. - `APERC/tags/s14_insight_store_trait_multi_backend_landscape.toml` -- four-backend landscape; method scope. - `APERC/tags/s14_learn_with_txn_async_friction.toml` -- with_txn options. - `../legal-chaingraph/APERC/KB/architecture_antipatterns.md` -- the 18-lesson source. - `../legal-chaingraph/APERC/tags/learn-LCG-ARCH-02.toml` -- compaction. - `../legal-chaingraph/APERC/tags/learn-LCG-ARCH-03.toml` -- tri-state null. - `../legal-chaingraph/APERC/tags/learn-LCG-ARCH-06.toml` -- revocation. - `../legal-chaingraph/APERC/tags/insight-LCG-ARCH-01.toml` -- meta-principle. - `CLAUDE.md` -- project rules; locked surfaces; library-not-platform discipline. - `docs/payload-v2.md` -- v2 payload signing design. - `docs/threat-model.md` -- existing trusty threat model (cross-reference for TCB). End of memo.