eric/Trusty
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
Trusty/docs/trusty-postgres-design.md

59 KiB
Raw Permalink Blame History

trusty-postgres -- Design Memo

Status: DRAFT. Pre-implementation. Author: design pass over the LCG architecture-antipatterns lessons, the Store trait surface, and the trusty-sqlite reference implementation. Reader: Eric.

This memo defines the contract, schema, and architectural decisions for trusty-postgres BEFORE any code is written. The point of writing it now is the meta-principle in insight-LCG-ARCH-01: features that "we will add in v2" become 5-10x more expensive to retrofit than to build in from day one. Every section below names the lesson it draws from so the design's lineage is auditable.

The reference implementation to mirror is crates/adapters/trusty-sqlite/src/store.rs:1-955 and crates/adapters/trusty-sqlite/src/schema.rs:1-49. The trait being implemented lives at crates/core/trusty-store/src/lib.rs:22-48.

1. Goal and non-goals

Goal

trusty-postgres is a sibling adapter crate (likely separate Gitea repo Modo-Melius/trusty-postgres) that implements trusty_store::Store and the higher-level Trusty operations against PostgreSQL 14+ with optional TimescaleDB extension. It targets the deployment tier named in s14_insight_store_trait_multi_backend_landscape.toml: "mid-scale cloud, time-series query acceleration on received_at via hypertable."

The product framing comes from s14_insight_oid_anchored_chain_identity.toml: trusty-postgres is the "OID-anchored TrustyDB" surface for enterprise buyers who already run Postgres + PKI infrastructure. The pitch is "your existing Postgres, plus cryptographic proof of stream completeness/authenticity," not "a new database." This framing forecloses any temptation to bundle application logic, distributed coordination, or marketplace primitives into the crate. It is a Store adapter and the higher-level Trusty methods, nothing more.

Non-goals

  • Not a replacement for trusty-sqlite. SQLite remains the embedded / edge / single-process reference. Postgres targets multi-writer, fleet-scale, or organizationally-required-Postgres deployments.
  • Not a distributed-coordination layer. Multi-region, multi-master, or consensus-flavored deployments belong in a hypothetical trusty-scylladb or above. Postgres logical replication is fine if the operator wants it; we do not invent any cross-cluster chain reconciliation primitive here.
  • Not an analytics surface. DuckDB / columnar read paths are a separate adapter per the four-tier landscape tag.
  • Not a migration of the locked v1 byte-shape. The point of the Store trait is that trusty-core primitives are untouched. Any change that would shift v1 bytes is out of scope by definition; see s5_invariant_v1_byte_shape_preserved and CLAUDE.md "Don't Touch."
  • Not a TCB-redefining crate. The trusted computing base added by this adapter is: postgres + libpq + the chosen Rust driver. Document this on page one of the README (per learn-LCG-ARCH-13 / TCB manifest discipline) and let it sit there as a deliberate cost.

Relationship to Distant Star (DS)

trusty-postgres is orthogonal to the DS network protocol (dspm, dsnl, dsutils, ds-cli -- the Monorepo-scratch stack). DS is a UDP gossip transport with X-Wing KEM, exponential-decay hop probability, and tag-based knowledge-network routing; it transports data, it does not store it. trusty-postgres stores data behind the Store trait. The two compose, they do not overlap.

Concrete composition points (relevant so this crate's API does not accidentally foreclose them):

  • trusty-ds-fs (#10) and trusty-ds-network (#12) may use trusty-postgres as their persistence layer when a DS node is deployed in the mid-scale cloud tier rather than at the edge. The Postgres adapter must therefore expose a stream_anchors_since and a ProofBundle export surface that DS handlers can call without coupling to any internal detail.
  • Advertise-don't-copy pattern from the DS README. DS nodes advertise file existence (tag + hash) and serve the file only on demand. The same pattern applies to chain proofs: a DS-integrated trusty-postgres node should advertise (chain_id, tip_hash) and serve the ProofBundle on pull, not push full chains through the gossip layer. This shapes the read API: prefer iterator-based proof export over "materialize the whole bundle in memory."
  • Key distribution is NOT DS's job per the S14 SSH-paradigm decision (s14_decision_decouple_key_distribution_from_ds_handshake). DS session keys (X-Wing) and trusty verification keys (SipHash + Ed25519 via #17 trusty-keydist) live on separate tracks. trusty-postgres stores the SipHash keys in the producer_identity table (section 8); it does not participate in any DS handshake.

Out of scope for this memo but noted: a future trusty-ds-postgres integration crate (separate from trusty-postgres) would implement the CapabilityHandler trait from dspm and wire the Store trait to DS marketplace messages. That crate is not trusty-postgres.

2. Scope: trait methods + higher-level Trusty methods

Store trait surface

The 7 methods at crates/core/trusty-store/src/lib.rs:22-48:

  • last_row(entity_id)
  • insert_row(&RowInsert)
  • iter_chain(entity_id) -> Iterator<StoreResult<StoredRow>>
  • get_tip(chain_id)
  • get_tip_for_entity(entity_id)
  • set_tip(&TipUpsert)
  • kv_cross_check()
  • with_txn(&mut dyn FnMut(&mut dyn Store) -> StoreResult<()>)

All 7 must be implemented. iter_chain is the one with non-trivial ergonomics in async-Postgres land -- see section 7 (with_txn) and section 9 (concurrency).

Higher-level Trusty methods (NOT in the Store trait today)

Per s14_insight_store_trait_multi_backend_landscape.toml, four higher-level methods live in trusty-sqlite::Trusty rather than behind the trait. They must be re-implemented in the Postgres adapter:

  • append / append_signed / append_with_method / append_with_method_signed -- the orchestration around compute_record + write_envelope + append_hash_index + set_tip. Source: store.rs:152-283.
  • append_batch(&[RowSpec]) -> Vec<AppendResult> -- single-transaction amortization. Source: store.rs:528-553.
  • verify(entity_id) -> VerifyReport -- iterate provenance_envelope in id-asc order, reconstruct ChainLinks, hand to trusty_core::verify_links. Source: store.rs:467-500.
  • breaks_since, liveness_check, anchors_since, export_bundle, history, integrity, anchor -- the read/observability surface. Source: store.rs:502-836.

Estimated 300-400 lines of mechanical work after the trait is implemented, per the multi-backend tag.

Out of scope for v1 of trusty-postgres

  • tk_* binaries / APERC tooling -- no adapter-side hooks.
  • Schema migrations between trusty-postgres versions. Document the v1 schema as locked by analogy to the locked v1 byte-shape; any v2 schema work is a future memo. (See section 11 cross-ref.)
  • Connection pooling configuration as a public API. Use the driver's pool with sane defaults; expose the pool object only if a downstream user demands it.

3. Schema design

Three-table layout, Postgres dialect

Mirror crates/adapters/trusty-sqlite/src/schema.rs:1-49 one-to-one in structure. The three-table layout is locked per CLAUDE.md "Don't Touch": provenance_envelope, hash_index, chain_tips. The KV cross-check (integrity method) depends on both chain_tips and hash_index agreeing on (entity_id, chain_id, current_tip_hash).

Postgres dialect translation:

  • INTEGER PRIMARY KEY AUTOINCREMENT -> BIGSERIAL PRIMARY KEY or BIGINT GENERATED BY DEFAULT AS IDENTITY. Use the latter; identity columns are SQL-standard and easier to reset for tests.
  • TEXT -> TEXT (Postgres TEXT is unbounded VARCHAR; same semantics).
  • INTEGER for sequence_int -> BIGINT. Sequence numbers are i64 in the Rust types.
  • Indexes: same set as SQLite. Add a partial index CREATE INDEX idx_pe_break ON provenance_envelope(entity_id, detected_at) WHERE event_type = 'BREAK' to accelerate breaks_since. The SQLite version scans; Postgres can short-circuit with the partial index.
  • raw_payload TEXT -> raw_payload TEXT. Do NOT use JSONB. The payload is a canonical byte-stable JSON string that the verifier reads back and recomputes hashes over. JSONB normalizes whitespace and key order, which would silently break v1 byte-shape verification. This is a v1-byte-shape-preservation invariant; document it inline in the migration SQL.

TimescaleDB hypertable on received_at

Decision: support TimescaleDB as an optional Cargo feature (features = ["timescale"]), default off. When enabled, the migration runs SELECT create_hypertable('provenance_envelope', 'received_at', chunk_time_interval => INTERVAL '7 days').

Reasoning:

  • The IoT/sensor tier (per project_trusty_role.md in memory) is the primary trusty-postgres adopter. Time-series query acceleration on received_at is the right enterprise pitch.
  • Hypertables do not change row identity, primary keys, or read/write semantics from the chain's perspective. The hash chain is independent of physical chunking.
  • Default-off avoids forcing the TimescaleDB dependency on operators who do not have it. The adapter must work on stock Postgres 14+.
  • One constraint: chain_tips and hash_index are NOT hypertables. They are small lookup tables; chunking by time would hurt, not help.

OPEN QUESTION: do we want a separate received_at column type (TIMESTAMPTZ) versus the SQLite TEXT-ISO8601 string used today? Recommendation: yes, store as TIMESTAMPTZ for query performance, but expose as RFC3339 string at the Store trait boundary so the trait is dialect-neutral. This is purely an internal storage choice and does not affect chain bytes. Confirm with Eric.

4. Tri-state null (apply learn-LCG-ARCH-03)

The tag is unambiguous: "wherever absence of a value might be legitimate, encode three states (PRESENT, ABSENT, UNKNOWN) explicitly." SQL NULL alone conflates ABSENT and UNKNOWN.

Audit of nullable columns in the trusty schema:

Column NULL means today Tri-state interpretation
provenance_envelope.row_hash "BREAK row, no hash applies" ABSENT (BREAK rows structurally have no hash). Two-state is fine.
provenance_envelope.anchor_hash same as row_hash ABSENT. Two-state is fine.
provenance_envelope.raw_payload same as row_hash ABSENT. Two-state is fine.
provenance_envelope.field_name/old_value/new_value "non-BREAK row, fields don't apply" ABSENT. Two-state is fine.
chain_tips.current_tip_hash TODAY ambiguous NEEDS TRI-STATE. See below.
chain_tips.chain_id "no chain bound to this entity yet" UNKNOWN/ABSENT ambiguity. See below.

The cold-start case is the one that bites:

An OID-bound producer is provisioned (its OID is registered, its Ed25519 pubkey is in producer_identity) but it has not yet appended a single row.

In trusty-sqlite, this state is unrepresentable: chain_tips rows are created on first append, so an unappended producer simply has no row. That conflates "we know this OID exists and has appended nothing" with "we have never heard of this OID." For an enterprise PKI integration that pre-registers devices, this matters.

Decision: introduce explicit cold-start representation in the trusty-postgres schema:

  • chain_tips.current_tip_hash: encoded as tip_hash TEXT paired with tip_state TEXT NOT NULL CHECK (tip_state IN ('UNKNOWN', 'COLD', 'WARM')). UNKNOWN = pre-provisioned, never appended. COLD = appended once but the chain was sealed by a break and a new chain is pending. WARM = chain is live and tip_hash is the current valid head.
  • chain_tips.chain_id: NULL only when tip_state = 'UNKNOWN'. Otherwise NOT NULL.
  • New optional column producer_oid TEXT (see section 8).

Per learn-LCG-ARCH-03, prefer enum/check-constrained columns over sentinel-value tricks. The pair (tip_hash, tip_state) makes every get_tip result explicit about which of the three states it is in.

This forces a trait-level decision: should TipRow (defined at crates/core/trusty-store/src/lib.rs:60-66) carry a tip_state field?

Recommendation: YES, additively. Add tip_state: TipState to TipRow and TipUpsert with a default WARM for backward compatibility with trusty-sqlite. trusty-sqlite would always return WARM (its existing semantics) and trusty-postgres would return the true state. Verify this does not break any existing trusty-sqlite test before merging.

OPEN QUESTION: does this trait-level addition rise to the level of needing v2 of the Store trait, or can it be additive on v1? Eric to decide. Recommendation: additive with a default; this matches the v2-payload-additive precedent in project_trusty_v2_signing_design.md.

5. Compaction strategy (apply learn-LCG-ARCH-02)

The tag is explicit: "chain format supports compaction snapshots FROM v1. Archive boundaries are themselves attested. No retrofit possible without breaking historical verifiability."

The append-only chain has no semantic notion of compaction today. trusty-sqlite grows unbounded; the IoT-fleet pitch implies hundreds of devices each appending hundreds of rows per day, which crosses "too large to verify in a reasonable time" within months.

What compaction means for an append-only hash chain

Three architectural options:

Option A: snapshot tables. Periodically materialize (chain_id, sequence_int_high_water, snapshot_root_hash, snapshot_payload) into a chain_snapshot table. The snapshot_root_hash is computed canonically from the slice it covers (a Merkle root of all row_hashes in the slice, or simpler: a SipHash over the concatenation, with domain separation). The snapshot row is itself a valid chain envelope of event_type = 'SNAPSHOT'. After a snapshot is committed, archived slices can be moved to a cold-storage table or dropped, and verifiers can resume mid-chain by trusting the snapshot.

Option B: TCB manifest checkpoint. Maintain a chain_checkpoint table with (chain_id, sequence_int, row_hash, checkpoint_attestation). Verifiers can prune their verification window to "from the most recent checkpoint forward" and trust earlier state by checkpoint signature. Lighter weight than snapshots; does not enable archival, only verification acceleration.

Option C: Anchor checkpoints. Repurpose existing per-row anchor_hash as the compaction primitive. Periodically aggregate anchors over a window into a single aggregated_anchor and publish it externally (S3, public ledger, whatever). Verifiers spot-check against the aggregated anchor. This is the lowest-cost option and mirrors Certificate Transparency Merkle-tree consistency proofs.

Decision

Ship Option B (CHECKPOINT envelope, verification-layer only) as the v1 primitive in trusty-postgres. Reasoning:

  • Issue #13 explicitly says "MUST NOT change v1 byte-shape. Checkpoints are a verification-layer construct, not a payload change." Option A (SNAPSHOT) adds a new event_type which, while additive, is a semantic change to the chain contract and requires a new trusty-core aggregation primitive -- that is a larger design pass than v1 of trusty-postgres should attempt.
  • Option B keeps the existing chain contract untouched. A checkpoint is a signed (chain_id, sequence_int, row_hash, attestation) tuple written to a separate chain_checkpoint table. Verifiers can prune their verify window to "from the most recent valid checkpoint forward." No new envelope type, no new canonical bytes, no trusty-core change beyond exposing the checkpointing API in a higher-level module.
  • Archival (#5) and row-aggregation compaction (Option A) are deferred to v2. Both build on top of the checkpoint primitive once it is proven in production.
  • Issues #5 and #13 resolve cleanly: #13 is the v1 feature; #5 is the v2 feature that uses checkpoints as archive boundaries.
  • Terminology lock: the word compaction is retired from this crate's vocabulary in favor of checkpoint (for v1) and archive (for v2). No ambiguity.

Schema: add a chain_checkpoint table:

chain_checkpoint (
    id               BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    chain_id         TEXT NOT NULL,
    sequence_int     BIGINT NOT NULL,
    row_hash         TEXT NOT NULL,
    checkpoint_at    TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    attested_by      TEXT NOT NULL,             -- OID or pubkey fingerprint
    attestation_sig  BYTEA NOT NULL,            -- Ed25519 over (chain_id, sequence_int, row_hash)
    UNIQUE (chain_id, sequence_int)
)

API on Trusty:

  • checkpoint(chain_id, attestation_key) -> CheckpointRow -- walks the chain to head, computes a fresh checkpoint at head, writes the row.
  • verify_since_checkpoint(chain_id) -> VerifyReport -- reads the most recent valid checkpoint, iterates forward from there, same report shape as verify today.

This is all adapter-level; no trusty-core change required for v1. Archival and row-aggregation compaction become separate design memos when the archival product case (#5) is ready to ship.

6. Revocation semantics (apply learn-LCG-ARCH-06)

The tag is blunt: "Every system that said 'we will figure revocation out in v2' paid for it in v2 with interest." Existing trusty memory treats revocation as application-layer. Re-examine for OID-anchored deployment:

The case under OID anchoring

A producer device has OID 1.3.6.1.4.1.<arc>.<leaf> and an Ed25519 keypair. The chain is signed (v2 payload) per project_trusty_v2_signing_design.md. Mid-stream, the device's key is revoked -- compromised hardware, retired device, or operator policy.

Three sub-cases:

Sub-case (a): retroactive revocation of past rows. Should rows appended before the revocation timestamp still verify? If yes, the chain remains valid up to the revocation point and is sealed there. If no, the entire chain is poisoned by retroactive distrust.

Sub-case (b): forward block on new appends. After revocation, any append signed by the revoked key must fail. This is structurally the chain-break primitive: a revoked-key append is a break, sealed chain ends, no new genesis under the same OID.

Sub-case (c): key rotation. Revoke old key, install new key, continue chain under new key. Requires a chain-of-custody attestation linking old-key-tip to new-key-genesis.

Recommendation

Adopt the Cert-Transparency stance: chains are append-only and revocation is forward-only. Sub-case (a) is rejected: past rows remain verifiable forever. Sub-case (b) is the default behavior (the existing chain-break primitive handles it for free if the condition library checks signing-key validity; that condition needs to exist). Sub-case (c) is a v2 feature involving a KEY_ROTATION envelope and an attestation chain to the new key.

Schema: add a producer_revocation table:

producer_revocation (
    producer_oid TEXT NOT NULL,
    revoked_at TIMESTAMPTZ NOT NULL,
    revoked_pubkey BYTEA NOT NULL,
    reason TEXT NOT NULL,
    attested_by_pubkey BYTEA,           -- who authorized the revocation
    revocation_envelope_id BIGINT       -- FK to a provenance_envelope row
)

The revocation event is itself a chain envelope of event_type = 'REVOCATION' so the act of revoking is part of the auditable chain. This satisfies both the append-only invariant and the auditability requirement.

Per the LCG-ARCH-06 runtime invariant: "revocation of an upstream capability invalidates derived attestations within N blocks. Testable via assert_invariants.py: insert edge, attest, revoke edge, verify attestation fails on re-check." Translate to a trusty-postgres integration test: append signed rows, write revocation envelope, attempt new append signed by revoked key, assert it produces a chain-break.

OPEN QUESTION: where does the condition that checks "is this signing key currently revoked?" live? Options: (i) a new core-level Condition in trusty-core, (ii) a Postgres-adapter-only Condition. Recommendation: (i), so the semantics are uniform across backends. Schedule as part of the trusty-core SNAPSHOT-primitive memo (section 5) -- same memo, same set of additive trusty-core changes.

7. with_txn decision

Source of friction: s14_learn_with_txn_async_friction.toml. Sync trait fn with_txn(&mut self, f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>) -> StoreResult<()> does not compose with async Postgres drivers (sqlx, tokio-postgres) because closures cannot be async without boxing a Future and dealing with lifetime/pinning.

Three options enumerated in the tag:

  • (a) block_on in adapter. Keep the sync trait. The Postgres adapter wraps a tokio runtime internally and block_ons every call. Pros: zero changes to trusty-sqlite, zero changes to the trait. Cons: the adapter's caller cannot be in an async context without nested runtime issues; or we use the synchronous-postgres driver (libpq directly via postgres crate) and skip async entirely.

  • (b) Boxed async closure. Add an async variant of with_txn to the trait: async fn with_txn_async(&mut self, f: BoxFuture<...>) -> StoreResult<()>. Pros: native async. Cons: trait object methods cannot be generic AND cannot be async-fn-in-trait without macro support; ergonomics are awful for callers.

  • (c) Split into begin / commit / rollback. Drop the closure pattern entirely. Trait gains begin_txn, commit_txn, rollback_txn. Pros: composes with both sync and async. Cons: callers lose RAII safety (forget to commit and the txn leaks); trusty-sqlite must be updated too.

Recommendation

Option (a) -- block_on with the synchronous postgres crate (not sqlx/tokio-postgres) -- for v1 of trusty-postgres. Reasoning:

  1. The synchronous postgres crate (built on tokio-postgres but exposing a sync API) is mature, widely used, and avoids the nested-runtime problem entirely. No block_on needed -- the driver is sync from the user's perspective.
  2. The trait stays unchanged. trusty-sqlite stays unchanged. Other adapters (DuckDB, ScyllaDB) can make their own sync-vs-async choice without us locking the trait shape.
  3. Async-Postgres consumers who want native tokio integration can wait for a v2 trait or implement Option (c) themselves; this is not a v1 requirement for the IoT/compliance buyer pitch.

However, document the trait friction explicitly in trusty-postgres README so the next person who tries to wire it into an async axum/tonic server is not surprised. Add a worked example of using tokio::task::spawn_blocking to bridge.

Defer Option (c) to a future trait-v2 pass. When we get serious about ScyllaDB or another genuinely-async backend, revisit. At that point all adapters get updated together. Eric to decide whether to formally schedule that pass; not blocking trusty-postgres v1.

OPEN QUESTION: confirm the postgres (sync wrapper) crate is the right choice over rust-postgres direct or diesel. Recommendation: postgres crate; it is the de facto standard sync Postgres client for Rust and has the same maintainer as tokio-postgres.

8. OID + chain_id binding (apply s14_insight_oid_anchored_chain_identity)

The OID-anchored model is the entire commercial reason this crate exists. Get this right at v1.

The binding

Per the insight tag, the model is:

  • An OID (e.g. 1.3.6.1.4.1.<vendor-arc>.<device-leaf>) identifies a producer (device, sensor, agent).
  • That OID is the canonical chain identity. Either chain_id IS the OID, or chain_id is derivable from the OID + a generation counter.
  • An Ed25519 pubkey is bound to the OID via a known_producers TOFU registry (per the SSH-paradigm tag).

The trusty-sqlite chain_id format today is chain_<entity_id>_<6-digit-gen> (see make_chain_id, used at store.rs:299 and parsed at store.rs:965-975). Eric: this format is fine for SQLite where entity_id is opaque, but for OID-anchored Postgres deployments we want OID legibility.

Schema

Add a producer_identity table, separate from the three locked tables:

producer_identity (
    producer_oid TEXT PRIMARY KEY,
    ed25519_pubkey BYTEA NOT NULL,
    siphash_key BYTEA NOT NULL,            -- derived per SSH-paradigm tag
    registered_at TIMESTAMPTZ NOT NULL,
    display_name TEXT,
    metadata JSONB                          -- ok to use JSONB here:
                                            -- not part of chain bytes
)

And add a producer_oid TEXT column to chain_tips, foreign-keyed to producer_identity.producer_oid when present (nullable for backward compat with non-OID deployments).

Decision: do NOT change the chain_id format string. The format chain_<entity_id>_<6-digit-gen> is locked by trusty-core's make_chain_id (at crates/core/trusty-core/src/chain.rs:100). Note: parse_generation currently lives in the SQLite adapter at crates/adapters/trusty-sqlite/src/store.rs:965; it should be promoted to trusty-core as a pre-requisite of this work so trusty-postgres does not copy-paste. Additive change; no v1 byte-shape impact.

Instead:

  • For OID deployments, set entity_id to the OID string. make_chain_id will produce chain_1.3.6.1.4.1.<arc>.<leaf>_000001, which is human-readable and OID-legible.
  • The producer_oid column on chain_tips denormalizes the OID for indexing/joins to producer_identity.

Reconciliation with Gitea #14

Issue #14 proposes chain_id = H(domain_sep || pubkey || nonce). Taken literally this contradicts the decision above. Reading the full issue body resolves the conflict: #14 is explicitly additive -- "Existing v1 chains without attribution remain verifiable." The two models coexist:

  • chain_id string stays as make_chain_id(entity_id, gen) for human legibility, log grep-ability, and OID integration.
  • Cryptographic binding moves into (a) the v2/v3 genesis-row signature (the Ed25519 signature over the signing-view commits to chain_id via the row_hash), plus (b) the producer_identity(producer_oid, ed25519_pubkey, ...) row that proves which key is allowed to sign for this OID.

A rival chain with the same chain_id string cannot produce a valid genesis signature without the private key, so the "malicious party presents a rival chain" threat in #14 is defended against by the signature chain, not by the id format. #14 becomes "require genesis sig + producer_identity lookup before trusting any append," not "change the chain_id derivation." No trusty-core change to make_chain_id.

The producer_identity table is the local cache equivalent of an SSH known_hosts file (per the SSH-paradigm tag) -- not a CA, just a TOFU registry. Whether OIDs cross-reference to X.509 cert chains is application-layer and out of scope here.

OPEN QUESTION: should the producer_identity table be optional (feature-gated) or always created? Recommendation: always created, but rows are optional. Adapters that do not use OID-anchoring simply leave it empty. Cost is one empty table; benefit is one schema for all deployments.

OPEN QUESTION: OID validation -- enforce the dotted-numeric format at the column level (CHECK constraint) or accept arbitrary strings? Recommendation: accept arbitrary strings. trusty-postgres should not become an OID validator; that is the producer's responsibility. We can add a CHECK later if a misuse pattern emerges.

9. Concurrency model

The core difference from SQLite

trusty-sqlite is single-writer by SQLite's nature (WAL mode plus BEGIN IMMEDIATE, see store.rs:535). The chain primitive's correctness depends on one fundamental invariant: between reading the tip and writing the next row, no other writer mutates the tip. SQLite gives us this trivially.

Postgres allows multi-writer. Two clients calling append() for the same entity_id concurrently can both:

  1. Read the same tip.
  2. Compute next sequence = tip.sequence + 1.
  3. Insert two rows with the same sequence and the same prev_hash.

Result: chain integrity violation. Both rows verify in isolation (both have correct prev_hash) but the chain has forked. Verification will eventually catch it (sequence collision in iter_chain) but silent damage is done.

Decision: per-entity row-level lock at append time

Use Postgres advisory locks keyed on entity_id:

SELECT pg_advisory_xact_lock(hashtext($1));   -- $1 = entity_id

Inside the transaction, take an advisory lock derived from entity_id (via hashtext or a dedicated 64-bit hash). The lock is held for the duration of the transaction and released on commit or rollback. Two appenders for the same entity serialize; appenders for different entities run concurrently.

Rationale:

  • Advisory locks are cheap (no row materialization), session-scoped or transaction-scoped, and Postgres-native.
  • They serialize exactly the operations that need serialization: the read-tip / compute / write-row / update-tip cycle for a single entity.
  • Different entities are independent; no global lock, so multi-entity fleets scale.

Alternative considered: SELECT ... FOR UPDATE on chain_tips rows. This works but materializes a row lock on every append even when no contention exists. Advisory locks are more direct.

Alternative considered: optimistic concurrency with a retry loop on sequence-collision unique-violation. Possible but complicates the adapter (retry loops, error mapping). Defer.

Test invariants

Add a concurrent-append test: spawn N threads each appending K rows to the same entity_id. Assert: total rows = NK, sequences are 0..NK-1 with no gaps, verify passes, kv_cross_check passes.

OPEN QUESTION: should the advisory lock be transaction-scoped (pg_advisory_xact_lock) or session-scoped (pg_advisory_lock + explicit unlock)? Recommendation: transaction-scoped. Releases automatically on commit/rollback; no leak risk.

10. Migration story: trusty-sqlite -> trusty-postgres

The product use case: a customer runs trusty-sqlite at the edge for a year, then promotes their archive to trusty-postgres for long-term audit. This must work without recomputing chain hashes.

Byte-shape preservation

The locked v1 byte-shape (see CLAUDE.md "Don't Touch") guarantees that raw_payload strings are byte-identical regardless of backend. Migration is a row-by-row copy with no recomputation:

  1. SELECT all rows from sqlite provenance_envelope ordered by id.
  2. INSERT into postgres provenance_envelope preserving every column verbatim. The id column re-issues from the postgres identity sequence; that is fine because chain semantics depend on sequence_int + prev_hash + chain_id, not on the surrogate id.
  3. SELECT all rows from sqlite hash_index, INSERT into postgres hash_index verbatim.
  4. SELECT all rows from sqlite chain_tips, INSERT into postgres chain_tips verbatim. Set tip_state = 'WARM' for all migrated rows; the migration tool cannot reconstruct cold/unknown state.
  5. Run verify for every entity_id on the postgres side; assert reports match the sqlite side row-for-row.
  6. Run kv_cross_check on the postgres side; assert OK.

Anchor hashes

Anchor hashes are NOT recomputed -- they are byte-identical between backends because they are derived from the locked v1 primitives. Copy them verbatim. If the post-migration verify pass fails on anchor mismatch, the bug is in the migration tool, not in the chain.

Implementation

Ship a trusty-migrate binary in the trusty-postgres crate:

trusty-migrate \
    --from sqlite://path/to/file.db \
    --to postgres://user:pass@host/db \
    --verify

--verify runs the full verify pass on both sides post-migration and asserts byte-identity. Exit non-zero on any mismatch.

OPEN QUESTION: do we want bidirectional migration (postgres -> sqlite for offline audit)? Recommendation: yes, but defer to a v2 of the migration tool. v1 ships sqlite -> postgres only. The inverse direction has the same byte-shape guarantee so it is mechanical to add later.

11. Testing strategy

The bar to merge trusty-postgres v1:

Required test suites

  1. All trusty-store testing-feature tests pass. The Store trait ships a conformance suite under crates/core/trusty-store/src/testing.rs (per the existing #[cfg(feature = "testing")] module, see crates/core/trusty-store/src/lib.rs:15-16). The v1 conformance suite is 6 tests, enumerated explicitly:

    • insert_and_read_back -- insert a row; last_row returns it.
    • iter_chain_ordered -- insert 3 rows; iterator yields them in sequence order with no gaps.
    • tip_roundtrip -- set_tip + get_tip_for_entity returns same row; get_tip(chain_id) agrees.
    • kv_cross_check_clean -- fresh insert produces OK report.
    • with_txn_commit -- inserts inside with_txn closure persist.
    • with_txn_rollback -- Err return from closure rolls back.

    trusty-postgres must pass all 6 identically to trusty-sqlite and trusty-memory.

  2. v1 byte-shape conformance. Add a test that appends N rows via trusty-postgres, exports a ProofBundle, and asserts every row_hash and anchor_hash matches the same sequence appended via trusty-sqlite with the same SecretKey, same payload, same timestamps. This is the runtime version of s5_invariant_v1_byte_shape_preserved for the Postgres adapter.

  3. Multi-writer concurrency. Per section 9: N threads, K rows each, single entity. Assert no sequence gaps, no duplicate sequences, verify passes.

  4. Cold-start tri-state. Provision a producer_identity row with no chain_tips row; assert get_tip_for_entity returns tip_state = UNKNOWN. After first append, assert tip_state = WARM.

  5. Revocation flow. Append signed rows, write a REVOCATION envelope, attempt append signed by revoked key, assert chain-break.

  6. Compaction round-trip. Append N rows, run compact() at high_water = N/2, assert verifier can still verify the full chain (using the snapshot envelope to skip archived rows).

  7. Migration tool round-trip. Append M rows to sqlite, migrate to postgres, run verify on both; assert reports identical.

  8. TimescaleDB feature gate. With --features timescale, the migration runs create_hypertable and inserts/queries succeed. Without the feature, the same tests pass on stock Postgres.

Test infrastructure

Use testcontainers-rs or equivalent to spin a Postgres container per test run. Do not depend on a developer-local Postgres install; CI must be hermetic.

OPEN QUESTION: should the conformance suite in crates/core/trusty-store/src/testing.rs be expanded with the tri-state, concurrency, and OID tests so all backends run them? Recommendation: yes, but in a follow-up trusty-store PR. Initially add them inline in trusty-postgres; promote to the conformance suite once stable and once trusty-sqlite is updated to support tri-state TipState (default WARM).

11b. Cross-cutting policy decisions (added post-research pass)

Decisions on four policies that applied across multiple sections and were surfaced in the S14-addendum research pass. These are the memo's current position; any of them can be re-opened but all four are in scope as implementation-ready defaults.

Payload version accept policy

trusty-postgres accepts reads of v1, v2, and v3 payloads (dispatch already lives in trusty-core via row_hash_for_version and extract_payload_version, confirmed at crates/core/trusty-core/src/chain.rs:52,211). For new appends the adapter writes v3 only. This guarantees:

  • The trusty-migrate sqlite -> postgres tool copies v1/v2 rows byte-verbatim; no recomputation, no upgrade of existing bytes.
  • All new rows benefit from domain separation.
  • compute_record_signed is v2-only today (handoff acknowledged gap). trusty-postgres signed-append uses v2 bytes until compute_record_signed_multi or a v3-signed path lands in trusty-core -- tracked as a pre-req of #16 (patch-policy multi-sig).

Schema versioning discipline

Schema migrations are planned, not forbidden. A schema_meta table ships in v1 carrying (key TEXT PRIMARY KEY, value TEXT) with a single row ('schema_version', '1'). Any future schema change ships with:

  • A new version number written to schema_meta.
  • An idempotent forward migration tested end-to-end.
  • A design memo at docs/trusty-postgres-schema-vN.md documenting the change.

Same rigor as payload v1/v2/v3 work. Do not pretend the schema is locked; pretending is how drift happens.

External anchor publish

App-level, not adapter-level. trusty-postgres exposes two read APIs that downstream crates use to build publish pipelines:

  • stream_anchors_since(chain_id, sequence) -> impl Iterator<AnchorRow>
  • export_bundle_since(chain_id, sequence) -> ProofBundle

No S3, Git, CT-log, or DS-transport dependency in the adapter. Retention/transport pipeline (#5) is a separate crate (likely trusty-archive or the DS integration crate) that consumes these iterators. Keeps trusty-postgres's TCB and dependency graph thin.

Observability

tracing spans only. Every public method on the adapter emits a span with chain_id, entity_id, and operation name. No prometheus/opentelemetry crate dependency -- consumers plug in any tracing_subscriber exporter they want (tracing-opentelemetry, tracing-prometheus-exporter, etc.) at their layer. Zero-cost when no subscriber is attached.

Span namespacing: trusty_postgres::append, trusty_postgres::verify, trusty_postgres::with_txn, trusty_postgres::checkpoint. Include advisory-lock wait time as a span event so lock contention is visible to the subscriber.

11c. Multi-tenant federated deployment (real target)

The original memo framed trusty-postgres as a generic mid-scale-cloud Store adapter. The actual product goal is tighter: a federated shared knowledge base spanning up to ~50 users, each with many agents writing concurrently, glued between user nodes by Distant Star, with the existing kb-mesh content (pins, synthesis artifacts, architectural decisions) materialized as trusty chains. This section captures the deployment shape implied by that goal; it supersedes no prior decision but adds concrete structure where the original memo was generic.

Deployment shape

  • Per-user Postgres schema. Each user gets a Postgres schema (namespace) named for their OID arc leaf, e.g. usr_eric, usr_partner, usr_agent03. Inside that schema the three-table layout (provenance_envelope, hash_index, chain_tips) is replicated. Schema-per-user gives hard isolation (an agent cannot accidentally write to another user's rows), per-schema index locality, and straightforward per-user backup/archive.
  • Cross-user schema in public. One schema-wide table set: public.producer_identity (OID -> Ed25519 pubkey + SipHash key + metadata, the known_producers registry), public.schema_meta (global schema version), public.received_bundles (inbound DS-delivered bundles awaiting or completed verification).
  • Cross-user read view. public.all_provenance_envelope AS SELECT ... FROM usr_eric.provenance_envelope UNION ALL ... FROM usr_partner.provenance_envelope ... regenerated whenever a user is provisioned or deprovisioned. Read-side queries spanning all users (e.g. "every pin tagged trusty across the mesh") go through this view; write-side never touches it.
  • One chain per (user, agent, topic) tuple. entity_id encodes <user_oid_leaf>/<agent_oid_leaf>/<topic>. make_chain_id on top of that produces chain_<user>/<agent>/<topic>_000001. Topic is caller-chosen (session id, KB pin topic, sensor stream name, etc.).

Ownership and authorization

  • Every agent has its own Ed25519 keypair and an OID leaf under its user's arc. producer_identity rows bind (OID -> pubkey).
  • Append is rejected at the adapter if the signing key does not match the producer_identity row for the target chain's OID. This turns the two-party "one writer per chain" rule into a general N-party invariant enforced at write time.
  • Agents provisioned/deprovisioned by a user-level operator flow (out of scope for this crate; in scope for a future trusty-admin tool).

Concurrency at scale

  • Advisory lock per entity_id inside each user's schema. hashtext(entity_id) already makes different entities contention- free; per-entity serialization matters only when two of one user's agents race the same chain.
  • Connection pool (pgbouncer or equivalent) required in any deployment exceeding ~100 concurrent agents. Memo-level recommendation: transaction-pooling mode with a pool of max_connections / N_users per user's app tier.
  • Postgres's own MVCC handles the 50-user + multi-agent load comfortably. The bottleneck is per-entity advisory locks, not the database. Measured scaling will come from real deployment; design-time assumption is that the chain-break rate stays O(errors) not O(normal appends).

Federation via Distant Star

  • Each user runs their own Postgres + DS node. Writes land locally under the user's schema.
  • DS gossips ProofBundles peer-to-peer. Advertise-don't-copy: a DS advertisement carries (chain_id, tip_hash, size_hint); bundles are pulled on demand.
  • Inbound bundles land in public.received_bundles with: bundle_id, source_producer_oid, chain_id, tip_hash, received_at, verify_status, verify_detail, bundle_blob (bytea). Verification runs against producer_identity (did the claimed OID sign this?); failed bundles stay with verify_status = 'REJECTED' and a reason -- not silently dropped, not replayed.
  • A user's view of "what partner has" is a query against received_bundles WHERE source_producer_oid IN (partner's OIDs). Historical verification remains local: even if the partner goes offline or leaves the federation, bundles already ingested are still cryptographically verifiable.

kb-mesh migration

The existing kb-mesh primitive (ephemeral pins + promoted patterns) maps cleanly:

  • Each pin becomes a chain row with event_type = 'PIN' and payload data carrying topic, turns, source_session, synthesis_level, pin body.
  • action = 'create' | 'update' | 'overwrite' | 'close' | 'promoted' from INDEX.jsonl becomes extraction_method or a structured field inside data.
  • Per-repo kb-mesh pins roll into a per-user chain named something like usr_<repo>/agent_<session>/kb-mesh-pins. INDEX.jsonl becomes redundant (SELECT ... FROM ...provenance_envelope WHERE event_type = 'PIN' replaces it).
  • Promoted patterns (ephemeral pin -> durable .kb/patterns/ file) become CHECKPOINT envelopes over the pin chain + a supplementary write to a pattern chain.

A dedicated trusty-kb-mesh (or trusty-kb) crate owns the mapping; trusty-postgres is the storage. Gitea issue to follow.

Changes to earlier sections this implies

  • §1 add non-goal: trusty-postgres is not a tenant manager; user provisioning and schema creation live in a future trusty-admin crate or operator script.
  • §4 (tri-state null): tip_state is per-user; cold-start state is per-agent per-chain, not global.
  • §8 (OID + chain_id): chain_id convention tightened -- <user>/<agent>/<topic> inside the template. entity_id encodes the same. Unchanged at the byte level; just a convention on the strings.
  • §9 (concurrency): advisory lock is per-entity within a schema, not global. Hash collision risk on hashtext(entity_id) across schemas is irrelevant because different schemas are different critical sections anyway.
  • §10 (migration): trusty-migrate sqlite -> postgres must now specify a target schema (or default to public for single-user deployments).

Open questions specific to the multi-tenant shape

  • How do we test 50-user + many-agent concurrency before deployment? Probably a load-gen harness that spins N schemas + K async appenders per schema. Out of scope for trusty-postgres v1; in scope for a trusty-load test binary later.
  • How do schema migrations roll out across N user schemas without blocking active writers? Partial answer: ALTER runs per-schema in sequence, advisory-locked against that schema's writers. Full answer needs a real migration tool.
  • Bundle ingestion rate limits per source producer? Probably yes, to prevent a rogue peer from flooding; app-level concern, not adapter.

12. Open questions and explicit deferrals

Items needing Eric's input before implementation. Items marked RESOLVED were decided in the post-research pass; strike-through anything that later changes.

  1. RESOLVED: TipState additive trait change (S16). Approved as v1-compatible additive change with default WARM. Precedent: v2 signing (project_trusty_v2_signing_design) added fields to PayloadFields and a new top-level compute_record_signed without breaking v1 callers; the same additive-default pattern applies to trait fields. TipState is store metadata, not chain-payload bytes, so v1 byte-shape is unaffected by construction.

  2. RESOLVED: Compaction primitive scope (section 5). Decision: ship CHECKPOINT (verification-layer only) for v1; archival and row-aggregation compaction are v2. No trusty-core change. Retired the word "compaction" from this crate's vocabulary.

  3. RESOLVED: Revocation Condition placement (S16). trusty-core. Uniform across backends prevents two adapters from diverging on what counts as revoked, which would silently fork chain verification semantics. The cost (a small hash-set lookup in the core verify path) is negligible vs. the consistency win.

  4. RESOLVED: Sync vs async driver (S16). Sync postgres crate for v1 confirmed. Trait stays unchanged; with_txn is already validated. Defer Option (c) trait-v2 split until ScyllaDB or similar makes async unavoidable.

  5. RESOLVED: OID validation (S16). Enforce dotted-numeric format at column level via CHECK constraint (chain_id ~ '^[0-9]+(\\.[0-9]+)*$' or equivalent on the producer_oid column). OID format is the trust spine of federation; arbitrary strings invite ad-hoc IDs that defeat the cryptographic identity model. Fail fast at the boundary.

  6. RESOLVED: TimescaleDB column type (S16). Use TIMESTAMPTZ internally; trait surface stays RFC3339 strings. Indexing on TEXT timestamps is bad; conversion is transparent at the adapter boundary; conformance test enforces the RFC3339 round-trip. No byte-shape impact (timestamps live in payload data, not envelope columns subject to v1 lock).

  7. RESOLVED: Bidirectional migration (S16). v1 ships sqlite -> postgres only. postgres -> sqlite is deferred to v2 (or never). Federated multi-tenant is the production target; nobody migrates back from federation to a single-file edge backend.

  8. RESOLVED: Conformance suite expansion (S16). Promote new tests into trusty-store::testing AFTER the first stable trusty-postgres release. Keeps churn local during development; promotion gated on proven stability.

  9. RESOLVED: Repo location (S16). Separate Gitea repo mm-trusty/trusty-postgres (already pushed via subtree split, commit 427c7a7), depending on trusty-core and trusty-store via path or git dep. Matches the workspace shape already in flight; keeps trusty-core/sqlite TCB minimal; users pick path/git/crates.io per deployment.

  10. RESOLVED: TCB manifest (S16). trusty-postgres README page-one section. Matches learn-LCG-ARCH-13 recommendation; one place, explicit, no separate doc to drift.

  11. RESOLVED: chain_id derivation (section 8). Decision: keep make_chain_id string template; move cryptographic binding into genesis-row signature + producer_identity table. #14 becomes "require genesis sig + identity lookup," additive only, no trusty-core change to make_chain_id.

  12. RESOLVED: payload version accept policy (section 11b). Decision: accept v1/v2/v3 reads; new appends are v3.

  13. RESOLVED: schema versioning discipline (section 11b). Decision: plan migrations; ship schema_meta table in v1; same rigor as payload-version work.

  14. RESOLVED: external anchor publish (section 11b). Decision: app-level. Adapter exposes stream_anchors_since and export_bundle_since read APIs; publishing lives in a downstream crate.

  15. RESOLVED: observability (section 11b). Decision: tracing spans only, no prometheus/otel dependency. Consumers plug in any subscriber layer.

  16. RESOLVED: parse_generation location (S16). Promote to trusty-core as part of the Tier 1 refactor (s14_insight_general_library_improvements). Pure string parser, no byte-shape involvement. Every future backend would re-copy-paste otherwise.

  17. RESOLVED: compute_record_signed v3 gap (S16). compute_record_signed now version-dispatches by fields.payload_version: v1/v2 callers stay byte-identical (v1 upgrades to v2 per S5 behavior), v3 callers get the domain-tagged signed path via row_hash_for_version / anchor_hash_for_version. Two new tests: v3_signed_record_verifies and v3_tampered_signature_fails_verify. 50 trusty-core tests pass (was 48). HARD BLOCKER lifted; §13 P1/P4/P7 unblocked. compute_record_signed_multi (tied to #16) remains a separate follow-on for N-signer multi-sig.

  18. First-class edges (Graphiti P6, deferred from §13). Cross- user / cross-agent provenance today rides on the chain_id string template (<user>/<agent>/<topic>). Graphiti models edges as their own chained relation with endpoint refs + source-episode pointer, which would make federation traces queryable. Conflicts mildly with the §8 chain-id-as-template legibility decision and adds a parallel structure. Needs its own scoping memo before any code; not adopted in §13.

13. Patterns lifted from Cozo + Graphiti

Cross-reference of design patterns from CozoDB (embedded Datalog graph DB; pure Rust; HNSW + per-relation time-travel) and Graphiti (Zep's temporal context graph; episode-based provenance) against the trusty-postgres KB. Each pattern is mapped to its existing anchor in the design memo or APERC tag substrate. All eight patterns are additive to v1 byte-shape -- no change to compute_record, build_payload, or the three locked tables' chained columns.

13.1 Substrate

P1 -- Stored-relation registry (Cozo). A chained_relations meta table where each app-level relation opts in to chaining via a row + trigger calling trusty_append. Existing anchor: s15_insight_kb_surface_taxonomy_for_trusty_kb_adapter mandates "four clean mapping functions" (policy / decisions / tags / pins) and "one entry point per concern." A registry formalizes that mandate -- the four kb-mesh surfaces become registry rows instead of bespoke crate-level glue. Slots into §3 schema and §11c kb-mesh chain mapping.

P5 -- Episode as canonical append unit (Graphiti). provenance_envelope already is the episode; event_type already names the kind (PIN / DECISION / POLICY_ROW per s15_insight). Vocabulary alignment only -- glossary edit in §1 and docstring edits referring to rows as "episodes." No schema delta.

P6 -- First-class edges (Graphiti). DEFERRED. See §12 OQ #18. Heavy add; conflicts with chain-id-as-string-template in §8; needs its own scoping memo.

13.2 Temporality

P3 -- Per-relation time-travel (Cozo bitemporal). Today trusty has system-time only (chain_timestamp); supersession is a payload-level concept (supersedes: [prior_row_hash] per s15_insight). Add a companion table:

CREATE TABLE bitemporal_index (
  chain_id   TEXT NOT NULL,
  row_hash   TEXT NOT NULL,
  valid_range tstzrange NOT NULL,
  PRIMARY KEY (chain_id, row_hash),
  FOREIGN KEY (chain_id, row_hash)
    REFERENCES provenance_envelope (chain_id, row_hash)
);
CREATE INDEX bitemporal_index_range
  ON bitemporal_index USING GIST (chain_id, valid_range);

Critical invariant: provenance_envelope columns and the build_payload byte-shape are NEVER modified by P3. The bitemporal index is a sidecar projection keyed by (chain_id, row_hash). Conformance suite vs. trusty-sqlite stays green.

P8 -- Bi-temporal supersession projection (Graphiti). Ships with P3. A Postgres view current_facts materializes "facts whose valid_range contains now()," excluding superseded rows. The supersession event itself remains a chained envelope per s15_insight (never mutate prior rows); P8 just makes the projection queryable.

13.3 Retrieval (unblocks the trusty-monitor-daemon pin)

P4 -- HNSW vector indices on chained relations (Cozo). Adopt pgvector. Per chained relation that opts in (via the registry from P1), an embedding vector(N) column on a sidecar table keyed by (chain_id, row_hash) plus an HNSW index. Existing anchor: pin ../.kb-mesh/pins/trusty/trusty-monitor-daemon-chat-capture.md asks for "captures chat as unstructured data, auto-tags" -- auto-tag-over-chain is exactly pgvector + HNSW. Currently no path; P4 is the path.

P7 -- Hybrid retrieval contract (Graphiti). One query API combining vector similarity + tsvector keyword + recursive graph traversal, with chain-verify result as a returned column on every row. Existing anchor: s15_insight specifies pre-write federation search as kb_search(keywords) -> Vec<RowRef> across the user's schema + received_bundles. P7 extends the signature to kb_search(keywords?, embedding?, graph_filter?) -> Vec<VerifiedRowRef>. Single API surface for both the kb-mesh adapter (s15_insight) and the trusty-monitor-daemon pin.

13.4 Query API

P2 -- Datalog-shaped chain queries via recursive CTEs. Existing anchor: s14_insight_general_library_improvements_from_postgres_impl item (1) -- promote history, breaks_since, liveness_check, integrity to trusty-store as a TrustyExt trait. P2 reframes that refactor: the trusty-store side ships a named-relation query library; each backend provides the templates (recursive CTEs on Postgres, WITH RECURSIVE on SQLite, Datalog rules on a future Cozo adapter). Same surface, backend-specific implementation.

13.5 Adoption summary

Pattern KB anchor Verdict
P1 s15_insight_kb_surface_taxonomy Adopt -- completes four-surface mandate
P2 s14_insight_general_library_improvements_from_postgres_impl Adopt -- reframes promote-to-store refactor
P3 s15_insight_kb_surface_taxonomy (supersession), §11c Adopt -- companion table only
P4 pin trusty-monitor-daemon-chat-capture, §11c federation Adopt -- direct unblocker
P5 provenance_envelope already exists Adopt -- vocabulary only
P6 §8 chain-id-as-template, §11c federation Defer -- §12 OQ #18
P7 s15_insight_kb_surface_taxonomy (kb_search spec), pin Adopt -- direct extension
P8 s15_insight_kb_surface_taxonomy (supersession decision) Adopt -- bundles with P3

13.6 Upstream blockers and locked-rule statement

  • v3 signed-path gap (§12 OQ #17, escalated to HARD BLOCKER in s15_decision_multitenant_shared_kb_shape) is upstream of P1 / P4 / P7. Federated hybrid retrieval over received_bundles is meaningless without verifiable v3 signed rows. Resolve OQ #17 before implementing §13.
  • Locked-rule check. P1 adds a meta table. P2 ships pure query library code. P3 adds a sidecar table keyed by (chain_id, row_hash); never modifies provenance_envelope. P4 adds a sidecar table per opted-in relation; never modifies provenance_envelope. P5 is documentation. P7 is API composition over P1 / P3 / P4. P8 is a view. None touches compute_record, build_payload, or the three core locked columns. Conformance suite vs. trusty-sqlite remains the gate.

Appendix A. Lessons applied -- index

Lesson Section Where in design
insight-LCG-ARCH-01 meta every section "build it in v1 because retrofit is 5-10x more expensive"
learn-LCG-ARCH-02 compaction 5 snapshot tables in v1; SNAPSHOT envelope; chain_compaction_state
learn-LCG-ARCH-03 tri-state 4 tip_state column; TipState enum on the trait; cold-start representation
learn-LCG-ARCH-06 revocation 6 producer_revocation table; REVOCATION envelope; runtime invariant test
learn-LCG-ARCH-13 TCB 1, 12 TCB statement in README page one; postgres + libpq + driver named explicitly
Trusty insight Section Where
s14_insight_oid_anchored_chain_identity 8 producer_identity table; OID-as-entity_id pattern
s14_insight_store_trait_multi_backend_landscape 2 scope of higher-level Trusty methods to re-implement
s14_learn_with_txn_async_friction 7 Option (a) sync postgres crate; trait unchanged
s5_invariant_v1_byte_shape_preserved 3, 11 TEXT not JSONB for raw_payload; conformance test vs sqlite
project_trusty_v2_signing_design 6, 4 additive-only precedent for trait additions

Appendix B. Files referenced

  • crates/core/trusty-store/src/lib.rs:22-48 -- Store trait definition.
  • crates/core/trusty-store/src/lib.rs:54-115 -- LastRow, TipRow, StoredRow, RowInsert, TipUpsert, KvCrossCheckReport.
  • crates/adapters/trusty-sqlite/src/store.rs:1-955 -- reference adapter implementation including all higher-level Trusty methods.
  • crates/adapters/trusty-sqlite/src/schema.rs:1-49 -- reference schema (three tables, locked).
  • APERC/tags/s14_insight_oid_anchored_chain_identity.toml -- OID-anchored model.
  • APERC/tags/s14_insight_store_trait_multi_backend_landscape.toml -- four-backend landscape; method scope.
  • APERC/tags/s14_learn_with_txn_async_friction.toml -- with_txn options.
  • ../legal-chaingraph/APERC/KB/architecture_antipatterns.md -- the 18-lesson source.
  • ../legal-chaingraph/APERC/tags/learn-LCG-ARCH-02.toml -- compaction.
  • ../legal-chaingraph/APERC/tags/learn-LCG-ARCH-03.toml -- tri-state null.
  • ../legal-chaingraph/APERC/tags/learn-LCG-ARCH-06.toml -- revocation.
  • ../legal-chaingraph/APERC/tags/insight-LCG-ARCH-01.toml -- meta-principle.
  • CLAUDE.md -- project rules; locked surfaces; library-not-platform discipline.
  • docs/payload-v2.md -- v2 payload signing design.
  • docs/threat-model.md -- existing trusty threat model (cross-reference for TCB).

End of memo.

Reference in New Issue View Git Blame Copy Permalink