commit 693f3bef1ea9a8b3e715eb215442c512c9afe8e6 Author: modomelius Date: Wed Sep 9 18:31:00 2026 -0400 Trusty: provenance library workspace (initial import) diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..089c702 --- /dev/null +++ b/.gitignore @@ -0,0 +1,18 @@ +/target +**/target +/.claude +*.pdb +*.dll.exp +*.dll.lib +.DS_Store +Thumbs.db +*.swp +*.swo +*~ +.tk-local/ + +# APERC session substrate (local-only) +/APERC/ + +# local credentials +/.secrets/ diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..b89a48b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,187 @@ +# Trusty -- Claude Code Context + +## What This Is +Trusty is a pure-Rust provenance system built around the proprietary +chain-break-as-event primitive. It was originally ported from a Python reference +(archived at `~/Downloads/4-5/tkg_complete/tkg/trusty_lib/...`), but that +codebase is historical context only -- the active system is 100% Rust and +the locked v1 byte-shape is pinned by invariant tests in `trusty-core`. +Owner: MM Advisory LLC / Eric Gottlieb. Proprietary license. + +Three crates: +- **`trusty-core`** -- pure library. Dual hash (SipHash-2-4 + SHA-256), chain primitives, payload builder, verify_chain, chain-break-as-event. No storage dependency. +- **`trusty-sqlite`** -- rusqlite adapter. Three-table schema (`provenance_envelope`, `hash_index`, `chain_tips`) with KV cross-check. `Trusty::{open, append, verify, anchor, history, integrity}`. +- **`trusty-sqlite-ext`** -- loadable SQLite extension (`cdylib`). `sqlite3_trusty_init()` exposes SQL functions (`trusty_append`, `trusty_verify`, `trusty_anchor`). + +## Design Rules +- **v1 byte-shape is locked.** Any change to hash inputs, chain payload format, or anchor semantics must leave the invariant tests green (`s5_invariant_v1_byte_shape_preserved`, `v1_*` payload/chain tests, conformance crate). Do not "simplify" or "optimize" the canonical primitives. +- **Pure Rust.** No Python dependency at runtime. No shelling out. The historical Python port-source is archival only. +- **ASCII-only comments.** Cross-platform patch safety. +- **Functional style where it fits.** Immutable data as design principle. The chain is append-only by definition. +- **Dual hash is non-negotiable.** SipHash-2-4 (keyed fingerprint) + SHA-256 (cryptographic anchor). Do not collapse to single hash. +- **Chain-break-as-event** is the proprietary primitive. Treat as core IP. Do not document its semantics in README-style docs meant for public consumption. +- **Trusty is a library, not a platform.** The product is audit-trail-as-a-service (cheap, low-resource, high-margin commodity; door-opener for IoT/sensor/compliance buyers). Application-layer work (digital twin, KCG, NNsight swarm, agent-mesh) lives in sibling crates/repos that depend on `trusty-core` -- same pattern as `legal-chaingraph/` and `thread-keeper-tauri/`. Do not bundle application logic into the locked crates. When evaluating a feature, prefer those that improve the service surface (throughput, verifier ergonomics, retention, domain separation, bundle contracts) over those that expand trusty into application territory. + +## Don't Touch (locked) +- `trusty-core` chain primitives (`append`, `verify_chain`, payload builder). Canonical port. +- `trusty-core` hash functions. Byte-identical to Python reference. +- v1 payload canonical bytes. `compute_record`, `PayloadFields::new_v1`, and the `build_payload` top-level ordering are all v1-stable. v2 is purely additive (`compute_record_signed`, `PAYLOAD_VERSION_V2`, populated `signatures` field via the signing-view pattern). Any v2+ work MUST preserve v1 byte-shape exactly -- see `s5_invariant_v1_byte_shape_preserved` tag and `memory/project_trusty_v2_signing_design.md`. +- Three-table SQLite schema in `trusty-sqlite` -- `provenance_envelope`, `hash_index`, `chain_tips`. Do not collapse, do not drop the KV cross-check. +- Historical Python port-source (`~/Downloads/4-5/tkg_complete/tkg/trusty_lib/...`) -- archival only; the Rust invariant tests are the active gate. + +## APERC Session Discipline + +Every Claude Code session in this repo follows the APERC loop. No exceptions. + +### On session start +1. Read this `CLAUDE.md`. +2. Read `APERC/boot/S_BOOT.md` for the current session's scope (currently `S1_BOOT.md` for the first real session after bootstrap). +3. Read `memory/MEMORY.md` index (always in context) and whichever referenced files are relevant to today's goal. +4. Run a baseline tag count: `tk_kb stats --aperc=APERC` (or count `APERC/tags/*.toml` directly). +5. **Cleanup doublecheck**: quick pass on whatever the prior session's close ritual should have tidied -- stale boot stubs, handoff naming drift, open_audits that should've been closed. If you find something, flag it as the first cycle (hygiene) before the real work. +6. State: session ID, baseline tags, goal, files you plan to touch. +7. Wait for Eric's go before any edit. + +### During work +- **Max 5 file edits per cycle.** After 5: stop, show what changed, explain what you did, present plan for next cycle, WAIT for approval. +- Never read large files whole. Use Grep, Glob, offset/limit Reads, head/tail on single files. +- If you touch a file not in your plan, stop and re-plan. +- Commit green states before risky operations (test run, DB migration, hash-primitive edit). + +### Before finishing + +**Pre-close cleanup pass** (runs before the mechanical close ritual below). Eric's standing request: cleanup is part of close, not an afterthought. Sweep for: +- Stale references in `APERC/boot/S*_BOOT.md` (e.g., auto-stub paths from other projects). +- Handoff block naming drift in `APERC/state/LATEST_HANDOFF.toml`. +- Open_audits lines that got resolved this session -- move to `[closed_audits]` so future boots don't re-read stale noise. +- Memory entries (`MEMORY.md` + referenced files) that need updates for the work you just shipped. +- CLAUDE.md design-invariant notes for new locked surfaces (e.g., v2 additive-only rule). +- Any new session-scope `DELTA_LOG_S*.toml` you wrote as a KB write-ahead log. + +Do cleanup at end of session, AND double-check at start of next session as a belt-and-braces pass -- sometimes a cleanup item only becomes obvious on re-read. + +**Mechanical close**: +- List every file you changed. +- List every decision you made and why. +- Tag findings as `[LEARN]`, `[MISTAKE]`, `[INVARIANT]`, `[DECISION]`, `[INSIGHT]` -- filed to `APERC/tags/` as individual `*.toml` files. +- Run tests if you touched any Rust file. Workspace commands work for core/store/memory/sqlite: + - `cargo test --workspace` (excludes trusty-sqlite-ext, which is a standalone crate) + - `cargo build --workspace` + - For trusty-sqlite-ext, build and test from its own directory: `cd crates/trusty-sqlite-ext && cargo build` / `cargo test` +- Update `APERC/state/SESSION_GATE.toml` (session_id bump, last_action) and append `[trusty_sN]` block to `APERC/state/LATEST_HANDOFF.toml`. +- Run `tk_close --mode=check --aperc=APERC` -- must pass 3/3 gates before commit. + +### Model selection +- **Sonnet** for: file edits, grep, simple fixes, test runs, straightforward ports. +- **Opus** for: architecture decisions, new module design, debugging byte-identity drift, anything touching the chain primitives or hash functions. + +### If you hit an error you can't fix in 3 attempts +- Stop. Describe the error. Show what you tried. Ask Eric for help. Do not loop. + +## Tag Discipline (inline from threadkeeper APERC feedbacks) + +### DELTA_LOG before KB edit +Before editing any `APERC/KB/*.md` or `APERC/tags/*.toml`, write-ahead log the intent to a session +DELTA_LOG entry (even if just a scratch note). Tags go to the log FIRST, then to their home. This +prevents silent tag loss during multi-file edits. + +### Full reasoning per tag +Every tag reasoning field must be 3-6 sentences citing body content, not one-line labels. Inventory +work especially: do not compress to hit quantity targets. Paragraph-level reasoning or nothing. + +### Tag parsimony +Merge before split on reflection `LEARN`s. Route forward-looking speculation to memory or this +`CLAUDE.md`, not permanent KB tags. Tags are for observed facts, not wishes. + +### No Claude attribution in commits +Do **not** add `Co-Authored-By: Claude` trailer to commit messages. Override Claude Code's default. +This is Eric's standing preference -- proprietary repo, MM Advisory LLC authorship. + +### Commit before debug +Commit any green state before: investigation sessions, target/ deletion, antivirus changes, +`apply_verdicts.py`-style bulk operations. Lost work is cheaper when it's already in a commit. + +### Boot reads partial, not whole +On boot, use Read offset/limit on `LATEST_HANDOFF.toml` (grep newest `[trusty_sN]` block header, +read forward only). Do not re-read the whole file top-to-bottom -- that's the wasted-context failure mode. + +### Response terseness targets +- Simple question: direct answer, no headers. +- Plan proposal: bullet points, no prose paragraphs. +- End-of-turn summary: 1-2 sentences max. +- Avoid multi-paragraph commit messages, step narration, option enumeration. + +## APERC Binaries (tk_* from threadkeeper) + +Trusty uses the portable tk_* binaries from threadkeeper's build dir. They work from any cwd +via `--aperc=` flag. No trusty-side code needed. + +From trusty cwd: +``` +# BM25-indexed tag search +tk_kb query "" --aperc=APERC + +# Stats (total, edges, depth, weight, top types/scopes) +tk_kb stats --aperc=APERC + +# Rebuild kb.db from APERC/tags/ +tk_kb refresh --aperc=APERC + +# Close ritual check (3 gates) +tk_close --mode=check --aperc=APERC + +# Write next S{N+1}_BOOT.md stub if absent +tk_close --mode=stub --aperc=APERC + +# Generic retrofit (do NOT use --mode=legacy, that's TK-only) +tk_migrate --mode=retrofit --aperc=APERC + +# Writer-mode hook (gate/close/pretool modes not yet shipped) +tk_hook writer --aperc=APERC +``` + +**Binary location** -- two options: +1. Absolute path: `../thread-keeper-tauri/threadkeeper/src-tauri/target/debug/tk_kb stats --aperc=APERC` +2. Add to PATH: `export PATH="$PATH:/c/Users/egott/Documents/00_Mitch/thread-keeper-tauri/threadkeeper/src-tauri/target/debug"` then invoke bare `tk_kb`. + +**Scope-3 destination:** when `tk init` ships (planned dedicated session, see threadkeeper +`memory/project_tk_substrate_vision.md`), trusty will get its own installed `tk` binary and the +binary-path documentation above becomes obsolete. + +## Memory System +Memory lives at `C:\Users\egott\.claude\projects\C--Users-egott-Documents-00-Mitch-trusty\memory\`. +`MEMORY.md` is always in context (capped to ~200 lines). Individual memory files are loaded on demand. + +**What belongs in memory (not here):** +- User profile, role, preferences (user_eric.md) +- Per-project state (project_trusty.md) +- Session-to-session feedback that outgrows this file +- External-system pointers (reference_*.md) + +**What belongs here (not memory):** +- Stable project rules (design invariants, locked files, build commands) +- The APERC discipline itself (copied above for every new session to read at boot) +- Binary invocation recipes + +## Sibling Projects +- `thread-keeper-tauri/threadkeeper/` -- the desktop app this substrate was extracted from. Source of tk_* binaries until scope-3 ships. +- `legal-chaingraph/` -- KCG v4 + CH/IR legal graph work. First proof-of-portability (bootstrapped 2026-04-14, commit 46011ff). + +## Commit Style +- Short subject line (< 70 chars), imperative. +- Body: why > what. 1-3 bullets if needed. +- **No Claude attribution trailer.** +- Reference session ID in subject when applicable: `TRUSTY-S1: `. + +## Cross-session pins (kb-mesh option 1) + +In-flight memory shared across parallel Claude Code sessions lives at `../.kb-mesh/pins/`. Pull-model — nothing is auto-injected. Pins are ephemeral (not committed), complementary to APERC tags (durable, committed). + +```bash +python ../.kb-mesh/scripts/kb_pin.py # pin last turn +python ../.kb-mesh/scripts/kb_pin.py --last 3 # pin last 3 turns +python ../.kb-mesh/scripts/kb_recall.py # pull a pin into current context +python ../.kb-mesh/scripts/kb_ls.py # list pins +``` + +See `../.kb-mesh/README.md` for spec. diff --git a/Cargo.lock b/Cargo.lock new file mode 100644 index 0000000..36d2c23 --- /dev/null +++ b/Cargo.lock @@ -0,0 +1,1745 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 3 + +[[package]] +name = "ahash" +version = "0.8.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" +dependencies = [ + "cfg-if", + "once_cell", + "version_check", + "zerocopy", +] + +[[package]] +name = "android_system_properties" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311" +dependencies = [ + "libc", +] + +[[package]] +name = "anyhow" +version = "1.0.102" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c" + +[[package]] +name = "async-trait" +version = "0.1.89" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9035ad2d096bed7955a320ee7e2230574d28fd3c3a0f186cbea1ff3c7eed5dbb" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "autocfg" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8" + +[[package]] +name = "base64" +version = "0.22.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" + +[[package]] +name = "base64ct" +version = "1.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" + +[[package]] +name = "bitflags" +version = "2.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "843867be96c8daad0d758b57df9392b6d8d271134fce549de6ce169ff98a92af" + +[[package]] +name = "block-buffer" +version = "0.10.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" +dependencies = [ + "generic-array", +] + +[[package]] +name = "block-buffer" +version = "0.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cdd35008169921d80bc60d3d0ab416eecb028c4cd653352907921d95084790be" +dependencies = [ + "hybrid-array", +] + +[[package]] +name = "bumpalo" +version = "3.20.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d20789868f4b01b2f2caec9f5c4e0213b41e3e5702a50157d699ae31ced2fcb" + +[[package]] +name = "byteorder" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" + +[[package]] +name = "bytes" +version = "1.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33" + +[[package]] +name = "cc" +version = "1.2.59" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7a4d3ec6524d28a329fc53654bbadc9bdd7b0431f5d65f1a56ffb28a1ee5283" +dependencies = [ + "find-msvc-tools", + "shlex", +] + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "chacha20" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6f8d983286843e49675a4b7a2d174efe136dc93a18d69130dd18198a6c167601" +dependencies = [ + "cfg-if", + "cpufeatures 0.3.0", + "rand_core 0.10.1", +] + +[[package]] +name = "chrono" +version = "0.4.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c673075a2e0e5f4a1dde27ce9dee1ea4558c7ffe648f576438a20ca1d2acc4b0" +dependencies = [ + "iana-time-zone", + "num-traits", + "serde", + "windows-link 0.2.1", +] + +[[package]] +name = "cmov" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f88a43d011fc4a6876cb7344703e297c71dda42494fee094d5f7c76bf13f746" + +[[package]] +name = "const-oid" +version = "0.9.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" + +[[package]] +name = "const-oid" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" + +[[package]] +name = "core-foundation-sys" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" + +[[package]] +name = "cpufeatures" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +dependencies = [ + "libc", +] + +[[package]] +name = "cpufeatures" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b2a41393f66f16b0823bb79094d54ac5fbd34ab292ddafb9a0456ac9f87d201" +dependencies = [ + "libc", +] + +[[package]] +name = "crypto-common" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" +dependencies = [ + "generic-array", + "typenum", +] + +[[package]] +name = "crypto-common" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77727bb15fa921304124b128af125e7e3b968275d1b108b379190264f4423710" +dependencies = [ + "hybrid-array", +] + +[[package]] +name = "ctutils" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e" +dependencies = [ + "cmov", +] + +[[package]] +name = "curve25519-dalek" +version = "4.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be" +dependencies = [ + "cfg-if", + "cpufeatures 0.2.17", + "curve25519-dalek-derive", + "digest 0.10.7", + "fiat-crypto", + "rustc_version", + "subtle", + "zeroize", +] + +[[package]] +name = "curve25519-dalek-derive" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "der" +version = "0.7.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" +dependencies = [ + "const-oid 0.9.6", + "zeroize", +] + +[[package]] +name = "digest" +version = "0.10.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" +dependencies = [ + "block-buffer 0.10.4", + "crypto-common 0.1.7", +] + +[[package]] +name = "digest" +version = "0.11.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4850db49bf08e663084f7fb5c87d202ef91a3907271aff24a94eb97ff039153c" +dependencies = [ + "block-buffer 0.12.0", + "const-oid 0.10.2", + "crypto-common 0.2.1", + "ctutils", +] + +[[package]] +name = "ed25519" +version = "2.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53" +dependencies = [ + "pkcs8", + "signature", +] + +[[package]] +name = "ed25519-dalek" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9" +dependencies = [ + "curve25519-dalek", + "ed25519", + "rand_core 0.6.4", + "serde", + "sha2 0.10.9", + "subtle", + "zeroize", +] + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "fallible-iterator" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4443176a9f2c162692bd3d352d745ef9413eec5782a80d8fd6f8a1ac692a07f7" + +[[package]] +name = "fallible-iterator" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2acce4a10f12dc2fb14a218589d4f1f62ef011b2d0cc4b3cb1bba8e94da14649" + +[[package]] +name = "fallible-streaming-iterator" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7360491ce676a36bf9bb3c56c1aa791658183a54d2744120f27285738d90465a" + +[[package]] +name = "fastrand" +version = "2.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f1f227452a390804cdb637b74a86990f2a7d7ba4b7d5693aac9b4dd6defd8d6" + +[[package]] +name = "fiat-crypto" +version = "0.2.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d" + +[[package]] +name = "find-msvc-tools" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582" + +[[package]] +name = "foldhash" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2" + +[[package]] +name = "futures-channel" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d" +dependencies = [ + "futures-core", + "futures-sink", +] + +[[package]] +name = "futures-core" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d" + +[[package]] +name = "futures-sink" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893" + +[[package]] +name = "futures-task" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393" + +[[package]] +name = "futures-util" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6" +dependencies = [ + "futures-core", + "futures-sink", + "futures-task", + "pin-project-lite", +] + +[[package]] +name = "generic-array" +version = "0.14.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +dependencies = [ + "typenum", + "version_check", +] + +[[package]] +name = "getrandom" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" +dependencies = [ + "cfg-if", + "libc", + "wasi 0.11.1+wasi-snapshot-preview1", +] + +[[package]] +name = "getrandom" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555" +dependencies = [ + "cfg-if", + "libc", + "r-efi", + "rand_core 0.10.1", + "wasip2", + "wasip3", +] + +[[package]] +name = "hashbrown" +version = "0.14.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" +dependencies = [ + "ahash", +] + +[[package]] +name = "hashbrown" +version = "0.15.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1" +dependencies = [ + "foldhash", +] + +[[package]] +name = "hashbrown" +version = "0.16.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100" + +[[package]] +name = "hashlink" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ba4ff7128dee98c7dc9794b6a411377e1404dba1c97deb8d1a55297bd25d8af" +dependencies = [ + "hashbrown 0.14.5", +] + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "hex" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" + +[[package]] +name = "hmac" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6303bc9732ae41b04cb554b844a762b4115a61bfaa81e3e83050991eeb56863f" +dependencies = [ + "digest 0.11.2", +] + +[[package]] +name = "hybrid-array" +version = "0.4.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3944cf8cf766b40e2a1a333ee5e9b563f854d5fa49d6a8ca2764e97c6eddb214" +dependencies = [ + "typenum", +] + +[[package]] +name = "iana-time-zone" +version = "0.1.65" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470" +dependencies = [ + "android_system_properties", + "core-foundation-sys", + "iana-time-zone-haiku", + "js-sys", + "log", + "wasm-bindgen", + "windows-core", +] + +[[package]] +name = "iana-time-zone-haiku" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f" +dependencies = [ + "cc", +] + +[[package]] +name = "id-arena" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954" + +[[package]] +name = "indexmap" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "45a8a2b9cb3e0b0c1803dbb0758ffac5de2f425b23c28f518faabd9d805342ff" +dependencies = [ + "equivalent", + "hashbrown 0.16.1", + "serde", + "serde_core", +] + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "js-sys" +version = "0.3.95" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2964e92d1d9dc3364cae4d718d93f227e3abb088e747d92e0395bfdedf1c12ca" +dependencies = [ + "once_cell", + "wasm-bindgen", +] + +[[package]] +name = "leb128fmt" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2" + +[[package]] +name = "libc" +version = "0.2.184" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "48f5d2a454e16a5ea0f4ced81bd44e4cfc7bd3a507b61887c99fd3538b28e4af" + +[[package]] +name = "libredox" +version = "0.1.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e02f3bb43d335493c96bf3fd3a321600bf6bd07ed34bc64118e9293bdffea46c" +dependencies = [ + "libc", +] + +[[package]] +name = "libsqlite3-sys" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c10584274047cb335c23d3e61bcef8e323adae7c5c8c760540f73610177fc3f" +dependencies = [ + "cc", + "pkg-config", + "vcpkg", +] + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "lock_api" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" +dependencies = [ + "scopeguard", +] + +[[package]] +name = "log" +version = "0.4.29" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897" + +[[package]] +name = "md-5" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69b6441f590336821bb897fb28fc622898ccceb1d6cea3fde5ea86b090c4de98" +dependencies = [ + "cfg-if", + "digest 0.11.2", +] + +[[package]] +name = "memchr" +version = "2.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79" + +[[package]] +name = "mio" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "50b7e5b27aa02a74bac8c3f23f448f8d87ff11f92d3aac1a6ed369ee08cc56c1" +dependencies = [ + "libc", + "wasi 0.11.1+wasi-snapshot-preview1", + "windows-sys", +] + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + +[[package]] +name = "objc2-core-foundation" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536" +dependencies = [ + "bitflags", +] + +[[package]] +name = "objc2-system-configuration" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7216bd11cbda54ccabcab84d523dc93b858ec75ecfb3a7d89513fa22464da396" +dependencies = [ + "objc2-core-foundation", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "parking_lot" +version = "0.12.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" +dependencies = [ + "lock_api", + "parking_lot_core", +] + +[[package]] +name = "parking_lot_core" +version = "0.9.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall", + "smallvec", + "windows-link 0.2.1", +] + +[[package]] +name = "percent-encoding" +version = "2.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" + +[[package]] +name = "phf" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c1562dc717473dbaa4c1f85a36410e03c047b2e7df7f45ee938fbef64ae7fadf" +dependencies = [ + "phf_shared", + "serde", +] + +[[package]] +name = "phf_shared" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e57fef6bc5981e38c2ce2d63bfa546861309f875b8a75f092d1d54ae2d64f266" +dependencies = [ + "siphasher", +] + +[[package]] +name = "pin-project-lite" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" + +[[package]] +name = "pkcs8" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7" +dependencies = [ + "der", + "spki", +] + +[[package]] +name = "pkg-config" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7edddbd0b52d732b21ad9a5fab5c704c14cd949e5e9a1ec5929a24fded1b904c" + +[[package]] +name = "postgres" +version = "0.19.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aacf632d0554ff75f58183694f41dc8999c8a3a43a386994d0ec2d034f1dfbe1" +dependencies = [ + "bytes", + "fallible-iterator 0.2.0", + "futures-util", + "log", + "tokio", + "tokio-postgres", +] + +[[package]] +name = "postgres-protocol" +version = "0.6.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "56201207dac53e2f38e848e31b4b91616a6bb6e0c7205b77718994a7f49e70fc" +dependencies = [ + "base64", + "byteorder", + "bytes", + "fallible-iterator 0.2.0", + "hmac", + "md-5", + "memchr", + "rand 0.10.1", + "sha2 0.11.0", + "stringprep", +] + +[[package]] +name = "postgres-types" +version = "0.2.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8dc729a129e682e8d24170cd30ae1aa01b336b096cbb56df6d534ffec133d186" +dependencies = [ + "bytes", + "fallible-iterator 0.2.0", + "postgres-protocol", +] + +[[package]] +name = "ppv-lite86" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" +dependencies = [ + "zerocopy", +] + +[[package]] +name = "prettyplease" +version = "0.2.37" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b" +dependencies = [ + "proc-macro2", + "syn 2.0.117", +] + +[[package]] +name = "proc-macro2" +version = "1.0.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "rand" +version = "0.8.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ca0ecfa931c29007047d1bc58e623ab12e5590e8c7cc53200d5202b69266d8a" +dependencies = [ + "libc", + "rand_chacha", + "rand_core 0.6.4", +] + +[[package]] +name = "rand" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2e8e8bcc7961af1fdac401278c6a831614941f6164ee3bf4ce61b7edb162207" +dependencies = [ + "chacha20", + "getrandom 0.4.2", + "rand_core 0.10.1", +] + +[[package]] +name = "rand_chacha" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +dependencies = [ + "ppv-lite86", + "rand_core 0.6.4", +] + +[[package]] +name = "rand_core" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" +dependencies = [ + "getrandom 0.2.17", +] + +[[package]] +name = "rand_core" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" + +[[package]] +name = "redox_syscall" +version = "0.5.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" +dependencies = [ + "bitflags", +] + +[[package]] +name = "rusqlite" +version = "0.31.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b838eba278d213a8beaf485bd313fd580ca4505a00d5871caeb1457c55322cae" +dependencies = [ + "bitflags", + "fallible-iterator 0.3.0", + "fallible-streaming-iterator", + "hashlink", + "libsqlite3-sys", + "smallvec", +] + +[[package]] +name = "rustc_version" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92" +dependencies = [ + "semver", +] + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "rustversion" +version = "1.0.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d" + +[[package]] +name = "scopeguard" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" + +[[package]] +name = "semver" +version = "1.0.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" + +[[package]] +name = "serde" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "serde_json" +version = "1.0.149" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86" +dependencies = [ + "indexmap", + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "sha2" +version = "0.10.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" +dependencies = [ + "cfg-if", + "cpufeatures 0.2.17", + "digest 0.10.7", +] + +[[package]] +name = "sha2" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4" +dependencies = [ + "cfg-if", + "cpufeatures 0.3.0", + "digest 0.11.2", +] + +[[package]] +name = "shlex" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64" + +[[package]] +name = "signature" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de" +dependencies = [ + "rand_core 0.6.4", +] + +[[package]] +name = "siphasher" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2aa850e253778c88a04c3d7323b043aeda9d3e30d5971937c1855769763678e" + +[[package]] +name = "smallvec" +version = "1.15.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03" + +[[package]] +name = "socket2" +version = "0.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3a766e1110788c36f4fa1c2b71b387a7815aa65f88ce0229841826633d93723e" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "spki" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d" +dependencies = [ + "base64ct", + "der", +] + +[[package]] +name = "stringprep" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b4df3d392d81bd458a8a621b8bffbd2302a12ffe288a9d931670948749463b1" +dependencies = [ + "unicode-bidi", + "unicode-normalization", + "unicode-properties", +] + +[[package]] +name = "subtle" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292" + +[[package]] +name = "syn" +version = "2.0.117" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "3.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53e9bae58849f64dfa4f5d5ae372c8341f7305f82a3868709269343628b659a3" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "tempfile" +version = "3.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" +dependencies = [ + "fastrand", + "getrandom 0.4.2", + "once_cell", + "rustix", + "windows-sys", +] + +[[package]] +name = "thiserror" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "tinyvec" +version = "1.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3e61e67053d25a4e82c844e8424039d9745781b3fc4f32b8d55ed50f5f667ef3" +dependencies = [ + "tinyvec_macros", +] + +[[package]] +name = "tinyvec_macros" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20" + +[[package]] +name = "tokio" +version = "1.52.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b67dee974fe86fd92cc45b7a95fdd2f99a36a6d7b0d431a231178d3d670bbcc6" +dependencies = [ + "bytes", + "libc", + "mio", + "pin-project-lite", + "socket2", + "tokio-macros", + "windows-sys", +] + +[[package]] +name = "tokio-macros" +version = "2.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "tokio-postgres" +version = "0.7.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4dd8df5ef180f6364759a6f00f7aadda4fbbac86cdee37480826a6ff9f3574ce" +dependencies = [ + "async-trait", + "byteorder", + "bytes", + "fallible-iterator 0.2.0", + "futures-channel", + "futures-util", + "log", + "parking_lot", + "percent-encoding", + "phf", + "pin-project-lite", + "postgres-protocol", + "postgres-types", + "rand 0.10.1", + "socket2", + "tokio", + "tokio-util", + "whoami", +] + +[[package]] +name = "tokio-util" +version = "0.7.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ae9cec805b01e8fc3fd2fe289f89149a9b66dd16786abd8b19cfa7b48cb0098" +dependencies = [ + "bytes", + "futures-core", + "futures-sink", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "pin-project-lite", + "tracing-attributes", + "tracing-core", +] + +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "trusty-core" +version = "0.1.0" +dependencies = [ + "ed25519-dalek", + "hex", + "rand 0.8.6", + "serde", + "serde_json", + "sha2 0.10.9", + "siphasher", + "thiserror", + "zeroize", +] + +[[package]] +name = "trusty-memory" +version = "0.1.0" +dependencies = [ + "trusty-store", +] + +[[package]] +name = "trusty-policy" +version = "0.1.0" +dependencies = [ + "serde", + "serde_json", + "thiserror", +] + +[[package]] +name = "trusty-policy-audit" +version = "0.1.0" +dependencies = [ + "serde", + "serde_json", + "tempfile", + "trusty-core", + "trusty-policy", + "trusty-sqlite", +] + +[[package]] +name = "trusty-policy-conditions" +version = "0.1.0" +dependencies = [ + "serde_json", + "sha2 0.10.9", + "trusty-core", + "trusty-policy", + "trusty-sqlite", +] + +[[package]] +name = "trusty-policy-postgres" +version = "0.1.0" +dependencies = [ + "hex", + "postgres", + "serde_json", + "sha2 0.10.9", + "thiserror", + "trusty-policy", +] + +[[package]] +name = "trusty-postgres" +version = "0.1.0" +dependencies = [ + "chrono", + "hex", + "postgres", + "serde", + "serde_json", + "thiserror", + "tracing", + "trusty-core", + "trusty-store", +] + +[[package]] +name = "trusty-sqlite" +version = "0.1.0" +dependencies = [ + "chrono", + "hex", + "rusqlite", + "serde", + "serde_json", + "thiserror", + "trusty-core", + "trusty-store", +] + +[[package]] +name = "trusty-store" +version = "0.1.0" +dependencies = [ + "serde", + "serde_json", + "thiserror", + "trusty-core", +] + +[[package]] +name = "trusty-tokio" +version = "0.1.0" +dependencies = [ + "tokio", + "trusty-store", +] + +[[package]] +name = "typenum" +version = "1.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "562d481066bde0658276a35467c4af00bdc6ee726305698a55b86e61d7ad82bb" + +[[package]] +name = "unicode-bidi" +version = "0.3.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c1cb5db39152898a79168971543b1cb5020dff7fe43c8dc468b0885f5e29df5" + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "unicode-normalization" +version = "0.1.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5fd4f6878c9cb28d874b009da9e8d183b5abc80117c40bbd187a1fde336be6e8" +dependencies = [ + "tinyvec", +] + +[[package]] +name = "unicode-properties" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7df058c713841ad818f1dc5d3fd88063241cc61f49f5fbea4b951e8cf5a8d71d" + +[[package]] +name = "unicode-xid" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853" + +[[package]] +name = "vcpkg" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426" + +[[package]] +name = "version_check" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "wasi" +version = "0.14.7+wasi-0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "883478de20367e224c0090af9cf5f9fa85bed63a95c1abf3afc5c083ebc06e8c" +dependencies = [ + "wasip2", +] + +[[package]] +name = "wasip2" +version = "1.0.3+wasi-0.2.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "20064672db26d7cdc89c7798c48a0fdfac8213434a1186e5ef29fd560ae223d6" +dependencies = [ + "wit-bindgen 0.57.1", +] + +[[package]] +name = "wasip3" +version = "0.4.0+wasi-0.3.0-rc-2026-01-06" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5428f8bf88ea5ddc08faddef2ac4a67e390b88186c703ce6dbd955e1c145aca5" +dependencies = [ + "wit-bindgen 0.51.0", +] + +[[package]] +name = "wasite" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "66fe902b4a6b8028a753d5424909b764ccf79b7a209eac9bf97e59cda9f71a42" +dependencies = [ + "wasi 0.14.7+wasi-0.2.4", +] + +[[package]] +name = "wasm-bindgen" +version = "0.2.118" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0bf938a0bacb0469e83c1e148908bd7d5a6010354cf4fb73279b7447422e3a89" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.118" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eeff24f84126c0ec2db7a449f0c2ec963c6a49efe0698c4242929da037ca28ed" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.118" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d08065faf983b2b80a79fd87d8254c409281cf7de75fc4b773019824196c904" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 2.0.117", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.118" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5fd04d9e306f1907bd13c6361b5c6bfc7b3b3c095ed3f8a9246390f8dbdee129" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "wasm-encoder" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "990065f2fe63003fe337b932cfb5e3b80e0b4d0f5ff650e6985b1048f62c8319" +dependencies = [ + "leb128fmt", + "wasmparser", +] + +[[package]] +name = "wasm-metadata" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bb0e353e6a2fbdc176932bbaab493762eb1255a7900fe0fea1a2f96c296cc909" +dependencies = [ + "anyhow", + "indexmap", + "wasm-encoder", + "wasmparser", +] + +[[package]] +name = "wasmparser" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe" +dependencies = [ + "bitflags", + "hashbrown 0.15.5", + "indexmap", + "semver", +] + +[[package]] +name = "web-sys" +version = "0.3.95" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4f2dfbb17949fa2088e5d39408c48368947b86f7834484e87b73de55bc14d97d" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "whoami" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6a5b12f9df4f978d2cfdb1bd3bac52433f44393342d7ee9c25f5a1c14c0f45d" +dependencies = [ + "libc", + "libredox", + "objc2-system-configuration", + "wasite", + "web-sys", +] + +[[package]] +name = "windows-core" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c0fdd3ddb90610c7638aa2b3a3ab2904fb9e5cdbecc643ddb3647212781c4ae3" +dependencies = [ + "windows-implement", + "windows-interface", + "windows-link 0.1.3", + "windows-result", + "windows-strings", +] + +[[package]] +name = "windows-implement" +version = "0.60.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "windows-interface" +version = "0.59.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "windows-link" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e6ad25900d524eaabdbbb96d20b4311e1e7ae1699af4fb28c17ae66c80d798a" + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-result" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "56f42bd332cc6c8eac5af113fc0c1fd6a8fd2aa08a0119358686e5160d0586c6" +dependencies = [ + "windows-link 0.1.3", +] + +[[package]] +name = "windows-strings" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "56e6c93f3a0c3b36176cb1327a4958a0353d5d166c2a35cb268ace15e91d3b57" +dependencies = [ + "windows-link 0.1.3", +] + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link 0.2.1", +] + +[[package]] +name = "wit-bindgen" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7249219f66ced02969388cf2bb044a09756a083d0fab1e566056b04d9fbcaa5" +dependencies = [ + "wit-bindgen-rust-macro", +] + +[[package]] +name = "wit-bindgen" +version = "0.57.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e" + +[[package]] +name = "wit-bindgen-core" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ea61de684c3ea68cb082b7a88508a8b27fcc8b797d738bfc99a82facf1d752dc" +dependencies = [ + "anyhow", + "heck", + "wit-parser", +] + +[[package]] +name = "wit-bindgen-rust" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7c566e0f4b284dd6561c786d9cb0142da491f46a9fbed79ea69cdad5db17f21" +dependencies = [ + "anyhow", + "heck", + "indexmap", + "prettyplease", + "syn 2.0.117", + "wasm-metadata", + "wit-bindgen-core", + "wit-component", +] + +[[package]] +name = "wit-bindgen-rust-macro" +version = "0.51.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c0f9bfd77e6a48eccf51359e3ae77140a7f50b1e2ebfe62422d8afdaffab17a" +dependencies = [ + "anyhow", + "prettyplease", + "proc-macro2", + "quote", + "syn 2.0.117", + "wit-bindgen-core", + "wit-bindgen-rust", +] + +[[package]] +name = "wit-component" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2" +dependencies = [ + "anyhow", + "bitflags", + "indexmap", + "log", + "serde", + "serde_derive", + "serde_json", + "wasm-encoder", + "wasm-metadata", + "wasmparser", + "wit-parser", +] + +[[package]] +name = "wit-parser" +version = "0.244.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ecc8ac4bc1dc3381b7f59c34f00b67e18f910c2c0f50015669dde7def656a736" +dependencies = [ + "anyhow", + "id-arena", + "indexmap", + "log", + "semver", + "serde", + "serde_derive", + "serde_json", + "unicode-xid", + "wasmparser", +] + +[[package]] +name = "zerocopy" +version = "0.8.48" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eed437bf9d6692032087e337407a86f04cd8d6a16a37199ed57949d415bd68e9" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.48" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70e3cd084b1788766f53af483dd21f93881ff30d7320490ec3ef7526d203bad4" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "zeroize" +version = "1.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0" +dependencies = [ + "zeroize_derive", +] + +[[package]] +name = "zeroize_derive" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3c50655cbb0fe3fc43170059e702f1ce5e19b84cec58dc87b037a09935c2f328" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + +[[package]] +name = "zmij" +version = "1.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa" diff --git a/Cargo.toml b/Cargo.toml new file mode 100644 index 0000000..ae2c2f1 --- /dev/null +++ b/Cargo.toml @@ -0,0 +1,62 @@ +[workspace] +resolver = "2" +members = [ + # core primitives + interface + "crates/core/trusty-core", + "crates/core/trusty-store", + "crates/core/trusty-policy", + # storage backends + "crates/adapters/trusty-memory", + "crates/adapters/trusty-sqlite", + "crates/adapters/trusty-postgres", + "crates/adapters/trusty-tokio", + # policy <-> chain wiring + "crates/adapters/trusty-policy-audit", + "crates/adapters/trusty-policy-postgres", + "crates/adapters/trusty-policy-conditions", +] + +[workspace.package] +version = "0.1.0" +edition = "2021" +authors = ["MM Advisory LLC / Eric Gottlieb"] +license = "LicenseRef-Proprietary" +repository = "https://github.com/mmadvisory/trusty" +rust-version = "1.75" +# Published only to the PRIVATE Modo-Melius registry on this machine. +# The crates-io default is still refused: see the registry allowlist below. +publish = ["modo-melius"] + +[workspace.dependencies] +trusty-core = { path = "crates/core/trusty-core", version = "0.1.0" } +trusty-store = { path = "crates/core/trusty-store", version = "0.1.0" } +trusty-policy = { path = "crates/core/trusty-policy", version = "0.1.0" } +trusty-memory = { path = "crates/adapters/trusty-memory", version = "0.1.0" } +trusty-sqlite = { path = "crates/adapters/trusty-sqlite", version = "0.1.0" } +trusty-postgres = { path = "crates/adapters/trusty-postgres", version = "0.1.0" } +trusty-tokio = { path = "crates/adapters/trusty-tokio", version = "0.1.0" } +trusty-policy-audit = { path = "crates/adapters/trusty-policy-audit", version = "0.1.0" } +trusty-policy-postgres = { path = "crates/adapters/trusty-policy-postgres", version = "0.1.0" } +trusty-policy-conditions = { path = "crates/adapters/trusty-policy-conditions", version = "0.1.0" } + +serde = { version = "=1.0.228", features = ["derive"] } +serde_json = { version = "=1.0.149", features = ["preserve_order"] } +sha2 = "0.10" +siphasher = "1" +zeroize = { version = "1", features = ["zeroize_derive"] } +hex = "0.4" +rusqlite = { version = "0.31", features = ["bundled"] } +chrono = { version = "0.4", default-features = false, features = ["clock", "serde"] } +thiserror = "1" +ed25519-dalek = { version = "2", features = ["std", "rand_core"] } +rand = "0.8" +postgres = "0.19" +tokio = { version = "1", default-features = false } +tracing = "0.1" +toml = "0.8" + +[profile.release] +lto = "thin" +codegen-units = 1 +opt-level = 3 +strip = "symbols" diff --git a/README.md b/README.md new file mode 100644 index 0000000..d5c25f2 --- /dev/null +++ b/README.md @@ -0,0 +1,74 @@ +# Trusty + +Tamper-evident data provenance chain library. Pure Rust. Proprietary. +MM Advisory LLC / Eric Gottlieb. + +## Crates + +- **`trusty-core`** — pure library. Dual hash (SipHash-2-4 + SHA-256), + canonical payload (v1 + v2), Ed25519 signing trait, `compute_record` / + `compute_record_signed` / `verify_links`, chain-break-as-event primitive, + pluggable `Condition` trait. No storage dependency. +- **`trusty-store`** — backend-agnostic `Store` trait. Data types, + `KvCrossCheckReport`, transactional scope. Implemented by + `trusty-memory` and `trusty-sqlite`. +- **`trusty-memory`** — in-memory `Store` impl. Reference implementation + for tests / fuzzing / conformance. Not intended for persistence. +- **`trusty-sqlite`** — rusqlite adapter. Three-table schema + (`provenance_envelope`, `hash_index`, `chain_tips`) with KV cross-check. + `Trusty::{open, append, append_signed, append_with_method, + append_with_method_signed, verify, anchor, history, integrity}`. +- **`trusty-sqlite-ext`** — loadable SQLite extension (`cdylib`). + `sqlite3_trusty_init()` exposes SQL functions (`trusty_append`, + `trusty_verify`, `trusty_anchor`). Load from any sqlite3 binding in any + language. + +## Docs + +- [`docs/payload-v2.md`](docs/payload-v2.md) — canonical payload byte + format, v1 + v2, signing-view algorithm, verifier logic, locked + invariants. +- [`docs/threat-model.md`](docs/threat-model.md) — detection surfaces, + Ed25519 trust boundary, key-compromise scenarios, out-of-scope threats. +- [`docs/device-signer-provisioning.md`](docs/device-signer-provisioning.md) + — enroll / rotate / revoke pattern for devices signing into a chain, + with a runnable demo at `crates/trusty-sqlite/examples/device_signer.rs`. + +## Layout + +``` +trusty/ + Cargo.toml workspace + crates/ + core/ + trusty-core/ locked primitives (hash, chain, payload, bundle) + trusty-store/ backend-agnostic Store trait + adapters/ + trusty-memory/ in-memory backend (tests / ref impl) + trusty-sqlite/ rusqlite production backend + extensions/ + trusty-sqlite-ext/ SQLite loadable extension (cdylib, standalone) + docs/ format + threat-model specs +``` + +## Build + +Workspace-wide `cargo` commands are broken by a rusqlite feature conflict +(`loadable_extension` in `trusty-sqlite-ext` vs `bundled` in +`trusty-sqlite`). Use per-package form: + +``` +cargo build -p trusty-core -p trusty-store -p trusty-memory -p trusty-sqlite +cargo build -p trusty-sqlite-ext + +cargo test -p trusty-core -p trusty-store -p trusty-memory -p trusty-sqlite +cargo test -p trusty-sqlite-ext +``` + +The extension ships as `trusty_sqlite_ext.{dll,so,dylib}`. Load from +sqlite3: + +``` +sqlite> .load ./trusty_sqlite_ext +sqlite> SELECT trusty_append('company_acme', '{"status":"active"}', 'https://source'); +``` diff --git a/crates/adapters/trusty-memory/Cargo.toml b/crates/adapters/trusty-memory/Cargo.toml new file mode 100644 index 0000000..083aade --- /dev/null +++ b/crates/adapters/trusty-memory/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "trusty-memory" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "In-memory Store backend for Trusty. Reference implementation and test harness." + +[dependencies] +trusty-store = { workspace = true } + +[dev-dependencies] +trusty-store = { workspace = true, features = ["testing"] } diff --git a/crates/adapters/trusty-memory/src/lib.rs b/crates/adapters/trusty-memory/src/lib.rs new file mode 100644 index 0000000..2f6cff2 --- /dev/null +++ b/crates/adapters/trusty-memory/src/lib.rs @@ -0,0 +1,135 @@ +//! In-memory [`Store`](trusty_store::Store) backend for Trusty. +//! +//! Reference implementation of the `Store` trait. Intended for tests, fuzzing, +//! and conformance checks. Not suitable for persistent production use -- all data +//! is lost when the process exits. + +#![deny(clippy::all)] + +use std::collections::{BTreeMap, HashMap}; + +use trusty_store::{ + KvCrossCheckReport, LastRow, RowInsert, Store, StoreResult, StoredRow, TipRow, TipUpsert, +}; + +#[derive(Clone, Default)] +pub struct MemoryStore { + rows: BTreeMap, + next_id: i64, + tips_by_chain: HashMap, + tips_by_entity: HashMap, +} + +impl MemoryStore { + pub fn new() -> Self { + Self::default() + } +} + +impl Store for MemoryStore { + fn last_row(&self, entity_id: &str) -> StoreResult> { + let last = self + .rows + .values() + .rev() + .find(|r| r.entity_id == entity_id && r.event_type != "BREAK" && r.row_hash.is_some()) + .map(|r| LastRow { + row_hash: r.row_hash.clone().unwrap(), + sequence: r.sequence, + }); + Ok(last) + } + + fn insert_row(&mut self, row: &RowInsert) -> StoreResult { + self.next_id += 1; + let id = self.next_id; + let stored = StoredRow { + id, + entity_id: row.entity_id.clone(), + chain_id: row.chain_id.clone(), + event_type: row.event_type.clone(), + sequence: row.sequence, + prev_hash: row.prev_hash.clone(), + row_hash: row.row_hash.clone(), + anchor_hash: row.anchor_hash.clone(), + raw_payload: row.raw_payload.clone(), + detected_at: row.detected_at.clone(), + received_at: row.received_at.clone(), + source_url: row.source_url.clone(), + extraction_method: row.extraction_method.clone(), + }; + self.rows.insert(id, stored); + Ok(id) + } + + fn iter_chain<'a>( + &'a self, + entity_id: &str, + ) -> StoreResult> + 'a>> { + let entity = entity_id.to_string(); + let iter = self + .rows + .values() + .filter(move |r| r.entity_id == entity) + .cloned() + .map(Ok); + Ok(Box::new(iter)) + } + + fn get_tip(&self, chain_id: &str) -> StoreResult> { + Ok(self.tips_by_chain.get(chain_id).cloned()) + } + + fn get_tip_for_entity(&self, entity_id: &str) -> StoreResult> { + Ok(self.tips_by_entity.get(entity_id).cloned()) + } + + fn set_tip(&mut self, tip: &TipUpsert) -> StoreResult<()> { + let row = TipRow { + entity_id: tip.entity_id.clone(), + chain_id: tip.chain_id.clone(), + current_tip_hash: tip.current_tip_hash.clone(), + updated_at: String::new(), + }; + self.tips_by_chain.insert(tip.chain_id.clone(), row.clone()); + self.tips_by_entity.insert(tip.entity_id.clone(), row); + Ok(()) + } + + fn kv_cross_check(&self) -> StoreResult { + let kv1 = self.rows.values().filter(|r| r.row_hash.is_some()).count(); + let kv2 = self.tips_by_entity.len(); + let mut mismatches: Vec = Vec::new(); + for (entity, tip) in &self.tips_by_entity { + if let Some(hash) = &tip.current_tip_hash { + let found = self.rows.values().any(|r| { + r.row_hash.as_deref() == Some(hash.as_str()) && r.chain_id == tip.chain_id + }); + if !found { + mismatches.push(format!("tip_hash_not_in_rows: {}", entity)); + } + } + } + Ok(KvCrossCheckReport { + ok: mismatches.is_empty(), + kv1_total_hashes: kv1, + kv2_total_entities: kv2, + entities_checked: kv2, + mismatches, + }) + } + + fn with_txn( + &mut self, + f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>, + ) -> StoreResult<()> { + let snap = self.clone(); + match f(self) { + Ok(()) => Ok(()), + Err(e) => { + *self = snap; + Err(e) + } + } + } +} diff --git a/crates/adapters/trusty-memory/tests/conformance.rs b/crates/adapters/trusty-memory/tests/conformance.rs new file mode 100644 index 0000000..0cd2a8b --- /dev/null +++ b/crates/adapters/trusty-memory/tests/conformance.rs @@ -0,0 +1,8 @@ +use trusty_memory::MemoryStore; +use trusty_store::testing::run_conformance; + +#[test] +fn memory_store_passes_conformance() { + let mut store = MemoryStore::new(); + run_conformance(&mut store); +} diff --git a/crates/adapters/trusty-policy-audit/Cargo.toml b/crates/adapters/trusty-policy-audit/Cargo.toml new file mode 100644 index 0000000..e949bfb --- /dev/null +++ b/crates/adapters/trusty-policy-audit/Cargo.toml @@ -0,0 +1,19 @@ +[package] +name = "trusty-policy-audit" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "Trusty-backed PolicyAuditor adapter. Routes every PolicyEvent (Allow + Deny) into a trusty append-only chain so the policy decision trail is tamper-evident, replayable, and externally verifiable." + +[dependencies] +trusty-policy = { workspace = true } +trusty-sqlite = { workspace = true } +serde = { workspace = true } +serde_json = { workspace = true } + +[dev-dependencies] +trusty-core = { workspace = true } +tempfile = "3" diff --git a/crates/adapters/trusty-policy-audit/src/lib.rs b/crates/adapters/trusty-policy-audit/src/lib.rs new file mode 100644 index 0000000..55a7556 --- /dev/null +++ b/crates/adapters/trusty-policy-audit/src/lib.rs @@ -0,0 +1,262 @@ +//! Trusty-backed `PolicyAuditor`. +//! +//! Closes the policy <-> chain loop: every `PolicyEvent` emitted by +//! `trusty_policy::PolicyEngine::evaluate` (Allow OR Deny) is appended +//! to a `Trusty` chain. The `entity_id` of the chain row is the policy +//! scope (e.g. `agent.executor.v1`), so each registered FSM gets its +//! own per-entity chain. Tamper-evidence, ordering, and external +//! verifiability are inherited from `trusty-sqlite`. +//! +//! # Why a separate crate +//! +//! `trusty-policy` deliberately has no dependency on `trusty-core` / +//! `trusty-sqlite` (library-not-platform rule). Wiring the two +//! together lives here so consumers who want only the policy gate +//! without an audit chain pay nothing. +//! +//! # Quickstart +//! +//! ```no_run +//! use trusty_policy::{Fsm, PolicyEngine, ScopeKey, Transition}; +//! use trusty_sqlite::Trusty; +//! use trusty_core::SecretKey; +//! use trusty_policy_audit::TrustyAuditor; +//! +//! let trusty = Trusty::open_in_memory(SecretKey::new([7u8; 16])).unwrap(); +//! let auditor = TrustyAuditor::new(trusty); +//! +//! let scope = ScopeKey::new("agent.executor.v1"); +//! let mut fsm: Fsm<&'static str, &'static str> = Fsm::new("idle"); +//! fsm.allow(Transition::new("idle", "claim", "claimed")); +//! +//! let mut engine = PolicyEngine::new(auditor); +//! engine.register(scope.clone(), fsm).unwrap(); +//! let _ = engine.evaluate(&scope, &"idle", &"claim", Some("agent_001")); +//! // The chain row for entity_id="agent.executor.v1" now contains +//! // the PolicyEvent JSON. +//! ``` + +#![deny(clippy::all)] + +use std::sync::{Arc, Mutex}; + +use trusty_policy::{PolicyAuditor, PolicyEvent}; +use trusty_sqlite::Trusty; + +/// Source-URL marker stamped on every chained policy row. Consumers +/// can filter by this to separate policy events from other domain +/// events that may share a chain. +pub const POLICY_SOURCE_URL: &str = "policy://trusty-policy/event"; + +/// Extraction-method tag used when calling `Trusty::append_with_method`. +/// Mirrors the existing trusty convention; consumers reading the chain +/// can recognize policy rows by `extraction_method = "policy"`. +pub const POLICY_EXTRACTION_METHOD: &str = "policy"; + +/// `PolicyAuditor` that appends each `PolicyEvent` to a `Trusty` chain. +/// The `Trusty` handle is held behind `Arc>` so the auditor +/// is `Send + 'static` (the trait bound `PolicyEngine::new` requires) +/// while still allowing the caller to share the same chain across +/// other writers if needed. +/// +/// On audit failure (rare: disk full, schema corruption) the auditor +/// stores the most recent error in `last_error` and silently drops +/// the row. Callers who need hard-stop semantics on audit failure +/// should poll `last_error()` or wrap with their own checking layer. +/// We deliberately do NOT panic: a misbehaving audit sink must not +/// take down the policy engine during a deny path. +pub struct TrustyAuditor { + trusty: Arc>, + last_error: Arc>>, +} + +impl TrustyAuditor { + /// Wrap a `Trusty` handle. The auditor takes ownership; if you + /// need shared access, build the auditor with [`Self::from_shared`]. + pub fn new(trusty: Trusty) -> Self { + Self { + trusty: Arc::new(Mutex::new(trusty)), + last_error: Arc::new(Mutex::new(None)), + } + } + + /// Wrap an already-shared `Trusty` so the same chain can be + /// appended-to by other writers. + pub fn from_shared(trusty: Arc>) -> Self { + Self { + trusty, + last_error: Arc::new(Mutex::new(None)), + } + } + + /// Most recent audit error, if any. Cleared by [`Self::clear_error`]. + pub fn last_error(&self) -> Option { + self.last_error.lock().ok().and_then(|g| g.clone()) + } + + pub fn clear_error(&self) { + if let Ok(mut g) = self.last_error.lock() { + *g = None; + } + } + + /// Borrow the inner shared handle. Useful for the consumer to + /// run integrity / verification queries after the engine has + /// emitted events. + pub fn shared(&self) -> Arc> { + Arc::clone(&self.trusty) + } +} + +impl PolicyAuditor for TrustyAuditor { + fn audit(&mut self, event: &PolicyEvent) { + // Serialize the event as JSON. The chain row's `data` column + // is the canonical record; downstream verifiers re-parse it. + let payload = match serde_json::to_value(event) { + Ok(v) => v, + Err(e) => { + self.record_error(format!("serialize PolicyEvent: {e}")); + return; + } + }; + + // entity_id = scope. One chain per FSM scope. + let entity_id = event.scope.as_str(); + + let mut guard = match self.trusty.lock() { + Ok(g) => g, + Err(p) => { + self.record_error(format!("trusty mutex poisoned: {p}")); + return; + } + }; + + if let Err(e) = guard.append_with_method( + entity_id, + &payload, + POLICY_SOURCE_URL, + POLICY_EXTRACTION_METHOD, + ) { + self.record_error(format!("append_with_method: {e}")); + } + } +} + +impl TrustyAuditor { + fn record_error(&self, msg: String) { + if let Ok(mut g) = self.last_error.lock() { + *g = Some(msg); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use trusty_core::SecretKey; + use trusty_policy::{Fsm, PolicyEngine, ScopeKey, Transition}; + + fn key() -> SecretKey { + SecretKey::new([7u8; 16]) + } + + fn pipeline_fsm() -> Fsm<&'static str, &'static str> { + let mut f = Fsm::new("idle"); + f.allow(Transition::new("idle", "claim", "claimed")) + .allow(Transition::new("claimed", "execute", "running")) + .allow(Transition::new("running", "finish", "done")); + f + } + + #[test] + fn allow_emits_chain_row() { + let trusty = Trusty::open_in_memory(key()).unwrap(); + let auditor = TrustyAuditor::new(trusty); + let shared = auditor.shared(); + + let scope = ScopeKey::new("agent.executor.v1"); + let mut engine: PolicyEngine<&'static str, &'static str> = PolicyEngine::new(auditor); + engine.register(scope.clone(), pipeline_fsm()).unwrap(); + + let r = engine.evaluate(&scope, &"idle", &"claim", Some("agent_001")); + assert!(r.is_allow()); + + // Read back the chain via the shared handle. raw_payload is + // the full envelope; the PolicyEvent JSON lives at .data. + let guard = shared.lock().unwrap(); + let history = guard.history("agent.executor.v1", 100).unwrap(); + assert_eq!(history.len(), 1, "exactly one chained policy row"); + let raw = history[0].raw_payload.as_ref().expect("payload present"); + let envelope: serde_json::Value = serde_json::from_str(raw).unwrap(); + let payload = &envelope["data"]; + assert_eq!(payload["scope"], "agent.executor.v1"); + assert_eq!(payload["from"], "\"idle\""); + assert_eq!(payload["action"], "\"claim\""); + assert_eq!(payload["actor"], "agent_001"); + assert!(payload["result"].get("Allow").is_some()); + } + + #[test] + fn deny_also_emits_chain_row() { + let trusty = Trusty::open_in_memory(key()).unwrap(); + let auditor = TrustyAuditor::new(trusty); + let shared = auditor.shared(); + + let scope = ScopeKey::new("agent.executor.v1"); + let mut engine: PolicyEngine<&'static str, &'static str> = PolicyEngine::new(auditor); + engine.register(scope.clone(), pipeline_fsm()).unwrap(); + + // illegal: idle -> finish is undeclared + let r = engine.evaluate(&scope, &"idle", &"finish", None); + assert!(!r.is_allow()); + + let guard = shared.lock().unwrap(); + let history = guard.history("agent.executor.v1", 100).unwrap(); + assert_eq!(history.len(), 1, "deny is also chained"); + let envelope: serde_json::Value = + serde_json::from_str(history[0].raw_payload.as_ref().unwrap()).unwrap(); + assert!(envelope["data"]["result"].get("Deny").is_some()); + } + + #[test] + fn separate_scopes_get_separate_chains() { + let trusty = Trusty::open_in_memory(key()).unwrap(); + let auditor = TrustyAuditor::new(trusty); + let shared = auditor.shared(); + + let s_exec = ScopeKey::new("agent.executor.v1"); + let s_audit = ScopeKey::new("agent.auditor.v1"); + + let mut engine: PolicyEngine<&'static str, &'static str> = PolicyEngine::new(auditor); + engine.register(s_exec.clone(), pipeline_fsm()).unwrap(); + engine.register(s_audit.clone(), pipeline_fsm()).unwrap(); + + let _ = engine.evaluate(&s_exec, &"idle", &"claim", None); + let _ = engine.evaluate(&s_audit, &"idle", &"claim", None); + + let guard = shared.lock().unwrap(); + assert_eq!(guard.history("agent.executor.v1", 100).unwrap().len(), 1); + assert_eq!(guard.history("agent.auditor.v1", 100).unwrap().len(), 1); + } + + #[test] + fn chain_is_tamper_evident() { + let trusty = Trusty::open_in_memory(key()).unwrap(); + let auditor = TrustyAuditor::new(trusty); + let shared = auditor.shared(); + + let scope = ScopeKey::new("agent.executor.v1"); + let mut engine: PolicyEngine<&'static str, &'static str> = PolicyEngine::new(auditor); + engine.register(scope.clone(), pipeline_fsm()).unwrap(); + + let _ = engine.evaluate(&scope, &"idle", &"claim", None); + let _ = engine.evaluate(&scope, &"claimed", &"finish", None); // deny + let _ = engine.evaluate(&scope, &"claimed", &"execute", None); + let _ = engine.evaluate(&scope, &"running", &"finish", None); + + let guard = shared.lock().unwrap(); + let report = guard.integrity().unwrap(); + assert!(report.ok, "chain integrity must hold: {:?}", report); + assert_eq!(guard.history("agent.executor.v1", 100).unwrap().len(), 4); + } +} diff --git a/crates/adapters/trusty-policy-conditions/Cargo.toml b/crates/adapters/trusty-policy-conditions/Cargo.toml new file mode 100644 index 0000000..5b72663 --- /dev/null +++ b/crates/adapters/trusty-policy-conditions/Cargo.toml @@ -0,0 +1,18 @@ +[package] +name = "trusty-policy-conditions" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "Lets a declared policy FSM act as a trusty break Condition, so the rules for how data may legally change are authored as data and a deviation records which rule broke." + +[dependencies] +trusty-core = { workspace = true } +trusty-policy = { workspace = true } +serde_json = { workspace = true } +sha2 = { workspace = true } + +[dev-dependencies] +trusty-sqlite = { workspace = true } diff --git a/crates/adapters/trusty-policy-conditions/src/lib.rs b/crates/adapters/trusty-policy-conditions/src/lib.rs new file mode 100644 index 0000000..d8b237e --- /dev/null +++ b/crates/adapters/trusty-policy-conditions/src/lib.rs @@ -0,0 +1,528 @@ +//! Rules for how data may legally change, authored as data. +//! +//! # The idea +//! +//! Declare the transitions a datum is allowed to make. A compliant +//! mutation is valid and the chain continues. A deviation records a +//! **break naming the rule it violated**, and that break is signed, +//! travels in an exported bundle, and can be acted on by anyone. +//! +//! Almost all of this already existed. [`Condition`] evaluates state at +//! append time, [`BreakReason::PropertyViolated`] already carries a +//! rule's name, and `export_bundle` always includes BREAK rows. +//! Chain-break-as-event *was* this design. The one thing missing was +//! that the rules were compiled in: [`Condition`] is a Rust trait, so a +//! new rule meant a new build. +//! +//! The policy side had already solved that -- [`FsmSpec`] authored in +//! TOML, guards named as strings, and [`GuardRegistry`] refusing to +//! build on an unknown name. This crate is the join. +//! +//! # Guards see the payload +//! +//! The FSM is built over [`ConditionContext`], so a guard receives the +//! **data being appended**, not just chain metadata. A rule can +//! therefore be a real predicate over the value -- a threshold, a shape +//! check, a comparison against the previous row -- rather than only a +//! statement about the shape of the graph. +//! +//! # The envelope +//! +//! A row states the transition it is making, under `data.transition`: +//! +//! ```json +//! { "transition": { "from": "draft", "action": "submit", "to": "review" } } +//! ``` +//! +//! The condition checks that transition is declared, that any guard on +//! it passes, and that the stated destination is the one the FSM says it +//! should be. That last check matters: without it a writer could name a +//! legal transition and then record an arbitrary destination. +//! +//! # Rows that claim no transition +//! +//! By default they pass -- this condition judges rows that claim to be +//! transitions, and a chain usually carries other traffic too. +//! [`FsmCondition::strict`] reverses that, so an unlabelled row is +//! itself a violation. +//! +//! Which is right depends on whether the chain is dedicated to the +//! governed process. Getting it wrong in the lenient direction leaves a +//! hole a writer can walk through by omitting a field, so **strict is +//! the right default for a chain that governs one thing**, and the +//! lenient default exists only because most chains do not. +//! +//! # The marker crosses the boundary +//! +//! A break's reason travels in the exported bundle, so a party holding +//! only that bundle -- and no secret key, and no access to the machine +//! that recorded it -- can ask `ProofBundle::stance()` and get one of +//! three answers: clean, violated (with the rule named), or +//! unverifiable. That third answer is the point: a bundle that cannot be +//! checked must not read as clean. +//! +//! # One limitation, stated rather than discovered +//! +//! **A rule cannot be violated by the first row of a chain.** +//! Break-as-event seals an existing chain, and at genesis there is +//! nothing to seal, so the opening row is recorded whatever it claims. +//! If the opening state must also be governed, check it before the +//! first append -- the chain will not do it for you. +//! +//! # Why this crate and not another +//! +//! [`trusty_policy`] is deliberately substrate-free and does not depend +//! on [`trusty_core`]. Putting this in either would give one of them a +//! dependency it was designed not to have -- the same split, for the +//! same reason, as `trusty-policy-audit`. + +#![deny(clippy::all)] + +use serde_json::Value; +use trusty_core::conditions::{BreakReason, Condition, ConditionContext, ConditionOutcome}; +use sha2::{Digest, Sha256}; +use trusty_policy::{Fsm, FsmSpec, GuardRegistry, PolicyError}; + +/// Where the transition envelope lives inside `data`. +pub const TRANSITION_FIELD: &str = "transition"; + +/// A declared state machine, acting as a break condition. +pub struct FsmCondition { + fsm: Fsm, + rule: String, + /// The rule set's content hash, short form, when it was built from a + /// spec. Stamped into every break so a violation names the rule AND + /// the version of it that produced the verdict. + rule_hash: Option, + strict: bool, +} + +impl FsmCondition { + /// Build from an already-constructed FSM. + /// + /// `rule` is the name that appears in the recorded break. Make it + /// something an operator reading a chain six months later can look + /// up -- the whole point of `PropertyViolated` carrying a name is + /// that the break says *which* invariant broke. + pub fn new(fsm: Fsm, rule: impl Into) -> Self { + Self { + fsm, + rule: rule.into(), + rule_hash: None, + strict: false, + } + } + + /// The content hash of a rule set, hex, first 16 characters. + /// + /// Truncated because it travels inside every break reason and a full + /// digest would crowd out the part a human reads. 64 bits is ample + /// for telling two rule sets apart; it is not a security boundary, + /// since the full canonical form is what gets signed. + pub fn hash_of(spec: &FsmSpec) -> Result { + let canonical = spec.canonical_json()?; + Ok(hex_short(&Sha256::digest(canonical.as_bytes()))) + } + + /// The rule set version this condition enforces, if it was built + /// from a spec. + pub fn rule_hash(&self) -> Option<&str> { + self.rule_hash.as_deref() + } + + /// Build from a TOML spec, with guards resolved by name. + /// + /// A transition naming a guard the registry does not hold **refuses + /// to build**. That refusal is the feature: a policy that silently + /// runs without its guard is a policy that is not being enforced, + /// and it would look identical to one that is. + pub fn from_json( + json: &str, + registry: &mut GuardRegistry, + rule: impl Into, + ) -> Result { + let spec = FsmSpec::from_json(json)?; + let rule_hash = Self::hash_of(&spec)?; + let (_scope, fsm) = spec.build_fsm_with_guards(registry)?; + let mut built = Self::new(fsm, rule); + built.rule_hash = Some(rule_hash); + Ok(built) + } + + /// Treat a row that claims no transition as a violation. + /// + /// Correct when the chain is dedicated to the governed process. + /// Without it, omitting the envelope is a way around the rule. + pub fn strict(mut self) -> Self { + self.strict = true; + self + } + + fn violated(&self, detail: &str) -> ConditionOutcome { + // The rule set version rides along, so a reader of the break can + // tell WHICH `doc.review` produced it. Without it a rule could be + // loosened after the fact and the old marker would still verify + // against a rule that no longer says what it said. + let name = match &self.rule_hash { + Some(h) => format!("{}@{h}", self.rule), + None => self.rule.clone(), + }; + ConditionOutcome::Fail(BreakReason::PropertyViolated(format!("{name}:{detail}"))) + } +} + +/// The declared transition a row claims to be making. +struct Declared { + from: String, + action: String, + to: String, +} + +fn read_envelope(data: &Value) -> Option { + let t = data.get(TRANSITION_FIELD)?; + Some(Declared { + from: t.get("from")?.as_str()?.to_string(), + action: t.get("action")?.as_str()?.to_string(), + to: t.get("to")?.as_str()?.to_string(), + }) +} + +impl Condition for FsmCondition { + fn name(&self) -> &'static str { + "FsmCondition" + } + + fn evaluate(&self, ctx: &ConditionContext) -> ConditionOutcome { + let Some(declared) = read_envelope(&ctx.data) else { + return if self.strict { + // Distinguished from a malformed one on purpose: "you + // did not say" and "you said something impossible" are + // different operator problems. + self.violated("no-transition-declared") + } else { + ConditionOutcome::Pass + }; + }; + + // Exclusions first, and they cannot be overridden -- the same + // ordering the permission model uses, for the same reason. + if let Some(why) = self.fsm.excluded_by(&declared.from, &declared.action) { + return self.violated(&format!("excluded:{why}")); + } + + let Some(expected) = self.fsm.next_state(&declared.from, &declared.action) else { + return self.violated(&format!( + "undeclared-transition:{}--{}", + declared.from, declared.action + )); + }; + + if let Some(guard) = self.fsm.guard(&declared.from, &declared.action) { + if let Err(code) = guard.check(ctx) { + return self.violated(&format!("guard-rejected:{code}")); + } + } + + if *expected != declared.to { + // A writer naming a legal transition and then recording a + // destination of its own choosing is exactly what this + // stops. + return self.violated(&format!( + "wrong-destination:{}--{}->{} expected {expected}", + declared.from, declared.action, declared.to + )); + } + + ConditionOutcome::Pass + } +} + +fn hex_short(digest: &[u8]) -> String { + digest.iter().take(8).map(|b| format!("{b:02x}")).collect() +} + +/// The transition envelope, for a writer to embed in its payload. +pub fn transition(from: &str, action: &str, to: &str) -> Value { + serde_json::json!({ TRANSITION_FIELD: { "from": from, "action": action, "to": to } }) +} + +#[cfg(test)] +mod tests { + use super::*; + + const SPEC: &str = r#" +{ + "scope": "doc.review.v1", + "initial": "draft", + "transitions": [ + { + "from": "draft", + "action": "submit", + "to": "review" + }, + { + "from": "review", + "action": "approve", + "to": "published", + "guard": "big_enough" + } + ] +} +"#; + + pub(super) fn ctx(data: Value) -> ConditionContext { + ConditionContext { + entity_id: "e".into(), + chain_id: "c".into(), + sequence: 1, + stored_tip_hash: None, + computed_hash: None, + prev_hash: None, + last_sequence: Some(0), + source_url: String::new(), + extraction_method: "trusty".into(), + data, + } + } + + fn registry() -> GuardRegistry { + let mut r = GuardRegistry::new(); + r.insert("big_enough", |c: &ConditionContext| { + match c.data.get("size").and_then(Value::as_i64) { + Some(n) if n >= 10 => Ok(()), + Some(n) => Err(format!("too-small:{n}")), + None => Err("no-size".to_string()), + } + }); + r + } + + fn condition() -> FsmCondition { + FsmCondition::from_json(SPEC, &mut registry(), "doc.review").expect("build") + } + + /// ConditionOutcome does not derive PartialEq, and adding a derive to + /// a core type to suit a test is the wrong direction of dependency. + fn passed(outcome: ConditionOutcome) -> bool { + matches!(outcome, ConditionOutcome::Pass) + } + + fn reason(outcome: ConditionOutcome) -> String { + match outcome { + ConditionOutcome::Fail(BreakReason::PropertyViolated(r)) => r, + other => panic!("expected a property violation, got {other:?}"), + } + } + + #[test] + fn a_declared_transition_passes() { + let outcome = condition().evaluate(&ctx(transition("draft", "submit", "review"))); + assert!( + passed(outcome), + "a compliant mutation must leave the chain unbroken; that is what \ + makes a break mean something" + ); + } + + #[test] + fn an_undeclared_transition_breaks_the_chain_and_names_the_rule() { + let outcome = condition().evaluate(&ctx(transition("draft", "publish", "published"))); + let r = reason(outcome); + assert!( + r.starts_with("doc.review@") && r.contains(":undeclared-transition"), + "the break must name the rule that broke, or an operator reading \ + the chain later learns only that something went wrong: {r}" + ); + } + + #[test] + fn naming_a_legal_transition_does_not_licence_an_arbitrary_destination() { + let outcome = condition().evaluate(&ctx(transition("draft", "submit", "published"))); + let r = reason(outcome); + assert!( + r.contains("wrong-destination"), + "submit is legal from draft, but it goes to review; without this \ + check a writer could name a legal transition and then record \ + wherever it liked: {r}" + ); + } + + #[test] + fn a_guard_sees_the_payload_not_just_the_chain_metadata() { + let mut small = transition("review", "approve", "published"); + small["size"] = Value::from(3); + let r = reason(condition().evaluate(&ctx(small))); + assert!( + r.contains("guard-rejected:too-small:3"), + "the guard must be able to judge the DATA being appended -- a rule \ + that can only see chain metadata cannot express most invariants \ + worth having: {r}" + ); + + let mut big = transition("review", "approve", "published"); + big["size"] = Value::from(50); + assert!(passed(condition().evaluate(&ctx(big)))); + } + + #[test] + fn a_missing_guard_input_is_a_rejection_not_a_pass() { + let r = reason(condition().evaluate(&ctx(transition("review", "approve", "published")))); + assert!( + r.contains("guard-rejected:no-size"), + "absent evidence must not read as a satisfied guard, for the same \ + reason absent policy is not permission: {r}" + ); + } + + #[test] + fn an_unregistered_guard_refuses_to_build() { + let mut empty = GuardRegistry::::new(); + let built = FsmCondition::from_json(SPEC, &mut empty, "doc.review"); + assert!( + built.is_err(), + "a policy that silently runs without its guard is a policy that is \ + not being enforced, and it looks identical to one that is" + ); + } + + #[test] + fn an_unlabelled_row_passes_by_default_and_fails_under_strict() { + let plain = ctx(serde_json::json!({ "temperature": 21 })); + assert!( + passed(condition().evaluate(&plain)), + "most chains carry traffic this condition has no opinion about" + ); + + let r = reason(condition().strict().evaluate(&plain)); + assert!( + r.contains("no-transition-declared"), + "on a chain dedicated to the governed process, omitting the \ + envelope would otherwise be a way around the rule: {r}" + ); + } + + #[test] + fn a_malformed_envelope_is_reported_as_undeclared_not_as_absent() { + let half = ctx(serde_json::json!({ TRANSITION_FIELD: { "from": "draft" } })); + assert!( + passed(condition().evaluate(&half)), + "an envelope missing required fields is not a transition claim at \ + all; under strict it fails, and the two cases are distinguished \ + because 'you did not say' and 'you said something impossible' are \ + different operator problems" + ); + let r = reason(condition().strict().evaluate(&half)); + assert!(r.contains("no-transition-declared"), "{r}"); + } + + #[test] + fn an_exclusion_cannot_be_overridden_by_a_declared_transition() { + let mut fsm: Fsm = Fsm::new("draft".to_string()); + fsm.allow(trusty_policy::Transition::new( + "draft".to_string(), + "submit".to_string(), + "review".to_string(), + )); + fsm.exclude("submit".to_string(), "frozen-for-audit"); + let cond = FsmCondition::new(fsm, "doc.review"); + + let r = reason(cond.evaluate(&ctx(transition("draft", "submit", "review")))); + assert!( + r.contains("excluded:frozen-for-audit"), + "exclusions are checked before the transition is looked up, so a \ + declared transition cannot reopen what an exclusion shut -- the \ + same ordering the permission model uses: {r}" + ); + } +} + +#[cfg(test)] +mod rule_version_tests { + use super::*; + + const NARROW: &str = r#" +{ + "scope": "doc.review.v1", + "initial": "draft", + "transitions": [ + { "from": "draft", "action": "submit", "to": "review" } + ] +} +"#; + + const LOOSENED: &str = r#" +{ + "scope": "doc.review.v1", + "initial": "draft", + "transitions": [ + { "from": "draft", "action": "submit", "to": "review" }, + { "from": "draft", "action": "publish", "to": "published" } + ] +} +"#; + + fn built(spec: &str) -> FsmCondition { + FsmCondition::from_json(spec, &mut GuardRegistry::new(), "doc.review") + .expect("build") + } + + #[test] + fn loosening_the_rule_changes_the_version_the_break_cites() { + let narrow = built(NARROW); + let loosened = built(LOOSENED); + assert_ne!( + narrow.rule_hash(), + loosened.rule_hash(), + "a rule set that can be widened without changing its version is a \ + rule set a violation marker cannot honestly cite -- loosen it \ + after the fact and the old marker still appears to hold" + ); + } + + #[test] + fn the_same_rule_set_always_yields_the_same_version() { + assert_eq!( + built(NARROW).rule_hash(), + built(NARROW).rule_hash(), + "a version that varied between processes would make every marker \ + uncomparable, which is worse than having no version at all" + ); + } + + #[test] + fn a_hand_built_fsm_has_no_version_rather_than_a_made_up_one() { + let mut fsm: Fsm = Fsm::new("draft".to_string()); + fsm.allow(trusty_policy::Transition::new( + "draft".to_string(), + "submit".to_string(), + "review".to_string(), + )); + let cond = FsmCondition::new(fsm, "doc.review"); + assert_eq!( + cond.rule_hash(), + None, + "an FSM assembled in Rust has no canonical document to hash, and \ + inventing a version for it would be a claim nobody can check" + ); + } + + #[test] + fn a_versionless_rule_still_names_itself_in_the_break() { + let mut fsm: Fsm = Fsm::new("draft".to_string()); + fsm.allow(trusty_policy::Transition::new( + "draft".to_string(), + "submit".to_string(), + "review".to_string(), + )); + let outcome = FsmCondition::new(fsm, "doc.review") + .evaluate(&super::tests::ctx(transition("draft", "publish", "gone"))); + match outcome { + ConditionOutcome::Fail(BreakReason::PropertyViolated(r)) => assert!( + r.starts_with("doc.review:"), + "no version is not the same as no name; the rule must still \ + identify itself: {r}" + ), + other => panic!("expected a violation, got {other:?}"), + } + } +} diff --git a/crates/adapters/trusty-policy-conditions/tests/enforced.rs b/crates/adapters/trusty-policy-conditions/tests/enforced.rs new file mode 100644 index 0000000..5ad5288 --- /dev/null +++ b/crates/adapters/trusty-policy-conditions/tests/enforced.rs @@ -0,0 +1,303 @@ +//! Does a declared rule actually break a real chain? +//! +//! The unit tests check the condition's verdict. These check that the +//! verdict reaches the chain: that a compliant mutation leaves it +//! unbroken, a deviation records a break, and the break says which rule +//! broke rather than merely that something did. + +use trusty_core::{Ed25519Signer, SecretKey, Stance}; +use trusty_policy::GuardRegistry; +use trusty_policy_conditions::{transition, FsmCondition}; +use trusty_sqlite::Trusty; + +const SPEC: &str = r#" +{ + "scope": "doc.review.v1", + "initial": "draft", + "transitions": [ + { + "from": "draft", + "action": "submit", + "to": "review" + }, + { + "from": "review", + "action": "approve", + "to": "published", + "guard": "big_enough" + } + ] +} +"#; + +fn store() -> Trusty { + let mut registry = GuardRegistry::new(); + registry.insert("big_enough", |c: &trusty_core::ConditionContext| { + match c.data.get("size").and_then(serde_json::Value::as_i64) { + Some(n) if n >= 10 => Ok(()), + Some(n) => Err(format!("too-small:{n}")), + None => Err("no-size".to_string()), + } + }); + let condition = + FsmCondition::from_json(SPEC, &mut registry, "doc.review").expect("build the rule"); + + // The integrity defaults PLUS the declared rule: a policy rule is an + // addition to tamper detection, never a replacement for it. + let mut conditions = trusty_core::conditions::default_conditions(); + conditions.push(Box::new(condition)); + Trusty::open_in_memory_with_conditions(SecretKey::new([9u8; 16]), conditions) + .expect("store") +} + +/// Every recorded break reason, in order. +/// +/// Read through `breaks_since`, not from the exported bundle: a BREAK +/// row's reason lives in the `old_value`/`new_value` columns, and +/// `BundleRow` carries neither. So the reason does **not** travel to a +/// remote party today -- see the sibling issue on making the marker +/// actionable across a boundary. +fn break_reasons(trusty: &Trusty, entity: &str) -> Vec { + trusty + .breaks_since(entity, None) + .expect("breaks") + .into_iter() + .map(|b| b.break_reason) + .collect() +} + +/// Establish the chain with a compliant row. +/// +/// A rule cannot be violated by the FIRST row of a chain: break-as-event +/// seals an existing chain, and at genesis there is nothing to seal. So +/// every test that wants to observe a violation must first give the +/// chain something to break. +fn genesis(trusty: &mut Trusty, signer: &Ed25519Signer, entity: &str) { + trusty + .append_signed( + entity, + &transition("draft", "submit", "review"), + "test://", + signer, + ) + .expect("genesis"); +} + +#[test] +fn a_compliant_mutation_leaves_the_chain_unbroken() { + let mut trusty = store(); + let signer = Ed25519Signer::generate(); + + trusty + .append_signed( + "doc:1", + &transition("draft", "submit", "review"), + "test://", + &signer, + ) + .expect("append"); + + assert!( + break_reasons(&trusty, "doc:1").is_empty(), + "a declared transition must pass; if compliant data broke the chain, \ + a break would stop meaning anything" + ); + assert!( + trusty + .export_bundle("doc:1", None, None) + .expect("export") + .verify_public() + .continuous, + "and the chain must read as continuous" + ); +} + +#[test] +fn an_undeclared_mutation_breaks_the_chain_and_the_break_names_the_rule() { + let mut trusty = store(); + let signer = Ed25519Signer::generate(); + + trusty + .append_signed( + "doc:2", + &transition("draft", "submit", "review"), + "test://", + &signer, + ) + .expect("first"); + + // draft --publish--> published is not declared anywhere. + trusty + .append_signed( + "doc:2", + &transition("draft", "publish", "published"), + "test://", + &signer, + ) + .expect("second"); + + let reasons = break_reasons(&trusty, "doc:2"); + assert_eq!( + reasons.len(), + 1, + "the deviation must be recorded as an event, not rejected silently -- \ + chain-break-as-event is what makes the violation evidence rather than \ + an error somebody's log swallowed; got {reasons:?}" + ); + assert!( + reasons[0].starts_with("PROPERTY_VIOLATED:doc.review@") + && reasons[0].contains(":undeclared-transition"), + "the break must carry the rule's name, so an operator reading this \ + chain in six months learns WHICH invariant broke; got {}", + reasons[0] + ); +} + +#[test] +fn a_guard_rejection_names_the_guard_and_its_reason() { + let mut trusty = store(); + let signer = Ed25519Signer::generate(); + + genesis(&mut trusty, &signer, "doc:3"); + let mut small = transition("review", "approve", "published"); + small["size"] = serde_json::Value::from(2); + trusty + .append_signed("doc:3", &small, "test://", &signer) + .expect("append"); + + let reasons = break_reasons(&trusty, "doc:3"); + assert!( + reasons + .iter() + .any(|r| r.contains("guard-rejected:too-small:2")), + "a rule over the DATA must be able to break the chain and say what it \ + saw; a break that only said 'guard failed' would send someone back to \ + the source to find out which one; got {reasons:?}" + ); +} + +#[test] +fn the_marker_crosses_the_boundary_carrying_the_rule_name() { + let mut trusty = store(); + let signer = Ed25519Signer::generate(); + + genesis(&mut trusty, &signer, "doc:4"); + trusty + .append_signed( + "doc:4", + &transition("draft", "publish", "published"), + "test://", + &signer, + ) + .expect("append"); + + // Everything below is what a REMOTE party can do: it holds the + // bundle and no secret key, and never talks to the machine that + // recorded the break. + let bundle = trusty.export_bundle("doc:4", None, None).expect("export"); + + let violations = bundle.violations(); + assert_eq!( + violations.len(), + 1, + "the violation must be readable off the bundle alone; a marker only \ + the writer can read is not something anyone else can act on" + ); + assert_eq!( + violations[0] + .rule + .as_deref() + .map(|r| r.split('@').next().unwrap_or(r)), + Some("doc.review"), + "and it must name the RULE, not merely report that something broke. \ + Refuse, quarantine or inspect are different decisions and the rule is \ + what tells them apart; got {:?}", + violations[0] + ); + + match bundle.stance() { + Stance::Violated { violations } => { + assert_eq!(violations.len(), 1, "one break, one violation") + } + other => panic!("expected a violation stance, got {other:?}"), + } + assert!( + !bundle.stance().is_clean(), + "a datum carrying a signed violation must never be waved through" + ); +} + +#[test] +fn a_clean_bundle_reads_as_clean_and_a_tampered_one_does_not() { + let mut trusty = store(); + let signer = Ed25519Signer::generate(); + genesis(&mut trusty, &signer, "doc:6"); + + let clean = trusty.export_bundle("doc:6", None, None).expect("export"); + assert_eq!( + clean.stance(), + Stance::Clean, + "a compliant chain must read as clean, or the marker means nothing" + ); + + // The same bundle, edited after export. + let mut forged = clean.clone(); + for row in forged.rows.iter_mut() { + if let Some(payload) = row.raw_payload.as_mut() { + *payload = payload.replace("submit", "publish"); + } + } + match forged.stance() { + Stance::Unverifiable { .. } => {} + other => panic!( + "a bundle that no longer matches its signatures must report \ + Unverifiable -- counting zero violations and looking spotless is \ + exactly the failure the third answer exists to prevent; got {other:?}" + ), + } + assert!( + !forged.stance().is_clean(), + "and unverifiable is not a permit" + ); +} + +#[test] +fn a_break_with_no_rule_is_still_reported() { + // Tamper detection and sequence gaps are real breaks with no + // invariant to name. They must still surface, with rule: None, + // rather than being filtered out for lacking a name. + let v = trusty_core::Violation { + entity_id: "e".into(), + chain_id: "c".into(), + rule: None, + reason: "TAMPER_DETECTED".into(), + }; + assert!( + v.rule.is_none() && !v.reason.is_empty(), + "a break without a rule name is still a break, and dropping it would \ + hide the most serious kind" + ); +} + +#[test] +fn a_declared_transition_to_the_wrong_destination_still_breaks() { + let mut trusty = store(); + let signer = Ed25519Signer::generate(); + + genesis(&mut trusty, &signer, "doc:5"); + trusty + .append_signed( + "doc:5", + &transition("draft", "submit", "published"), + "test://", + &signer, + ) + .expect("append"); + + let reasons = break_reasons(&trusty, "doc:5"); + assert!( + reasons.iter().any(|r| r.contains("wrong-destination")), + "submit is legal from draft, but it goes to review. Without this the \ + rule would check the move and ignore where it landed; got {reasons:?}" + ); +} diff --git a/crates/adapters/trusty-policy-postgres/Cargo.toml b/crates/adapters/trusty-policy-postgres/Cargo.toml new file mode 100644 index 0000000..95d05ae --- /dev/null +++ b/crates/adapters/trusty-policy-postgres/Cargo.toml @@ -0,0 +1,20 @@ +[package] +name = "trusty-policy-postgres" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "Compiles a trusty-policy Grant into a Postgres row-level-security decision. Grants are rows and the policy is static, so no SQL is ever built from a grant's contents." + +[dependencies] +trusty-policy = { workspace = true } +postgres = { workspace = true } +serde_json = { workspace = true } +sha2 = { workspace = true } +hex = { workspace = true } +thiserror = { workspace = true } + +[features] +default = [] diff --git a/crates/adapters/trusty-policy-postgres/examples/emit_ddl.rs b/crates/adapters/trusty-policy-postgres/examples/emit_ddl.rs new file mode 100644 index 0000000..b9fe614 --- /dev/null +++ b/crates/adapters/trusty-policy-postgres/examples/emit_ddl.rs @@ -0,0 +1,78 @@ +//! Print the permission model's schema as a plain `.sql` migration. +//! +//! A consumer whose migrations are files rather than a Rust `open()` +//! call — Supabase, sqlx, plain psql — needs the same DDL as text. +//! Emitting it from the same constants the library applies is the point: +//! two hand-maintained copies of an authorisation schema will diverge, +//! and the divergence will be discovered by something going wrong. +//! +//! ```text +//! cargo run -p trusty-policy-postgres --example emit_ddl > 0001_permission_model.sql +//! ``` +//! +//! Pass a table and its scope column to additionally emit the policies +//! that put that table under enforcement: +//! +//! ```text +//! cargo run -p trusty-policy-postgres --example emit_ddl -- public.calendar resource_scope +//! ``` + +use trusty_policy_postgres::permits::DDL_FUNCTIONS; +use trusty_policy_postgres::schema::{DDL_SCHEMA_VERSION_UPSERT, DDL_V1, SCHEMA_VERSION}; +use trusty_policy_postgres::store::policy_statements; + +fn main() { + println!("-- Generated by `cargo run -p trusty-policy-postgres --example emit_ddl`."); + println!("-- Source of truth is trusty-policy-postgres; edit there, regenerate here."); + println!("-- Applying this by hand and by GrantStore::open must agree, so do not"); + println!("-- edit this file directly."); + println!(); + + for stmt in DDL_V1 { + println!("{};", stmt.trim()); + println!(); + } + + for stmt in DDL_FUNCTIONS { + println!("{};", stmt.trim()); + println!(); + } + + // The library upserts this with a bind parameter; a file has no + // parameters, so the version is inlined. It is a constant, not + // input. + println!( + "{};", + DDL_SCHEMA_VERSION_UPSERT + .trim() + .replace("$1", &format!("'{SCHEMA_VERSION}'")) + ); + println!(); + + let args: Vec = std::env::args().skip(1).collect(); + match args.as_slice() { + [] => { + println!("-- No table given, so no policies were emitted."); + println!("-- Installing this schema protects nothing on its own: a table is"); + println!("-- guarded only once it has ENABLE + FORCE ROW LEVEL SECURITY and the"); + println!("-- four policies. Re-run with a table and its scope column, e.g."); + println!("-- ... --example emit_ddl -- public.calendar resource_scope"); + } + [table, column] => match policy_statements(table, column) { + Ok(stmts) => { + println!("-- Enforcement for {table}({column})."); + for stmt in stmts { + println!("{stmt}"); + } + } + Err(e) => { + eprintln!("error: {e}"); + std::process::exit(1); + } + }, + _ => { + eprintln!("usage: emit_ddl [ ]"); + std::process::exit(2); + } + } +} diff --git a/crates/adapters/trusty-policy-postgres/src/lib.rs b/crates/adapters/trusty-policy-postgres/src/lib.rs new file mode 100644 index 0000000..851261c --- /dev/null +++ b/crates/adapters/trusty-policy-postgres/src/lib.rs @@ -0,0 +1,282 @@ +//! Compiles a [`trusty_policy::Grant`] into a decision Postgres enforces. +//! +//! # The problem this exists to solve +//! +//! `trusty-policy` decides in Rust. Postgres decides in SQL. When those +//! two answers disagree, the database wins and nobody is told. The +//! permission model closes that by making one definition serve both: +//! a grant *compiles* to a row-level-security policy. [`Tier`] was +//! defined as the database's four verbs for exactly this reason, so the +//! compilation needs no translation layer -- and a translation layer is +//! where the two answers drift apart. +//! +//! # Grants are rows; the policy is static +//! +//! One `CREATE POLICY` per (table, verb), written once: +//! +//! ```sql +//! CREATE POLICY calendar_read ON calendar FOR SELECT +//! USING (trusty_policy.permits( +//! current_setting('trusty.subject', true), 'read', resource_scope)); +//! ``` +//! +//! Issuing a grant is then a parameterized `INSERT`, and revoking one is +//! an `UPDATE`. **No SQL is ever built from a grant's contents**, so a +//! scope string can never become syntax. The alternative -- a +//! `CREATE POLICY` per grant -- would require exactly that, and would +//! introduce the first dynamic SQL into a workspace where every +//! statement today is a static literal with `$1` placeholders. +//! +//! # What compiles, and what does not +//! +//! `scope`, `tier`, `excludes`, `expires_at` and `acts_for` reach the +//! database. `limits` -- rate, irreversible budget, dead man -- has no +//! RLS analogue and stays in [`trusty_policy::PolicyEngine`]. That gap +//! is reported, never silent: see [`CompilationReport`]. A compiler that +//! quietly drops a rate limit is worse than one that refuses. +//! +//! # Why a separate crate +//! +//! `trusty-policy` deliberately has no substrate. `CREATE POLICY` and +//! `current_setting` are Postgres dialect, so putting them in the engine +//! would give a substrate-free crate a substrate. Same split, and the +//! same reason, as `trusty-policy-audit`. + +#![deny(clippy::all)] + +pub mod permits; +pub mod schema; +pub mod store; + +use sha2::{Digest, Sha256}; +use trusty_policy::{Grant, Tier}; + +pub use permits::{ + DDL_ENSURE_PRINCIPAL, DDL_FUNCTIONS, DDL_REINSTATE_PRINCIPAL, DDL_PERMITS, MAX_DELEGATION_DEPTH, + SQL_RECONCILE_PRINCIPALS, +}; +pub use schema::{DDL_V1, SCHEMA_NAME, SCHEMA_VERSION}; +pub use store::{policy_statements, GrantStore, PolicySession, SUBJECT_SETTING}; + +/// A grant's identity: the SHA-256 of its RFC 8785 canonical JSON, hex +/// encoded. +/// +/// Content-addressed on purpose. Issuing the same grant twice is +/// idempotent rather than duplicative, and the id is precisely the bytes +/// a signature over the grant would cover -- so the row, the id and the +/// signed artifact can never disagree about what was granted. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub struct GrantId(String); + +impl GrantId { + /// Derive the id from a grant's canonical form. + pub fn of(grant: &Grant) -> Result { + let canonical = grant + .canonical_json() + .map_err(|e| PolicyPgError::Canonicalise(e.to_string()))?; + Ok(Self::of_canonical(&canonical)) + } + + /// Derive the id from canonical JSON that is already in hand. + pub fn of_canonical(canonical: &str) -> Self { + let digest = Sha256::digest(canonical.as_bytes()); + Self(hex::encode(digest)) + } + + pub fn as_str(&self) -> &str { + &self.0 + } +} + +impl std::fmt::Display for GrantId { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +/// What a grant carried that the database will not be enforcing. +/// +/// Returned alongside every issued grant so a caller cannot mistake +/// partial compilation for total. Every field named here is still +/// enforced -- by [`trusty_policy::PolicyEngine`], in the application -- +/// but *not* by Postgres, and a caller relying on the database alone +/// would be relying on something that is not there. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct CompilationReport { + /// Names of grant fields that did not reach the database. + pub uncompiled: Vec<&'static str>, +} + +impl CompilationReport { + /// Build the report for a grant. + pub fn of(grant: &Grant) -> Self { + let mut uncompiled = Vec::new(); + if grant.limits.rate_per_min.is_some() { + uncompiled.push("limits.rate_per_min"); + } + if grant.limits.irreversible_budget.is_some() { + uncompiled.push("limits.irreversible_budget"); + } + if grant.limits.dead_man_secs.is_some() { + uncompiled.push("limits.dead_man_secs"); + } + Self { uncompiled } + } + + /// True when every field of the grant reached the database, so the + /// database alone is a sufficient enforcer of it. + pub fn is_total(&self) -> bool { + self.uncompiled.is_empty() + } +} + +/// The lowercase token a [`Tier`] is stored and compared as. +/// +/// Distinct from [`Tier::sql_verb`], which gives the SQL keyword. The +/// two are 1:1 -- that is the whole design -- but they are not the same +/// string, and conflating them would put `SELECT` in a data column. +pub fn tier_token(tier: Tier) -> &'static str { + match tier { + Tier::Read => "read", + Tier::Append => "append", + Tier::Update => "update", + Tier::Delete => "delete", + } +} + +#[derive(Debug, thiserror::Error)] +pub enum PolicyPgError { + #[error("postgres: {0}")] + Postgres(#[from] postgres::Error), + #[error("could not canonicalise the grant: {0}")] + Canonicalise(String), + #[error( + "no relation named {0:?} exists, so enabling row-level security on it \ + would silently protect nothing" + )] + UnknownRelation(String), + #[error( + "no principal named {0:?} is registered, and a key bound to nobody is a key nothing can revoke" + )] + UnknownPrincipal(String), +} + +#[cfg(test)] +mod tests { + use super::*; + use trusty_policy::Limits; + + fn grant() -> Grant { + Grant { + why: Some("Sam's scheduling agent may propose, never book.".into()), + subject: "agent:01J".into(), + acts_for: Some("user:01H".into()), + scope: "board:9f2/calendar".into(), + tier: Tier::Append, + excludes: vec!["board:9f2/calendar/private".into()], + limits: Limits::default(), + expires_at: 1_788_000_000, + } + } + + #[test] + fn the_id_is_the_hash_of_the_canonical_form() { + let g = grant(); + let canonical = g.canonical_json().expect("canonical form"); + assert_eq!( + GrantId::of(&g).expect("id"), + GrantId::of_canonical(&canonical), + "the id must be derivable from the canonical bytes alone, or a \ + verifier holding only the signed artifact could not name the row" + ); + } + + #[test] + fn issuing_the_same_grant_twice_yields_one_id() { + assert_eq!( + GrantId::of(&grant()).expect("id"), + GrantId::of(&grant()).expect("id"), + "a content-addressed id makes re-issuing idempotent rather than \ + quietly accumulating duplicate permissions" + ); + } + + #[test] + fn changing_why_changes_the_id() { + let mut other = grant(); + other.why = Some("a different reason entirely".into()); + assert_ne!( + GrantId::of(&grant()).expect("id"), + GrantId::of(&other).expect("id"), + "the justification is inside the hash, not beside it: a grant \ + whose stated reason can be edited without changing its identity \ + is missing the half an auditor asks about" + ); + } + + #[test] + fn a_grant_without_limits_compiles_totally() { + let report = CompilationReport::of(&grant()); + assert!( + report.is_total(), + "with no limits set there is nothing the database cannot express, \ + so it alone is a sufficient enforcer; got {report:?}" + ); + } + + #[test] + fn a_grant_with_limits_reports_them_as_uncompiled() { + let mut g = grant(); + g.limits = Limits { + rate_per_min: Some(20), + irreversible_budget: Some(0), + dead_man_secs: Some(1800), + }; + let report = CompilationReport::of(&g); + assert!( + !report.is_total(), + "a rate limit that reached no enforcer must never look enforced" + ); + assert_eq!( + report.uncompiled, + vec![ + "limits.rate_per_min", + "limits.irreversible_budget", + "limits.dead_man_secs" + ], + "each unenforceable field must be named, so the caller knows which \ + enforcement it still owes the gate" + ); + } + + #[test] + fn the_stored_token_is_not_the_sql_verb() { + assert_eq!(tier_token(Tier::Read), "read"); + assert_eq!(Tier::Read.sql_verb(), "SELECT"); + for tier in [Tier::Read, Tier::Append, Tier::Update, Tier::Delete] { + assert_ne!( + tier_token(tier), + tier.sql_verb(), + "the data token and the SQL keyword are 1:1 but not identical; \ + conflating them would put a SQL keyword in a data column" + ); + } + } + + #[test] + fn every_tier_has_a_distinct_token() { + let tokens: Vec<_> = [Tier::Read, Tier::Append, Tier::Update, Tier::Delete] + .into_iter() + .map(tier_token) + .collect(); + let mut sorted = tokens.clone(); + sorted.sort_unstable(); + sorted.dedup(); + assert_eq!( + sorted.len(), + tokens.len(), + "two tiers sharing a token would let a grant for one silently \ + authorise the other" + ); + } +} diff --git a/crates/adapters/trusty-policy-postgres/src/permits.rs b/crates/adapters/trusty-policy-postgres/src/permits.rs new file mode 100644 index 0000000..e8625b7 --- /dev/null +++ b/crates/adapters/trusty-policy-postgres/src/permits.rs @@ -0,0 +1,508 @@ +//! The decision function every row-level-security policy calls. +//! +//! This is the compilation target. A `CREATE POLICY` written once per +//! (table, verb) delegates to [`DDL_PERMITS`], and grants are rows, so +//! issuing or revoking a permission never touches DDL and no SQL is ever +//! built from a grant's contents. +//! +//! The evaluation order is the permission model's, and the order is +//! load-bearing rather than incidental: +//! +//! 1. **Exclusion wins**, checked first, across *all* of the subject's +//! live grants regardless of tier. An exclusion can never be +//! overridden by a wider grant -- so it cannot be evaluated after +//! coverage, or a second grant would reopen what the first shut. +//! 2. **Expiry** is wall clock, independent of whether work finished. +//! 3. **Coverage** -- some live grant's scope contains the resource. +//! 4. **Tier** matches exactly. +//! 5. **Delegation intersects**: an agent is never a principal in its +//! own right, so the principal it acts for must independently permit +//! the same thing. +//! 6. Anything unmatched, NULL or unreadable is **false**. Absent policy +//! is not permission. + +/// Scope containment. +/// +/// `starts_with`, never `LIKE`: a scope containing `%` or `_` would +/// otherwise widen itself, and the scope string is attacker-influenced +/// in exactly the deployments this is for. +pub const DDL_COVERS: &str = r#" +CREATE OR REPLACE FUNCTION trusty_policy.covers(scope TEXT, resource TEXT) +RETURNS BOOLEAN +LANGUAGE sql IMMUTABLE PARALLEL SAFE +AS $fn$ + SELECT scope IS NOT NULL + AND resource IS NOT NULL + AND (resource = scope OR starts_with(resource, scope || '/')) +$fn$ +"#; + +/// How many `acts_for` hops are followed before the answer is refusal. +/// +/// A delegation cycle -- A acts for B, B acts for A -- would otherwise +/// recurse until the stack gave out, and a stack overflow inside a +/// policy predicate is an availability bug in the authorisation path. +/// Refusing past the bound keeps the failure mode deny-by-default. +pub const MAX_DELEGATION_DEPTH: i32 = 4; + +/// The decision function. `SECURITY DEFINER` with a pinned +/// `search_path`, so the application role can consult grants without +/// being able to read or write them directly. +pub const DDL_PERMITS: &str = r#" +CREATE OR REPLACE FUNCTION trusty_policy.permits( + subject TEXT, + tier TEXT, + resource TEXT, + depth INT DEFAULT 0 +) RETURNS BOOLEAN +LANGUAGE plpgsql +STABLE +SECURITY DEFINER +SET search_path = trusty_policy, pg_temp +AS $fn$ +DECLARE + g RECORD; +BEGIN + -- Deny by default. An unreadable or missing subject refuses; it + -- does not fall back to something wider. + IF subject IS NULL OR subject = '' + OR tier IS NULL OR resource IS NULL THEN + RETURN FALSE; + END IF; + + -- A delegation cycle is refused, not followed. + IF depth > 4 THEN + RETURN FALSE; + END IF; + + -- 0. Identity before permission. A grant held by a principal who + -- was never registered is a grant to nobody, and one held by a + -- revoked principal must stop at once rather than grant by grant + -- -- revoking them one at a time is a race with whoever is using + -- them. This is what makes "kill the agent's key, the person is + -- untouched" true in practice. + IF NOT EXISTS ( + SELECT 1 + FROM trusty_policy.principal p + WHERE p.id = permits.subject + AND p.revoked_at IS NULL + ) THEN + RETURN FALSE; + END IF; + + -- 1. Exclusion wins, and is checked before coverage so that a + -- second, wider grant cannot reopen what an exclusion shut. + -- Deliberately NOT filtered by tier: an exclusion is a statement + -- about a resource, not about a verb. + IF EXISTS ( + SELECT 1 + FROM trusty_policy.permission_grant pgr, + LATERAL unnest(pgr.excludes) AS ex(ref) + WHERE pgr.subject = permits.subject + AND pgr.revoked_at IS NULL + AND trusty_policy.covers(ex.ref, permits.resource) + ) THEN + RETURN FALSE; + END IF; + + -- 2, 3, 4. A live grant, unexpired by the wall clock, of exactly + -- this tier, whose scope covers the resource. + FOR g IN + SELECT pgr.acts_for + FROM trusty_policy.permission_grant pgr + WHERE pgr.subject = permits.subject + AND pgr.tier = permits.tier + AND pgr.revoked_at IS NULL + AND pgr.expires_at > now() + AND trusty_policy.covers(pgr.scope, permits.resource) + LOOP + -- 5. effective = grant(agent) INTERSECT permission(acts_for). + -- A grant naming no principal stands on its own. + IF g.acts_for IS NULL THEN + RETURN TRUE; + END IF; + + IF EXISTS ( + SELECT 1 + FROM trusty_policy.delegation d + WHERE d.agent_id = permits.subject + AND d.acts_for_id = g.acts_for + AND d.revoked_at IS NULL + AND (d.expires_at IS NULL OR d.expires_at > now()) + ) AND trusty_policy.permits( + g.acts_for, permits.tier, permits.resource, depth + 1 + ) THEN + RETURN TRUE; + END IF; + END LOOP; + + -- 6. No matching grant is a refusal. + RETURN FALSE; +END; +$fn$ +"#; + +/// Applied after [`crate::schema::DDL_V1`], in the same transaction. +pub const DDL_FUNCTIONS: &[&str] = &[ + DDL_COVERS, + DDL_PERMITS, + DDL_ENSURE_PRINCIPAL, + DDL_REINSTATE_PRINCIPAL, +]; + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn the_decision_function_pins_its_search_path() { + assert!( + DDL_PERMITS.contains("SET search_path = trusty_policy, pg_temp"), + "a SECURITY DEFINER function without a pinned search_path is a \ + privilege escalation: anyone able to set search_path could shadow \ + permission_grant with a table of their own" + ); + } + + #[test] + fn security_definer_never_appears_without_a_pinned_search_path() { + for ddl in DDL_FUNCTIONS { + if ddl.contains("SECURITY DEFINER") { + assert!( + ddl.contains("SET search_path ="), + "every SECURITY DEFINER function must pin search_path, not \ + just the one that happens to be checked above: {ddl}" + ); + } + } + } + + #[test] + fn scope_containment_never_uses_like() { + for ddl in DDL_FUNCTIONS { + assert!( + !ddl.to_uppercase().contains(" LIKE "), + "LIKE would let a scope containing % or _ widen itself, and the \ + scope string is attacker-influenced: {ddl}" + ); + } + assert!( + DDL_COVERS.contains("starts_with(resource, scope || '/')"), + "containment must be a literal prefix test on a path separator" + ); + } + + #[test] + fn a_prefix_that_is_not_a_path_segment_does_not_match() { + // covers('a/b', 'a/bc') must be false: 'a/bc' is not inside + // 'a/b'. The separator in the concatenation is what enforces it. + assert!( + DDL_COVERS.contains("scope || '/'"), + "without the separator, scope 'a/b' would cover 'a/bcd', silently \ + granting a sibling resource whose name merely starts the same way" + ); + } + + #[test] + fn exclusion_is_evaluated_before_coverage() { + let exclusion = DDL_PERMITS + .find("unnest(pgr.excludes)") + .expect("the exclusion check must exist"); + let coverage = DDL_PERMITS + .find("FOR g IN") + .expect("the coverage loop must exist"); + assert!( + exclusion < coverage, + "an exclusion checked after coverage could be overridden by a wider \ + grant, which is exactly what the model forbids" + ); + } + + #[test] + fn exclusions_are_not_filtered_by_tier() { + let start = DDL_PERMITS + .find("unnest(pgr.excludes)") + .expect("the exclusion check must exist"); + let end = DDL_PERMITS[start..] + .find("RETURN FALSE") + .expect("the exclusion check must refuse") + + start; + assert!( + !DDL_PERMITS[start..end].contains("pgr.tier"), + "an exclusion is a statement about a resource, not about a verb; \ + filtering it by tier would let a read grant expose what a write \ + grant excluded" + ); + } + + + #[test] + fn an_unregistered_or_revoked_principal_refuses() { + assert!( + DDL_PERMITS.contains("FROM trusty_policy.principal p"), + "a grant whose subject was never registered is a grant to nobody, and deny-by-default says that refuses" + ); + let start = DDL_PERMITS + .find("FROM trusty_policy.principal p") + .expect("the principal check must exist"); + assert!( + DDL_PERMITS[start..].starts_with( + "FROM trusty_policy.principal p + WHERE p.id = permits.subject + AND p.revoked_at IS NULL" + ), + "revoking a principal must stop every grant it holds at once, not leave them to be revoked one at a time" + ); + } + + #[test] + fn identity_is_checked_before_any_grant_is_read() { + let identity = DDL_PERMITS + .find("FROM trusty_policy.principal p") + .expect("the principal check must exist"); + let exclusion = DDL_PERMITS + .find("unnest(pgr.excludes)") + .expect("the exclusion check must exist"); + assert!( + identity < exclusion, + "asking who someone is before asking what they may do is both cheaper and the right order to reason about; a revoked principal should never reach the grant table at all" + ); + } + + #[test] + fn a_delegation_cycle_is_refused_rather_than_followed() { + assert!( + DDL_PERMITS.contains("depth > 4"), + "A acts for B and B acts for A would recurse until the stack gave \ + out, and a stack overflow inside a policy predicate is an \ + availability bug in the authorisation path" + ); + assert_eq!( + MAX_DELEGATION_DEPTH, 4, + "the documented bound and the bound in the SQL must agree" + ); + } + + #[test] + fn the_delegated_side_is_checked_independently() { + assert!( + DDL_PERMITS.contains("trusty_policy.permits(\n g.acts_for"), + "the principal must be asked the same question, not merely looked \ + up: an agent is never a principal in its own right" + ); + } + + #[test] + fn a_revoked_delegation_stops_permitting() { + let start = DDL_PERMITS + .find("FROM trusty_policy.delegation d") + .expect("the delegation lookup must exist"); + let window = &DDL_PERMITS[start..]; + assert!( + window.contains("d.revoked_at IS NULL"), + "revocation must take effect at the decision, not merely be recorded" + ); + assert!( + window.contains("d.expires_at IS NULL OR d.expires_at > now()"), + "an expired delegation must stop permitting; NULL means no expiry, \ + which is not the same as expired" + ); + } + + #[test] + fn expiry_is_wall_clock_and_revocation_is_immediate() { + assert!( + DDL_PERMITS.contains("pgr.expires_at > now()"), + "expiry is wall clock, independent of whether work finished" + ); + assert!( + DDL_PERMITS.contains("pgr.revoked_at IS NULL"), + "a revoked grant must stop permitting without any DDL running" + ); + } + + #[test] + fn a_null_or_empty_subject_refuses() { + assert!( + DDL_PERMITS.contains("subject IS NULL OR subject = ''"), + "current_setting('trusty.subject', true) returns NULL when unset \ + and '' in some paths; either must refuse, because absent policy \ + is not permission" + ); + } + + #[test] + fn the_function_is_stable_not_volatile() { + assert!( + DDL_PERMITS.contains("STABLE"), + "a VOLATILE function cannot be used in an RLS predicate efficiently \ + and would be re-evaluated per row without caching" + ); + } + + #[test] + fn nothing_here_is_created_outside_the_trusty_policy_schema() { + for ddl in DDL_FUNCTIONS { + assert!( + ddl.contains("FUNCTION trusty_policy."), + "an unqualified function would land in whatever search_path \ + happened to be set: {ddl}" + ); + } + } +} + +/// Create a principal if missing, and refresh its display name. +/// +/// The portable half of enrolment. A consumer whose users arrive through +/// its own auth table hangs a thin trigger on that table and calls this; +/// the trigger is the consumer's, because only they know which table +/// their users live in, but the logic is here so both sides cannot +/// drift. +/// +/// `SECURITY DEFINER` with a pinned `search_path`, for the same reason +/// [`DDL_PERMITS`] is: without the pin, anyone able to set `search_path` +/// could shadow `principal` with a table of their own. +/// +/// **It does not un-revoke.** An earlier version did, on the reasoning +/// that re-enrolment is the same decision said again. That reasoning +/// holds for a person deliberately re-enrolling and fails for an +/// automated repair job -- and the function cannot tell them apart. +/// Reconciliation calls this same verb, so un-revoking here meant a +/// revoked principal whose auth row still existed would be resurrected +/// by the next backfill, silently undoing a deliberate act. Bringing +/// someone back is [`DDL_REINSTATE_PRINCIPAL`], which a trigger has no +/// reason to reach. +pub const DDL_ENSURE_PRINCIPAL: &str = r#" +CREATE OR REPLACE FUNCTION trusty_policy.ensure_principal( + p_id TEXT, + p_kind TEXT, + p_display TEXT DEFAULT '' +) RETURNS VOID +LANGUAGE plpgsql +SECURITY DEFINER +SET search_path = trusty_policy, pg_temp +AS $fn$ +BEGIN + INSERT INTO trusty_policy.principal (id, kind, display) + VALUES (p_id, p_kind, COALESCE(p_display, '')) + ON CONFLICT (id) DO UPDATE + SET display = EXCLUDED.display; +END; +$fn$ +"#; + +/// Bring a revoked principal back, deliberately. +/// +/// Separate from [`DDL_ENSURE_PRINCIPAL`] because "create if missing" +/// and "bring someone back" are different intentions, and only one of +/// them should be reachable from a trigger. Revocation is an act someone +/// may have to justify; undoing it should be equally deliberate, and +/// equally visible in whatever called it. +/// +/// Returns whether a revoked principal was actually reinstated, so a +/// caller reinstating someone already live is told rather than +/// reassured. +pub const DDL_REINSTATE_PRINCIPAL: &str = r#" +CREATE OR REPLACE FUNCTION trusty_policy.reinstate_principal(p_id TEXT) +RETURNS BOOLEAN +LANGUAGE plpgsql +SECURITY DEFINER +SET search_path = trusty_policy, pg_temp +AS $fn$ +DECLARE + changed INT; +BEGIN + UPDATE trusty_policy.principal + SET revoked_at = NULL + WHERE id = p_id AND revoked_at IS NOT NULL; + GET DIAGNOSTICS changed = ROW_COUNT; + RETURN changed = 1; +END; +$fn$ +"#; + +/// Principals that should exist and do not, or exist and are revoked. +/// +/// **Must always return zero rows.** `permits()` refuses for a subject +/// that is not a registered live principal, so a missed enrolment is not +/// degraded access -- it is a total lockout, with nothing in any log +/// saying why. This is the query that catches it before a person does. +/// +/// Takes a table and column to reconcile against, so a consumer can +/// point it at whatever holds its users. Returns the ids that are +/// missing and the ones that are revoked, separately: they need +/// different fixes. +pub const SQL_RECONCILE_PRINCIPALS: &str = r#" +SELECT src.id, + CASE WHEN p.id IS NULL THEN 'missing' ELSE 'revoked' END AS problem + FROM (SELECT $1::text[] AS ids) AS _ + CROSS JOIN LATERAL unnest(_.ids) AS src(id) + LEFT JOIN trusty_policy.principal p + ON p.id = src.id AND p.revoked_at IS NULL + WHERE p.id IS NULL +"#; + +#[cfg(test)] +mod enrolment_tests { + use super::*; + + #[test] + fn ensure_principal_pins_its_search_path() { + assert!( + DDL_ENSURE_PRINCIPAL.contains("SET search_path = trusty_policy, pg_temp"), + "a SECURITY DEFINER function without a pinned search_path lets \ + anyone who can set search_path shadow `principal` with a table of \ + their own" + ); + } + + #[test] + fn enrolment_cannot_resurrect_a_revoked_principal() { + assert!( + !DDL_ENSURE_PRINCIPAL.contains("revoked_at"), + "reconciliation calls this same verb, so un-revoking here would let a backfill resurrect a principal somebody deliberately revoked -- and permits() refuses a revoked principal, so it would restore everything at once" + ); + } + + #[test] + fn bringing_someone_back_is_a_separate_verb() { + assert!( + DDL_REINSTATE_PRINCIPAL.contains("revoked_at = NULL"), + "there must still be a way back, or revocation is a trapdoor" + ); + assert!( + DDL_REINSTATE_PRINCIPAL.contains("revoked_at IS NOT NULL"), + "and it must report whether it actually changed anything, so a caller reinstating someone already live is told rather than reassured" + ); + assert!( + DDL_REINSTATE_PRINCIPAL.contains("SET search_path = trusty_policy, pg_temp"), + "a SECURITY DEFINER function without a pinned search_path is a privilege escalation, and this one can restore authority" + ); + } + + #[test] + fn enrolment_is_idempotent() { + assert!( + DDL_ENSURE_PRINCIPAL.contains("ON CONFLICT (id) DO UPDATE"), + "a trigger fires more than once in practice; a second call must \ + not fail and must not duplicate" + ); + } + + #[test] + fn reconciliation_separates_missing_from_revoked() { + assert!( + SQL_RECONCILE_PRINCIPALS.contains("'missing'") + && SQL_RECONCILE_PRINCIPALS.contains("'revoked'"), + "the two need different fixes -- one is an enrolment that never \ + ran, the other a deliberate act somebody may need to justify -- \ + so a query reporting them alike hides the difference" + ); + assert!( + SQL_RECONCILE_PRINCIPALS.contains("revoked_at IS NULL"), + "a revoked principal is not a present one as far as permits() is \ + concerned, so reconciliation must count it as a problem" + ); + } +} diff --git a/crates/adapters/trusty-policy-postgres/src/schema.rs b/crates/adapters/trusty-policy-postgres/src/schema.rs new file mode 100644 index 0000000..5510508 --- /dev/null +++ b/crates/adapters/trusty-policy-postgres/src/schema.rs @@ -0,0 +1,312 @@ +//! Schema DDL for the grant store. +//! +//! Follows `trusty-postgres`'s idiom exactly: a `&[&str]` of +//! `CREATE ... IF NOT EXISTS` applied in one transaction on every open, +//! followed by a `schema_meta` version upsert. There is no migration +//! runner here and there should not be one -- the consumer's migration +//! tool owns *its* tables; this owns only the permission model's own. +//! +//! Everything lives in its own `trusty_policy` schema so it can never +//! collide with an application table, and so revoking write access to +//! the whole schema is one statement. + +pub const SCHEMA_VERSION: &str = "1"; + +/// The Postgres schema every object below lives in. +pub const SCHEMA_NAME: &str = "trusty_policy"; + +pub const DDL_V1: &[&str] = &[ + r#"CREATE SCHEMA IF NOT EXISTS trusty_policy"#, + // ---------------------------------------------------------------- + // principal -- one table, four kinds. + // + // `id` is TEXT holding the full reference ("agent:01J..."), not a + // bare UUID. The permission model's canonical grant JSON carries + // subjects in that prefixed form, and `Grant.subject` is a String; + // storing a bare uuid would mean parsing the prefix off on every + // write and re-attaching it on every read, which is exactly the + // translation layer this design exists to avoid. + // + // The CHECK keeps the prefix honest, so `kind` cannot drift from + // what the id claims. + // ---------------------------------------------------------------- + r#"CREATE TABLE IF NOT EXISTS trusty_policy.principal ( + id TEXT PRIMARY KEY, + kind TEXT NOT NULL CHECK (kind IN ('org','user','agent','program')), + display TEXT NOT NULL DEFAULT '', + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + -- The `key` edge. Points at trusty's producer registry + -- (producer_identity.producer_oid), which is the agent identity + -- system; there is not to be a second one. Deliberately no + -- foreign key: that table belongs to trusty-postgres, lives in + -- another schema, and may not be installed at all. The edge is + -- recorded here and resolved through lookup_producer. + producer_oid TEXT UNIQUE, + -- Revoking the identity itself, as distinct from revoking any + -- one grant. Kills every grant the principal holds at once, + -- which is what "kill the agent's key, the person is untouched" + -- has to mean in practice. + revoked_at TIMESTAMPTZ, + -- What KIND of thing this is, structurally, is `kind`. What kind + -- of ORGANISATION it is -- nonprofit, agency, prime, sub -- is + -- descriptive and belongs here, not there. + -- + -- Kept apart deliberately. Folded into `kind`, every new + -- organisation type becomes a change to permits(), and the + -- authorisation function ends up knowing about tax status. The + -- CHECK on `kind` is what keeps the two from being conflated by + -- accident. + org_type TEXT, + CONSTRAINT principal_id_matches_kind CHECK (starts_with(id, kind || ':')), + -- Only an org has an org type. A user with one is a modelling + -- mistake that would otherwise sit there looking deliberate. + CONSTRAINT org_type_only_on_orgs CHECK (org_type IS NULL OR kind = 'org') + )"#, + // Idempotent columns for an installation that predates them. + // CREATE TABLE IF NOT EXISTS silently does nothing when the table is + // already there, so a new column needs saying twice. + r#"ALTER TABLE trusty_policy.principal + ADD COLUMN IF NOT EXISTS producer_oid TEXT"#, + r#"ALTER TABLE trusty_policy.principal + ADD COLUMN IF NOT EXISTS revoked_at TIMESTAMPTZ"#, + r#"ALTER TABLE trusty_policy.principal + ADD COLUMN IF NOT EXISTS org_type TEXT"#, + r#"CREATE UNIQUE INDEX IF NOT EXISTS idx_principal_producer_oid + ON trusty_policy.principal(producer_oid)"#, + // ---------------------------------------------------------------- + // membership -- user belongs to org. + // ---------------------------------------------------------------- + r#"CREATE TABLE IF NOT EXISTS trusty_policy.membership ( + member_id TEXT NOT NULL REFERENCES trusty_policy.principal(id) ON DELETE CASCADE, + org_id TEXT NOT NULL REFERENCES trusty_policy.principal(id) ON DELETE CASCADE, + role TEXT NOT NULL DEFAULT '', + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + PRIMARY KEY (member_id, org_id) + )"#, + // ---------------------------------------------------------------- + // delegation -- agent acts for user | org. + // + // Deliberately NOT cascaded on delete: revoking a delegation is + // `revoked_at`, and losing the row would lose the record that the + // delegation ever existed. An audit asks "who could act for whom + // last March", and a deleted row cannot answer. + // ---------------------------------------------------------------- + r#"CREATE TABLE IF NOT EXISTS trusty_policy.delegation ( + agent_id TEXT NOT NULL REFERENCES trusty_policy.principal(id), + acts_for_id TEXT NOT NULL REFERENCES trusty_policy.principal(id), + granted_by TEXT REFERENCES trusty_policy.principal(id), + granted_at TIMESTAMPTZ NOT NULL DEFAULT now(), + expires_at TIMESTAMPTZ, + revoked_at TIMESTAMPTZ, + PRIMARY KEY (agent_id, acts_for_id) + )"#, + // ---------------------------------------------------------------- + // permission_grant -- the grant itself. + // + // Named `permission_grant`, not `grant`: GRANT is a reserved word in + // SQL, and a table that must be double-quoted at every call site is + // a footgun waiting for the one place someone forgets. + // + // `id` is the SHA-256 of `canonical_json`, so a grant's identity IS + // its content. Issuing the same grant twice is idempotent, and the + // id is exactly what a signature over the grant would cover. + // + // `canonical_json` is stored verbatim -- RFC 8785 bytes, not a JSONB + // column. JSONB renormalises whitespace and key order, which would + // destroy the very property the canonical form exists to provide. + // Same reasoning that keeps `raw_payload` TEXT in trusty-postgres. + // ---------------------------------------------------------------- + r#"CREATE TABLE IF NOT EXISTS trusty_policy.permission_grant ( + id TEXT PRIMARY KEY, + subject TEXT NOT NULL, + acts_for TEXT, + scope TEXT NOT NULL, + tier TEXT NOT NULL CHECK (tier IN ('read','append','update','delete')), + excludes TEXT[] NOT NULL DEFAULT '{}', + expires_at TIMESTAMPTZ NOT NULL, + why TEXT, + canonical_json TEXT NOT NULL, + issued_at TIMESTAMPTZ NOT NULL DEFAULT now(), + revoked_at TIMESTAMPTZ + )"#, + // Every permits() call filters on subject first, then tier. + r#"CREATE INDEX IF NOT EXISTS idx_grant_subject_tier + ON trusty_policy.permission_grant(subject, tier)"#, + // Exclusions are checked across ALL of a subject's grants, including + // ones whose tier does not match, so they need their own path in. + r#"CREATE INDEX IF NOT EXISTS idx_grant_subject + ON trusty_policy.permission_grant(subject)"#, + r#"CREATE TABLE IF NOT EXISTS trusty_policy.schema_meta ( + key TEXT PRIMARY KEY, + value TEXT NOT NULL + )"#, +]; + +/// Upserts the schema_version row. Runs on every open to declare the +/// version this build expects. Mirrors `trusty-postgres`. +pub const DDL_SCHEMA_VERSION_UPSERT: &str = r#" + INSERT INTO trusty_policy.schema_meta(key, value) VALUES ('schema_version', $1) + ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value +"#; + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn every_statement_is_idempotent() { + for stmt in DDL_V1 { + let upper = stmt.to_uppercase(); + assert!( + upper.contains("IF NOT EXISTS"), + "the schema is applied on every open, so a statement without \ + IF NOT EXISTS would fail the second time a process starts: {stmt}" + ); + } + } + + #[test] + fn nothing_is_created_outside_the_trusty_policy_schema() { + for stmt in DDL_V1 { + if stmt.to_uppercase().starts_with("CREATE SCHEMA") { + continue; + } + assert!( + stmt.contains("trusty_policy."), + "an unqualified object would land in whatever search_path \ + happened to be set, which is not a place to keep permissions: {stmt}" + ); + } + } + + #[test] + fn the_grant_table_avoids_the_reserved_word() { + let ddl = DDL_V1.join("\n").to_uppercase(); + assert!( + !ddl.contains("EXISTS TRUSTY_POLICY.GRANT "), + "GRANT is reserved in SQL; the table is permission_grant precisely \ + so no call site has to remember to double-quote it" + ); + } + + #[test] + fn the_canonical_form_is_stored_as_text_not_jsonb() { + let ddl = DDL_V1.join("\n"); + assert!( + ddl.contains("canonical_json TEXT"), + "JSONB renormalises whitespace and key order, which would destroy \ + the exact property RFC 8785 canonicalisation exists to give" + ); + assert!( + !ddl.to_uppercase().contains("JSONB"), + "no column here may be JSONB, for the same reason" + ); + } + + #[test] + fn tier_is_constrained_to_the_four_verbs() { + let ddl = DDL_V1.join("\n"); + assert!( + ddl.contains("tier IN ('read','append','update','delete')"), + "a tier outside the four is a permission nobody can enforce; \ + the database should refuse to store one" + ); + } + + #[test] + fn a_principals_kind_cannot_drift_from_its_id() { + let ddl = DDL_V1.join("\n"); + assert!( + ddl.contains("principal_id_matches_kind"), + "if kind and the id prefix can disagree, an audit line reading \ + 'agent:...' whose kind says 'user' is unreadable" + ); + } + + + + #[test] + fn org_type_is_descriptive_and_kind_is_structural() { + let ddl = DDL_V1.join(" +"); + assert!( + ddl.contains("org_type"), + "an organisation's type is descriptive; folded into `kind` it would make every new organisation type a change to permits()" + ); + assert!( + ddl.contains("kind IN ('org','user','agent','program')"), + "and `kind` must stay the closed structural set, or the two collapse back into one column" + ); + assert!( + ddl.contains("org_type_only_on_orgs"), + "a user carrying an org type is a modelling mistake that would otherwise sit there looking deliberate" + ); + } + + #[test] + fn permits_never_needs_to_read_org_type() { + // Structural, not a runtime check: the decision function selects + // its own columns, and org_type is not among them. If it ever + // is, authorisation has started depending on what sort of + // organisation someone is, which is the coupling this split + // exists to prevent. + assert!( + !crate::permits::DDL_PERMITS.contains("org_type"), + "permits() must not read org_type" + ); + } + + #[test] + fn the_key_edge_points_at_the_producer_registry() { + let ddl = DDL_V1.join(" +"); + assert!( + ddl.contains("producer_oid"), + "the third edge -- principal has an ed25519 key -- must be recorded, and it must point at trusty's producer registry rather than at a second identity system" + ); + assert!( + !ddl.contains("ed25519_pubkey"), + "storing the key material here would BE the second identity system; only the reference to the registry belongs here" + ); + } + + #[test] + fn a_principal_can_be_revoked_as_a_whole() { + let ddl = DDL_V1.join(" +"); + assert!( + ddl.contains("revoked_at TIMESTAMPTZ"), + "killing an agent's key must stop every grant it holds at once; revoking them one at a time is a race with whoever is using them" + ); + } + + #[test] + fn a_column_added_after_v1_is_added_idempotently_too() { + let ddl = DDL_V1.join(" +"); + for col in ["producer_oid", "revoked_at"] { + assert!( + ddl.contains(&format!("ADD COLUMN IF NOT EXISTS {col}")), + "CREATE TABLE IF NOT EXISTS does nothing when the table already exists, so a column added later never reaches an existing installation unless it is also ALTERed in: {col}" + ); + } + } + + #[test] + fn a_delegation_is_revoked_not_deleted() { + let delegation = DDL_V1 + .iter() + .find(|s| s.contains("trusty_policy.delegation")) + .expect("the delegation table must exist"); + assert!( + delegation.contains("revoked_at"), + "revocation must be recorded, not effected by deletion" + ); + assert!( + !delegation.contains("ON DELETE CASCADE"), + "cascading a delegation away would erase the record that it ever \ + existed, and 'who could act for whom last March' is exactly what \ + an audit asks" + ); + } +} diff --git a/crates/adapters/trusty-policy-postgres/src/store.rs b/crates/adapters/trusty-policy-postgres/src/store.rs new file mode 100644 index 0000000..3602bc5 --- /dev/null +++ b/crates/adapters/trusty-policy-postgres/src/store.rs @@ -0,0 +1,616 @@ +//! Issuing grants, and putting a table under their protection. +//! +//! Three things live here: +//! +//! - [`GrantStore`] — issue and revoke grants. Both are parameterized +//! statements against rows; neither runs DDL. +//! - [`PolicySession`] — set the subject for one transaction, so a +//! policy predicate has someone to ask about. +//! - [`GrantStore::enforce`] — the one place a table name reaches SQL, +//! and the only DDL a consumer runs after install. + +use postgres::{Client, NoTls, Transaction}; +use trusty_policy::{Grant, Tier}; + +use crate::permits::DDL_FUNCTIONS; +use crate::schema::{DDL_SCHEMA_VERSION_UPSERT, DDL_V1, SCHEMA_VERSION}; +use crate::{tier_token, CompilationReport, GrantId, PolicyPgError}; + +/// The GUC a policy predicate reads the acting subject from. +/// +/// Set transaction-locally by [`PolicySession::begin`]. Transaction +/// local, not session local, so a pooled connection handed to the next +/// request cannot still be carrying the last one's identity. +pub const SUBJECT_SETTING: &str = "trusty.subject"; + +/// Reads and writes grants. +pub struct GrantStore { + client: Client, +} + +impl GrantStore { + /// Connect and apply the schema idempotently. + pub fn open(conn_str: &str) -> Result { + let mut client = Client::connect(conn_str, NoTls)?; + apply_schema(&mut client)?; + Ok(Self { client }) + } + + /// Adopt an existing connection, applying the schema to it. + pub fn adopt(mut client: Client) -> Result { + apply_schema(&mut client)?; + Ok(Self { client }) + } + + pub fn client(&mut self) -> &mut Client { + &mut self.client + } + + /// Record a grant, and say what about it the database will not be + /// enforcing. + /// + /// Idempotent: the id is the hash of the canonical form, so issuing + /// the same grant twice leaves one row rather than two permissions. + /// Re-issuing a previously revoked grant un-revokes it, which is the + /// same decision said again. + pub fn issue(&mut self, grant: &Grant) -> Result<(GrantId, CompilationReport), PolicyPgError> { + let canonical = grant + .canonical_json() + .map_err(|e| PolicyPgError::Canonicalise(e.to_string()))?; + let id = GrantId::of_canonical(&canonical); + let expires = expires_at_sql(grant.expires_at); + + self.client.execute( + "INSERT INTO trusty_policy.permission_grant + (id, subject, acts_for, scope, tier, excludes, + expires_at, why, canonical_json) + VALUES ($1, $2, $3, $4, $5, $6, to_timestamp($7), $8, $9) + ON CONFLICT (id) DO UPDATE SET revoked_at = NULL", + &[ + &id.as_str(), + &grant.subject, + &grant.acts_for, + &grant.scope, + &tier_token(grant.tier), + &grant.excludes, + &expires, + &grant.why, + &canonical, + ], + )?; + + Ok((id, CompilationReport::of(grant))) + } + + /// Register a principal, or update its display name. + /// + /// `kind` must be one of `org`, `user`, `agent`, `program`, and the + /// id must carry it as a prefix — `agent:01J...`. The database + /// enforces both; this returns the error rather than papering over + /// it, because a principal whose kind disagrees with its id makes + /// every audit line that mentions it unreadable. + /// + /// Registering an already-revoked principal reinstates it, which is + /// the same decision said again. + pub fn register_principal( + &mut self, + id: &str, + kind: &str, + display: &str, + ) -> Result<(), PolicyPgError> { + self.client.execute( + "INSERT INTO trusty_policy.principal (id, kind, display) + VALUES ($1, $2, $3) + ON CONFLICT (id) DO UPDATE + SET display = EXCLUDED.display, revoked_at = NULL", + &[&id, &kind, &display], + )?; + Ok(()) + } + + /// Record the `key` edge: this principal is that producer. + /// + /// `producer_oid` refers to trusty's producer registry — + /// `register_producer` / `lookup_producer` / `revoke_producer` in + /// `trusty-postgres`. That registry is the agent identity system and + /// there is not to be a second one, so no key material is stored + /// here; only the reference. + /// + /// There is deliberately no foreign key. `producer_identity` belongs + /// to a different crate and schema and may not be installed at all, + /// and a policy engine that refuses to start without a chain store + /// would be a policy engine with a substrate. + pub fn bind_key(&mut self, principal: &str, producer_oid: &str) -> Result<(), PolicyPgError> { + let n = self.client.execute( + "UPDATE trusty_policy.principal SET producer_oid = $2 WHERE id = $1", + &[&principal, &producer_oid], + )?; + if n == 0 { + return Err(PolicyPgError::UnknownPrincipal(principal.to_string())); + } + Ok(()) + } + + /// The producer this principal is, if the edge has been recorded. + /// + /// Resolving it to an actual public key goes through the producer + /// registry's own `lookup_producer`. Doing that here would mean + /// reimplementing half of it. + pub fn producer_oid_of(&mut self, principal: &str) -> Result, PolicyPgError> { + let row = self.client.query_opt( + "SELECT producer_oid FROM trusty_policy.principal WHERE id = $1", + &[&principal], + )?; + Ok(row.and_then(|r| r.get::<_, Option>(0))) + } + + /// Revoke a principal, and with it every grant the principal holds. + /// + /// This is the blunt instrument, and it is meant to be. Revoking + /// grants one at a time is a race with whoever is using them; the + /// point of "kill the agent's key, the person is untouched" is that + /// one action ends the agent's authority entirely while leaving the + /// principal it acted for alone. + /// + /// The grant rows survive. An audit asks what was permitted last + /// March, and a deleted row cannot answer. + /// + /// Returns whether a live principal was actually revoked. + pub fn revoke_principal(&mut self, id: &str) -> Result { + let n = self.client.execute( + "UPDATE trusty_policy.principal + SET revoked_at = now() + WHERE id = $1 AND revoked_at IS NULL", + &[&id], + )?; + Ok(n == 1) + } + + /// Record that `agent` may act for `acts_for`. + /// + /// The delegation edge. Without it, an agent's grant naming a + /// principal is inert: [`GrantStore::permits`] requires both the + /// edge and the principal's own permission. + pub fn delegate(&mut self, agent: &str, acts_for: &str) -> Result<(), PolicyPgError> { + self.client.execute( + "INSERT INTO trusty_policy.delegation (agent_id, acts_for_id) + VALUES ($1, $2) + ON CONFLICT (agent_id, acts_for_id) DO UPDATE SET revoked_at = NULL", + &[&agent, &acts_for], + )?; + Ok(()) + } + + /// End a delegation without touching either side's grants. + pub fn revoke_delegation(&mut self, agent: &str, acts_for: &str) -> Result { + let n = self.client.execute( + "UPDATE trusty_policy.delegation SET revoked_at = now() + WHERE agent_id = $1 AND acts_for_id = $2 AND revoked_at IS NULL", + &[&agent, &acts_for], + )?; + Ok(n == 1) + } + + /// Stop a grant permitting, without running any DDL. + /// + /// Returns whether a live grant was actually revoked, so a caller + /// revoking something already gone is told rather than reassured. + /// The row stays: an audit asks what was permitted last March, and a + /// deleted row cannot answer. + pub fn revoke(&mut self, id: &GrantId) -> Result { + let n = self.client.execute( + "UPDATE trusty_policy.permission_grant + SET revoked_at = now() + WHERE id = $1 AND revoked_at IS NULL", + &[&id.as_str()], + )?; + Ok(n == 1) + } + + /// Ask the database the same question a policy predicate asks. + /// + /// For diagnostics and tests. Enforcement does not go through here — + /// it goes through the policy — but a caller debugging a refusal + /// wants to ask directly rather than infer from an empty result set. + pub fn permits( + &mut self, + subject: &str, + tier: Tier, + resource: &str, + ) -> Result { + let row = self.client.query_one( + "SELECT trusty_policy.permits($1, $2, $3)", + &[&subject, &tier_token(tier), &resource], + )?; + Ok(row.get::<_, Option>(0).unwrap_or(false)) + } + + /// Put one table under grant enforcement. + /// + /// `scope_column` names the column holding each row's resource + /// reference — the value a grant's scope is tested against. + /// + /// This is the only place an identifier reaches SQL text. It is not + /// escaped by hand: the server is asked to resolve and quote it, via + /// `to_regclass` and `format('%I')`, and a name that does not + /// resolve to a real table is refused rather than quoted. Escaping + /// is a thing you can get subtly wrong; existence is not. + /// + /// Returns the statements executed, so a caller can log or review + /// exactly what was applied. + pub fn enforce( + &mut self, + table: &str, + scope_column: &str, + ) -> Result, PolicyPgError> { + let row = self + .client + .query_opt( + "SELECT format('%I.%I', n.nspname, c.relname), + format('%I', a.attname), + c.relname + FROM pg_class c + JOIN pg_namespace n ON n.oid = c.relnamespace + JOIN pg_attribute a ON a.attrelid = c.oid + WHERE c.oid = to_regclass($1) + AND c.relkind = 'r' + AND a.attname = $2 + AND a.attnum > 0 + AND NOT a.attisdropped", + &[&table, &scope_column], + )? + .ok_or_else(|| PolicyPgError::UnknownRelation(format!("{table}.{scope_column}")))?; + + let qualified: String = row.get(0); + let column: String = row.get(1); + let bare: String = row.get(2); + + let mut applied = Vec::new(); + + // FORCE is not optional. Plain ENABLE is bypassed by the table + // owner, and the owner is who most connections are. Without + // FORCE every policy below is decorative. + for stmt in [ + format!("ALTER TABLE {qualified} ENABLE ROW LEVEL SECURITY"), + format!("ALTER TABLE {qualified} FORCE ROW LEVEL SECURITY"), + ] { + self.client.batch_execute(&stmt)?; + applied.push(stmt); + } + + for tier in [Tier::Read, Tier::Append, Tier::Update, Tier::Delete] { + let policy = policy_name(&bare, tier); + let quoted: String = self + .client + .query_one("SELECT format('%I', $1::text)", &[&policy])? + .get(0); + + // DROP first so enforce() is idempotent: re-running it after + // a definition change must converge, not fail. + let drop = format!("DROP POLICY IF EXISTS {quoted} ON {qualified}"); + self.client.batch_execute(&drop)?; + applied.push(drop); + + let create = create_policy_sql("ed, &qualified, &column, tier); + self.client.batch_execute(&create)?; + applied.push(create); + } + + Ok(applied) + } +} + +/// The four policy statements that put `table` under grant enforcement, +/// plus the two `ALTER TABLE` lines that make them bite. +/// +/// Pure: it touches no database and creates nothing. Use it to generate +/// a migration for a table that does not exist yet, where +/// [`GrantStore::enforce`] cannot help because it resolves the name +/// against `pg_class` first. +/// +/// Because there is no catalogue to resolve against, identifiers are +/// checked by shape instead, and refused rather than escaped. The output +/// is SQL text destined for a file a person reviews; that is a good +/// reason to be strict about it, not a reason to relax. +pub fn policy_statements( + table: &str, + scope_column: &str, +) -> Result, PolicyPgError> { + let qualified = check_identifier(table)?; + let column = check_identifier(scope_column)?; + let bare = table.rsplit('.').next().unwrap_or(table).to_string(); + + let mut out = vec![ + format!("ALTER TABLE {qualified} ENABLE ROW LEVEL SECURITY;"), + format!("ALTER TABLE {qualified} FORCE ROW LEVEL SECURITY;"), + ]; + for tier in [Tier::Read, Tier::Append, Tier::Update, Tier::Delete] { + let policy = policy_name(&bare, tier); + out.push(format!( + "DROP POLICY IF EXISTS {policy} ON {qualified};" + )); + out.push(format!( + "{};", + create_policy_sql(&policy, &qualified, &column, tier) + )); + } + Ok(out) +} + +/// Accept an unquoted SQL identifier, optionally schema-qualified. +/// +/// Deliberately narrow: lowercase ASCII, digits and underscore, not +/// starting with a digit, at most one dot. Anything else is refused +/// rather than quoted, because a name needing quotes in a generated +/// migration is a name worth a human deciding about. +fn check_identifier(name: &str) -> Result { + let refuse = || PolicyPgError::UnknownRelation(name.to_string()); + if name.is_empty() || name.len() > 128 || name.matches('.').count() > 1 { + return Err(refuse()); + } + for part in name.split('.') { + let mut chars = part.chars(); + match chars.next() { + Some(c) if c.is_ascii_lowercase() || c == '_' => {} + _ => return Err(refuse()), + } + if !chars.all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '_') { + return Err(refuse()); + } + } + Ok(name.to_string()) +} + +/// The policy statement for one (table, verb). +/// +/// `INSERT` takes `WITH CHECK` and no `USING`; `DELETE` and `SELECT` +/// take `USING` and no `WITH CHECK`; `UPDATE` takes both, because a row +/// may be moved out of a permitted scope as easily as into one. Getting +/// this wrong is the classic RLS hole — a policy that guards which rows +/// you may change but not what you may change them into. +fn create_policy_sql(policy: &str, table: &str, column: &str, tier: Tier) -> String { + let predicate = format!( + "trusty_policy.permits(current_setting('{SUBJECT_SETTING}', true), '{}', {column})", + tier_token(tier) + ); + let verb = tier.sql_verb(); + match tier { + Tier::Append => { + format!("CREATE POLICY {policy} ON {table} FOR {verb} WITH CHECK ({predicate})") + } + Tier::Update => format!( + "CREATE POLICY {policy} ON {table} FOR {verb} \ + USING ({predicate}) WITH CHECK ({predicate})" + ), + Tier::Read | Tier::Delete => { + format!("CREATE POLICY {policy} ON {table} FOR {verb} USING ({predicate})") + } + } +} + +/// Deterministic policy name for a (table, verb), so re-running +/// [`GrantStore::enforce`] replaces rather than accumulates. +fn policy_name(table: &str, tier: Tier) -> String { + format!("trusty_{table}_{}", tier_token(tier)) +} + +/// Grant expiry crosses as seconds since the epoch and is turned into a +/// `timestamptz` by `to_timestamp` server-side. +/// +/// `Grant.expires_at` is an `i64` of unix seconds; the column is +/// `TIMESTAMPTZ`. Converting in Rust would mean picking a timezone, and +/// the one thing worse than a wrong expiry is a wrong expiry that looks +/// right in one region. +fn expires_at_sql(expires_at: i64) -> f64 { + expires_at as f64 +} + +fn apply_schema(client: &mut Client) -> Result<(), PolicyPgError> { + let mut txn = client.transaction()?; + for stmt in DDL_V1 { + txn.batch_execute(stmt)?; + } + for stmt in DDL_FUNCTIONS { + txn.batch_execute(stmt)?; + } + txn.execute(DDL_SCHEMA_VERSION_UPSERT, &[&SCHEMA_VERSION])?; + txn.commit()?; + Ok(()) +} + +/// A transaction carrying the identity of whoever is acting. +/// +/// Every policy predicate reads `current_setting('trusty.subject')`. +/// Outside a `PolicySession` that setting is unset, the predicate sees +/// NULL, and every policy refuses — which is the correct default: code +/// that forgets to say who it is gets nothing, rather than everything. +pub struct PolicySession<'a> { + txn: Transaction<'a>, +} + +impl<'a> PolicySession<'a> { + /// Begin a transaction acting as `subject`. + pub fn begin(client: &'a mut Client, subject: &str) -> Result { + let mut txn = client.transaction()?; + // `true` is is_local: the setting dies with the transaction, so + // a pooled connection cannot carry one request's identity into + // the next. + txn.execute( + "SELECT set_config($1, $2, true)", + &[&SUBJECT_SETTING, &subject], + )?; + Ok(Self { txn }) + } + + pub fn txn(&mut self) -> &mut Transaction<'a> { + &mut self.txn + } + + pub fn commit(self) -> Result<(), PolicyPgError> { + self.txn.commit()?; + Ok(()) + } + + pub fn rollback(self) -> Result<(), PolicyPgError> { + self.txn.rollback()?; + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn insert_takes_with_check_and_no_using() { + let sql = create_policy_sql("p", "t", "scope", Tier::Append); + assert!( + sql.contains("WITH CHECK") && !sql.contains("USING"), + "an INSERT policy has no existing row to test, so USING is not just \ + redundant, it is rejected by Postgres: {sql}" + ); + assert!(sql.contains("FOR INSERT"), "got {sql}"); + } + + #[test] + fn update_takes_both_using_and_with_check() { + let sql = create_policy_sql("p", "t", "scope", Tier::Update); + assert!( + sql.contains("USING") && sql.contains("WITH CHECK"), + "USING alone guards which rows may be changed but not what they may \ + be changed INTO, so a row could be moved out of a permitted scope. \ + That is the classic RLS hole: {sql}" + ); + } + + #[test] + fn read_and_delete_take_using_and_no_with_check() { + for tier in [Tier::Read, Tier::Delete] { + let sql = create_policy_sql("p", "t", "scope", tier); + assert!( + sql.contains("USING") && !sql.contains("WITH CHECK"), + "neither reading nor deleting produces a new row to check: {sql}" + ); + } + } + + #[test] + fn each_policy_asks_about_its_own_tier() { + for tier in [Tier::Read, Tier::Append, Tier::Update, Tier::Delete] { + let sql = create_policy_sql("p", "t", "scope", tier); + assert!( + sql.contains(&format!("'{}'", tier_token(tier))), + "a policy for one verb asking about another would let a grant \ + to read authorise a delete: {sql}" + ); + assert!( + sql.contains(&format!("FOR {}", tier.sql_verb())), + "the SQL verb and the tier token must stay in step: {sql}" + ); + } + } + + #[test] + fn the_predicate_reads_the_subject_from_the_transaction() { + let sql = create_policy_sql("p", "t", "scope", Tier::Read); + assert!( + sql.contains("current_setting('trusty.subject', true)"), + "the second argument must be `true` (missing_ok), or an unset \ + subject raises instead of refusing, and an error is not a denial: \ + {sql}" + ); + } + + #[test] + fn an_identifier_that_would_need_quoting_is_refused() { + for bad in [ + "public.\"; DROP TABLE x; --", + "public.tab le", + "Public.Table", + "9lives", + "a.b.c", + "", + "tab'le", + ] { + assert!( + check_identifier(bad).is_err(), + "a name needing quotes in a generated migration is a name worth a human deciding about, not one worth escaping: {bad:?}" + ); + } + assert!(check_identifier("public.calendar_event").is_ok()); + assert!(check_identifier("calendar").is_ok()); + } + + #[test] + fn generated_statements_force_row_level_security() { + let stmts = policy_statements("public.calendar", "resource_scope").expect("emit"); + assert!( + stmts.iter().any(|s| s.contains("FORCE ROW LEVEL SECURITY")), + "ENABLE alone is bypassed by the table owner, so a migration emitting only ENABLE would read as protection and provide none" + ); + assert_eq!( + stmts.len(), + 2 + 4 * 2, + "two ALTERs, then a DROP and a CREATE for each of the four verbs" + ); + } + + #[test] + fn generated_statements_match_what_enforce_would_run() { + let generated = policy_statements("public.calendar", "resource_scope").expect("emit"); + // enforce() builds the same CREATE POLICY text, minus the + // trailing semicolons a file needs. If these ever diverge, a + // migration and a live call would mean different things. + let direct = create_policy_sql( + &policy_name("calendar", Tier::Read), + "public.calendar", + "resource_scope", + Tier::Read, + ); + assert!( + generated.iter().any(|s| s.trim_end_matches(';') == direct), + "the emitted migration and enforce() must produce the same policy, or the file a reviewer reads is not the rule the database gets" + ); + } + + #[test] + fn policy_names_are_deterministic_per_table_and_verb() { + assert_eq!(policy_name("calendar", Tier::Read), "trusty_calendar_read"); + assert_ne!( + policy_name("calendar", Tier::Read), + policy_name("calendar", Tier::Delete), + "one name per (table, verb) is what makes enforce() converge on \ + re-run instead of accumulating policies" + ); + } + + #[test] + fn a_grants_contents_never_appear_in_policy_sql() { + let sql = create_policy_sql("p", "t", "scope", Tier::Read); + for grant_shaped in ["board:9f2", "agent:01J", "user:01H"] { + assert!( + !sql.contains(grant_shaped), + "the policy is written once and knows nothing about any \ + particular grant; if a grant's contents reached this string, \ + a scope could become syntax: {sql}" + ); + } + } + + #[test] + fn the_subject_setting_is_transaction_local() { + // Guards the literal in PolicySession::begin. `false` here would + // leave the identity on a pooled connection after the request + // that set it had finished. + let src = include_str!("store.rs"); + assert!( + src.contains("&[&SUBJECT_SETTING, &subject],"), + "set_config must be called with the constant, not an ad-hoc string" + ); + assert!( + src.contains("\"SELECT set_config($1, $2, true)\""), + "is_local must be true, or a pooled connection carries one \ + request's identity into the next" + ); + } +} diff --git a/crates/adapters/trusty-policy-postgres/tests/integration.rs b/crates/adapters/trusty-policy-postgres/tests/integration.rs new file mode 100644 index 0000000..ef91181 --- /dev/null +++ b/crates/adapters/trusty-policy-postgres/tests/integration.rs @@ -0,0 +1,883 @@ +//! Behavioural tests against a real Postgres. +//! +//! Everything in the crate's unit tests asserts properties of SQL +//! *text*. These assert what the database actually does with it, which +//! is the only claim worth making about an authorisation boundary. +//! +//! Skipped, not failed, when `TRUSTY_POLICY_PG_URL` is unset, so +//! `cargo test --workspace` stays green on a machine with no database. +//! Same convention as `trusty-postgres`'s suite. +//! +//! ```text +//! TRUSTY_POLICY_PG_URL=postgresql://postgres:postgres@127.0.0.1:54322/postgres \ +//! cargo test -p trusty-policy-postgres --test integration +//! ``` +//! +//! **The tests act as a non-superuser role.** A superuser bypasses row +//! level security entirely, so a suite that ran as `postgres` would pass +//! against a table with no protection at all. + +use postgres::Client; +use trusty_policy::{Grant, Limits, Tier}; +use trusty_policy_postgres::{CompilationReport, GrantId, GrantStore, PolicyPgError}; + +fn url() -> Option { + std::env::var("TRUSTY_POLICY_PG_URL").ok() +} + +/// Serialises the suite against one database. +/// +/// Every `GrantStore::open` applies the schema, and concurrent +/// `CREATE OR REPLACE FUNCTION` on the same function collides in +/// `pg_proc` with "tuple concurrently updated". The tests also create +/// and drop roles, which are cluster-global however carefully the table +/// names are scoped. +/// +/// Taken here rather than left to `--test-threads=1`, because a suite +/// that is only green when invoked a particular way is one plain +/// `cargo test` away from looking broken. +fn db_lock() -> std::sync::MutexGuard<'static, ()> { + static LOCK: std::sync::OnceLock> = std::sync::OnceLock::new(); + LOCK.get_or_init(|| std::sync::Mutex::new(())) + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) +} + +/// Distinct names per test, so tests neither collide nor depend on +/// order. +struct Fixture { + table: String, + role: String, +} + +impl Fixture { + fn new(tag: &str) -> Self { + Self { + table: format!("rls_probe_{tag}"), + role: format!("rls_role_{tag}"), + } + } + + /// A protected table, a non-superuser role that can reach it, and no + /// grants at all. + fn set_up(&self, store: &mut GrantStore) -> Vec { + let (table, role) = (&self.table, &self.role); + let drop_role = drop_role_sql(role); + store + .client() + .batch_execute(&format!( + "DROP TABLE IF EXISTS public.{table}; + CREATE TABLE public.{table} ( + id SERIAL PRIMARY KEY, + resource_scope TEXT NOT NULL, + note TEXT NOT NULL + ); + INSERT INTO public.{table}(resource_scope, note) VALUES + ('board:9f2/calendar/team', 'visible'), + ('board:9f2/calendar/private', 'excluded'), + ('board:other/calendar', 'someone else'); + {drop_role} + CREATE ROLE {role} NOSUPERUSER NOBYPASSRLS; + -- SET ROLE requires membership. Supabase's `postgres` is + -- not a true superuser, so the connecting user has to be + -- granted the role it intends to act as. + GRANT {role} TO CURRENT_USER; + GRANT USAGE ON SCHEMA public TO {role}; + GRANT SELECT, INSERT, UPDATE, DELETE ON public.{table} TO {role}; + GRANT USAGE, SELECT ON SEQUENCE public.{table}_id_seq TO {role}; + GRANT USAGE ON SCHEMA trusty_policy TO {role};" + )) + .expect("fixture set-up"); + + store + .enforce(&format!("public.{table}"), "resource_scope") + .expect("enforce") + } + + fn tear_down(&self, store: &mut GrantStore) { + let (table, role) = (&self.table, &self.role); + let drop_role = drop_role_sql(role); + let _ = store.client().batch_execute(&format!( + "DROP TABLE IF EXISTS public.{table}; + {drop_role}" + )); + let _ = store + .client() + .batch_execute("DELETE FROM trusty_policy.permission_grant"); + } + + /// Read the table the way the application would: as the unprivileged + /// role, with a subject set for the transaction only. + fn read_as(&self, client: &mut Client, subject: &str) -> Vec { + let mut txn = client.transaction().expect("txn"); + txn.execute(&format!("SET LOCAL ROLE {}", self.role), &[]) + .expect("assume role"); + txn.execute( + "SELECT set_config('trusty.subject', $1, true)", + &[&subject], + ) + .expect("set subject"); + let rows = txn + .query( + &format!("SELECT note FROM public.{} ORDER BY id", self.table), + &[], + ) + .expect("select"); + let notes = rows.iter().map(|r| r.get::<_, String>(0)).collect(); + txn.rollback().expect("rollback"); + notes + } +} + +/// Drop a role only if it exists. +/// +/// `DROP ROLE IF EXISTS` tolerates absence, but `DROP OWNED BY` does not +/// -- and the grants a role holds must go before the role can. So both +/// have to sit behind one existence check rather than relying on +/// IF EXISTS. +fn drop_role_sql(role: &str) -> String { + format!( + "DO $do$ BEGIN + IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = '{role}') THEN + -- In PG16+, ADMIN OPTION on a role is not the same as + -- holding its privileges, and DROP OWNED BY needs the + -- latter. Creating the role is not enough. + EXECUTE 'GRANT {role} TO CURRENT_USER'; + EXECUTE 'DROP OWNED BY {role}'; + EXECUTE 'DROP ROLE {role}'; + END IF; + END $do$;" + ) +} + +/// Issue a grant, registering its principals first. +/// +/// `permits()` refuses for a subject that is not a registered, live +/// principal — identity before permission — so a test that only issued +/// a grant would be testing the registration check rather than the rule +/// it names. +trait Registering { + fn issue_registered( + &mut self, + grant: &Grant, + ) -> Result<(GrantId, CompilationReport), PolicyPgError>; +} + +impl Registering for GrantStore { + fn issue_registered( + &mut self, + grant: &Grant, + ) -> Result<(GrantId, CompilationReport), PolicyPgError> { + for who in [Some(&grant.subject), grant.acts_for.as_ref()].into_iter().flatten() { + let kind = who.split(':').next().unwrap_or("agent"); + self.register_principal(who, kind, "")?; + } + self.issue(grant) + } +} + +fn grant(subject: &str, scope: &str, tier: Tier) -> Grant { + Grant { + why: Some("integration fixture".into()), + subject: subject.into(), + acts_for: None, + scope: scope.into(), + tier, + excludes: Vec::new(), + limits: Limits::default(), + // Far future; the expiry test overrides this deliberately. + expires_at: 4_102_444_800, + } +} + +// --------------------------------------------------------------------- +// The acceptance criterion. +// --------------------------------------------------------------------- + +#[test] +fn a_read_the_grant_excludes_fails_at_the_database() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("excl"); + fx.set_up(&mut store); + + let mut g = grant("agent:excl", "board:9f2/calendar", Tier::Read); + g.excludes = vec!["board:9f2/calendar/private".into()]; + store.issue_registered(&g).expect("issue"); + + let seen = fx.read_as(store.client(), "agent:excl"); + fx.tear_down(&mut store); + + assert!( + !seen.contains(&"excluded".to_string()), + "an excluded resource must be unreadable AT THE DATABASE, not merely \ + refused by an application check that could be bypassed; saw {seen:?}" + ); +} + +#[test] +fn an_exclusion_is_not_overridden_by_a_wider_grant() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("wider"); + fx.set_up(&mut store); + + let mut narrow = grant("agent:wider", "board:9f2/calendar/team", Tier::Read); + narrow.excludes = vec!["board:9f2/calendar/private".into()]; + store.issue_registered(&narrow).expect("issue narrow"); + + // Issued afterwards, covering strictly more, and carrying no + // exclusion of its own. + let wide = grant("agent:wider", "board:9f2", Tier::Read); + store.issue_registered(&wide).expect("issue wide"); + + let seen = fx.read_as(store.client(), "agent:wider"); + fx.tear_down(&mut store); + + assert!( + !seen.contains(&"excluded".to_string()), + "adding a wider grant later must not reopen what an exclusion shut; \ + saw {seen:?}" + ); + assert!( + seen.contains(&"visible".to_string()), + "the wider grant should still permit everything it covers that is not \ + excluded; saw {seen:?}" + ); +} + +#[test] +fn absent_policy_is_not_permission() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("deny"); + fx.set_up(&mut store); + + let with_no_grant = fx.read_as(store.client(), "agent:nobody"); + let with_no_subject = fx.read_as(store.client(), ""); + fx.tear_down(&mut store); + + assert!( + with_no_grant.is_empty(), + "a subject with no grant must see nothing, not fall back to something \ + wider; saw {with_no_grant:?}" + ); + assert!( + with_no_subject.is_empty(), + "code that forgets to say who it is must get nothing rather than \ + everything; saw {with_no_subject:?}" + ); +} + +#[test] +fn a_scope_containing_a_like_metacharacter_does_not_widen_itself() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("meta"); + fx.set_up(&mut store); + + // Under LIKE semantics '%' matches everything and this would read + // the whole table. + store + .issue(&grant("agent:meta", "%", Tier::Read)) + .expect("issue"); + let wildcard = fx.read_as(store.client(), "agent:meta"); + + store + .issue(&grant("agent:meta2", "board:9f2/calendar/_eam", Tier::Read)) + .expect("issue"); + let underscore = fx.read_as(store.client(), "agent:meta2"); + + fx.tear_down(&mut store); + + assert!( + wildcard.is_empty(), + "a scope of '%' is a scope literally named '%', not a wildcard; saw \ + {wildcard:?}" + ); + assert!( + underscore.is_empty(), + "'_' must not match an arbitrary character, or a near-miss scope would \ + silently grant its neighbour; saw {underscore:?}" + ); +} + +#[test] +fn a_sibling_whose_name_merely_starts_the_same_is_not_covered() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("sib"); + fx.set_up(&mut store); + + // 'board:9f2/calendar' must NOT cover 'board:9f2/calendar/team' via a + // bare prefix test that ignores the separator -- it does cover it + // here, legitimately -- but 'board:o' must not cover 'board:other'. + store + .issue(&grant("agent:sib", "board:o", Tier::Read)) + .expect("issue"); + let seen = fx.read_as(store.client(), "agent:sib"); + fx.tear_down(&mut store); + + assert!( + seen.is_empty(), + "containment must break on a path separator; without it, 'board:o' \ + would grant 'board:other/calendar'; saw {seen:?}" + ); +} + +#[test] +fn an_expired_grant_stops_permitting_at_the_wall_clock() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("exp"); + fx.set_up(&mut store); + + let mut g = grant("agent:exp", "board:9f2/calendar", Tier::Read); + g.expires_at = 1_000_000_000; // 2001, comfortably past + store.issue_registered(&g).expect("issue"); + + let seen = fx.read_as(store.client(), "agent:exp"); + fx.tear_down(&mut store); + + assert!( + seen.is_empty(), + "expiry is wall clock and independent of whether work finished; saw \ + {seen:?}" + ); +} + +#[test] +fn revoking_a_grant_takes_effect_without_ddl() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("rev"); + fx.set_up(&mut store); + + let g = grant("agent:rev", "board:9f2/calendar", Tier::Read); + let (id, _) = store.issue_registered(&g).expect("issue"); + let before = fx.read_as(store.client(), "agent:rev"); + + let revoked = store.revoke(&id).expect("revoke"); + let after = fx.read_as(store.client(), "agent:rev"); + + let twice = store.revoke(&id).expect("revoke again"); + fx.tear_down(&mut store); + + assert!( + !before.is_empty(), + "the grant must permit something before revocation, or this test \ + proves nothing" + ); + assert!(revoked, "revoking a live grant must report that it did so"); + assert!( + after.is_empty(), + "revocation must take effect at the next decision, with no DDL run; \ + saw {after:?}" + ); + assert!( + !twice, + "revoking what is already revoked must report that nothing changed, \ + rather than reassuring the caller" + ); +} + +#[test] +fn an_agent_cannot_exceed_the_principal_it_acts_for() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("deleg"); + fx.set_up(&mut store); + + store + .client() + .batch_execute( + "INSERT INTO trusty_policy.principal(id, kind) VALUES + ('agent:d1', 'agent'), ('user:d1', 'user') + ON CONFLICT DO NOTHING; + INSERT INTO trusty_policy.delegation(agent_id, acts_for_id) VALUES + ('agent:d1', 'user:d1') + ON CONFLICT DO NOTHING;", + ) + .expect("principals"); + + // The agent's own grant is wider than the principal's. + let mut wide = grant("agent:d1", "board:9f2", Tier::Read); + wide.acts_for = Some("user:d1".into()); + store.issue_registered(&wide).expect("issue agent"); + store + .issue(&grant("user:d1", "board:9f2/calendar/team", Tier::Read)) + .expect("issue principal"); + + let seen = fx.read_as(store.client(), "agent:d1"); + + // Now revoke the delegation and try again. + store + .client() + .batch_execute( + "UPDATE trusty_policy.delegation SET revoked_at = now() + WHERE agent_id = 'agent:d1'", + ) + .expect("revoke delegation"); + let after_revoke = fx.read_as(store.client(), "agent:d1"); + + store + .client() + .batch_execute("DELETE FROM trusty_policy.delegation; DELETE FROM trusty_policy.principal") + .expect("clean principals"); + fx.tear_down(&mut store); + + assert_eq!( + seen, + vec!["visible".to_string()], + "the agent must be held to the intersection: its own grant covers \ + board:9f2 entirely, but the principal only permits the team \ + calendar; saw {seen:?}" + ); + assert!( + after_revoke.is_empty(), + "revoking the delegation must stop the agent acting at all, without \ + touching either grant; saw {after_revoke:?}" + ); +} + +#[test] +fn a_grant_for_one_verb_does_not_authorise_another() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("verb"); + fx.set_up(&mut store); + + // Delete is the most destructive tier. If tiers implied one another + // by ordering, this would also confer read. + store + .issue(&grant("agent:verb", "board:9f2/calendar", Tier::Delete)) + .expect("issue"); + let seen = fx.read_as(store.client(), "agent:verb"); + fx.tear_down(&mut store); + + assert!( + seen.is_empty(), + "tier maps 1:1 onto the SQL verbs and deny-by-default governs the rest, \ + so a grant to delete must not silently confer the ability to read \ + everything deletable; saw {seen:?}" + ); +} + +#[test] +fn enforce_is_idempotent_and_forces_row_level_security() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("force"); + let first = fx.set_up(&mut store); + + // Re-running must converge, not fail on an existing policy. + let second = store + .enforce(&format!("public.{}", fx.table), "resource_scope") + .expect("second enforce"); + + let forced: bool = store + .client() + .query_one( + "SELECT relforcerowsecurity FROM pg_class WHERE oid = to_regclass($1)", + &[&format!("public.{}", fx.table)], + ) + .expect("pg_class") + .get(0); + let policies: i64 = store + .client() + .query_one( + "SELECT count(*) FROM pg_policies + WHERE schemaname = 'public' AND tablename = $1", + &[&fx.table], + ) + .expect("pg_policies") + .get(0); + + fx.tear_down(&mut store); + + assert_eq!(first, second, "enforce must be deterministic across runs"); + assert!( + forced, + "without FORCE, the table owner bypasses every policy and the \ + protection is decorative" + ); + assert_eq!( + policies, 4, + "one policy per verb, replaced rather than accumulated on re-run" + ); +} + +#[test] +fn an_unknown_relation_is_refused_rather_than_quoted() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + + assert!( + store.enforce("public.no_such_table_here", "resource_scope").is_err(), + "enabling row level security on a table that does not exist would \ + silently protect nothing" + ); + assert!( + store.enforce("public.no_such_table_here", "'; DROP TABLE x; --").is_err(), + "an identifier that does not resolve must be refused, never escaped \ + and used" + ); +} + +#[test] +fn issuing_the_same_grant_twice_leaves_one_row() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let g = grant("agent:idem", "board:9f2/calendar", Tier::Read); + + let (first, report) = store.issue_registered(&g).expect("issue"); + let (second, _) = store.issue_registered(&g).expect("issue again"); + + let count: i64 = store + .client() + .query_one( + "SELECT count(*) FROM trusty_policy.permission_grant WHERE subject = $1", + &[&"agent:idem"], + ) + .expect("count") + .get(0); + let _ = store + .client() + .batch_execute("DELETE FROM trusty_policy.permission_grant"); + + assert_eq!(first, second, "identity is content, so it must not vary"); + assert_eq!( + count, 1, + "a content-addressed id makes re-issuing idempotent rather than \ + quietly accumulating duplicate permissions" + ); + assert!( + report.is_total(), + "this grant sets no limits, so nothing about it went unenforced" + ); +} + +#[test] +fn the_stored_canonical_form_reproduces_the_id() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let g = grant("agent:canon", "board:9f2/calendar", Tier::Read); + let (id, _) = store.issue_registered(&g).expect("issue"); + + let stored: String = store + .client() + .query_one( + "SELECT canonical_json FROM trusty_policy.permission_grant WHERE id = $1", + &[&id.as_str()], + ) + .expect("select") + .get(0); + let _ = store + .client() + .batch_execute("DELETE FROM trusty_policy.permission_grant"); + + assert_eq!( + GrantId::of_canonical(&stored), + id, + "the bytes in the row must be the bytes that were hashed, or a \ + verifier holding the signed grant could not find or check its row" + ); +} + +#[test] +fn a_registered_principal_with_no_grant_still_sees_nothing() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("reg"); + fx.set_up(&mut store); + + // Registration is identity, not permission. Without this test, + // `absent_policy_is_not_permission` would pass merely because the + // subject was unregistered, and the deny-by-default rule it is named + // for would go untested. + store + .register_principal("agent:registered", "agent", "has an identity, no grant") + .expect("register"); + + let seen = fx.read_as(store.client(), "agent:registered"); + fx.tear_down(&mut store); + + assert!( + seen.is_empty(), + "being someone is not the same as being allowed something; saw {seen:?}" + ); +} + +#[test] +fn an_unregistered_principal_refuses_even_holding_a_grant() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("unreg"); + fx.set_up(&mut store); + + // Issued WITHOUT registering: a grant to nobody. + store + .issue(&grant("agent:ghost", "board:9f2/calendar", Tier::Read)) + .expect("issue"); + let seen = fx.read_as(store.client(), "agent:ghost"); + fx.tear_down(&mut store); + + assert!( + seen.is_empty(), + "a grant held by a principal who was never registered is a grant to \ + nobody, and absent policy is not permission; saw {seen:?}" + ); +} + +#[test] +fn revoking_a_principal_stops_every_grant_it_holds_at_once() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("prev"); + fx.set_up(&mut store); + + // Two separate grants, so this tests the blunt instrument rather + // than a single revocation wearing a different hat. + store + .issue_registered(&grant("agent:doomed", "board:9f2/calendar", Tier::Read)) + .expect("issue one"); + store + .issue_registered(&grant("agent:doomed", "board:other", Tier::Read)) + .expect("issue two"); + let before = fx.read_as(store.client(), "agent:doomed"); + + let revoked = store.revoke_principal("agent:doomed").expect("revoke"); + let after = fx.read_as(store.client(), "agent:doomed"); + let twice = store.revoke_principal("agent:doomed").expect("revoke again"); + + let survivors: i64 = store + .client() + .query_one( + "SELECT count(*) FROM trusty_policy.permission_grant WHERE subject = $1", + &[&"agent:doomed"], + ) + .expect("count") + .get(0); + fx.tear_down(&mut store); + + assert_eq!( + before.len(), + 3, + "both grants must permit something first, or this proves nothing; \ + saw {before:?}" + ); + assert!(revoked, "revoking a live principal must report that it did so"); + assert!( + after.is_empty(), + "killing the key must end the agent's authority entirely, not grant \ + by grant -- revoking them one at a time races whoever is using them; \ + saw {after:?}" + ); + assert!(!twice, "revoking what is already revoked must report no change"); + assert_eq!( + survivors, 2, + "the grant rows must survive revocation; an audit asks what was \ + permitted last March and a deleted row cannot answer" + ); +} + +#[test] +fn revoking_an_agent_leaves_the_principal_it_acted_for_untouched() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("sever"); + fx.set_up(&mut store); + + let mut agent = grant("agent:s1", "board:9f2", Tier::Read); + agent.acts_for = Some("user:s1".into()); + store.issue_registered(&agent).expect("issue agent"); + store + .issue_registered(&grant("user:s1", "board:9f2/calendar/team", Tier::Read)) + .expect("issue principal"); + store.delegate("agent:s1", "user:s1").expect("delegate"); + + let agent_before = fx.read_as(store.client(), "agent:s1"); + store.revoke_principal("agent:s1").expect("revoke agent"); + let agent_after = fx.read_as(store.client(), "agent:s1"); + let principal_after = fx.read_as(store.client(), "user:s1"); + + store + .client() + .batch_execute("DELETE FROM trusty_policy.delegation") + .expect("clean"); + fx.tear_down(&mut store); + + assert!(!agent_before.is_empty(), "the agent must start able to act"); + assert!( + agent_after.is_empty(), + "killing the agent's key must end its authority; saw {agent_after:?}" + ); + assert_eq!( + principal_after, + vec!["visible".to_string()], + "and the person behind it must be untouched -- that asymmetry is the \ + entire point of an agent never being a principal in its own right; \ + saw {principal_after:?}" + ); +} + +#[test] +fn the_key_edge_records_a_reference_and_not_a_second_registry() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + + store + .register_principal("agent:keyed", "agent", "") + .expect("register"); + assert_eq!( + store.producer_oid_of("agent:keyed").expect("lookup"), + None, + "an unbound principal has no producer, and None is not the same as \ + a producer named empty string" + ); + + store + .bind_key("agent:keyed", "1.3.6.1.4.1.99999.1") + .expect("bind"); + let bound = store.producer_oid_of("agent:keyed").expect("lookup"); + + let unknown = store.bind_key("agent:missing", "1.3.6.1.4.1.99999.2"); + + let _ = store + .client() + .batch_execute("DELETE FROM trusty_policy.principal WHERE id = 'agent:keyed'"); + + assert_eq!( + bound.as_deref(), + Some("1.3.6.1.4.1.99999.1"), + "the edge must round-trip; resolving it to a public key is the \ + producer registry's job, not this crate's" + ); + assert!( + unknown.is_err(), + "a key bound to a principal that does not exist is a key nothing can \ + revoke, so binding it must fail rather than create a dangling edge" + ); +} + +#[test] +fn a_delegation_must_be_recorded_before_an_agent_can_use_it() { + let Some(url) = url() else { + eprintln!("skipped: TRUSTY_POLICY_PG_URL unset"); + return; + }; + let _db = db_lock(); + let mut store = GrantStore::open(&url).expect("open"); + let fx = Fixture::new("edge"); + fx.set_up(&mut store); + + let mut agent = grant("agent:e1", "board:9f2/calendar", Tier::Read); + agent.acts_for = Some("user:e1".into()); + store.issue_registered(&agent).expect("issue agent"); + store + .issue_registered(&grant("user:e1", "board:9f2/calendar", Tier::Read)) + .expect("issue principal"); + + // Both grants exist and agree. The edge does not. + let without_edge = fx.read_as(store.client(), "agent:e1"); + store.delegate("agent:e1", "user:e1").expect("delegate"); + let with_edge = fx.read_as(store.client(), "agent:e1"); + store + .revoke_delegation("agent:e1", "user:e1") + .expect("revoke delegation"); + let after_revoke = fx.read_as(store.client(), "agent:e1"); + + store + .client() + .batch_execute("DELETE FROM trusty_policy.delegation") + .expect("clean"); + fx.tear_down(&mut store); + + assert!( + without_edge.is_empty(), + "two matching grants are not a delegation; the authority to act for \ + someone has to be recorded, or it is merely claimed; saw \ + {without_edge:?}" + ); + assert!( + !with_edge.is_empty(), + "with the edge recorded, the intersection should permit; saw {with_edge:?}" + ); + assert!( + after_revoke.is_empty(), + "and withdrawing it must take effect without touching either grant; \ + saw {after_revoke:?}" + ); +} diff --git a/crates/adapters/trusty-postgres/Cargo.toml b/crates/adapters/trusty-postgres/Cargo.toml new file mode 100644 index 0000000..78760c7 --- /dev/null +++ b/crates/adapters/trusty-postgres/Cargo.toml @@ -0,0 +1,27 @@ +[package] +name = "trusty-postgres" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "PostgreSQL adapter for Trusty. Three-table schema mirroring trusty-sqlite with optional TimescaleDB feature, OID-anchored producer identity, and CHECKPOINT verification primitive." + +[dependencies] +trusty-core = { workspace = true } +trusty-store = { workspace = true } +postgres = { workspace = true } +serde = { workspace = true } +serde_json = { workspace = true } +chrono = { workspace = true } +thiserror = { workspace = true } +tracing = { workspace = true } +hex = { workspace = true } + +[dev-dependencies] +trusty-store = { workspace = true, features = ["testing"] } + +[features] +default = [] +timescale = [] diff --git a/crates/adapters/trusty-postgres/src/error.rs b/crates/adapters/trusty-postgres/src/error.rs new file mode 100644 index 0000000..1a0816d --- /dev/null +++ b/crates/adapters/trusty-postgres/src/error.rs @@ -0,0 +1,39 @@ +use thiserror::Error; +use trusty_store::StoreError; + +#[derive(Debug, Error)] +pub enum PostgresError { + #[error("postgres driver error: {0}")] + Driver(#[from] postgres::Error), + + #[error("serialization error: {0}")] + Serialization(#[from] serde_json::Error), + + #[error("malformed chain_id: {0}")] + MalformedChainId(String), + + #[error("schema migration error: {0}")] + Migration(String), + + #[error("integrity violation: {0}")] + IntegrityViolation(String), + + #[error("not found: {0}")] + NotFound(String), + + #[error("trusty-core error: {0}")] + Core(#[from] trusty_core::TrustyError), +} + +pub type PostgresResult = Result; + +impl From for StoreError { + fn from(err: PostgresError) -> Self { + match err { + PostgresError::NotFound(s) => StoreError::NotFound(s), + PostgresError::IntegrityViolation(s) => StoreError::IntegrityViolation(s), + PostgresError::Serialization(e) => StoreError::Serialization(e.to_string()), + other => StoreError::Backend(other.to_string()), + } + } +} diff --git a/crates/adapters/trusty-postgres/src/lib.rs b/crates/adapters/trusty-postgres/src/lib.rs new file mode 100644 index 0000000..e7a1b01 --- /dev/null +++ b/crates/adapters/trusty-postgres/src/lib.rs @@ -0,0 +1,31 @@ +//! PostgreSQL adapter for Trusty. +//! +//! Implements the `trusty_store::Store` trait against PostgreSQL 14+ using +//! the synchronous `postgres` crate. Mirrors the three-table schema of +//! `trusty-sqlite` (`provenance_envelope`, `hash_index`, `chain_tips`) with +//! Postgres dialect and adds: +//! - `chain_checkpoint` table for verification-layer checkpoints (#13). +//! - `producer_identity` table for OID-anchored producer identity (#14, #22). +//! - `schema_meta` table for planned schema migration versioning. +//! - Optional TimescaleDB hypertable on `received_at` via `timescale` feature. +//! +//! Design memo: `docs/trusty-postgres-design.md`. +//! +//! TCB note: the trusted computing base added by this adapter is Postgres + +//! libpq + the `postgres` Rust driver. Trusty chain math is backend-agnostic; +//! v1 byte-shape is preserved by copying `raw_payload` verbatim as `TEXT` +//! (never `JSONB`, which would normalize whitespace and key order). + +#![deny(clippy::all)] + +pub mod error; +pub mod schema; +pub mod store; +pub mod trusty; + +pub use error::{PostgresError, PostgresResult}; +pub use store::TrustyPostgres; +pub use trusty::{ + AnchorInfo, AppendResult, BreakEvent, CheckpointRow, HistoryRow, IntegrityMismatch, + IntegrityReport, ProducerIdentity, RevocationRecord, RowSpec, SilentEntity, Trusty, +}; diff --git a/crates/adapters/trusty-postgres/src/schema.rs b/crates/adapters/trusty-postgres/src/schema.rs new file mode 100644 index 0000000..ec3ae9c --- /dev/null +++ b/crates/adapters/trusty-postgres/src/schema.rs @@ -0,0 +1,144 @@ +//! Schema DDL for trusty-postgres. +//! +//! Mirrors the three-table layout of trusty-sqlite (provenance_envelope, +//! hash_index, chain_tips) in Postgres dialect. Adds chain_checkpoint, +//! producer_identity, and schema_meta. See docs/trusty-postgres-design.md. +//! +//! raw_payload MUST be TEXT, never JSONB -- JSONB normalizes whitespace +//! and key order, which would break v1 byte-shape verification. + +pub const SCHEMA_VERSION: &str = "1"; + +pub const DDL_V1: &[&str] = &[ + // ---------------------------------------------------------------- + // Core three-table schema (mirrors trusty-sqlite). + // ---------------------------------------------------------------- + r#"CREATE TABLE IF NOT EXISTS provenance_envelope ( + id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, + chain_id TEXT NOT NULL, + entity_id TEXT NOT NULL, + event_type TEXT NOT NULL, + field_name TEXT, + old_value TEXT, + new_value TEXT, + source_url TEXT NOT NULL DEFAULT '', + detected_at TEXT NOT NULL, + received_at TEXT NOT NULL, + extraction_method TEXT NOT NULL DEFAULT 'trusty', + prev_hash TEXT NOT NULL, + raw_payload TEXT, + row_hash TEXT, + anchor_hash TEXT, + sequence_int BIGINT NOT NULL DEFAULT 0 + )"#, + r#"CREATE INDEX IF NOT EXISTS idx_pe_entity ON provenance_envelope(entity_id)"#, + r#"CREATE INDEX IF NOT EXISTS idx_pe_chain ON provenance_envelope(chain_id)"#, + r#"CREATE INDEX IF NOT EXISTS idx_pe_entity_chain ON provenance_envelope(entity_id, chain_id)"#, + r#"CREATE INDEX IF NOT EXISTS idx_pe_row_hash ON provenance_envelope(row_hash)"#, + // Partial index accelerates breaks_since on Postgres (SQLite scans). + r#"CREATE INDEX IF NOT EXISTS idx_pe_break ON provenance_envelope(entity_id, detected_at) + WHERE event_type = 'BREAK'"#, + r#"CREATE TABLE IF NOT EXISTS hash_index ( + id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, + row_hash TEXT NOT NULL UNIQUE, + chain_id TEXT NOT NULL, + sequence_int BIGINT NOT NULL, + entity_id TEXT NOT NULL, + recorded_at TEXT NOT NULL + )"#, + r#"CREATE INDEX IF NOT EXISTS idx_hi_entity ON hash_index(entity_id)"#, + r#"CREATE INDEX IF NOT EXISTS idx_hi_chain ON hash_index(chain_id)"#, + r#"CREATE TABLE IF NOT EXISTS chain_tips ( + entity_id TEXT PRIMARY KEY, + current_tip_hash TEXT, + chain_id TEXT, + tip_state TEXT NOT NULL DEFAULT 'WARM' + CHECK (tip_state IN ('UNKNOWN','COLD','WARM')), + updated_at TEXT NOT NULL + )"#, + r#"CREATE INDEX IF NOT EXISTS idx_ct_chain ON chain_tips(chain_id)"#, + // ---------------------------------------------------------------- + // Producer identity (#14, #22) -- OID-anchored. Nullable in the + // sense that rows are optional; table always exists. + // ---------------------------------------------------------------- + r#"CREATE TABLE IF NOT EXISTS producer_identity ( + producer_oid TEXT PRIMARY KEY, + -- Key material in the format `key_format` declares. Named for + -- the raw Ed25519 case it was built for; a COSE credential from + -- a WebAuthn authenticator lives here too, and key_format is + -- what says which. Renaming the column would break every + -- existing installation for a cosmetic gain. + ed25519_pubkey BYTEA NOT NULL, + siphash_key BYTEA NOT NULL, + registered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + display_name TEXT, + metadata JSONB, + -- How ed25519_pubkey is encoded, and therefore how a signature + -- from it must be checked. A user enrolling through WebAuthn + -- registers a COSE key whose signature covers + -- authenticatorData || SHA-256(clientDataJSON), NOT the message, + -- so it cannot be checked as a bare signature. One column and + -- one verifier made that mistake available. + key_format TEXT NOT NULL DEFAULT 'ed25519-raw' + CHECK (key_format IN ('ed25519-raw','cose')) + )"#, + // Existing installations predate the column; CREATE TABLE IF NOT + // EXISTS does nothing for them, so it needs saying twice. + r#"ALTER TABLE producer_identity + ADD COLUMN IF NOT EXISTS key_format TEXT NOT NULL DEFAULT 'ed25519-raw'"#, + // ---------------------------------------------------------------- + // Producer revocation (memo §6 / learn-LCG-ARCH-06) -- forward- + // only revocation per the Cert-Transparency stance: rows signed + // BEFORE revoked_at remain verifiable; rows signed AT or AFTER + // are rejected. The revocation event itself is also written as a + // chain envelope (event_type='REVOCATION') so the act is part of + // the audit chain. revocation_row_hash references that envelope. + // ---------------------------------------------------------------- + r#"CREATE TABLE IF NOT EXISTS producer_revocation ( + producer_oid TEXT PRIMARY KEY, + revoked_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + revoked_pubkey BYTEA NOT NULL, + reason TEXT NOT NULL, + attested_by_pubkey BYTEA NOT NULL, + revocation_row_hash TEXT NOT NULL + )"#, + // ---------------------------------------------------------------- + // Checkpoints (#13) -- verification-layer only; no row deletion. + // ---------------------------------------------------------------- + r#"CREATE TABLE IF NOT EXISTS chain_checkpoint ( + id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, + chain_id TEXT NOT NULL, + sequence_int BIGINT NOT NULL, + row_hash TEXT NOT NULL, + checkpoint_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + attested_by TEXT NOT NULL, + attestation_sig BYTEA NOT NULL, + UNIQUE (chain_id, sequence_int) + )"#, + r#"CREATE INDEX IF NOT EXISTS idx_ckpt_chain ON chain_checkpoint(chain_id)"#, + // ---------------------------------------------------------------- + // Schema meta (planned migration versioning). + // ---------------------------------------------------------------- + r#"CREATE TABLE IF NOT EXISTS schema_meta ( + key TEXT PRIMARY KEY, + value TEXT NOT NULL + )"#, +]; + +/// Upserts the schema_version row. Runs on every open to declare the +/// version this build expects. +pub const DDL_SCHEMA_VERSION_UPSERT: &str = r#" + INSERT INTO schema_meta(key, value) VALUES ('schema_version', $1) + ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value +"#; + +/// TimescaleDB hypertable creation (guarded by the `timescale` feature +/// flag at the call site). Uses `if_not_exists` so it is idempotent. +#[cfg(feature = "timescale")] +pub const DDL_TIMESCALE_HYPERTABLE: &str = r#" + SELECT create_hypertable( + 'provenance_envelope', 'received_at', + chunk_time_interval => INTERVAL '7 days', + if_not_exists => TRUE + ) +"#; diff --git a/crates/adapters/trusty-postgres/src/store.rs b/crates/adapters/trusty-postgres/src/store.rs new file mode 100644 index 0000000..a375e45 --- /dev/null +++ b/crates/adapters/trusty-postgres/src/store.rs @@ -0,0 +1,463 @@ +//! `Store` trait implementation for PostgreSQL. +//! +//! Uses the synchronous `postgres` crate (libpq-backed, not tokio-postgres) +//! per the `with_txn` sync-trait decision in docs/trusty-postgres-design.md +//! section 7. Advisory locks serialize per-entity appends; different entities +//! run concurrently. +//! +//! Interior mutability: the `postgres` crate's `Client::query*` methods +//! require `&mut Client`, but the `Store` trait exposes several read-only +//! methods (`last_row`, `iter_chain`, `get_tip`, `get_tip_for_entity`, +//! `kv_cross_check`) that take `&self`. We wrap `Client` in `RefCell` to +//! reconcile. This matches how `rusqlite::Connection` provides `&self` +//! query APIs internally; we just make the bookkeeping explicit. +//! `RefCell` is `Send` (not `Sync`), which satisfies the +//! `Store: Send` bound. + +use std::cell::RefCell; + +use chrono::SecondsFormat; +use postgres::{Client, NoTls, Transaction}; +use tracing::instrument; +use trusty_store::{ + KvCrossCheckReport, LastRow, RowInsert, Store, StoreError, StoreResult, StoredRow, TipRow, + TipUpsert, +}; + +use crate::error::PostgresError; +use crate::schema::{DDL_SCHEMA_VERSION_UPSERT, DDL_V1, SCHEMA_VERSION}; + +pub struct TrustyPostgres { + client: RefCell, +} + +impl TrustyPostgres { + /// Open a connection and apply the v1 schema idempotently. Accepts any + /// standard libpq connection string, e.g. + /// `host=localhost user=postgres dbname=trusty password=...`. + pub fn open(conn_str: &str) -> Result { + let mut client = Client::connect(conn_str, NoTls)?; + apply_schema(&mut client)?; + Ok(Self { + client: RefCell::new(client), + }) + } +} + +fn apply_schema(client: &mut Client) -> Result<(), PostgresError> { + let mut txn = client.transaction()?; + for stmt in DDL_V1 { + txn.batch_execute(stmt)?; + } + txn.execute(DDL_SCHEMA_VERSION_UPSERT, &[&SCHEMA_VERSION])?; + txn.commit()?; + + #[cfg(feature = "timescale")] + { + use crate::schema::DDL_TIMESCALE_HYPERTABLE; + client.batch_execute(DDL_TIMESCALE_HYPERTABLE)?; + } + + Ok(()) +} + +fn now_iso_micros() -> String { + chrono::Utc::now().to_rfc3339_opts(SecondsFormat::Micros, true) +} + +fn backend(e: E) -> StoreError { + StoreError::Backend(e.to_string()) +} + +// ------------------------------------------------------------ +// Row mapping helpers +// ------------------------------------------------------------ + +fn row_to_last(row: &postgres::Row) -> LastRow { + LastRow { + row_hash: row.get::<_, String>(0), + sequence: row.get::<_, i64>(1), + } +} + +fn row_to_tip(row: &postgres::Row) -> TipRow { + TipRow { + entity_id: row.get::<_, String>(0), + chain_id: row.get::<_, Option>(1).unwrap_or_default(), + current_tip_hash: row.get::<_, Option>(2), + updated_at: row.get::<_, String>(3), + } +} + +fn row_to_stored(row: &postgres::Row) -> StoredRow { + StoredRow { + id: row.get::<_, i64>(0), + entity_id: row.get::<_, String>(1), + chain_id: row.get::<_, String>(2), + event_type: row.get::<_, String>(3), + sequence: row.get::<_, i64>(4), + prev_hash: row.get::<_, String>(5), + row_hash: row.get::<_, Option>(6), + anchor_hash: row.get::<_, Option>(7), + raw_payload: row.get::<_, Option>(8), + detected_at: row.get::<_, String>(9), + received_at: row.get::<_, String>(10), + source_url: row.get::<_, String>(11), + extraction_method: row.get::<_, String>(12), + } +} + +const SQL_SELECT_LAST_ROW: &str = " + SELECT row_hash, sequence_int FROM provenance_envelope + WHERE entity_id = $1 AND event_type != 'BREAK' AND row_hash IS NOT NULL + ORDER BY id DESC LIMIT 1"; + +const SQL_SELECT_CHAIN: &str = " + SELECT id, entity_id, chain_id, event_type, sequence_int, prev_hash, + row_hash, anchor_hash, raw_payload, detected_at, received_at, + source_url, extraction_method + FROM provenance_envelope + WHERE entity_id = $1 + ORDER BY id ASC"; + +const SQL_SELECT_TIP_BY_CHAIN: &str = " + SELECT entity_id, chain_id, current_tip_hash, updated_at + FROM chain_tips WHERE chain_id = $1"; + +const SQL_SELECT_TIP_BY_ENTITY: &str = " + SELECT entity_id, chain_id, current_tip_hash, updated_at + FROM chain_tips WHERE entity_id = $1"; + +const SQL_INSERT_ROW: &str = " + INSERT INTO provenance_envelope ( + chain_id, entity_id, event_type, source_url, detected_at, + received_at, extraction_method, prev_hash, raw_payload, + row_hash, anchor_hash, sequence_int + ) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12) + RETURNING id"; + +const SQL_INSERT_HASH_INDEX: &str = " + INSERT INTO hash_index (row_hash, chain_id, sequence_int, entity_id, recorded_at) + VALUES ($1,$2,$3,$4,$5) + ON CONFLICT (row_hash) DO NOTHING"; + +const SQL_UPSERT_TIP: &str = " + INSERT INTO chain_tips (entity_id, current_tip_hash, chain_id, updated_at) + VALUES ($1, $2, $3, $4) + ON CONFLICT (entity_id) DO UPDATE SET + current_tip_hash = EXCLUDED.current_tip_hash, + chain_id = EXCLUDED.chain_id, + updated_at = EXCLUDED.updated_at"; + +// ------------------------------------------------------------ +// Store trait impl +// ------------------------------------------------------------ + +impl Store for TrustyPostgres { + #[instrument(skip(self), fields(entity_id = %entity_id))] + fn last_row(&self, entity_id: &str) -> StoreResult> { + let mut client = self.client.borrow_mut(); + let row = client + .query_opt(SQL_SELECT_LAST_ROW, &[&entity_id]) + .map_err(backend)?; + Ok(row.as_ref().map(row_to_last)) + } + + #[instrument(skip(self, row), fields(entity_id = %row.entity_id, seq = row.sequence))] + fn insert_row(&mut self, row: &RowInsert) -> StoreResult { + let client = self.client.get_mut(); + let id: i64 = client + .query_one( + SQL_INSERT_ROW, + &[ + &row.chain_id, + &row.entity_id, + &row.event_type, + &row.source_url, + &row.detected_at, + &row.received_at, + &row.extraction_method, + &row.prev_hash, + &row.raw_payload, + &row.row_hash, + &row.anchor_hash, + &row.sequence, + ], + ) + .map_err(backend)? + .get(0); + + if let Some(rh) = &row.row_hash { + client + .execute( + SQL_INSERT_HASH_INDEX, + &[ + rh, + &row.chain_id, + &row.sequence, + &row.entity_id, + &row.received_at, + ], + ) + .map_err(backend)?; + } + + Ok(id) + } + + fn iter_chain<'a>( + &'a self, + entity_id: &str, + ) -> StoreResult> + 'a>> { + let mut client = self.client.borrow_mut(); + let rows = client + .query(SQL_SELECT_CHAIN, &[&entity_id]) + .map_err(backend)?; + let mapped: Vec> = + rows.iter().map(|r| Ok(row_to_stored(r))).collect(); + Ok(Box::new(mapped.into_iter())) + } + + fn get_tip(&self, chain_id: &str) -> StoreResult> { + let mut client = self.client.borrow_mut(); + let row = client + .query_opt(SQL_SELECT_TIP_BY_CHAIN, &[&chain_id]) + .map_err(backend)?; + Ok(row.as_ref().map(row_to_tip)) + } + + fn get_tip_for_entity(&self, entity_id: &str) -> StoreResult> { + let mut client = self.client.borrow_mut(); + let row = client + .query_opt(SQL_SELECT_TIP_BY_ENTITY, &[&entity_id]) + .map_err(backend)?; + Ok(row.as_ref().map(row_to_tip)) + } + + #[instrument(skip(self, tip), fields(entity_id = %tip.entity_id, chain_id = %tip.chain_id))] + fn set_tip(&mut self, tip: &TipUpsert) -> StoreResult<()> { + let client = self.client.get_mut(); + client + .execute( + SQL_UPSERT_TIP, + &[ + &tip.entity_id, + &tip.current_tip_hash, + &tip.chain_id, + &now_iso_micros(), + ], + ) + .map_err(backend)?; + Ok(()) + } + + fn kv_cross_check(&self) -> StoreResult { + let mut client = self.client.borrow_mut(); + + // Count sides. + let kv1_total: i64 = client + .query_one("SELECT COUNT(*) FROM hash_index", &[]) + .map_err(backend)? + .get(0); + let kv2_total: i64 = client + .query_one("SELECT COUNT(*) FROM chain_tips", &[]) + .map_err(backend)? + .get(0); + + // Cross-check: every chain_tip's current_tip_hash must exist in hash_index + // under the same (entity_id, chain_id). + let rows = client + .query( + "SELECT ct.entity_id, ct.chain_id, ct.current_tip_hash + FROM chain_tips ct + WHERE ct.current_tip_hash IS NOT NULL", + &[], + ) + .map_err(backend)?; + + let mut mismatches: Vec = Vec::new(); + let mut entities_checked = 0usize; + for row in rows.iter() { + entities_checked += 1; + let entity_id: String = row.get(0); + let chain_id: Option = row.get(1); + let tip_hash: String = row.get(2); + let chain_id = chain_id.unwrap_or_default(); + + let ok: Option = client + .query_opt( + "SELECT 1 FROM hash_index + WHERE row_hash = $1 AND entity_id = $2 AND chain_id = $3", + &[&tip_hash, &entity_id, &chain_id], + ) + .map_err(backend)? + .map(|r| r.get(0)); + if ok.is_none() { + mismatches.push(format!( + "entity={} chain={} tip={} not in hash_index", + entity_id, chain_id, tip_hash + )); + } + } + + Ok(KvCrossCheckReport { + ok: mismatches.is_empty(), + kv1_total_hashes: kv1_total as usize, + kv2_total_entities: kv2_total as usize, + entities_checked, + mismatches, + }) + } + + #[instrument(skip(self, f))] + fn with_txn( + &mut self, + f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>, + ) -> StoreResult<()> { + let client = self.client.get_mut(); + let mut txn = client.transaction().map_err(backend)?; + let result = { + let mut wrapper = TxnStore { + txn: RefCell::new(&mut txn), + }; + f(&mut wrapper) + // `wrapper` dropped here, releasing the &mut txn borrow. + }; + match result { + Ok(()) => txn.commit().map_err(backend), + Err(e) => { + let _ = txn.rollback(); + Err(e) + } + } + } +} + +// ------------------------------------------------------------ +// Transaction-scoped Store wrapper (for with_txn) +// +// Two lifetimes: 'a is how long this wrapper holds the &mut, 'c is +// the underlying connection's lifetime (parameter on Transaction). +// Conflating them (impl<'t> with Transaction<'t>) causes the borrow +// checker to think the outer Transaction is still borrowed when we +// try to commit() after f() returns. Splitting them lets the +// wrapper's lifetime end at scope close while Transaction keeps +// living on the outer stack frame. +// ------------------------------------------------------------ + +struct TxnStore<'t, 'c: 't> { + txn: RefCell<&'t mut Transaction<'c>>, +} + +impl<'t, 'c: 't> Store for TxnStore<'t, 'c> { + fn last_row(&self, entity_id: &str) -> StoreResult> { + let mut txn = self.txn.borrow_mut(); + let row = txn + .query_opt(SQL_SELECT_LAST_ROW, &[&entity_id]) + .map_err(backend)?; + Ok(row.as_ref().map(row_to_last)) + } + + fn insert_row(&mut self, row: &RowInsert) -> StoreResult { + let mut txn = self.txn.borrow_mut(); + let id: i64 = txn + .query_one( + SQL_INSERT_ROW, + &[ + &row.chain_id, + &row.entity_id, + &row.event_type, + &row.source_url, + &row.detected_at, + &row.received_at, + &row.extraction_method, + &row.prev_hash, + &row.raw_payload, + &row.row_hash, + &row.anchor_hash, + &row.sequence, + ], + ) + .map_err(backend)? + .get(0); + if let Some(rh) = &row.row_hash { + txn.execute( + SQL_INSERT_HASH_INDEX, + &[ + rh, + &row.chain_id, + &row.sequence, + &row.entity_id, + &row.received_at, + ], + ) + .map_err(backend)?; + } + Ok(id) + } + + fn iter_chain<'a>( + &'a self, + entity_id: &str, + ) -> StoreResult> + 'a>> { + let mut txn = self.txn.borrow_mut(); + let rows = txn + .query(SQL_SELECT_CHAIN, &[&entity_id]) + .map_err(backend)?; + let mapped: Vec> = + rows.iter().map(|r| Ok(row_to_stored(r))).collect(); + Ok(Box::new(mapped.into_iter())) + } + + fn get_tip(&self, chain_id: &str) -> StoreResult> { + let mut txn = self.txn.borrow_mut(); + let row = txn + .query_opt(SQL_SELECT_TIP_BY_CHAIN, &[&chain_id]) + .map_err(backend)?; + Ok(row.as_ref().map(row_to_tip)) + } + + fn get_tip_for_entity(&self, entity_id: &str) -> StoreResult> { + let mut txn = self.txn.borrow_mut(); + let row = txn + .query_opt(SQL_SELECT_TIP_BY_ENTITY, &[&entity_id]) + .map_err(backend)?; + Ok(row.as_ref().map(row_to_tip)) + } + + fn set_tip(&mut self, tip: &TipUpsert) -> StoreResult<()> { + let mut txn = self.txn.borrow_mut(); + txn.execute( + SQL_UPSERT_TIP, + &[ + &tip.entity_id, + &tip.current_tip_hash, + &tip.chain_id, + &now_iso_micros(), + ], + ) + .map_err(backend)?; + Ok(()) + } + + fn kv_cross_check(&self) -> StoreResult { + // Inside a txn we skip the cross-check -- the relevant checks happen + // at txn commit time via Postgres constraints. Return an OK stub. + Ok(KvCrossCheckReport { + ok: true, + kv1_total_hashes: 0, + kv2_total_entities: 0, + entities_checked: 0, + mismatches: Vec::new(), + }) + } + + fn with_txn( + &mut self, + _f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>, + ) -> StoreResult<()> { + Err(StoreError::Backend( + "nested with_txn on an already-open transaction is not supported".into(), + )) + } +} diff --git a/crates/adapters/trusty-postgres/src/trusty.rs b/crates/adapters/trusty-postgres/src/trusty.rs new file mode 100644 index 0000000..6f43d86 --- /dev/null +++ b/crates/adapters/trusty-postgres/src/trusty.rs @@ -0,0 +1,1520 @@ +//! Higher-level `Trusty` API for PostgreSQL. +//! +//! Mirrors the `trusty_sqlite::Trusty` surface for the happy-path +//! feature set: `open`, `append` (unsigned with chain-break detection), +//! `verify`, `history`, `integrity`, `breaks_since`, `liveness_check`. +//! +//! Deliberate gaps (tracked; out of scope for this cycle): +//! - `append_signed` / `append_with_method_signed` -- v2-only signed path +//! lives in trusty-core; v3 signed path is a separate trusty-core task. +//! - `append_batch` -- single-transaction batch wrapper, trivial follow-up. +//! - `anchor`, `anchors_since` -- pure-query additions. +//! - `export_bundle` -- ProofBundle construction from rows; pending. +//! - CHECKPOINT API (#13) -- separate module. +//! - Per-entity advisory lock at append time (memo section 9). The +//! batch path holds a transaction lock; the single-append path holds +//! a session lock for the whole read-tip/compute/insert sequence, +//! which is not otherwise atomic. + +use chrono::SecondsFormat; +use tracing::instrument; +use trusty_core::{ + checkpoint_signing_bytes, compute_record, compute_record_signed, default_conditions, evaluate, + make_chain_id, parse_generation as core_parse_generation, verify_links, AnchorEntry, BundleRow, + ChainLink, Condition, ConditionContext, ConditionOutcome, Ed25519Verifier, EventType, + PayloadFields, ProofBundle, SecretKey, Signer, Verifier, VerifyError, VerifyErrorType, + VerifyReport, GENESIS_SENTINEL, +}; + +// Higher-level result types live in trusty-store (S16, promoted from +// trusty-sqlite so every backend reuses the same shape). Re-exported +// here so existing trusty-postgres consumers keep working. +pub use trusty_store::{ + AnchorInfo, AppendResult, BreakEvent, CheckpointRow, HistoryRow, IntegrityMismatch, + IntegrityReport, ProducerIdentity, RevocationRecord, RowSpec, SilentEntity, +}; + +use crate::error::{PostgresError, PostgresResult}; +use crate::schema::{DDL_SCHEMA_VERSION_UPSERT, DDL_V1, SCHEMA_VERSION}; + +pub struct Trusty { + client: postgres::Client, + key: SecretKey, + conditions: Vec>, +} + +struct InternalLastRow { + row_hash: String, + sequence_int: i64, +} + +struct InternalTipRow { + chain_id: String, + current_tip_hash: Option, +} + +impl Trusty { + pub fn open(conn_str: &str, key: SecretKey) -> PostgresResult { + let mut client = postgres::Client::connect(conn_str, postgres::NoTls)?; + apply_schema(&mut client)?; + Ok(Self { + client, + key, + conditions: default_conditions(), + }) + } + + pub fn open_with_conditions( + conn_str: &str, + key: SecretKey, + conditions: Vec>, + ) -> PostgresResult { + let mut client = postgres::Client::connect(conn_str, postgres::NoTls)?; + apply_schema(&mut client)?; + Ok(Self { + client, + key, + conditions, + }) + } + + #[instrument(skip(self, data), fields(entity_id = %entity_id, src = %source_url))] + pub fn append( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + ) -> PostgresResult { + self.append_with_method(entity_id, data, source_url, "trusty") + } + + /// Signed append (v2 payload bytes by default; v3 if the caller + /// constructs `PayloadFields::new_v3`). Mirrors trusty-sqlite's + /// `append_signed`. Unblocks federated ProofBundle exchange per + /// `s15_decision_multitenant_shared_kb_shape` -- v2-only signatures + /// cannot be mixed across federation peers safely; v3 signed path + /// shipped in S16 cycle 2 (`compute_record_signed` now version- + /// dispatches). + pub fn append_signed( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + signer: &dyn Signer, + ) -> PostgresResult { + self.append_with_method_inner(entity_id, data, source_url, "trusty", Some(signer)) + } + + pub fn append_with_method( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + ) -> PostgresResult { + self.append_with_method_inner(entity_id, data, source_url, extraction_method, None) + } + + /// Signed variant of `append_with_method`. See `append_signed`. + pub fn append_with_method_signed( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + signer: &dyn Signer, + ) -> PostgresResult { + self.append_with_method_inner(entity_id, data, source_url, extraction_method, Some(signer)) + } + + /// Hold a per-entity lock for the duration of `f`. + /// + /// Appending is read-tip, compute, insert. Those three steps are not + /// atomic, so two writers on the same entity can read the same tip + /// and both build on it -- one chain, two rows claiming the same + /// predecessor. The batch path has taken a lock since it was + /// written; the single-append path never did, and the module header + /// said so: "add before any multi-writer deployment". + /// + /// A SESSION lock rather than the batch path's transaction lock, + /// because the append helpers each use `self.client` directly and + /// threading a transaction through all four is a larger change than + /// the bug warrants. Postgres releases session locks when the + /// connection drops, so a process dying mid-append does not strand + /// it; what this must get right is releasing on the error path, + /// which is why the result is captured before unlocking rather than + /// returned through `?`. + /// + /// `hashtext` reduces the entity id to a lock key. Collisions are + /// rare and benign: two unrelated entities serialise, which is slow, + /// never wrong. + fn with_entity_lock( + &mut self, + entity_id: &str, + f: impl FnOnce(&mut Self) -> PostgresResult, + ) -> PostgresResult { + self.client + .execute("SELECT pg_advisory_lock(hashtext($1)::bigint)", &[&entity_id])?; + let result = f(self); + let unlocked = self + .client + .execute("SELECT pg_advisory_unlock(hashtext($1)::bigint)", &[&entity_id]); + // A failed unlock must not mask the caller's error, but it must + // not be swallowed either: the connection is now holding a lock + // nobody will release until it drops. + match (result, unlocked) { + (Ok(v), Ok(_)) => Ok(v), + (Ok(_), Err(e)) => Err(PostgresError::IntegrityViolation(format!( + "append succeeded but the entity lock for {entity_id:?} could not be released: {e}" + ))), + (Err(e), _) => Err(e), + } + } + + fn append_with_method_inner( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + signer: Option<&dyn Signer>, + ) -> PostgresResult { + let entity = entity_id.to_string(); + let data = data.clone(); + let source_url = source_url.to_string(); + let extraction_method = extraction_method.to_string(); + self.with_entity_lock(entity_id, move |me| { + me.append_locked(&entity, &data, &source_url, &extraction_method, signer) + }) + } + + fn append_locked( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + signer: Option<&dyn Signer>, + ) -> PostgresResult { + let detected_at = now_iso(); + let received_at = now_iso_micros(); + let tip = self.read_tip(entity_id)?; + + let tip = match tip { + None => { + return self.create_genesis( + entity_id, + 1, + data, + source_url, + extraction_method, + &detected_at, + &received_at, + None, + None, + signer, + ); + } + Some(t) => t, + }; + + let chain_id = tip.chain_id.clone(); + let last = self.read_last(entity_id, &chain_id)?; + let (sequence, last_sequence, computed_hash) = match &last { + Some(l) => ( + l.sequence_int + 1, + Some(l.sequence_int), + Some(l.row_hash.clone()), + ), + None => (0, None, None), + }; + + let ctx = ConditionContext { + entity_id: entity_id.to_string(), + chain_id: chain_id.clone(), + sequence, + stored_tip_hash: tip.current_tip_hash.clone(), + computed_hash, + prev_hash: tip.current_tip_hash.clone(), + last_sequence, + source_url: source_url.to_string(), + extraction_method: extraction_method.to_string(), + data: data.clone(), + }; + + match evaluate(&self.conditions, &ctx) { + ConditionOutcome::Pass => { + let prev_hash = last + .as_ref() + .map(|l| l.row_hash.clone()) + .unwrap_or_else(|| GENESIS_SENTINEL.to_string()); + self.write_mutation( + entity_id, + &chain_id, + data, + source_url, + extraction_method, + &detected_at, + &received_at, + &prev_hash, + sequence, + signer, + ) + } + ConditionOutcome::Fail(reason) => { + let last_valid_hash = last + .as_ref() + .map(|l| l.row_hash.clone()) + .unwrap_or_else(|| GENESIS_SENTINEL.to_string()); + let code = reason.as_code().to_string(); + self.handle_break( + entity_id, + &chain_id, + &code, + &last_valid_hash, + data, + source_url, + extraction_method, + &detected_at, + &received_at, + signer, + ) + } + } + } + + #[allow(clippy::too_many_arguments)] + fn create_genesis( + &mut self, + entity_id: &str, + generation: u32, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + detected_at: &str, + received_at: &str, + sealed_chain: Option, + break_reason: Option, + signer: Option<&dyn Signer>, + ) -> PostgresResult { + let chain_id = make_chain_id(entity_id, generation); + let prev_hash = GENESIS_SENTINEL.to_string(); + let sequence: i64 = 0; + + let fields = PayloadFields::new_v1( + entity_id, + chain_id.clone(), + data.clone(), + source_url, + detected_at, + received_at, + extraction_method, + prev_hash.clone(), + sequence, + ); + let computed = match signer { + Some(s) => compute_record_signed(&fields, &self.key, s)?, + None => compute_record(&fields, &self.key)?, + }; + + self.write_envelope( + &chain_id, + entity_id, + EventType::Genesis.as_str(), + None, + None, + None, + source_url, + detected_at, + received_at, + extraction_method, + &prev_hash, + Some(&computed.row_hash), + Some(&computed.anchor_hash), + sequence, + Some(&computed.raw_payload), + )?; + self.append_hash_index(&computed.row_hash, &chain_id, sequence, entity_id)?; + self.set_tip(entity_id, &chain_id, Some(&computed.row_hash))?; + + let chain_break = sealed_chain.is_some(); + Ok(AppendResult { + entity_id: entity_id.to_string(), + chain_id, + event_type: EventType::Genesis.as_str().to_string(), + chain_break, + break_reason, + sealed_chain, + row_hash: computed.row_hash, + anchor_hash: computed.anchor_hash, + sequence, + is_genesis: !chain_break, + }) + } + + #[allow(clippy::too_many_arguments)] + fn write_mutation( + &mut self, + entity_id: &str, + chain_id: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + detected_at: &str, + received_at: &str, + prev_hash: &str, + sequence: i64, + signer: Option<&dyn Signer>, + ) -> PostgresResult { + let fields = PayloadFields::new_v1( + entity_id, + chain_id, + data.clone(), + source_url, + detected_at, + received_at, + extraction_method, + prev_hash, + sequence, + ); + let computed = match signer { + Some(s) => compute_record_signed(&fields, &self.key, s)?, + None => compute_record(&fields, &self.key)?, + }; + + self.write_envelope( + chain_id, + entity_id, + EventType::Mutation.as_str(), + None, + None, + None, + source_url, + detected_at, + received_at, + extraction_method, + prev_hash, + Some(&computed.row_hash), + Some(&computed.anchor_hash), + sequence, + Some(&computed.raw_payload), + )?; + self.append_hash_index(&computed.row_hash, chain_id, sequence, entity_id)?; + self.set_tip(entity_id, chain_id, Some(&computed.row_hash))?; + + Ok(AppendResult { + entity_id: entity_id.to_string(), + chain_id: chain_id.to_string(), + event_type: EventType::Mutation.as_str().to_string(), + chain_break: false, + break_reason: None, + sealed_chain: None, + row_hash: computed.row_hash, + anchor_hash: computed.anchor_hash, + sequence, + is_genesis: false, + }) + } + + #[allow(clippy::too_many_arguments)] + fn handle_break( + &mut self, + entity_id: &str, + old_chain_id: &str, + break_reason: &str, + last_valid_hash: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + detected_at: &str, + received_at: &str, + signer: Option<&dyn Signer>, + ) -> PostgresResult { + self.write_envelope( + old_chain_id, + entity_id, + EventType::Break.as_str(), + Some("__chain_break__"), + Some(last_valid_hash), + Some(break_reason), + source_url, + detected_at, + received_at, + extraction_method, + last_valid_hash, + None, + None, + -1, + None, + )?; + + let current_gen = parse_generation(old_chain_id)?; + let next_gen = current_gen + 1; + + let mut result = self.create_genesis( + entity_id, + next_gen, + data, + source_url, + extraction_method, + detected_at, + received_at, + Some(old_chain_id.to_string()), + Some(break_reason.to_string()), + signer, + )?; + result.event_type = "BREAK".to_string(); + result.is_genesis = false; + Ok(result) + } + + /// Single-transaction batch wrapper around `append_with_method`. + /// Mirrors `trusty_sqlite::Trusty::append_batch`. All rows commit + /// or all rows roll back. Per-row advisory lock is held inside + /// the surrounding transaction so concurrent batches targeting + /// the same entity serialize without blocking on others (memo §9). + /// + /// Signed batches: use per-row `append_signed` in a loop until a + /// signed-batch surface lands. + pub fn append_batch(&mut self, specs: &[RowSpec]) -> PostgresResult> { + if specs.is_empty() { + return Ok(Vec::new()); + } + self.client.batch_execute("BEGIN")?; + let mut out = Vec::with_capacity(specs.len()); + for spec in specs { + // Per-entity advisory lock inside the batch transaction. + // pg_advisory_xact_lock auto-releases on COMMIT or + // ROLLBACK so cleanup is implicit. hashtext() reduces + // entity_id to an int4 lock key; collisions are rare and + // benign (two distinct entities serialize, never wrong). + if let Err(e) = self + .client + .execute( + "SELECT pg_advisory_xact_lock(hashtext($1)::bigint)", + &[&spec.entity_id], + ) + .map_err(PostgresError::from) + { + let _ = self.client.batch_execute("ROLLBACK"); + return Err(e); + } + match self.append_with_method(spec.entity_id, spec.data, spec.source_url, "trusty") { + Ok(r) => out.push(r), + Err(e) => { + let _ = self.client.batch_execute("ROLLBACK"); + return Err(e); + } + } + } + self.client.batch_execute("COMMIT")?; + Ok(out) + } + + /// Most recent anchor commitment for `entity_id` (chain_id + + /// tip_hash, plus a server-side `anchored_at` timestamp). + /// Returns None if the entity has never been seen. Mirrors + /// `trusty_sqlite::Trusty::anchor`. + pub fn anchor(&mut self, entity_id: &str) -> PostgresResult> { + let row = self.client.query_opt( + "SELECT chain_id, current_tip_hash FROM chain_tips WHERE entity_id = $1", + &[&entity_id], + )?; + Ok(row.map(|r| AnchorInfo { + entity_id: entity_id.to_string(), + chain_id: r.get::<_, String>(0), + tip_hash: r.get::<_, Option>(1), + anchored_at: now_iso(), + })) + } + + /// Per-row anchor commitments for `entity_id` whose `detected_at` + /// is strictly greater than `since` (or all of them when `since` + /// is None). Insertion order (id ASC). BREAK rows are excluded; + /// they are structural events, not commitments. Pairs with + /// `breaks_since` for a two-feed model. Mirrors + /// `trusty_sqlite::Trusty::anchors_since`. + pub fn anchors_since( + &mut self, + entity_id: &str, + since: Option<&str>, + ) -> PostgresResult> { + let rows = self.client.query( + "SELECT sequence_int, anchor_hash, detected_at + FROM provenance_envelope + WHERE entity_id = $1 + AND row_hash IS NOT NULL + AND anchor_hash IS NOT NULL + AND ($2::text IS NULL OR detected_at > $2) + ORDER BY id ASC", + &[&entity_id, &since], + )?; + Ok(rows + .iter() + .map(|r| AnchorEntry { + sequence: r.get::<_, i64>(0), + anchor_hash: r.get::<_, String>(1), + detected_at: r.get::<_, String>(2), + }) + .collect()) + } + + /// Package a slice of the chain into a self-contained + /// `ProofBundle`. Non-BREAK rows are filtered by `sequence` in + /// `[since, until]` (inclusive; unbounded on either side when + /// None). BREAK rows for the entity are always included so + /// chain-break-as-event proof survives export. Anchors for the + /// same slice are attached. The resulting bundle can be handed + /// to `ProofBundle::verify(key)` without database access. + /// Mirrors `trusty_sqlite::Trusty::export_bundle` -- foundation + /// for federated ProofBundle exchange (s15_decision_multitenant + /// _shared_kb_shape) and external-verifier handoff. + pub fn export_bundle( + &mut self, + entity_id: &str, + since: Option, + until: Option, + ) -> PostgresResult { + let lo = since.unwrap_or(i64::MIN); + let hi = until.unwrap_or(i64::MAX); + + let rows = self.client.query( + "SELECT chain_id, event_type, sequence_int, prev_hash, + row_hash, anchor_hash, raw_payload, detected_at, new_value + FROM provenance_envelope + WHERE entity_id = $1 + AND ( + (event_type != 'BREAK' AND sequence_int >= $2 AND sequence_int <= $3) + OR event_type = 'BREAK' + ) + ORDER BY id ASC", + &[&entity_id, &lo, &hi], + )?; + + let mut bundle_rows: Vec = Vec::with_capacity(rows.len()); + let mut anchors: Vec = Vec::new(); + for r in rows.iter() { + let chain_id: String = r.get(0); + let event_type: String = r.get(1); + let sequence: i64 = r.get(2); + let prev_hash: String = r.get(3); + let row_hash: Option = r.get(4); + let anchor_hash: Option = r.get(5); + let raw_payload: Option = r.get(6); + let detected_at: Option = r.get(7); + // A BREAK row keeps its reason in new_value; carrying it out + // is what makes the marker readable off the machine that + // recorded it. + let break_reason: Option = if event_type == "BREAK" { + r.get(8) + } else { + None + }; + + if event_type != "BREAK" { + if let Some(ah) = anchor_hash.clone() { + anchors.push(AnchorEntry { + sequence, + anchor_hash: ah, + detected_at: detected_at.clone().unwrap_or_default(), + }); + } + } + + bundle_rows.push( + BundleRow::new(entity_id, chain_id, event_type, sequence, prev_hash) + .with_hashes(row_hash, anchor_hash) + .with_payload(raw_payload) + .with_break_reason(break_reason), + ); + } + + Ok(ProofBundle::new( + entity_id.to_string(), + now_iso(), + bundle_rows, + anchors, + )) + } + + /// Register (or update) a producer identity row. OID is the + /// primary key; calling with the same OID overwrites the + /// pubkey + siphash_key + display_name (UPSERT). This is the + /// federation-auth onboarding path: a new agent generates an + /// Ed25519 keypair + SipHash key, the operator registers them + /// here under the agent's OID, and from then on attestations + /// signed by that agent are verifiable. + pub fn register_producer( + &mut self, + producer_oid: &str, + ed25519_pubkey: &[u8], + siphash_key: &[u8], + display_name: Option<&str>, + ) -> PostgresResult<()> { + self.register_producer_with_format( + producer_oid, + ed25519_pubkey, + siphash_key, + display_name, + trusty_core::KeyFormat::Ed25519Raw, + ) + } + + /// Register a producer, declaring how its key material is encoded. + /// + /// A `user` enrolling through WebAuthn registers a COSE credential + /// and must say so. Verification then refuses the bare-signature + /// path for that producer rather than failing in a way that looks + /// like a wrong key. + pub fn register_producer_with_format( + &mut self, + producer_oid: &str, + ed25519_pubkey: &[u8], + siphash_key: &[u8], + display_name: Option<&str>, + key_format: trusty_core::KeyFormat, + ) -> PostgresResult<()> { + self.client.execute( + "INSERT INTO producer_identity + (producer_oid, ed25519_pubkey, siphash_key, display_name, key_format) + VALUES ($1, $2, $3, $4, $5) + ON CONFLICT (producer_oid) DO UPDATE SET + ed25519_pubkey = EXCLUDED.ed25519_pubkey, + siphash_key = EXCLUDED.siphash_key, + display_name = EXCLUDED.display_name, + key_format = EXCLUDED.key_format", + &[ + &producer_oid, + &ed25519_pubkey, + &siphash_key, + &display_name, + &key_format.as_str(), + ], + )?; + Ok(()) + } + + /// Read back a producer identity row by OID. Returns None when + /// the OID is not registered. Used by `verify_since_checkpoint` + /// (and any caller building federation auth) to fetch the + /// canonical Ed25519 pubkey for verifying attestations. + pub fn lookup_producer( + &mut self, + producer_oid: &str, + ) -> PostgresResult> { + let row = self.client.query_opt( + "SELECT producer_oid, ed25519_pubkey, siphash_key, key_format, + registered_at::text, display_name, metadata::text + FROM producer_identity + WHERE producer_oid = $1", + &[&producer_oid], + )?; + Ok(match row { + Some(r) => { + let metadata_str: Option = r.get(6); + let metadata = metadata_str + .map(|s| serde_json::from_str::(&s)) + .transpose()?; + let format_str: String = r.get(3); + Some(ProducerIdentity { + producer_oid: r.get(0), + ed25519_pubkey: r.get(1), + siphash_key: r.get(2), + key_format: trusty_core::KeyFormat::parse(&format_str) + .map_err(|e| PostgresError::IntegrityViolation(e.to_string()))?, + registered_at: r.get(4), + display_name: r.get(5), + metadata, + }) + } + None => None, + }) + } + + /// Revoke a registered producer. Memo §6 / learn-LCG-ARCH-06: + /// forward-only revocation, CT-stance. After revocation, any + /// checkpoint or row signed AT or AFTER `revoked_at` by this + /// producer is rejected by verify_since_checkpoint; rows signed + /// BEFORE remain verifiable forever (append-only invariant). + /// + /// Two writes happen, ordered: + /// 1. A REVOCATION envelope is appended to the per-user + /// `revocations` chain via `append_signed`, so the act of + /// revoking is itself part of an authenticated chain. + /// 2. The `producer_revocation` row is UPSERTed for fast + /// "is_revoked?" lookups by verify_since_checkpoint. + /// + /// Re-revoking an already-revoked OID overwrites the row with a + /// new `revoked_at` timestamp + reason + attestor + envelope + /// reference. The earlier revocation envelope stays in the + /// chain (append-only); only the index row is replaced. + /// + /// Sub-case (c) key rotation per memo §6 is NOT supported in + /// this v1 -- a revoked OID stays revoked. Rotation is its own + /// envelope type (KEY_ROTATION) plus chain-of-custody work. + pub fn revoke_producer( + &mut self, + producer_oid: &str, + reason: &str, + attestor: &dyn Signer, + ) -> PostgresResult { + let identity = self.lookup_producer(producer_oid)?.ok_or_else(|| { + PostgresError::NotFound(format!( + "revoke_producer: {} not in producer_identity", + producer_oid + )) + })?; + + let attested_by_pubkey = attestor.public_key_bytes(); + + // 1. Append a REVOCATION envelope on the per-user "revocations" + // chain. event_type stays as the standard append flow's + // Mutation/Genesis (since we cannot inject a custom + // event_type via append_signed today); the payload data + // encodes the revocation specifics. This keeps the chain- + // break-as-event pipeline clean while still committing + // the revocation to a verifiable chain. + let revocation_data = serde_json::json!({ + "kind": "REVOCATION", + "producer_oid": producer_oid, + "revoked_pubkey_hex": hex::encode(&identity.ed25519_pubkey), + "reason": reason, + "attested_by_pubkey_hex": hex::encode(&attested_by_pubkey), + }); + let envelope = self.append_signed("revocations", &revocation_data, "revoke", attestor)?; + + // 2. UPSERT the producer_revocation index row. + let row = self.client.query_one( + "INSERT INTO producer_revocation + (producer_oid, revoked_pubkey, reason, attested_by_pubkey, revocation_row_hash) + VALUES ($1, $2, $3, $4, $5) + ON CONFLICT (producer_oid) DO UPDATE SET + revoked_at = NOW(), + revoked_pubkey = EXCLUDED.revoked_pubkey, + reason = EXCLUDED.reason, + attested_by_pubkey = EXCLUDED.attested_by_pubkey, + revocation_row_hash = EXCLUDED.revocation_row_hash + RETURNING producer_oid, revoked_at::text, revoked_pubkey, + reason, attested_by_pubkey, revocation_row_hash", + &[ + &producer_oid, + &identity.ed25519_pubkey, + &reason, + &attested_by_pubkey, + &envelope.row_hash, + ], + )?; + + Ok(RevocationRecord { + producer_oid: row.get(0), + revoked_at: row.get(1), + revoked_pubkey: row.get(2), + reason: row.get(3), + attested_by_pubkey: row.get(4), + revocation_row_hash: row.get(5), + }) + } + + /// Read back a producer's revocation row (None if not revoked). + pub fn lookup_revocation( + &mut self, + producer_oid: &str, + ) -> PostgresResult> { + let row = self.client.query_opt( + "SELECT producer_oid, revoked_at::text, revoked_pubkey, + reason, attested_by_pubkey, revocation_row_hash + FROM producer_revocation WHERE producer_oid = $1", + &[&producer_oid], + )?; + Ok(row.map(|r| RevocationRecord { + producer_oid: r.get(0), + revoked_at: r.get(1), + revoked_pubkey: r.get(2), + reason: r.get(3), + attested_by_pubkey: r.get(4), + revocation_row_hash: r.get(5), + })) + } + + /// Write a fresh checkpoint at the current chain tip for + /// `entity_id`. Walks the chain to head, signs + /// `checkpoint_signing_bytes(chain_id, sequence, row_hash)` with + /// `signer`, and inserts a row into `chain_checkpoint`. + /// `attested_by` typically names the attestor's OID (matching + /// the future `producer_identity` lookup). + /// + /// Verification-layer only -- chain payload bytes are NEVER + /// touched. Memo §5 / Gitea #13. Resolves the v1 CHECKPOINT API + /// scope on trusty-postgres. + pub fn checkpoint( + &mut self, + entity_id: &str, + attested_by: &str, + signer: &dyn Signer, + ) -> PostgresResult { + let tip = self.read_tip(entity_id)?.ok_or_else(|| { + PostgresError::NotFound(format!("checkpoint: no tip for entity {}", entity_id)) + })?; + let last = self.read_last(entity_id, &tip.chain_id)?.ok_or_else(|| { + PostgresError::NotFound(format!("checkpoint: chain_id {} has no rows", tip.chain_id)) + })?; + let row_hash = last.row_hash.clone(); + let sequence_int = last.sequence_int; + + let to_sign = checkpoint_signing_bytes(&tip.chain_id, sequence_int, &row_hash); + let sig = signer.sign(&to_sign); + + let row = self.client.query_one( + "INSERT INTO chain_checkpoint + (chain_id, sequence_int, row_hash, attested_by, attestation_sig) + VALUES ($1, $2, $3, $4, $5) + RETURNING id, chain_id, sequence_int, row_hash, + checkpoint_at::text, attested_by, attestation_sig", + &[&tip.chain_id, &sequence_int, &row_hash, &attested_by, &sig], + )?; + + Ok(CheckpointRow { + id: row.get(0), + chain_id: row.get(1), + sequence_int: row.get(2), + row_hash: row.get(3), + checkpoint_at: row.get(4), + attested_by: row.get(5), + attestation_sig: row.get(6), + }) + } + + /// Most recent checkpoint for `chain_id` (id DESC, take 1) or + /// None if no checkpoint exists. Caller is responsible for + /// signature verification against `producer_identity` -- this + /// API is the row reader. + pub fn latest_checkpoint(&mut self, chain_id: &str) -> PostgresResult> { + let row = self.client.query_opt( + "SELECT id, chain_id, sequence_int, row_hash, + checkpoint_at::text, attested_by, attestation_sig + FROM chain_checkpoint + WHERE chain_id = $1 + ORDER BY id DESC + LIMIT 1", + &[&chain_id], + )?; + Ok(row.map(|r| CheckpointRow { + id: r.get(0), + chain_id: r.get(1), + sequence_int: r.get(2), + row_hash: r.get(3), + checkpoint_at: r.get(4), + attested_by: r.get(5), + attestation_sig: r.get(6), + })) + } + + /// Verify the chain for `entity_id` from the most recent valid + /// checkpoint forward. Falls back to full verify when no + /// checkpoint exists. The window-pruning is the whole reason + /// checkpoints exist (memo §5): on a long-lived chain the + /// verify cost stays bounded by the checkpoint cadence rather + /// than the chain length. + /// + /// Authentication: the checkpoint's `attested_by` MUST resolve + /// to a registered `producer_identity` row, AND the + /// `attestation_sig` MUST verify under that row's + /// `ed25519_pubkey` over the canonical + /// `checkpoint_signing_bytes(chain_id, sequence_int, row_hash)`. + /// On either failure, the function returns a non-ok + /// `VerifyReport` with a `SignatureInvalid` entry rather than + /// silently falling back to full verify -- a malformed or + /// unauthenticated checkpoint is a federation-trust event the + /// caller must see (s15 multi-tenant federation requirement). + pub fn verify_since_checkpoint(&mut self, entity_id: &str) -> PostgresResult { + let tip = match self.read_tip(entity_id)? { + Some(t) => t, + None => return self.verify(entity_id), + }; + let ckpt = match self.latest_checkpoint(&tip.chain_id)? { + Some(c) => c, + None => return self.verify(entity_id), + }; + + // Authenticate the checkpoint before trusting its window. + let identity = self.lookup_producer(&ckpt.attested_by)?; + let identity = match identity { + Some(id) => id, + None => { + return Ok(VerifyReport { + entity_id: entity_id.to_string(), + rows_valid: false, + // The checkpoint was refused, so no rows were walked + // and continuity is UNESTABLISHED rather than proved. + // False, with no breaks listed: fail closed, because + // absent evidence is not evidence of absence. + continuous: false, + // Nothing was walked, so no ceiling was reached either. + limit_reached: false, + breaks: Vec::new(), + row_count: 0, + errors: vec![VerifyError { + sequence: ckpt.sequence_int, + chain_id: ckpt.chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "checkpoint attested_by={} not in producer_identity", + ckpt.attested_by + ), + }, + }], + }); + } + }; + + let signing_view = + checkpoint_signing_bytes(&ckpt.chain_id, ckpt.sequence_int, &ckpt.row_hash); + if Ed25519Verifier::verify( + &identity.ed25519_pubkey, + &signing_view, + &ckpt.attestation_sig, + ) + .is_err() + { + return Ok(VerifyReport { + entity_id: entity_id.to_string(), + rows_valid: false, + // Refused, not walked -- see the note on the sibling + // early return above. Continuity is unestablished. + continuous: false, + // Nothing was walked, so no ceiling was reached either. + limit_reached: false, + breaks: Vec::new(), + row_count: 0, + errors: vec![VerifyError { + sequence: ckpt.sequence_int, + chain_id: ckpt.chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "checkpoint attestation_sig invalid for attested_by={}", + ckpt.attested_by + ), + }, + }], + }); + } + + // Forward-only revocation check (memo §6): if the attestor is + // revoked AND the checkpoint was written AT or AFTER revocation, + // reject. Checkpoints written BEFORE revocation remain valid + // (CT stance: past attestations stay verifiable). + if let Some(rev) = self.lookup_revocation(&ckpt.attested_by)? { + if ckpt.checkpoint_at.as_str() >= rev.revoked_at.as_str() { + return Ok(VerifyReport { + entity_id: entity_id.to_string(), + rows_valid: false, + // The checkpoint was refused, so no rows were walked + // and continuity is UNESTABLISHED rather than proved. + // False, with no breaks listed: fail closed, because + // absent evidence is not evidence of absence. + continuous: false, + // Nothing was walked, so no ceiling was reached either. + limit_reached: false, + breaks: Vec::new(), + row_count: 0, + errors: vec![VerifyError { + sequence: ckpt.sequence_int, + chain_id: ckpt.chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "checkpoint attested_by={} revoked at {} (checkpoint_at={})", + ckpt.attested_by, rev.revoked_at, ckpt.checkpoint_at + ), + }, + }], + }); + } + } + + let rows = self.client.query( + "SELECT chain_id, event_type, sequence_int, prev_hash, row_hash, anchor_hash, raw_payload + FROM provenance_envelope + WHERE entity_id = $1 + AND sequence_int >= $2 + ORDER BY id ASC", + &[&entity_id, &ckpt.sequence_int], + )?; + let mut links: Vec = Vec::with_capacity(rows.len()); + for r in rows.iter() { + links.push(ChainLink { + entity_id: entity_id.to_string(), + chain_id: r.get::<_, String>(0), + event_type: r.get::<_, String>(1), + sequence: r.get::<_, i64>(2), + prev_hash: r.get::<_, String>(3), + row_hash: r.get::<_, Option>(4), + anchor_hash: r.get::<_, Option>(5), + raw_payload: r.get::<_, Option>(6), + }); + } + Ok(verify_links(entity_id, &links, &self.key)) + } + + #[instrument(skip(self), fields(entity_id = %entity_id))] + pub fn verify(&mut self, entity_id: &str) -> PostgresResult { + let rows = self.client.query( + "SELECT chain_id, event_type, sequence_int, prev_hash, row_hash, anchor_hash, raw_payload + FROM provenance_envelope WHERE entity_id = $1 ORDER BY id ASC", + &[&entity_id], + )?; + let mut links: Vec = Vec::with_capacity(rows.len()); + for r in rows.iter() { + links.push(ChainLink { + entity_id: entity_id.to_string(), + chain_id: r.get::<_, String>(0), + event_type: r.get::<_, String>(1), + sequence: r.get::<_, i64>(2), + prev_hash: r.get::<_, String>(3), + row_hash: r.get::<_, Option>(4), + anchor_hash: r.get::<_, Option>(5), + raw_payload: r.get::<_, Option>(6), + }); + } + Ok(verify_links(entity_id, &links, &self.key)) + } + + /// Same as `verify`, but ALSO gates each signed row's signature + /// pubkey against the registered `producer_identity` for + /// `expected_oid`. Closes the federation-trust loop at the row + /// level: today's `verify_links` accepts any pubkey embedded in + /// the row's signatures array; this method asserts every signed + /// row was signed by the producer the caller expects. + /// + /// Forward-only revocation per memo §6: a row whose `detected_at + /// >= revoked_at` is rejected as `SignatureInvalid`. Rows whose + /// `detected_at < revoked_at` remain valid (CT stance). + /// + /// Errors that go in the report (NOT as a Result error) so the + /// caller can see the full picture: + /// - `SignatureInvalid { detail: "expected_oid not in producer_identity" }` + /// on the genesis row when the OID isn't registered. + /// - `SignatureInvalid { detail: "row signed by unauthorized pubkey ..." }` + /// on any row whose sig pubkey != registered pubkey. + /// - `SignatureInvalid { detail: "row signed AT or AFTER revocation ..." }` + /// on rows past revocation. + /// + /// Plus all errors `verify_links` produces for chain-integrity reasons. + pub fn verify_with_producer( + &mut self, + entity_id: &str, + expected_oid: &str, + ) -> PostgresResult { + // 1. Resolve the expected pubkey + revocation state up front. + let identity = self.lookup_producer(expected_oid)?; + let revocation = self.lookup_revocation(expected_oid)?; + + // 2. Pull rows including detected_at + raw_payload for sig parsing. + let rows = self.client.query( + "SELECT chain_id, event_type, sequence_int, prev_hash, row_hash, + anchor_hash, raw_payload, detected_at + FROM provenance_envelope WHERE entity_id = $1 ORDER BY id ASC", + &[&entity_id], + )?; + + let mut links: Vec = Vec::with_capacity(rows.len()); + let mut auth_errors: Vec = Vec::new(); + + for r in rows.iter() { + let chain_id: String = r.get(0); + let event_type: String = r.get(1); + let sequence: i64 = r.get(2); + let prev_hash: String = r.get(3); + let row_hash: Option = r.get(4); + let anchor_hash: Option = r.get(5); + let raw_payload: Option = r.get(6); + let detected_at: String = r.get(7); + + // Auth gate: if this row carries signatures, every sig's + // pubkey must equal the registered pubkey, AND the row + // must not be past revocation. + if let Some(raw) = raw_payload.as_deref() { + if let Some(sig_pks) = parse_signature_pubkeys(raw) { + for pk_hex in sig_pks { + let pk_bytes = match hex::decode(&pk_hex) { + Ok(b) => b, + Err(e) => { + auth_errors.push(VerifyError { + sequence, + chain_id: chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!("row signature pk hex malformed: {}", e), + }, + }); + continue; + } + }; + + match &identity { + None => { + auth_errors.push(VerifyError { + sequence, + chain_id: chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "expected_oid={} not in producer_identity", + expected_oid + ), + }, + }); + } + Some(id) if pk_bytes != id.ed25519_pubkey => { + auth_errors.push(VerifyError { + sequence, + chain_id: chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "row signed by unauthorized pubkey (expected_oid={}); pk_hex={}", + expected_oid, pk_hex + ), + }, + }); + } + _ => {} + } + + if let Some(rev) = &revocation { + if detected_at.as_str() >= rev.revoked_at.as_str() { + auth_errors.push(VerifyError { + sequence, + chain_id: chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "row signed AT or AFTER revocation (expected_oid={} revoked_at={} detected_at={})", + expected_oid, rev.revoked_at, detected_at + ), + }, + }); + } + } + } + } + } + + links.push(ChainLink { + entity_id: entity_id.to_string(), + chain_id, + event_type, + sequence, + prev_hash, + row_hash, + anchor_hash, + raw_payload, + }); + } + + // 3. Run normal verify_links and merge errors. + let mut report = verify_links(entity_id, &links, &self.key); + if !auth_errors.is_empty() { + report.rows_valid = false; + report.errors.extend(auth_errors); + } + Ok(report) + } + + pub fn history(&mut self, entity_id: &str, limit: i64) -> PostgresResult> { + let rows = self.client.query( + "SELECT id, chain_id, entity_id, event_type, sequence_int, detected_at, source_url, + row_hash, anchor_hash, raw_payload + FROM provenance_envelope WHERE entity_id = $1 ORDER BY id ASC LIMIT $2", + &[&entity_id, &limit], + )?; + Ok(rows + .iter() + .map(|r| HistoryRow { + id: r.get::<_, i64>(0), + chain_id: r.get::<_, String>(1), + entity_id: r.get::<_, String>(2), + event_type: r.get::<_, String>(3), + sequence: r.get::<_, i64>(4), + detected_at: r.get::<_, String>(5), + source_url: r.get::<_, String>(6), + row_hash: r.get::<_, Option>(7), + anchor_hash: r.get::<_, Option>(8), + raw_payload: r.get::<_, Option>(9), + }) + .collect()) + } + + pub fn breaks_since( + &mut self, + entity_id: &str, + since: Option<&str>, + ) -> PostgresResult> { + let rows = self.client.query( + "SELECT chain_id, old_value, new_value, detected_at + FROM provenance_envelope + WHERE entity_id = $1 AND event_type = 'BREAK' + AND ($2::text IS NULL OR detected_at > $2) + ORDER BY id ASC", + &[&entity_id, &since], + )?; + Ok(rows + .iter() + .map(|r| BreakEvent { + entity_id: entity_id.to_string(), + chain_id: r.get::<_, String>(0), + sealed_at_hash: r.get::<_, Option>(1).unwrap_or_default(), + break_reason: r.get::<_, Option>(2).unwrap_or_default(), + detected_at: r.get::<_, String>(3), + }) + .collect()) + } + + pub fn liveness_check( + &mut self, + entity_id: Option<&str>, + max_gap_secs: i64, + ) -> PostgresResult> { + // detected_at is stored as ISO-8601 TEXT; we parse at query time. + // For v1 we keep it simple: let Postgres do the arithmetic via + // to_timestamp + NOW() diff. + let rows = self.client.query( + "SELECT entity_id, + MAX(detected_at) AS last_seen, + EXTRACT(EPOCH FROM (NOW() - MAX(detected_at)::timestamptz)) AS gap_secs + FROM provenance_envelope + WHERE event_type != 'BREAK' + AND ($1::text IS NULL OR entity_id = $1) + GROUP BY entity_id + HAVING EXTRACT(EPOCH FROM (NOW() - MAX(detected_at)::timestamptz)) > $2", + &[&entity_id, &(max_gap_secs as f64)], + )?; + Ok(rows + .iter() + .map(|r| SilentEntity { + entity_id: r.get::<_, String>(0), + last_seen: r.get::<_, String>(1), + gap_secs: r.get::<_, f64>(2), + }) + .collect()) + } + + pub fn integrity(&mut self) -> PostgresResult { + let kv1_total: i64 = self + .client + .query_one("SELECT COUNT(*) FROM hash_index", &[])? + .get(0); + let kv2_total: i64 = self + .client + .query_one("SELECT COUNT(*) FROM chain_tips", &[])? + .get(0); + + let rows = self.client.query( + "SELECT ct.entity_id, ct.chain_id, ct.current_tip_hash + FROM chain_tips ct WHERE ct.current_tip_hash IS NOT NULL", + &[], + )?; + let mut mismatches: Vec = Vec::new(); + let mut entities_checked = 0usize; + for r in rows.iter() { + entities_checked += 1; + let entity_id: String = r.get(0); + let chain_id: Option = r.get(1); + let tip_hash: String = r.get(2); + let chain_id = chain_id.unwrap_or_default(); + let ok = self.client.query_opt( + "SELECT 1 FROM hash_index + WHERE row_hash = $1 AND entity_id = $2 AND chain_id = $3", + &[&tip_hash, &entity_id, &chain_id], + )?; + if ok.is_none() { + mismatches.push(IntegrityMismatch { + entity_id, + reason: "tip_not_in_hash_index".to_string(), + detail: Some(format!("chain={} tip={}", chain_id, tip_hash)), + }); + } + } + Ok(IntegrityReport { + ok: mismatches.is_empty(), + entities_checked, + kv1_total_hashes: kv1_total as usize, + kv2_total_entities: kv2_total as usize, + mismatches, + }) + } + + // -------------------------------------------------------------- + // Internal helpers + // -------------------------------------------------------------- + + fn read_tip(&mut self, entity_id: &str) -> PostgresResult> { + let row = self.client.query_opt( + "SELECT chain_id, current_tip_hash FROM chain_tips WHERE entity_id = $1", + &[&entity_id], + )?; + Ok(row.map(|r| InternalTipRow { + chain_id: r.get::<_, Option>(0).unwrap_or_default(), + current_tip_hash: r.get::<_, Option>(1), + })) + } + + fn read_last( + &mut self, + entity_id: &str, + chain_id: &str, + ) -> PostgresResult> { + let row = self.client.query_opt( + "SELECT row_hash, sequence_int FROM provenance_envelope + WHERE entity_id = $1 AND chain_id = $2 AND event_type != 'BREAK' + AND row_hash IS NOT NULL + ORDER BY id DESC LIMIT 1", + &[&entity_id, &chain_id], + )?; + Ok(row.map(|r| InternalLastRow { + row_hash: r.get::<_, String>(0), + sequence_int: r.get::<_, i64>(1), + })) + } + + #[allow(clippy::too_many_arguments)] + fn write_envelope( + &mut self, + chain_id: &str, + entity_id: &str, + event_type: &str, + field_name: Option<&str>, + old_value: Option<&str>, + new_value: Option<&str>, + source_url: &str, + detected_at: &str, + received_at: &str, + extraction_method: &str, + prev_hash: &str, + row_hash: Option<&str>, + anchor_hash: Option<&str>, + sequence: i64, + raw_payload: Option<&str>, + ) -> PostgresResult<()> { + self.client.execute( + "INSERT INTO provenance_envelope ( + chain_id, entity_id, event_type, + field_name, old_value, new_value, + source_url, detected_at, received_at, extraction_method, + prev_hash, row_hash, anchor_hash, sequence_int, raw_payload + ) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15)", + &[ + &chain_id, + &entity_id, + &event_type, + &field_name, + &old_value, + &new_value, + &source_url, + &detected_at, + &received_at, + &extraction_method, + &prev_hash, + &row_hash, + &anchor_hash, + &sequence, + &raw_payload, + ], + )?; + Ok(()) + } + + fn append_hash_index( + &mut self, + row_hash: &str, + chain_id: &str, + sequence: i64, + entity_id: &str, + ) -> PostgresResult<()> { + self.client.execute( + "INSERT INTO hash_index (row_hash, chain_id, sequence_int, entity_id, recorded_at) + VALUES ($1,$2,$3,$4,$5) + ON CONFLICT (row_hash) DO NOTHING", + &[ + &row_hash, + &chain_id, + &sequence, + &entity_id, + &now_iso_micros(), + ], + )?; + Ok(()) + } + + fn set_tip( + &mut self, + entity_id: &str, + chain_id: &str, + current_tip_hash: Option<&str>, + ) -> PostgresResult<()> { + self.client.execute( + "INSERT INTO chain_tips (entity_id, current_tip_hash, chain_id, updated_at) + VALUES ($1,$2,$3,$4) + ON CONFLICT (entity_id) DO UPDATE SET + current_tip_hash = EXCLUDED.current_tip_hash, + chain_id = EXCLUDED.chain_id, + updated_at = EXCLUDED.updated_at", + &[&entity_id, ¤t_tip_hash, &chain_id, &now_iso_micros()], + )?; + Ok(()) + } +} + +// ------------------------------------------------------------ +// Freestanding helpers +// ------------------------------------------------------------ + +fn apply_schema(client: &mut postgres::Client) -> PostgresResult<()> { + let mut txn = client.transaction()?; + for stmt in DDL_V1 { + txn.batch_execute(stmt)?; + } + txn.execute(DDL_SCHEMA_VERSION_UPSERT, &[&SCHEMA_VERSION])?; + txn.commit()?; + + #[cfg(feature = "timescale")] + { + use crate::schema::DDL_TIMESCALE_HYPERTABLE; + client.batch_execute(DDL_TIMESCALE_HYPERTABLE)?; + } + Ok(()) +} + +fn now_iso() -> String { + chrono::Utc::now().to_rfc3339_opts(SecondsFormat::Secs, true) +} + +fn now_iso_micros() -> String { + chrono::Utc::now().to_rfc3339_opts(SecondsFormat::Micros, true) +} + +/// Wrap the trusty-core `parse_generation` so adapter call sites stay +/// in PostgresResult and translate None to MalformedChainId. The pure +/// parser was promoted to trusty-core in S16 (resolves §12 OQ #16). +fn parse_generation(chain_id: &str) -> PostgresResult { + core_parse_generation(chain_id) + .ok_or_else(|| PostgresError::MalformedChainId(chain_id.to_string())) +} + +/// Pull the `signatures` array out of a v2/v3 raw_payload and return +/// the hex-encoded pubkey from each entry (`"pk"` field). Returns +/// None if the payload is unparseable, the field is missing, or the +/// array is empty (i.e., this is an unsigned v1 row). On per-entry +/// malformation, that entry is skipped silently -- callers that want +/// to surface malformed entries should inspect raw_payload directly. +fn parse_signature_pubkeys(raw_payload: &str) -> Option> { + let v: serde_json::Value = serde_json::from_str(raw_payload).ok()?; + let arr = v.get("signatures")?.as_array()?; + if arr.is_empty() { + return None; + } + let pks: Vec = arr + .iter() + .filter_map(|s| s.get("pk")?.as_str().map(|s| s.to_string())) + .collect(); + if pks.is_empty() { + None + } else { + Some(pks) + } +} diff --git a/crates/adapters/trusty-postgres/tests/conformance.rs b/crates/adapters/trusty-postgres/tests/conformance.rs new file mode 100644 index 0000000..6d9887f --- /dev/null +++ b/crates/adapters/trusty-postgres/tests/conformance.rs @@ -0,0 +1,55 @@ +//! Runs the shared `Store` conformance suite against a live Postgres. +//! +//! Gated behind the TRUSTY_PG_URL env var. Set it to a libpq connection +//! string that points at a disposable test DB, e.g.: +//! +//! TRUSTY_PG_URL="host=localhost user=postgres password=postgres dbname=trusty_test" +//! cargo test -p trusty-postgres --test conformance -- --nocapture +//! +//! If TRUSTY_PG_URL is not set, the test prints a skip notice and exits +//! clean. This keeps the default `cargo test --workspace` hermetic. + +use trusty_postgres::TrustyPostgres; +use trusty_store::testing::run_conformance; + +const ENV_URL: &str = "TRUSTY_PG_URL"; + +fn reset_schema(store: &mut TrustyPostgres) { + // Drop-and-recreate so the suite starts from a known-empty state. + // Done by connecting to the same DB and issuing the drops directly. + let url = std::env::var(ENV_URL).expect("TRUSTY_PG_URL must be set"); + let mut client = postgres::Client::connect(&url, postgres::NoTls).expect("connect"); + client + .batch_execute( + "DROP TABLE IF EXISTS chain_checkpoint CASCADE; + DROP TABLE IF EXISTS producer_identity CASCADE; + DROP TABLE IF EXISTS chain_tips CASCADE; + DROP TABLE IF EXISTS hash_index CASCADE; + DROP TABLE IF EXISTS provenance_envelope CASCADE; + DROP TABLE IF EXISTS schema_meta CASCADE;", + ) + .expect("drop tables"); + let _ = store; // store gets recreated below +} + +#[test] +fn postgres_passes_store_conformance() { + let url = match std::env::var(ENV_URL) { + Ok(v) => v, + Err(_) => { + eprintln!( + "SKIP: set {} to run the conformance suite against a live Postgres", + ENV_URL + ); + return; + } + }; + + // Initial open to ensure the tables exist. + let mut store = TrustyPostgres::open(&url).expect("open"); + reset_schema(&mut store); + // Re-open so apply_schema runs against a clean DB. + let mut store = TrustyPostgres::open(&url).expect("re-open"); + + run_conformance(&mut store); +} diff --git a/crates/adapters/trusty-postgres/tests/integration.rs b/crates/adapters/trusty-postgres/tests/integration.rs new file mode 100644 index 0000000..d0ee0a5 --- /dev/null +++ b/crates/adapters/trusty-postgres/tests/integration.rs @@ -0,0 +1,705 @@ +//! TRUSTY_PG_URL-gated integration tests for the higher-level +//! `Trusty` surface on Postgres. Pays down the test debt accumulated +//! across S16 cycles 4-7 (append_signed, append_batch + advisory +//! lock, anchor, anchors_since, export_bundle, checkpoint, +//! latest_checkpoint, verify_since_checkpoint). +//! +//! Setup mirrors `tests/conformance.rs`: when TRUSTY_PG_URL is unset +//! the tests print SKIP and exit clean so `cargo test --workspace` +//! stays hermetic. Set it to a libpq connection string pointing at a +//! disposable test DB to actually run them. + +use serde_json::json; +use trusty_core::{Ed25519Signer, ProofBundle, SecretKey, Signer}; +use trusty_postgres::{RowSpec, Trusty}; + +const ENV_URL: &str = "TRUSTY_PG_URL"; + +fn maybe_url() -> Option { + std::env::var(ENV_URL).ok() +} + +fn drop_tables(url: &str) { + let mut client = postgres::Client::connect(url, postgres::NoTls).expect("connect"); + client + .batch_execute( + "DROP TABLE IF EXISTS chain_checkpoint CASCADE; + DROP TABLE IF EXISTS producer_revocation CASCADE; + DROP TABLE IF EXISTS producer_identity CASCADE; + DROP TABLE IF EXISTS chain_tips CASCADE; + DROP TABLE IF EXISTS hash_index CASCADE; + DROP TABLE IF EXISTS provenance_envelope CASCADE; + DROP TABLE IF EXISTS schema_meta CASCADE;", + ) + .expect("drop tables"); +} + +fn open_clean(url: &str) -> Trusty { + drop_tables(url); + let key = SecretKey::from_bytes(b"0123456789abcdef").expect("key"); + Trusty::open(url, key).expect("open Trusty") +} + +fn signer() -> Ed25519Signer { + let seed = [9u8; 32]; + Ed25519Signer::from_secret_bytes(&seed).expect("signer") +} + +fn skip_if_no_url() -> Option { + let url = maybe_url(); + if url.is_none() { + eprintln!( + "SKIP: set {} to a Postgres connection string to run integration tests", + ENV_URL + ); + } + url +} + +#[test] +fn append_signed_writes_v2_payload_and_verifies() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + let r = t + .append_signed("dev1", &json!({"v": 1}), "src", &s) + .expect("append_signed"); + assert!(!r.chain_break); + assert_eq!(r.sequence, 0); + assert!(r.is_genesis); + + // Payload should be v2 with a populated signature object. Read + // raw_payload back from the row directly to confirm. + let mut client = postgres::Client::connect(&url, postgres::NoTls).expect("connect"); + let row = client + .query_one( + "SELECT raw_payload FROM provenance_envelope + WHERE entity_id = $1 ORDER BY id ASC LIMIT 1", + &[&"dev1"], + ) + .expect("read row"); + let raw: String = row.get(0); + assert!(raw.contains("\"payload_version\":2"), "raw = {}", raw); + assert!(raw.contains("\"alg\":\"ed25519\""), "raw = {}", raw); + + let report = t.verify("dev1").expect("verify"); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 1); +} + +#[test] +fn append_batch_is_all_or_nothing_and_advisory_lock_present() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + + let d0 = json!({"v": 0}); + let d1 = json!({"v": 1}); + let d2 = json!({"v": 2}); + let specs = vec![ + RowSpec { + entity_id: "fleet-a", + data: &d0, + source_url: "src", + }, + RowSpec { + entity_id: "fleet-a", + data: &d1, + source_url: "src", + }, + RowSpec { + entity_id: "fleet-b", + data: &d2, + source_url: "src", + }, + ]; + let results = t.append_batch(&specs).expect("append_batch"); + assert_eq!(results.len(), 3); + assert!(t.verify("fleet-a").expect("verify a").rows_valid); + assert!(t.verify("fleet-b").expect("verify b").rows_valid); + + // Empty batch is a no-op. + let empty: Vec = Vec::new(); + let none = t.append_batch(&empty).expect("empty batch"); + assert!(none.is_empty()); +} + +#[test] +fn anchor_and_anchors_since_round_trip() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + + for i in 0..3 { + t.append("dev2", &json!({"i": i}), "src").expect("append"); + } + + let anchor = t.anchor("dev2").expect("anchor").expect("some"); + assert_eq!(anchor.entity_id, "dev2"); + assert!(anchor.chain_id.starts_with("chain_dev2_")); + assert!(anchor.tip_hash.is_some()); + + let entries = t.anchors_since("dev2", None).expect("anchors_since"); + // Three appends -> three rows -> three anchor commitments. + assert_eq!(entries.len(), 3); + // Each entry has a distinct sequence and non-empty hash. + for (i, e) in entries.iter().enumerate() { + assert_eq!(e.sequence, i as i64); + assert!(!e.anchor_hash.is_empty()); + } + + // since-filter uses strict-greater-than on detected_at; a value + // beyond all rows yields the empty list. + let none = t + .anchors_since("dev2", Some("9999-01-01T00:00:00Z")) + .expect("anchors_since since=future"); + assert!(none.is_empty()); +} + +#[test] +fn export_bundle_round_trips_through_proof_bundle_verify() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let key = SecretKey::from_bytes(b"0123456789abcdef").expect("key"); + + for i in 0..4 { + t.append("dev3", &json!({"i": i}), "src").expect("append"); + } + + let bundle: ProofBundle = t.export_bundle("dev3", None, None).expect("export_bundle"); + let report = bundle.verify(&key); + assert!( + report.rows_valid, + "bundle verify failed: {:?}", + report.errors + ); + assert_eq!(report.row_count, 4); + + // Sequence-bounded slice still verifies. + let slice = t + .export_bundle("dev3", Some(1), Some(2)) + .expect("export_bundle slice"); + let slice_report = slice.verify(&key); + assert!(slice_report.rows_valid, "{:?}", slice_report.errors); + assert_eq!(slice_report.row_count, 2); +} + +#[test] +fn checkpoint_writes_then_latest_returns_it() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + for i in 0..3 { + t.append("dev4", &json!({"i": i}), "src").expect("append"); + } + + let ckpt = t.checkpoint("dev4", "oid:1.2.3", &s).expect("checkpoint"); + assert_eq!(ckpt.attested_by, "oid:1.2.3"); + assert_eq!(ckpt.sequence_int, 2); + assert!(!ckpt.row_hash.is_empty()); + assert!(!ckpt.attestation_sig.is_empty()); + + let latest = t + .latest_checkpoint(&ckpt.chain_id) + .expect("latest_checkpoint") + .expect("some"); + assert_eq!(latest.id, ckpt.id); + assert_eq!(latest.sequence_int, ckpt.sequence_int); + assert_eq!(latest.row_hash, ckpt.row_hash); + assert_eq!(latest.attestation_sig, ckpt.attestation_sig); +} + +#[test] +fn verify_since_checkpoint_falls_back_when_no_checkpoint() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + + for i in 0..2 { + t.append("dev5", &json!({"i": i}), "src").expect("append"); + } + + // No checkpoint written -> verify_since_checkpoint must return + // the same shape as a full verify. + let full = t.verify("dev5").expect("verify"); + let pruned = t.verify_since_checkpoint("dev5").expect("vsc"); + assert_eq!(full.rows_valid, pruned.rows_valid); + assert_eq!(full.row_count, pruned.row_count); +} + +#[test] +fn verify_since_checkpoint_after_checkpoint_still_passes() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + // S16 cycle 9: verify_since_checkpoint authenticates the + // checkpoint against producer_identity before trusting the + // window-prune. Register the attestor first. + t.register_producer( + "oid:test", + &s.public_key_bytes(), + b"siphash_key_bytes_16", + Some("test attestor"), + ) + .expect("register producer"); + + for i in 0..3 { + t.append("dev6", &json!({"i": i}), "src").expect("append"); + } + t.checkpoint("dev6", "oid:test", &s).expect("checkpoint"); + // Append more rows after the checkpoint. + for i in 3..5 { + t.append("dev6", &json!({"i": i}), "src").expect("append"); + } + + let report = t.verify_since_checkpoint("dev6").expect("vsc"); + assert!(report.rows_valid, "{:?}", report.errors); + // Verify covers checkpoint.sequence (=2) forward -> rows 2,3,4 + // -> 3 rows. + assert_eq!(report.row_count, 3); +} + +#[test] +fn producer_identity_register_and_lookup_round_trip() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + let pk = s.public_key_bytes(); + t.register_producer("oid:1.2.3", &pk, b"sip-key-1", Some("agent-a")) + .expect("register"); + + let got = t + .lookup_producer("oid:1.2.3") + .expect("lookup") + .expect("some"); + assert_eq!(got.producer_oid, "oid:1.2.3"); + assert_eq!(got.ed25519_pubkey, pk); + assert_eq!(got.siphash_key, b"sip-key-1"); + assert_eq!(got.display_name.as_deref(), Some("agent-a")); + + // Re-register same OID is an UPSERT. + let s2 = trusty_core::Ed25519Signer::from_secret_bytes(&[3u8; 32]).expect("s2"); + let pk2 = s2.public_key_bytes(); + t.register_producer("oid:1.2.3", &pk2, b"sip-key-2", None) + .expect("re-register"); + let updated = t + .lookup_producer("oid:1.2.3") + .expect("lookup2") + .expect("some"); + assert_eq!(updated.ed25519_pubkey, pk2); + assert_eq!(updated.siphash_key, b"sip-key-2"); + assert!(updated.display_name.is_none()); + + // Unknown OID yields None. + assert!(t.lookup_producer("oid:nope").expect("lookup3").is_none()); +} + +#[test] +fn verify_since_checkpoint_rejects_unknown_attestor() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + // Append rows + write checkpoint WITHOUT registering the + // attestor first. The checkpoint row is in the table but + // attested_by has no producer_identity row. + for i in 0..2 { + t.append("dev7", &json!({"i": i}), "src").expect("append"); + } + t.checkpoint("dev7", "oid:ghost", &s) + .expect("checkpoint write succeeds (lookup happens at verify)"); + + let report = t.verify_since_checkpoint("dev7").expect("vsc"); + assert!( + !report.rows_valid, + "expected failure for unregistered attestor" + ); + assert_eq!(report.errors.len(), 1); + let detail = match &report.errors[0].kind { + trusty_core::VerifyErrorType::SignatureInvalid { detail } => detail.clone(), + other => panic!("expected SignatureInvalid, got {:?}", other), + }; + assert!( + detail.contains("not in producer_identity"), + "detail = {}", + detail + ); +} + +#[test] +fn verify_with_producer_accepts_signed_rows_by_registered_producer() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + t.register_producer("oid:test", &s.public_key_bytes(), b"sip-key", None) + .expect("register"); + + // Three signed rows by the registered producer. + for i in 0..3 { + t.append_signed("dev-rsa1", &json!({"i": i}), "src", &s) + .expect("append_signed"); + } + + let report = t + .verify_with_producer("dev-rsa1", "oid:test") + .expect("verify_with_producer"); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 3); +} + +#[test] +fn verify_with_producer_rejects_row_signed_by_unauthorized_pubkey() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let registered = signer(); + let imposter = trusty_core::Ed25519Signer::from_secret_bytes(&[77u8; 32]).expect("imp"); + + t.register_producer("oid:test", ®istered.public_key_bytes(), b"sip-key", None) + .expect("register"); + + // Row 0 signed by the registered producer (legitimate). + t.append_signed("dev-rsa2", &json!({"i": 0}), "src", ®istered) + .expect("append_signed legit"); + // Row 1 signed by the imposter -- chain math still verifies, but + // verify_with_producer must flag it because the pk doesn't match + // producer_identity for oid:test. + t.append_signed("dev-rsa2", &json!({"i": 1}), "src", &imposter) + .expect("append_signed imposter"); + + let report = t + .verify_with_producer("dev-rsa2", "oid:test") + .expect("verify_with_producer"); + assert!(!report.rows_valid, "expected rejection of unauthorized pk"); + let unauthorized = report + .errors + .iter() + .find(|e| match &e.kind { + trusty_core::VerifyErrorType::SignatureInvalid { detail } => { + detail.contains("unauthorized pubkey") + } + _ => false, + }) + .expect("expected an 'unauthorized pubkey' SignatureInvalid"); + assert_eq!(unauthorized.sequence, 1); +} + +#[test] +fn verify_with_producer_rejects_row_after_revocation() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + t.register_producer("oid:test", &s.public_key_bytes(), b"sip-key", None) + .expect("register"); + + // Revoke FIRST, then append. The row's detected_at will be + // strictly greater than revoked_at, so it must reject. + t.revoke_producer("oid:test", "compromise", &s) + .expect("revoke"); + std::thread::sleep(std::time::Duration::from_millis(50)); + t.append_signed("dev-rsa3", &json!({"i": 0}), "src", &s) + .expect("append_signed after revoke"); + + let report = t + .verify_with_producer("dev-rsa3", "oid:test") + .expect("verify_with_producer"); + assert!(!report.rows_valid); + let after_rev = report.errors.iter().find(|e| match &e.kind { + trusty_core::VerifyErrorType::SignatureInvalid { detail } => { + detail.contains("AT or AFTER revocation") + } + _ => false, + }); + assert!( + after_rev.is_some(), + "expected post-revocation rejection; errors: {:?}", + report.errors + ); +} + +#[test] +fn verify_with_producer_accepts_rows_signed_before_revocation() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + t.register_producer("oid:test", &s.public_key_bytes(), b"sip-key", None) + .expect("register"); + + // Append FIRST, then revoke. CT stance: past rows stay valid. + for i in 0..2 { + t.append_signed("dev-rsa4", &json!({"i": i}), "src", &s) + .expect("append_signed"); + } + std::thread::sleep(std::time::Duration::from_millis(50)); + t.revoke_producer("oid:test", "later compromise", &s) + .expect("revoke"); + + let report = t + .verify_with_producer("dev-rsa4", "oid:test") + .expect("verify_with_producer"); + assert!( + report.rows_valid, + "expected pre-revocation rows to remain valid; errors: {:?}", + report.errors + ); + assert_eq!(report.row_count, 2); +} + +#[test] +fn revoke_producer_round_trip_and_lookup() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + t.register_producer("oid:test", &s.public_key_bytes(), b"sip-key", None) + .expect("register"); + + // Not revoked yet. + assert!(t.lookup_revocation("oid:test").expect("lookup").is_none()); + + // Revoke. Returns RevocationRecord; producer_revocation row exists. + let rec = t + .revoke_producer("oid:test", "key compromise", &s) + .expect("revoke"); + assert_eq!(rec.producer_oid, "oid:test"); + assert_eq!(rec.reason, "key compromise"); + assert_eq!(rec.revoked_pubkey, s.public_key_bytes()); + assert_eq!(rec.attested_by_pubkey, s.public_key_bytes()); + assert!(!rec.revocation_row_hash.is_empty()); + + // Lookup returns the same record. + let got = t + .lookup_revocation("oid:test") + .expect("lookup") + .expect("some"); + assert_eq!(got.producer_oid, rec.producer_oid); + assert_eq!(got.revocation_row_hash, rec.revocation_row_hash); + + // The REVOCATION envelope is on the per-user 'revocations' chain + // and verifies via the standard verify path. + let report = t.verify("revocations").expect("verify revocations chain"); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 1); + + // Re-revoking the same OID UPSERTs (single row per OID). + let rec2 = t + .revoke_producer("oid:test", "second reason", &s) + .expect("re-revoke"); + assert_eq!(rec2.reason, "second reason"); + // Two REVOCATION envelopes on chain (append-only); one + // producer_revocation index row (UPSERT). + let report2 = t + .verify("revocations") + .expect("verify revocations chain v2"); + assert!(report2.rows_valid); + assert_eq!(report2.row_count, 2); +} + +#[test] +fn revoke_producer_unknown_oid_errors() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + let err = t.revoke_producer("oid:ghost", "no such producer", &s); + assert!(err.is_err(), "expected NotFound for unregistered OID"); +} + +#[test] +fn verify_since_checkpoint_rejects_revoked_attestor_after_revocation() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + t.register_producer("oid:test", &s.public_key_bytes(), b"sip-key", None) + .expect("register"); + + for i in 0..3 { + t.append("dev-rev1", &json!({"i": i}), "src") + .expect("append"); + } + + // Revoke FIRST, then write checkpoint. Per CT stance, this + // checkpoint's checkpoint_at >= revoked_at, so it must reject. + t.revoke_producer("oid:test", "compromise", &s) + .expect("revoke"); + t.checkpoint("dev-rev1", "oid:test", &s) + .expect("checkpoint write succeeds (verify happens later)"); + + let report = t.verify_since_checkpoint("dev-rev1").expect("vsc"); + assert!(!report.rows_valid, "expected rejection of revoked attestor"); + let detail = match &report.errors[0].kind { + trusty_core::VerifyErrorType::SignatureInvalid { detail } => detail.clone(), + other => panic!("expected SignatureInvalid, got {:?}", other), + }; + assert!(detail.contains("revoked at"), "detail = {}", detail); +} + +#[test] +fn verify_since_checkpoint_accepts_attestor_revoked_after_checkpoint() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let s = signer(); + + t.register_producer("oid:test", &s.public_key_bytes(), b"sip-key", None) + .expect("register"); + + for i in 0..3 { + t.append("dev-rev2", &json!({"i": i}), "src") + .expect("append"); + } + + // Checkpoint FIRST, then revoke. Per CT stance, this checkpoint's + // checkpoint_at < revoked_at, so it must still accept. + t.checkpoint("dev-rev2", "oid:test", &s) + .expect("checkpoint"); + // Sleep briefly to ensure a strictly-greater revoked_at. + std::thread::sleep(std::time::Duration::from_millis(50)); + t.revoke_producer("oid:test", "later compromise", &s) + .expect("revoke"); + + let report = t.verify_since_checkpoint("dev-rev2").expect("vsc"); + assert!( + report.rows_valid, + "expected past checkpoint to remain valid; errors: {:?}", + report.errors + ); +} + +#[test] +fn verify_since_checkpoint_rejects_tampered_attestation_sig() { + let Some(url) = skip_if_no_url() else { return }; + let mut t = open_clean(&url); + let attestor = signer(); + let imposter = trusty_core::Ed25519Signer::from_secret_bytes(&[42u8; 32]).expect("imp"); + + // Register the *attestor*'s pubkey under the OID, but write the + // checkpoint signed by the *imposter*. Sig verification must + // fail. + t.register_producer( + "oid:test", + &attestor.public_key_bytes(), + b"siphash-key-XYZ", + None, + ) + .expect("register"); + + for i in 0..2 { + t.append("dev8", &json!({"i": i}), "src").expect("append"); + } + t.checkpoint("dev8", "oid:test", &imposter) + .expect("checkpoint with imposter signer"); + + let report = t.verify_since_checkpoint("dev8").expect("vsc"); + assert!(!report.rows_valid, "expected failure for tampered sig"); + let detail = match &report.errors[0].kind { + trusty_core::VerifyErrorType::SignatureInvalid { detail } => detail.clone(), + other => panic!("expected SignatureInvalid, got {:?}", other), + }; + assert!( + detail.contains("attestation_sig invalid"), + "detail = {}", + detail + ); +} + +// --------------------------------------------------------------------- +// Per-entity lock on the single-append path. +// --------------------------------------------------------------------- + +/// Two writers, two connections, one entity, at the same time. +/// +/// Appending is read-tip, compute, insert, and those three steps are not +/// atomic. Without a per-entity lock both writers read the same tip and +/// both build on it, producing two rows that claim the same predecessor +/// -- a fork in a structure whose whole promise is that there is one +/// line. +/// +/// The batch path has held a lock since it was written. The single +/// path never did; the module header said "add before any multi-writer +/// deployment", and nothing was enforcing it. +#[test] +fn concurrent_appends_to_one_entity_do_not_fork_the_chain() { + let Some(url) = maybe_url() else { + eprintln!("SKIP: {ENV_URL} unset"); + return; + }; + let entity = "concurrency-probe"; + + // A unique entity per run, so a previous run cannot make this pass + // and no cleanup accessor is needed on the public surface. + let entity = &format!( + "{entity}-{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + ); + + // Apply the schema ONCE. Four threads each running + // CREATE TABLE IF NOT EXISTS collide in the catalog with "tuple + // concurrently updated" -- a real property of concurrent DDL, and + // not what this test is about. + { + let key = SecretKey::from_bytes(b"0123456789abcdef").expect("key"); + Trusty::open(&url, key).expect("apply schema"); + } + + const WRITERS: usize = 4; + const EACH: usize = 6; + + let handles: Vec<_> = (0..WRITERS) + .map(|w| { + let url = url.clone(); + let entity = entity.clone(); + std::thread::spawn(move || { + let key = SecretKey::from_bytes(b"0123456789abcdef").expect("key"); + let mut t = Trusty::open(&url, key).expect("open"); + for i in 0..EACH { + t.append(&entity, &json!({ "writer": w, "i": i }), "test://") + .expect("append"); + } + }) + }) + .collect(); + for h in handles { + h.join().expect("writer panicked"); + } + + let key = SecretKey::from_bytes(b"0123456789abcdef").expect("key"); + let mut t = Trusty::open(&url, key).expect("open"); + let bundle = t.export_bundle(entity, None, None).expect("export"); + let rows: Vec<_> = bundle + .rows + .iter() + .filter(|r| r.event_type != "BREAK") + .collect(); + + assert_eq!( + rows.len(), + WRITERS * EACH, + "every append must land exactly once" + ); + + // The real assertion: no two rows may claim the same predecessor. + // That is what a lost update looks like from the outside. + let mut prevs: Vec<&str> = rows.iter().map(|r| r.prev_hash.as_str()).collect(); + prevs.sort_unstable(); + let before = prevs.len(); + prevs.dedup(); + assert_eq!( + prevs.len(), + before, + "two rows claim the same prev_hash, so two writers read the same tip \ + and both built on it -- the chain forked, which is exactly what the \ + per-entity lock exists to prevent" + ); + + let report = t.verify(entity).expect("verify"); + assert!( + report.rows_valid, + "and the chain must still verify end to end: {report:?}" + ); +} diff --git a/crates/adapters/trusty-sqlite/Cargo.toml b/crates/adapters/trusty-sqlite/Cargo.toml new file mode 100644 index 0000000..fad63ba --- /dev/null +++ b/crates/adapters/trusty-sqlite/Cargo.toml @@ -0,0 +1,22 @@ +[package] +name = "trusty-sqlite" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "SQLite adapter for Trusty. Three-table schema (provenance_envelope, hash_index, chain_tips) with KV cross-check." + +[dependencies] +trusty-core = { workspace = true } +trusty-store = { workspace = true } +rusqlite = { workspace = true } +serde = { workspace = true } +serde_json = { workspace = true } +chrono = { workspace = true } +thiserror = { workspace = true } +hex = { workspace = true } + +[dev-dependencies] +trusty-store = { workspace = true, features = ["testing"] } diff --git a/crates/adapters/trusty-sqlite/examples/bench_append.rs b/crates/adapters/trusty-sqlite/examples/bench_append.rs new file mode 100644 index 0000000..d827aa8 --- /dev/null +++ b/crates/adapters/trusty-sqlite/examples/bench_append.rs @@ -0,0 +1,83 @@ +// ============================================================ +// EXAMPLE: bench_append +// PURPOSE: Measure fsync-amortization speedup of append_batch over +// a loop of append() calls on a file-backed SQLite DB. +// Intended for ad-hoc inspection, not CI regression +// tracking (no criterion / no gate). Prints rows/sec + +// speedup ratio. +// +// RUN: cargo run -p trusty-sqlite --release --example bench_append +// (release mode matters: debug rusqlite is much slower) +// ============================================================ + +use serde_json::json; +use std::time::Instant; +use trusty_core::SecretKey; +use trusty_sqlite::{RowSpec, Trusty}; + +const N: usize = 1000; + +fn tmp_db_path(tag: &str) -> std::path::PathBuf { + let mut p = std::env::temp_dir(); + let nanos = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_nanos()) + .unwrap_or(0); + p.push(format!("trusty_bench_{tag}_{nanos}.db")); + p +} + +fn key() -> SecretKey { + SecretKey::new(*b"0123456789abcdef") +} + +fn bench_single_row(path: &std::path::Path) -> f64 { + let mut t = Trusty::open(path, key()).expect("open"); + // Warmup: one genesis so we are measuring mutations, not first-row setup. + t.append("acme", &json!({"warmup": true}), "s") + .expect("warmup"); + let start = Instant::now(); + for i in 0..N { + t.append("acme", &json!({"i": i}), "s").expect("append ok"); + } + let elapsed = start.elapsed().as_secs_f64(); + N as f64 / elapsed +} + +fn bench_batch(path: &std::path::Path) -> f64 { + let mut t = Trusty::open(path, key()).expect("open"); + t.append("acme", &json!({"warmup": true}), "s") + .expect("warmup"); + let values: Vec = (0..N).map(|i| json!({"i": i})).collect(); + let specs: Vec = values + .iter() + .map(|v| RowSpec { + entity_id: "acme", + data: v, + source_url: "s", + }) + .collect(); + let start = Instant::now(); + t.append_batch(&specs).expect("batch ok"); + let elapsed = start.elapsed().as_secs_f64(); + N as f64 / elapsed +} + +fn main() { + let p1 = tmp_db_path("single"); + let p2 = tmp_db_path("batch"); + println!("trusty-sqlite append benchmark (N = {N}, file-backed, WAL + FULL sync)"); + println!("single-row db : {}", p1.display()); + println!("batch db : {}", p2.display()); + println!(); + + let single_rps = bench_single_row(&p1); + let batch_rps = bench_batch(&p2); + let speedup = batch_rps / single_rps; + + println!("single-row append : {single_rps:>10.1} rows/sec"); + println!("append_batch : {batch_rps:>10.1} rows/sec"); + println!("speedup : {speedup:>10.2}x"); + println!(); + println!("(cleanup: rm {} {})", p1.display(), p2.display()); +} diff --git a/crates/adapters/trusty-sqlite/examples/device_signer.rs b/crates/adapters/trusty-sqlite/examples/device_signer.rs new file mode 100644 index 0000000..db477dd --- /dev/null +++ b/crates/adapters/trusty-sqlite/examples/device_signer.rs @@ -0,0 +1,195 @@ +// ============================================================ +// EXAMPLE: device_signer +// PURPOSE: Walk the device-signer provisioning pattern end to end. +// Enroll -> sign -> rotate -> revoke. Demonstrates how an +// application keeps a signer_registry alongside trusty's +// chain tables, using a second rusqlite::Connection to +// the same file. +// +// See `docs/device-signer-provisioning.md` for the conceptual doc. +// +// RUN: cargo run -p trusty-sqlite --example device_signer +// ============================================================ + +use rusqlite::{params, Connection}; +use serde_json::{json, Value}; +use std::path::PathBuf; +use trusty_core::{Ed25519Signer, Ed25519Verifier, SecretKey, Signer, Verifier}; +use trusty_sqlite::Trusty; + +fn tmp_db_path() -> PathBuf { + let mut p = std::env::temp_dir(); + let nanos = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_nanos()) + .unwrap_or(0); + p.push(format!("trusty_device_signer_demo_{nanos}.db")); + p +} + +fn now_iso() -> String { + chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Secs, true) +} + +fn init_registry(conn: &Connection) { + conn.execute_batch( + "CREATE TABLE IF NOT EXISTS signer_registry ( + device_id TEXT PRIMARY KEY, + public_key TEXT NOT NULL, + enrolled_at TEXT NOT NULL, + rotated_at TEXT, + revoked_at TEXT, + note TEXT + ); + CREATE INDEX IF NOT EXISTS idx_sr_pk ON signer_registry(public_key);", + ) + .expect("registry schema"); +} + +fn enroll(conn: &Connection, device_id: &str, pk_hex: &str, note: &str) { + conn.execute( + "INSERT INTO signer_registry (device_id, public_key, enrolled_at, note) + VALUES (?1, ?2, ?3, ?4)", + params![device_id, pk_hex, now_iso(), note], + ) + .expect("enroll"); + println!(" [enrolled] device={device_id} pk={pk_hex:.16}... ({note})"); +} + +fn rotate(conn: &Connection, old_device_id: &str, new_device_id: &str, new_pk_hex: &str) { + conn.execute( + "UPDATE signer_registry SET rotated_at = ?1 WHERE device_id = ?2", + params![now_iso(), old_device_id], + ) + .expect("mark rotated"); + enroll(conn, new_device_id, new_pk_hex, "post-rotation"); + println!(" [rotated] {old_device_id} -> {new_device_id}"); +} + +fn revoke(conn: &Connection, device_id: &str, reason: &str) { + conn.execute( + "UPDATE signer_registry SET revoked_at = ?1, note = ?2 WHERE device_id = ?3", + params![now_iso(), format!("REVOKED: {reason}"), device_id], + ) + .expect("revoke"); + println!(" [revoked] device={device_id} reason={reason}"); +} + +fn trust_status_for_pk(conn: &Connection, pk_hex: &str) -> String { + conn.query_row( + "SELECT device_id, enrolled_at, rotated_at, revoked_at + FROM signer_registry WHERE public_key = ?1", + params![pk_hex], + |r| { + let dev: String = r.get(0)?; + let enrolled: String = r.get(1)?; + let rotated: Option = r.get(2)?; + let revoked: Option = r.get(3)?; + Ok(match (rotated, revoked) { + (_, Some(rev)) => format!("REVOKED({dev}, at={rev})"), + (Some(rot), _) => format!("ROTATED({dev}, at={rot}) -- old-key rows still verify"), + _ => format!("ACTIVE({dev}, since={enrolled})"), + }) + }, + ) + .unwrap_or_else(|_| "UNKNOWN (pk not in registry)".to_string()) +} + +fn sign_and_append(t: &mut Trusty, entity: &str, data: &Value, signer: &Ed25519Signer) { + let r = t + .append_signed(entity, data, "sensor://demo", signer) + .expect("signed append"); + println!( + " [appended] seq={} row_hash={:.16}... pk={:.16}...", + r.sequence, + r.row_hash, + hex::encode(signer.public_key_bytes()) + ); +} + +fn main() { + let path = tmp_db_path(); + println!("trusty device-signer provisioning demo"); + println!("db: {}\n", path.display()); + + // Trusty owns one Connection; the app opens a second one for the + // registry. WAL mode (set by trusty-sqlite's schema) allows this + // cleanly. + let mut t = Trusty::open(&path, SecretKey::new(*b"0123456789abcdef")).expect("open trusty"); + let app_conn = Connection::open(&path).expect("open app conn"); + init_registry(&app_conn); + + // --- 1. Enroll device_001 ------------------------------------- + println!("step 1: enroll device_001"); + let signer_v1 = Ed25519Signer::generate(); + let pk_v1_hex = hex::encode(signer_v1.public_key_bytes()); + enroll(&app_conn, "device_001", &pk_v1_hex, "initial enrollment"); + + // --- 2. Sign and append with v1 key --------------------------- + println!("\nstep 2: append two signed rows with v1 key"); + sign_and_append(&mut t, "sensor_a", &json!({"temp": 21.4}), &signer_v1); + sign_and_append(&mut t, "sensor_a", &json!({"temp": 21.5}), &signer_v1); + + // --- 3. Rotate: new key, new registry entry ------------------- + println!("\nstep 3: rotate device_001 -> device_001_v2"); + let signer_v2 = Ed25519Signer::generate(); + let pk_v2_hex = hex::encode(signer_v2.public_key_bytes()); + rotate(&app_conn, "device_001", "device_001_v2", &pk_v2_hex); + + // --- 4. Append more rows with v2 key -------------------------- + println!("\nstep 4: append two more signed rows with v2 key"); + sign_and_append(&mut t, "sensor_a", &json!({"temp": 21.7}), &signer_v2); + sign_and_append(&mut t, "sensor_a", &json!({"temp": 21.9}), &signer_v2); + + // --- 5. Revoke v1 (simulated compromise) ---------------------- + println!("\nstep 5: simulated compromise of v1 secret, revoke it"); + revoke(&app_conn, "device_001", "secret leaked from retired device"); + + // --- 6. Verify chain: math is still sound --------------------- + println!("\nstep 6: chain verify -- cryptography is unaffected by registry policy"); + let report = t.verify("sensor_a").expect("verify"); + println!( + " chain verify: ok={} row_count={} (revocation is app-policy, not chain-policy)", + report.rows_valid, report.row_count + ); + + // --- 7. Registry overlay: per-row trust report ---------------- + println!("\nstep 7: registry-overlay trust report for each chain row"); + let bundle = t.export_bundle("sensor_a", None, None).expect("bundle"); + for row in &bundle.rows { + if let Some(raw) = &row.raw_payload { + let parsed: Value = serde_json::from_str(raw).expect("payload json"); + let pk_hex = parsed + .get("signatures") + .and_then(|s| s.as_array()) + .and_then(|a| a.first()) + .and_then(|o| o.get("pk")) + .and_then(|v| v.as_str()); + match pk_hex { + Some(pk) => { + println!( + " seq={} event={} -> {}", + row.sequence, + row.event_type, + trust_status_for_pk(&app_conn, pk) + ); + } + None => { + println!( + " seq={} event={} -> (unsigned v1 row)", + row.sequence, row.event_type + ); + } + } + } + } + + // --- 8. Sanity: Ed25519Verifier round-trips on an isolated sig + println!("\nstep 8: sanity -- Ed25519Verifier on the v2 public key works end-to-end"); + let msg = b"independent verifier check"; + let sig = signer_v2.sign(msg); + Ed25519Verifier::verify(&signer_v2.public_key_bytes(), msg, &sig).expect("independent verify"); + println!(" standalone verify: ok"); + + println!("\ndone. (cleanup: rm {})", path.display()); +} diff --git a/crates/adapters/trusty-sqlite/examples/twin_coupling.rs b/crates/adapters/trusty-sqlite/examples/twin_coupling.rs new file mode 100644 index 0000000..ac98c5e --- /dev/null +++ b/crates/adapters/trusty-sqlite/examples/twin_coupling.rs @@ -0,0 +1,186 @@ +// ============================================================ +// EXAMPLE: twin_coupling +// PURPOSE: Demonstrate the data-convention pattern for linking a +// digital-twin computation row back to the sensor rows +// that fed it. Sensor chain and twin chain are independent +// trusty entities. The twin row carries a `source_refs` +// array in its `data` field so an external verifier can +// check the linkage from two ProofBundles alone -- no +// extra tables, no schema changes. +// +// See `docs/twin-coupling.md` for the conceptual doc. +// +// RUN: cargo run -p trusty-sqlite --example twin_coupling +// ============================================================ + +use serde_json::{json, Value}; +use std::path::PathBuf; +use trusty_core::SecretKey; +use trusty_sqlite::Trusty; + +fn tmp_db_path() -> PathBuf { + let mut p = std::env::temp_dir(); + let nanos = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_nanos()) + .unwrap_or(0); + p.push(format!("trusty_twin_coupling_demo_{nanos}.db")); + p +} + +fn main() { + let path = tmp_db_path(); + println!("trusty twin-coupling demo"); + println!("db: {}\n", path.display()); + + let sip_key = SecretKey::new(*b"0123456789abcdef"); + let mut t = Trusty::open(&path, sip_key.clone()).expect("open trusty"); + + // ---------------------------------------------------------------- + // step 1: append sensor readings to the sensor chain + // ---------------------------------------------------------------- + println!("step 1: append three sensor readings"); + + let r0 = t + .append("sensor-001", &json!({"temp_c": 21.1}), "iot://sensor-001") + .expect("append r0"); + println!( + " seq={} chain_id={:.16}... row_hash={}", + r0.sequence, r0.chain_id, r0.row_hash + ); + + let r1 = t + .append("sensor-001", &json!({"temp_c": 21.4}), "iot://sensor-001") + .expect("append r1"); + println!( + " seq={} chain_id={:.16}... row_hash={}", + r1.sequence, r1.chain_id, r1.row_hash + ); + + let r2 = t + .append("sensor-001", &json!({"temp_c": 21.6}), "iot://sensor-001") + .expect("append r2"); + println!( + " seq={} chain_id={:.16}... row_hash={}", + r2.sequence, r2.chain_id, r2.row_hash + ); + + // ---------------------------------------------------------------- + // step 2: twin computation cites the three sensor rows by + // chain_id + sequence + row_hash (data convention) + // ---------------------------------------------------------------- + println!("\nstep 2: append twin computation row with source_refs"); + + let avg = (21.1_f64 + 21.4 + 21.6) / 3.0; + let twin_data = json!({ + "computation": "rolling_avg_temp_c", + "result": (avg * 100.0).round() / 100.0, + "source_refs": [ + {"chain_id": r0.chain_id, "sequence": r0.sequence, "row_hash": r0.row_hash}, + {"chain_id": r1.chain_id, "sequence": r1.sequence, "row_hash": r1.row_hash}, + {"chain_id": r2.chain_id, "sequence": r2.sequence, "row_hash": r2.row_hash}, + ] + }); + + let tw = t + .append("twin-001", &twin_data, "omniverse://twin-001") + .expect("append twin"); + println!(" seq={} row_hash={}", tw.sequence, tw.row_hash); + + // ---------------------------------------------------------------- + // step 3: verify both chains independently + // ---------------------------------------------------------------- + println!("\nstep 3: verify both chains"); + + let sensor_report = t.verify("sensor-001").expect("verify sensor"); + println!( + " sensor-001: ok={} rows={}", + sensor_report.rows_valid, sensor_report.row_count + ); + assert!(sensor_report.rows_valid, "sensor chain verification failed"); + + let twin_report = t.verify("twin-001").expect("verify twin"); + println!( + " twin-001: ok={} rows={}", + twin_report.rows_valid, twin_report.row_count + ); + assert!(twin_report.rows_valid, "twin chain verification failed"); + + // ---------------------------------------------------------------- + // step 4: external-verifier cross-chain check + // Export both ProofBundles; walk twin source_refs and + // confirm each referenced row_hash is present and matches + // in the sensor bundle. + // ---------------------------------------------------------------- + println!("\nstep 4: cross-chain verify from ProofBundles"); + + let sensor_bundle = t + .export_bundle("sensor-001", None, None) + .expect("sensor bundle"); + let twin_bundle = t + .export_bundle("twin-001", None, None) + .expect("twin bundle"); + + let sensor_verify = sensor_bundle.verify(&sip_key); + let twin_verify = twin_bundle.verify(&sip_key); + println!( + " sensor bundle verify: ok={} rows={}", + sensor_verify.rows_valid, sensor_verify.row_count + ); + println!( + " twin bundle verify: ok={} rows={}", + twin_verify.rows_valid, twin_verify.row_count + ); + assert!(sensor_verify.rows_valid); + assert!(twin_verify.rows_valid); + + // Walk twin rows; for each source_ref, confirm the sensor bundle + // contains a row at that sequence with a matching row_hash. + let mut refs_checked = 0usize; + for twin_row in twin_bundle.rows.iter().filter(|r| r.sequence >= 0) { + let payload: Value = twin_row + .raw_payload + .as_ref() + .and_then(|p| serde_json::from_str(p).ok()) + .expect("twin row has no payload"); + + let data = payload.get("data").expect("payload has no data"); + let refs = match data.get("source_refs").and_then(|v| v.as_array()) { + Some(r) => r, + None => continue, + }; + + for sr in refs { + let ref_chain_id = sr["chain_id"].as_str().expect("chain_id"); + let ref_seq = sr["sequence"].as_i64().expect("sequence"); + let ref_hash = sr["row_hash"].as_str().expect("row_hash"); + + let matched = sensor_bundle.rows.iter().find(|r| { + r.chain_id == ref_chain_id + && r.sequence == ref_seq + && r.row_hash.as_deref() == Some(ref_hash) + }); + + match matched { + Some(m) => { + println!( + " source_ref ok: seq={} row_hash={} (sensor chain_id={:.16}...)", + m.sequence, ref_hash, ref_chain_id + ); + refs_checked += 1; + } + None => { + panic!( + "source_ref not found in sensor bundle: chain_id={} seq={} row_hash={}", + ref_chain_id, ref_seq, ref_hash + ); + } + } + } + } + + println!("\n {refs_checked} source_refs verified across ProofBundles"); + assert_eq!(refs_checked, 3, "expected 3 source_refs checked"); + + println!("\ndone. (cleanup: rm {})", path.display()); +} diff --git a/crates/adapters/trusty-sqlite/src/error.rs b/crates/adapters/trusty-sqlite/src/error.rs new file mode 100644 index 0000000..c1dec03 --- /dev/null +++ b/crates/adapters/trusty-sqlite/src/error.rs @@ -0,0 +1,30 @@ +use thiserror::Error; + +#[derive(Debug, Error)] +pub enum SqliteError { + #[error("sqlite error: {0}")] + Sqlite(#[from] rusqlite::Error), + #[error("trusty-core error: {0}")] + Core(#[from] trusty_core::TrustyError), + #[error("json error: {0}")] + Json(#[from] serde_json::Error), + #[error("chain tip missing row_hash for entity {0}")] + MissingTipHash(String), + #[error("malformed chain id: {0}")] + MalformedChainId(String), + #[error("not found: {0}")] + NotFound(String), + /// The producer enrolled a credential this path cannot check. + /// + /// Separate from a failed verification on purpose: a failure invites + /// "wrong key, try another", while this says the check itself is the + /// wrong one for that credential. + #[error("unsupported credential: {0}")] + UnsupportedCredential(String), + #[error( + "store is signed-only: an unsigned append to entity {0} was refused. Use append_signed / append_with_method_signed." + )] + UnsignedAppendRefused(String), +} + +pub type SqliteResult = Result; diff --git a/crates/adapters/trusty-sqlite/src/lib.rs b/crates/adapters/trusty-sqlite/src/lib.rs new file mode 100644 index 0000000..f8ba1e8 --- /dev/null +++ b/crates/adapters/trusty-sqlite/src/lib.rs @@ -0,0 +1,26 @@ +//! SQLite-backed Trusty adapter -- the production storage backend. +//! +//! Three-table schema (`provenance_envelope`, `hash_index`, `chain_tips`) with +//! KV cross-check. Entry point is [`Trusty`], which wraps a rusqlite connection +//! and exposes the full append / verify / query surface. +//! +//! # Key APIs +//! - [`Trusty::append`] / [`Trusty::append_batch`] -- write records (batch gives ~21x throughput) +//! - [`Trusty::verify`] -- check chain integrity for an entity +//! - [`Trusty::breaks_since`] -- poll chain-break events (tamper / gap alerts) +//! - [`Trusty::export_bundle`] -- package a chain slice as a [`ProofBundle`](trusty_core::ProofBundle) for external verification +//! - [`Trusty::append_signed`] -- v2 Ed25519-signed append + +#![deny(clippy::all)] + +pub mod error; +pub mod schema; +pub mod sqlite_store; +pub mod store; + +pub use error::{SqliteError, SqliteResult}; +pub use sqlite_store::SqliteStore; +pub use store::{ + AnchorInfo, AppendResult, BreakEvent, CheckpointRow, HistoryRow, IntegrityMismatch, + IntegrityReport, ProducerIdentity, RevocationRecord, RowSpec, SilentEntity, Trusty, +}; diff --git a/crates/adapters/trusty-sqlite/src/schema.rs b/crates/adapters/trusty-sqlite/src/schema.rs new file mode 100644 index 0000000..8b1e902 --- /dev/null +++ b/crates/adapters/trusty-sqlite/src/schema.rs @@ -0,0 +1,90 @@ +pub const SCHEMA_SQL: &str = r#" +PRAGMA journal_mode = WAL; +PRAGMA synchronous = FULL; +PRAGMA foreign_keys = ON; + +CREATE TABLE IF NOT EXISTS provenance_envelope ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + chain_id TEXT NOT NULL, + entity_id TEXT NOT NULL, + event_type TEXT NOT NULL, + field_name TEXT, + old_value TEXT, + new_value TEXT, + source_url TEXT NOT NULL DEFAULT '', + detected_at TEXT NOT NULL, + extraction_method TEXT NOT NULL DEFAULT 'trusty', + prev_hash TEXT NOT NULL, + raw_payload TEXT, + row_hash TEXT, + anchor_hash TEXT, + sequence_int INTEGER NOT NULL DEFAULT 0 +); + +CREATE INDEX IF NOT EXISTS idx_pe_entity ON provenance_envelope(entity_id); +CREATE INDEX IF NOT EXISTS idx_pe_chain ON provenance_envelope(chain_id); +CREATE INDEX IF NOT EXISTS idx_pe_entity_chain ON provenance_envelope(entity_id, chain_id); +CREATE INDEX IF NOT EXISTS idx_pe_row_hash ON provenance_envelope(row_hash); + +CREATE TABLE IF NOT EXISTS hash_index ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + row_hash TEXT NOT NULL UNIQUE, + chain_id TEXT NOT NULL, + sequence_int INTEGER NOT NULL, + entity_id TEXT NOT NULL, + recorded_at TEXT NOT NULL +); + +CREATE INDEX IF NOT EXISTS idx_hi_entity ON hash_index(entity_id); +CREATE INDEX IF NOT EXISTS idx_hi_chain ON hash_index(chain_id); + +CREATE TABLE IF NOT EXISTS chain_tips ( + entity_id TEXT PRIMARY KEY, + current_tip_hash TEXT, + chain_id TEXT, + updated_at TEXT NOT NULL +); + +CREATE INDEX IF NOT EXISTS idx_ct_chain ON chain_tips(chain_id); + +-- Federation primitives mirrored from trusty-postgres (S17 cycle 4). +-- BLOB replaces BYTEA; TEXT (RFC3339) replaces TIMESTAMPTZ; INTEGER +-- PK AUTOINCREMENT replaces BIGINT IDENTITY. Timestamps set by +-- application code via now_iso() (no DEFAULT CURRENT_TIMESTAMP -- +-- format would differ from RFC3339). + +CREATE TABLE IF NOT EXISTS producer_identity ( + producer_oid TEXT PRIMARY KEY, + ed25519_pubkey BLOB NOT NULL, + siphash_key BLOB NOT NULL, + registered_at TEXT NOT NULL, + display_name TEXT, + metadata TEXT, + -- See trusty-postgres: a COSE credential cannot be + -- checked as a bare signature, and this is what says which. + key_format TEXT NOT NULL DEFAULT 'ed25519-raw' + CHECK (key_format IN ('ed25519-raw','cose')) +); + +CREATE TABLE IF NOT EXISTS producer_revocation ( + producer_oid TEXT PRIMARY KEY, + revoked_at TEXT NOT NULL, + revoked_pubkey BLOB NOT NULL, + reason TEXT NOT NULL, + attested_by_pubkey BLOB NOT NULL, + revocation_row_hash TEXT NOT NULL +); + +CREATE TABLE IF NOT EXISTS chain_checkpoint ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + chain_id TEXT NOT NULL, + sequence_int INTEGER NOT NULL, + row_hash TEXT NOT NULL, + checkpoint_at TEXT NOT NULL, + attested_by TEXT NOT NULL, + attestation_sig BLOB NOT NULL, + UNIQUE (chain_id, sequence_int) +); + +CREATE INDEX IF NOT EXISTS idx_ckpt_chain ON chain_checkpoint(chain_id); +"#; diff --git a/crates/adapters/trusty-sqlite/src/sqlite_store.rs b/crates/adapters/trusty-sqlite/src/sqlite_store.rs new file mode 100644 index 0000000..ab120ba --- /dev/null +++ b/crates/adapters/trusty-sqlite/src/sqlite_store.rs @@ -0,0 +1,315 @@ +// ============================================================ +// MODULE: sqlite_store +// PURPOSE: Implements `trusty_store::Store` on top of the existing +// three-table schema used by the `Trusty` struct. Exposed +// so any code that wants a backend-agnostic handle can use +// it; `Trusty` itself still holds its own Connection for +// now and will be genericized in a follow-up. +// ============================================================ + +use rusqlite::{params, Connection, OptionalExtension}; +use std::path::Path; + +use trusty_store::{ + KvCrossCheckReport, LastRow, RowInsert, Store, StoreError, StoreResult, StoredRow, TipRow, + TipUpsert, +}; + +use crate::error::{SqliteError, SqliteResult}; +use crate::schema; + +pub struct SqliteStore { + conn: Connection, +} + +impl SqliteStore { + pub fn open>(path: P) -> SqliteResult { + let conn = Connection::open(path)?; + conn.execute_batch(schema::SCHEMA_SQL)?; + Ok(Self { conn }) + } + + pub fn open_in_memory() -> SqliteResult { + let conn = Connection::open_in_memory()?; + conn.execute_batch(schema::SCHEMA_SQL)?; + Ok(Self { conn }) + } + + pub fn conn(&self) -> &Connection { + &self.conn + } + + pub fn conn_mut(&mut self) -> &mut Connection { + &mut self.conn + } +} + +fn backend(e: E) -> StoreError { + StoreError::Backend(e.to_string()) +} + +impl Store for SqliteStore { + fn last_row(&self, entity_id: &str) -> StoreResult> { + let row = self + .conn + .query_row( + "SELECT row_hash, sequence_int FROM provenance_envelope + WHERE entity_id = ? AND event_type != 'BREAK' AND row_hash IS NOT NULL + ORDER BY id DESC LIMIT 1", + params![entity_id], + |r| { + Ok(LastRow { + row_hash: r.get(0)?, + sequence: r.get(1)?, + }) + }, + ) + .optional() + .map_err(backend)?; + Ok(row) + } + + fn insert_row(&mut self, row: &RowInsert) -> StoreResult { + self.conn + .execute( + "INSERT INTO provenance_envelope ( + chain_id, entity_id, event_type, + field_name, old_value, new_value, + source_url, detected_at, extraction_method, + prev_hash, row_hash, anchor_hash, sequence_int, raw_payload + ) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)", + params![ + row.chain_id, + row.entity_id, + row.event_type, + Option::<&str>::None, + Option::<&str>::None, + Option::<&str>::None, + row.source_url, + row.detected_at, + row.extraction_method, + row.prev_hash, + row.row_hash, + row.anchor_hash, + row.sequence, + row.raw_payload, + ], + ) + .map_err(backend)?; + let id = self.conn.last_insert_rowid(); + + if let Some(hash) = &row.row_hash { + self.conn + .execute( + "INSERT OR IGNORE INTO hash_index + (row_hash, chain_id, sequence_int, entity_id, recorded_at) + VALUES (?, ?, ?, ?, ?)", + params![ + hash, + row.chain_id, + row.sequence, + row.entity_id, + row.received_at, + ], + ) + .map_err(backend)?; + } + + Ok(id) + } + + fn iter_chain<'a>( + &'a self, + entity_id: &str, + ) -> StoreResult> + 'a>> { + let mut stmt = self + .conn + .prepare( + "SELECT id, entity_id, chain_id, event_type, sequence_int, prev_hash, + row_hash, anchor_hash, raw_payload, detected_at, source_url, + extraction_method + FROM provenance_envelope + WHERE entity_id = ? + ORDER BY id ASC", + ) + .map_err(backend)?; + let rows: Vec> = stmt + .query_map(params![entity_id], |r| { + Ok(StoredRow { + id: r.get(0)?, + entity_id: r.get(1)?, + chain_id: r.get(2)?, + event_type: r.get(3)?, + sequence: r.get(4)?, + prev_hash: r.get(5)?, + row_hash: r.get(6)?, + anchor_hash: r.get(7)?, + raw_payload: r.get(8)?, + detected_at: r.get(9)?, + received_at: String::new(), + source_url: r.get(10)?, + extraction_method: r.get(11)?, + }) + }) + .map_err(backend)? + .map(|r| r.map_err(backend)) + .collect(); + Ok(Box::new(rows.into_iter())) + } + + fn get_tip(&self, chain_id: &str) -> StoreResult> { + let row = self + .conn + .query_row( + "SELECT entity_id, chain_id, current_tip_hash, updated_at + FROM chain_tips WHERE chain_id = ?", + params![chain_id], + |r| { + Ok(TipRow { + entity_id: r.get(0)?, + chain_id: r.get::<_, Option>(1)?.unwrap_or_default(), + current_tip_hash: r.get(2)?, + updated_at: r.get(3)?, + }) + }, + ) + .optional() + .map_err(backend)?; + Ok(row) + } + + fn get_tip_for_entity(&self, entity_id: &str) -> StoreResult> { + let row = self + .conn + .query_row( + "SELECT entity_id, chain_id, current_tip_hash, updated_at + FROM chain_tips WHERE entity_id = ?", + params![entity_id], + |r| { + Ok(TipRow { + entity_id: r.get(0)?, + chain_id: r.get::<_, Option>(1)?.unwrap_or_default(), + current_tip_hash: r.get(2)?, + updated_at: r.get(3)?, + }) + }, + ) + .optional() + .map_err(backend)?; + Ok(row) + } + + fn set_tip(&mut self, tip: &TipUpsert) -> StoreResult<()> { + let updated_at = chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Micros, true); + self.conn + .execute( + "INSERT INTO chain_tips (entity_id, current_tip_hash, chain_id, updated_at) + VALUES (?, ?, ?, ?) + ON CONFLICT(entity_id) DO UPDATE SET + current_tip_hash = excluded.current_tip_hash, + chain_id = excluded.chain_id, + updated_at = excluded.updated_at", + params![ + tip.entity_id, + tip.current_tip_hash, + tip.chain_id, + updated_at, + ], + ) + .map_err(backend)?; + Ok(()) + } + + fn kv_cross_check(&self) -> StoreResult { + let kv1_total_hashes: i64 = self + .conn + .query_row("SELECT COUNT(*) FROM hash_index", [], |r| r.get(0)) + .map_err(backend)?; + + let mut tips_stmt = self + .conn + .prepare("SELECT entity_id, current_tip_hash, chain_id FROM chain_tips") + .map_err(backend)?; + let tip_rows: Vec<(String, Option, Option)> = tips_stmt + .query_map([], |r| { + Ok(( + r.get::<_, String>(0)?, + r.get::<_, Option>(1)?, + r.get::<_, Option>(2)?, + )) + }) + .map_err(backend)? + .collect::, _>>() + .map_err(backend)?; + + let mut mismatches: Vec = Vec::new(); + let mut entities_checked: usize = 0; + let kv2_total_entities = tip_rows.len(); + + for (entity_id, tip_hash, chain_id) in tip_rows { + let tip_hash = match tip_hash { + Some(h) => h, + None => continue, + }; + let located: Option = self + .conn + .query_row( + "SELECT chain_id FROM hash_index WHERE row_hash = ?", + params![tip_hash], + |r| r.get(0), + ) + .optional() + .map_err(backend)?; + match located { + None => mismatches.push(format!( + "tip_hash_not_in_index: entity={} hash={}", + entity_id, tip_hash + )), + Some(idx_chain) => { + if Some(idx_chain.as_str()) != chain_id.as_deref() { + mismatches.push(format!( + "chain_id_mismatch: entity={} expected={:?} got={}", + entity_id, chain_id, idx_chain + )); + } + } + } + entities_checked += 1; + } + + Ok(KvCrossCheckReport { + ok: mismatches.is_empty(), + kv1_total_hashes: kv1_total_hashes as usize, + kv2_total_entities, + entities_checked, + mismatches, + }) + } + + fn with_txn( + &mut self, + f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>, + ) -> StoreResult<()> { + self.conn + .execute_batch("BEGIN IMMEDIATE") + .map_err(backend)?; + match f(self) { + Ok(()) => { + self.conn.execute_batch("COMMIT").map_err(backend)?; + Ok(()) + } + Err(e) => { + let _ = self.conn.execute_batch("ROLLBACK"); + Err(e) + } + } + } +} + +// Allow callers that already hold a SqliteError-returning function +// to map into StoreError trivially. +impl From for StoreError { + fn from(e: SqliteError) -> Self { + StoreError::Backend(e.to_string()) + } +} diff --git a/crates/adapters/trusty-sqlite/src/store.rs b/crates/adapters/trusty-sqlite/src/store.rs new file mode 100644 index 0000000..0f82e75 --- /dev/null +++ b/crates/adapters/trusty-sqlite/src/store.rs @@ -0,0 +1,2363 @@ +use chrono::SecondsFormat; +use rusqlite::{params, Connection, OptionalExtension}; +use std::path::Path; + +use trusty_core::{ + checkpoint_signing_bytes, compute_record, compute_record_signed, default_conditions, evaluate, + make_chain_id, parse_generation as core_parse_generation, verify_links, verify_links_from, + AnchorEntry, BundleRow, ChainLink, Condition, ConditionContext, ConditionOutcome, + Ed25519Verifier, EventType, PayloadFields, ProofBundle, SecretKey, Signer, Verifier, + VerifyError, VerifyErrorType, VerifyReport, GENESIS_SENTINEL, +}; + +// Higher-level result types live in trusty-store (S16, promoted from +// this crate so trusty-postgres and any future backend reuse them). +// Re-exported here so existing trusty-sqlite consumers keep working. +pub use trusty_store::{ + AnchorInfo, AppendResult, BreakEvent, CheckpointRow, HistoryRow, IntegrityMismatch, + IntegrityReport, ProducerIdentity, RevocationRecord, RowSpec, SilentEntity, +}; + +use crate::error::{SqliteError, SqliteResult}; +use crate::schema; + +pub struct Trusty { + conn: Connection, + key: SecretKey, + conditions: Vec>, + /// Refuse appends that carry no signature. + /// + /// An unsigned row records content without recording WHO wrote it. Its + /// hashes still verify, so it looks fine to anyone reading the chain -- but + /// nothing attributes it, and an attacker able to write to the store can + /// produce one with perfectly correct hashes. Integrity without + /// authenticity is close to worthless against a hostile writer. + /// + /// Off by default so existing callers are unaffected; turn it on for any + /// chain whose rows are evidence. + signed_only: bool, +} + +struct LastRow { + row_hash: String, + sequence_int: i64, +} + +struct TipRow { + chain_id: String, + current_tip_hash: Option, +} + +impl Trusty { + pub fn open(path: impl AsRef, key: SecretKey) -> SqliteResult { + let conn = Connection::open(path)?; + conn.execute_batch(schema::SCHEMA_SQL)?; + Ok(Self { + conn, + key, + conditions: default_conditions(), + signed_only: false, + }) + } + + pub fn open_with_conditions( + path: impl AsRef, + key: SecretKey, + conditions: Vec>, + ) -> SqliteResult { + let conn = Connection::open(path)?; + conn.execute_batch(schema::SCHEMA_SQL)?; + Ok(Self { + conn, + key, + conditions, + signed_only: false, + }) + } + + pub fn open_in_memory(key: SecretKey) -> SqliteResult { + let conn = Connection::open_in_memory()?; + conn.execute_batch(schema::SCHEMA_SQL)?; + Ok(Self { + conn, + key, + conditions: default_conditions(), + signed_only: false, + }) + } + + /// In-memory store with custom conditions (the integrity defaults PLUS any + /// runtime monitors). The in-memory twin of `open_with_conditions`, so a + /// monitor can be exercised without a file. + pub fn open_in_memory_with_conditions( + key: SecretKey, + conditions: Vec>, + ) -> SqliteResult { + let conn = Connection::open_in_memory()?; + conn.execute_batch(schema::SCHEMA_SQL)?; + Ok(Self { + conn, + key, + conditions, + signed_only: false, + }) + } + + /// Refuse unsigned appends from now on. + /// + /// Existing unsigned rows are untouched -- rewriting history is exactly + /// what this crate exists to prevent. `PublicVerifyReport::fully_signed` + /// reports whether a chain already contains any. + pub fn set_signed_only(&mut self, signed_only: bool) { + self.signed_only = signed_only; + } + + /// Whether unsigned appends are currently refused. + pub fn is_signed_only(&self) -> bool { + self.signed_only + } + + pub fn append( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + ) -> SqliteResult { + self.append_with_method(entity_id, data, source_url, "trusty") + } + + /// v2-payload append: attach an Ed25519 signature (or any `Signer` + /// impl). Row content and chain semantics are otherwise identical + /// to `append`. Defaults `extraction_method` to "trusty". + pub fn append_signed( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + signer: &dyn Signer, + ) -> SqliteResult { + self.append_with_method_inner(entity_id, data, source_url, "trusty", Some(signer)) + } + + pub fn append_with_method( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + ) -> SqliteResult { + self.append_with_method_inner(entity_id, data, source_url, extraction_method, None) + } + + /// v2-payload variant of `append_with_method`. See `append_signed`. + pub fn append_with_method_signed( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + signer: &dyn Signer, + ) -> SqliteResult { + self.append_with_method_inner(entity_id, data, source_url, extraction_method, Some(signer)) + } + + fn append_with_method_inner( + &mut self, + entity_id: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + signer: Option<&dyn Signer>, + ) -> SqliteResult { + // Single choke point: every append path -- genesis, mutation, and the + // method-carrying variants -- funnels through here, so the check + // cannot be bypassed by picking a different entry point. + if self.signed_only && signer.is_none() { + return Err(SqliteError::UnsignedAppendRefused(entity_id.to_string())); + } + + let detected_at = now_iso(); + let received_at = now_iso_micros(); + let tip = self.read_tip(entity_id)?; + + let tip = match tip { + None => { + return self.create_genesis( + entity_id, + 1, + data, + source_url, + extraction_method, + &detected_at, + &received_at, + None, + None, + signer, + ); + } + Some(t) => t, + }; + + let chain_id = tip.chain_id.clone(); + let last = self.read_last(entity_id, &chain_id)?; + let (sequence, last_sequence, computed_hash) = match &last { + Some(l) => ( + l.sequence_int + 1, + Some(l.sequence_int), + Some(l.row_hash.clone()), + ), + None => (0, None, None), + }; + + let ctx = ConditionContext { + entity_id: entity_id.to_string(), + chain_id: chain_id.clone(), + sequence, + stored_tip_hash: tip.current_tip_hash.clone(), + computed_hash: computed_hash.clone(), + prev_hash: tip.current_tip_hash.clone(), + last_sequence, + source_url: source_url.to_string(), + extraction_method: extraction_method.to_string(), + // The condition can now read the measured payload (gap B). + data: data.clone(), + }; + + match evaluate(&self.conditions, &ctx) { + ConditionOutcome::Pass => { + let prev_hash = last + .as_ref() + .map(|l| l.row_hash.clone()) + .unwrap_or_else(|| GENESIS_SENTINEL.to_string()); + self.write_mutation( + entity_id, + &chain_id, + data, + source_url, + extraction_method, + &detected_at, + &received_at, + &prev_hash, + sequence, + signer, + ) + } + ConditionOutcome::Fail(reason) => { + let last_valid_hash = last + .as_ref() + .map(|l| l.row_hash.clone()) + .unwrap_or_else(|| GENESIS_SENTINEL.to_string()); + self.handle_break( + entity_id, + &chain_id, + &reason_code(&reason), + &last_valid_hash, + data, + source_url, + extraction_method, + &detected_at, + &received_at, + signer, + ) + } + } + } + + #[allow(clippy::too_many_arguments)] + fn create_genesis( + &mut self, + entity_id: &str, + generation: u32, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + detected_at: &str, + received_at: &str, + sealed_chain: Option, + break_reason: Option, + signer: Option<&dyn Signer>, + ) -> SqliteResult { + let chain_id = make_chain_id(entity_id, generation); + let prev_hash = GENESIS_SENTINEL.to_string(); + let sequence: i64 = 0; + + let fields = PayloadFields::new_v1( + entity_id, + chain_id.clone(), + data.clone(), + source_url, + detected_at, + received_at, + extraction_method, + prev_hash.clone(), + sequence, + ); + let computed = match signer { + Some(s) => compute_record_signed(&fields, &self.key, s)?, + None => compute_record(&fields, &self.key)?, + }; + + self.write_envelope( + &chain_id, + entity_id, + EventType::Genesis.as_str(), + None, + None, + None, + source_url, + detected_at, + extraction_method, + &prev_hash, + Some(&computed.row_hash), + Some(&computed.anchor_hash), + sequence, + Some(&computed.raw_payload), + )?; + self.append_hash_index(&computed.row_hash, &chain_id, sequence, entity_id)?; + self.set_tip(entity_id, &chain_id, Some(&computed.row_hash))?; + + let chain_break = sealed_chain.is_some(); + Ok(AppendResult { + entity_id: entity_id.to_string(), + chain_id, + event_type: EventType::Genesis.as_str().to_string(), + chain_break, + break_reason, + sealed_chain, + row_hash: computed.row_hash, + anchor_hash: computed.anchor_hash, + sequence, + is_genesis: !chain_break, + }) + } + + #[allow(clippy::too_many_arguments)] + fn write_mutation( + &mut self, + entity_id: &str, + chain_id: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + detected_at: &str, + received_at: &str, + prev_hash: &str, + sequence: i64, + signer: Option<&dyn Signer>, + ) -> SqliteResult { + let fields = PayloadFields::new_v1( + entity_id, + chain_id, + data.clone(), + source_url, + detected_at, + received_at, + extraction_method, + prev_hash, + sequence, + ); + let computed = match signer { + Some(s) => compute_record_signed(&fields, &self.key, s)?, + None => compute_record(&fields, &self.key)?, + }; + + self.write_envelope( + chain_id, + entity_id, + EventType::Mutation.as_str(), + None, + None, + None, + source_url, + detected_at, + extraction_method, + prev_hash, + Some(&computed.row_hash), + Some(&computed.anchor_hash), + sequence, + Some(&computed.raw_payload), + )?; + self.append_hash_index(&computed.row_hash, chain_id, sequence, entity_id)?; + self.set_tip(entity_id, chain_id, Some(&computed.row_hash))?; + + Ok(AppendResult { + entity_id: entity_id.to_string(), + chain_id: chain_id.to_string(), + event_type: EventType::Mutation.as_str().to_string(), + chain_break: false, + break_reason: None, + sealed_chain: None, + row_hash: computed.row_hash, + anchor_hash: computed.anchor_hash, + sequence, + is_genesis: false, + }) + } + + #[allow(clippy::too_many_arguments)] + fn handle_break( + &mut self, + entity_id: &str, + old_chain_id: &str, + break_reason: &str, + last_valid_hash: &str, + data: &serde_json::Value, + source_url: &str, + extraction_method: &str, + detected_at: &str, + received_at: &str, + signer: Option<&dyn Signer>, + ) -> SqliteResult { + self.write_envelope( + old_chain_id, + entity_id, + EventType::Break.as_str(), + Some("__chain_break__"), + Some(last_valid_hash), + Some(break_reason), + source_url, + detected_at, + extraction_method, + last_valid_hash, + None, + None, + -1, + None, + )?; + + let current_gen = parse_generation(old_chain_id)?; + let next_gen = current_gen + 1; + + let mut result = self.create_genesis( + entity_id, + next_gen, + data, + source_url, + extraction_method, + detected_at, + received_at, + Some(old_chain_id.to_string()), + Some(break_reason.to_string()), + signer, + )?; + result.event_type = "BREAK".to_string(); + result.is_genesis = false; + Ok(result) + } + + pub fn verify(&self, entity_id: &str) -> SqliteResult { + let mut stmt = self.conn.prepare( + "SELECT chain_id, event_type, sequence_int, prev_hash, row_hash, anchor_hash, raw_payload + FROM provenance_envelope + WHERE entity_id = ? + ORDER BY id ASC", + )?; + let rows = stmt.query_map(params![entity_id], |r| { + let chain_id: String = r.get(0)?; + let event_type: String = r.get(1)?; + let sequence: i64 = r.get(2)?; + let prev_hash: String = r.get(3)?; + let row_hash: Option = r.get(4)?; + let anchor_hash: Option = r.get(5)?; + let raw_payload: Option = r.get(6)?; + Ok(ChainLink { + entity_id: entity_id.to_string(), + chain_id, + event_type, + sequence, + prev_hash, + row_hash, + anchor_hash, + raw_payload, + }) + })?; + + let mut links: Vec = Vec::new(); + for link in rows { + links.push(link?); + } + + Ok(verify_links(entity_id, &links, &self.key)) + } + + pub fn anchor(&self, entity_id: &str) -> SqliteResult> { + let row: Option<(String, Option)> = self + .conn + .query_row( + "SELECT chain_id, current_tip_hash FROM chain_tips WHERE entity_id = ?", + params![entity_id], + |r| Ok((r.get(0)?, r.get(1)?)), + ) + .optional()?; + + Ok(row.map(|(chain_id, tip_hash)| AnchorInfo { + entity_id: entity_id.to_string(), + chain_id, + tip_hash, + anchored_at: now_iso(), + })) + } + + /// Append many rows inside a single transaction. Amortizes fsync + /// (WAL sync) cost across the whole batch; expected large + /// speedup on file-backed databases over a loop of single + /// `append` calls. Semantics match `append` per-row: a chain + /// break during the batch returns `Ok(AppendResult { + /// chain_break: true, .. })` for that row and continues -- it is + /// not a batch-level error. Only genuine errors (SQL failure, + /// crypto failure) roll the whole batch back. + pub fn append_batch(&mut self, specs: &[RowSpec]) -> SqliteResult> { + if specs.is_empty() { + return Ok(Vec::new()); + } + self.conn.execute_batch("BEGIN IMMEDIATE")?; + let mut out = Vec::with_capacity(specs.len()); + for spec in specs { + match self.append_with_method(spec.entity_id, spec.data, spec.source_url, "trusty") { + Ok(r) => out.push(r), + Err(e) => { + let _ = self.conn.execute_batch("ROLLBACK"); + return Err(e); + } + } + } + self.conn.execute_batch("COMMIT")?; + Ok(out) + } + + /// Return chain-break events for `entity_id` whose `detected_at` + /// is strictly greater than `since` (or all of them when `since` + /// is `None`). Events are in insertion order (id ASC). Pairs + /// with `anchors_since` for a two-feed model: anchors are the + /// happy-path commitments, breaks are the structural incidents. + /// Apps building alerting / incident dashboards over trusty + /// should subscribe to both. + pub fn breaks_since( + &self, + entity_id: &str, + since: Option<&str>, + ) -> SqliteResult> { + let mut stmt = self.conn.prepare( + "SELECT chain_id, old_value, new_value, detected_at + FROM provenance_envelope + WHERE entity_id = ?1 + AND event_type = 'BREAK' + AND (?2 IS NULL OR detected_at > ?2) + ORDER BY id ASC", + )?; + let rows = stmt.query_map(params![entity_id, since], |r| { + Ok(BreakEvent { + entity_id: entity_id.to_string(), + chain_id: r.get(0)?, + sealed_at_hash: r.get::<_, Option>(1)?.unwrap_or_default(), + break_reason: r.get::<_, Option>(2)?.unwrap_or_default(), + detected_at: r.get(3)?, + }) + })?; + let mut out = Vec::new(); + for row in rows { + out.push(row?); + } + Ok(out) + } + + /// Poll for entities that have gone silent. Returns every entity + /// whose most recent non-BREAK row was appended more than + /// `max_gap_secs` seconds ago. Pass `entity_id = Some(id)` to + /// check a single agent; pass `None` to sweep all known entities. + /// + /// Complement to `breaks_since`: breaks fire on tamper, liveness + /// fires on silence. Together they cover the full agent-health surface. + pub fn liveness_check( + &self, + entity_id: Option<&str>, + max_gap_secs: i64, + ) -> SqliteResult> { + let mut stmt = self.conn.prepare( + "SELECT entity_id, + MAX(detected_at) AS last_seen, + CAST((julianday('now') - julianday(MAX(detected_at))) * 86400 AS REAL) AS gap_secs + FROM provenance_envelope + WHERE event_type != 'BREAK' + AND (?1 IS NULL OR entity_id = ?1) + GROUP BY entity_id + HAVING gap_secs > ?2 + ORDER BY gap_secs DESC", + )?; + let rows = stmt.query_map(params![entity_id, max_gap_secs], |r| { + Ok(SilentEntity { + entity_id: r.get(0)?, + last_seen: r.get(1)?, + gap_secs: r.get(2)?, + }) + })?; + let mut out = Vec::new(); + for row in rows { + out.push(row?); + } + Ok(out) + } + + /// Return every per-row anchor commitment for `entity_id` whose + /// `detected_at` is strictly greater than `since` (or all of them + /// when `since` is `None`). Rows are returned in insertion order + /// (id ASC), which matches chain order for the rows that carry an + /// anchor_hash. BREAK rows (no row_hash, no anchor_hash) are + /// excluded; they are chain structural events, not commitments. + /// Intended use: periodic external publication of anchor hashes + /// so a third party can later spot-check a ProofBundle without + /// the database. + pub fn anchors_since( + &self, + entity_id: &str, + since: Option<&str>, + ) -> SqliteResult> { + let mut stmt = self.conn.prepare( + "SELECT sequence_int, anchor_hash, detected_at + FROM provenance_envelope + WHERE entity_id = ?1 + AND row_hash IS NOT NULL + AND anchor_hash IS NOT NULL + AND (?2 IS NULL OR detected_at > ?2) + ORDER BY id ASC", + )?; + let rows = stmt.query_map(params![entity_id, since], |r| { + let anchor_hash: String = r.get(1)?; + Ok(AnchorEntry { + sequence: r.get(0)?, + anchor_hash, + detected_at: r.get(2)?, + }) + })?; + let mut out = Vec::new(); + for row in rows { + out.push(row?); + } + Ok(out) + } + + /// Package a slice of the chain into a self-contained + /// `ProofBundle`. Non-BREAK rows are filtered by `sequence` in + /// `[since, until]` (inclusive; unbounded on either side when + /// `None`). BREAK rows for the entity are always included so + /// chain-break-as-event proof survives export. Anchors for the + /// same slice are attached. The resulting bundle can be handed + /// to `ProofBundle::verify(key)` without database access. + pub fn export_bundle( + &self, + entity_id: &str, + since: Option, + until: Option, + ) -> SqliteResult { + let lo = since.unwrap_or(i64::MIN); + let hi = until.unwrap_or(i64::MAX); + + let mut stmt = self.conn.prepare( + "SELECT chain_id, event_type, sequence_int, prev_hash, + row_hash, anchor_hash, raw_payload, detected_at, new_value + FROM provenance_envelope + WHERE entity_id = ?1 + AND ( + (event_type != 'BREAK' AND sequence_int >= ?2 AND sequence_int <= ?3) + OR event_type = 'BREAK' + ) + ORDER BY id ASC", + )?; + let mapped = stmt.query_map(params![entity_id, lo, hi], |r| { + let chain_id: String = r.get(0)?; + let event_type: String = r.get(1)?; + let sequence: i64 = r.get(2)?; + let prev_hash: String = r.get(3)?; + let row_hash: Option = r.get(4)?; + let anchor_hash: Option = r.get(5)?; + let raw_payload: Option = r.get(6)?; + let detected_at: Option = r.get(7)?; + // A BREAK row keeps its reason in new_value. Carrying it out + // is what lets a remote party learn WHICH rule broke rather + // than only that something did. + let break_reason: Option = if event_type == "BREAK" { + r.get(8)? + } else { + None + }; + Ok(( + BundleRow::new( + entity_id, + chain_id, + event_type.clone(), + sequence, + prev_hash, + ) + .with_hashes(row_hash.clone(), anchor_hash.clone()) + .with_payload(raw_payload) + .with_break_reason(break_reason), + event_type, + sequence, + anchor_hash, + detected_at.unwrap_or_default(), + )) + })?; + + let mut rows: Vec = Vec::new(); + let mut anchors: Vec = Vec::new(); + for m in mapped { + let (row, event_type, seq, anchor_hash, detected_at) = m?; + if event_type != "BREAK" { + if let Some(ah) = anchor_hash { + anchors.push(AnchorEntry { + sequence: seq, + anchor_hash: ah, + detected_at, + }); + } + } + rows.push(row); + } + + Ok(ProofBundle::new( + entity_id.to_string(), + now_iso(), + rows, + anchors, + )) + } + + pub fn history(&self, entity_id: &str, limit: i64) -> SqliteResult> { + let mut stmt = self.conn.prepare( + "SELECT id, chain_id, entity_id, event_type, sequence_int, detected_at, source_url, + row_hash, anchor_hash, raw_payload + FROM provenance_envelope + WHERE entity_id = ? AND event_type != 'BREAK' AND row_hash IS NOT NULL + ORDER BY id ASC LIMIT ?", + )?; + let rows = stmt.query_map(params![entity_id, limit], |r| { + Ok(HistoryRow { + id: r.get(0)?, + chain_id: r.get(1)?, + entity_id: r.get(2)?, + event_type: r.get(3)?, + sequence: r.get(4)?, + detected_at: r.get(5)?, + source_url: r.get(6)?, + row_hash: r.get(7)?, + anchor_hash: r.get(8)?, + raw_payload: r.get(9)?, + }) + })?; + let mut out = Vec::new(); + for row in rows { + out.push(row?); + } + Ok(out) + } + + pub fn integrity(&self) -> SqliteResult { + let kv1_total_hashes: i64 = + self.conn + .query_row("SELECT COUNT(*) FROM hash_index", [], |r| r.get(0))?; + + let mut tips_stmt = self + .conn + .prepare("SELECT entity_id, current_tip_hash, chain_id FROM chain_tips")?; + let tip_rows = tips_stmt.query_map([], |r| { + Ok(( + r.get::<_, String>(0)?, + r.get::<_, Option>(1)?, + r.get::<_, Option>(2)?, + )) + })?; + + let mut mismatches: Vec = Vec::new(); + let mut entities_checked: usize = 0; + let mut kv2_total_entities: usize = 0; + + for row in tip_rows { + let (entity_id, tip_hash, chain_id) = row?; + kv2_total_entities += 1; + let tip_hash = match tip_hash { + Some(h) => h, + None => continue, + }; + + let located: Option = self + .conn + .query_row( + "SELECT chain_id FROM hash_index WHERE row_hash = ?", + params![tip_hash], + |r| r.get(0), + ) + .optional()?; + + match located { + None => mismatches.push(IntegrityMismatch { + entity_id: entity_id.clone(), + reason: "tip_hash_not_in_index".to_string(), + detail: Some(tip_hash.clone()), + }), + Some(idx_chain) => { + if Some(idx_chain.as_str()) != chain_id.as_deref() { + mismatches.push(IntegrityMismatch { + entity_id: entity_id.clone(), + reason: "chain_id_mismatch".to_string(), + detail: Some(idx_chain), + }); + } + } + } + entities_checked += 1; + } + + Ok(IntegrityReport { + ok: mismatches.is_empty(), + entities_checked, + kv1_total_hashes: kv1_total_hashes as usize, + kv2_total_entities, + mismatches, + }) + } + + // ---------------------------------------------------------------- + // Federation primitives (S17 cycle 4) -- mirror of trusty-postgres. + // producer_identity (OID -> ed25519 pubkey + siphash key), + // producer_revocation (forward-only, CT stance), + // chain_checkpoint (verification-layer compaction primitive). + // Implementations + auth-gating semantics match trusty-postgres + // verbatim; SQL dialect is the only difference. Tested in the + // same #[cfg(test)] mod below. + // ---------------------------------------------------------------- + + pub fn register_producer( + &mut self, + producer_oid: &str, + ed25519_pubkey: &[u8], + siphash_key: &[u8], + display_name: Option<&str>, + ) -> SqliteResult<()> { + self.register_producer_with_format( + producer_oid, + ed25519_pubkey, + siphash_key, + display_name, + trusty_core::KeyFormat::Ed25519Raw, + ) + } + + /// Register a producer, declaring how its key material is encoded. + /// + /// A user enrolling through WebAuthn registers a COSE credential and + /// must say so; verification then refuses the bare-signature path + /// for that producer rather than failing in a way that reads like a + /// wrong key. Mirrors `trusty-postgres` -- a guarantee that holds on + /// one backend and not the other is worse than one that holds on + /// neither, because it works in testing. + pub fn register_producer_with_format( + &mut self, + producer_oid: &str, + ed25519_pubkey: &[u8], + siphash_key: &[u8], + display_name: Option<&str>, + key_format: trusty_core::KeyFormat, + ) -> SqliteResult<()> { + let registered_at = now_iso(); + self.conn.execute( + "INSERT INTO producer_identity + (producer_oid, ed25519_pubkey, siphash_key, registered_at, + display_name, key_format) + VALUES (?, ?, ?, ?, ?, ?) + ON CONFLICT(producer_oid) DO UPDATE SET + ed25519_pubkey = excluded.ed25519_pubkey, + siphash_key = excluded.siphash_key, + display_name = excluded.display_name, + key_format = excluded.key_format", + params![ + producer_oid, + ed25519_pubkey, + siphash_key, + registered_at, + display_name, + key_format.as_str() + ], + )?; + Ok(()) + } + + pub fn lookup_producer(&self, producer_oid: &str) -> SqliteResult> { + let row = self + .conn + .query_row( + "SELECT producer_oid, ed25519_pubkey, siphash_key, registered_at, + display_name, metadata, key_format + FROM producer_identity WHERE producer_oid = ?", + params![producer_oid], + |r| { + let metadata_str: Option = r.get(5)?; + let metadata = metadata_str + .and_then(|s| serde_json::from_str::(&s).ok()); + let format_str: String = r.get(6)?; + // An unparseable format is a refusal, never a + // fallback to raw: guessing wrong here means + // checking a WebAuthn credential as a bare + // signature. + let key_format = trusty_core::KeyFormat::parse(&format_str) + .map_err(|e| rusqlite::Error::FromSqlConversionFailure( + 6, + rusqlite::types::Type::Text, + Box::new(std::io::Error::new(std::io::ErrorKind::InvalidData, e.to_string())), + ))?; + Ok(ProducerIdentity { + producer_oid: r.get(0)?, + ed25519_pubkey: r.get(1)?, + siphash_key: r.get(2)?, + key_format, + registered_at: r.get(3)?, + display_name: r.get(4)?, + metadata, + }) + }, + ) + .optional()?; + Ok(row) + } + + pub fn revoke_producer( + &mut self, + producer_oid: &str, + reason: &str, + attestor: &dyn Signer, + ) -> SqliteResult { + let identity = self.lookup_producer(producer_oid)?.ok_or_else(|| { + SqliteError::NotFound(format!( + "revoke_producer: {} not in producer_identity", + producer_oid + )) + })?; + let attested_by_pubkey = attestor.public_key_bytes(); + + // 1. Append a REVOCATION envelope on the per-user "revocations" + // chain. Same pattern as trusty-postgres. + let revocation_data = serde_json::json!({ + "kind": "REVOCATION", + "producer_oid": producer_oid, + "revoked_pubkey_hex": hex::encode(&identity.ed25519_pubkey), + "reason": reason, + "attested_by_pubkey_hex": hex::encode(&attested_by_pubkey), + }); + let envelope = self.append_signed("revocations", &revocation_data, "src", attestor)?; + + // 2. UPSERT producer_revocation index row. + let revoked_at = now_iso(); + self.conn.execute( + "INSERT INTO producer_revocation + (producer_oid, revoked_at, revoked_pubkey, reason, + attested_by_pubkey, revocation_row_hash) + VALUES (?, ?, ?, ?, ?, ?) + ON CONFLICT(producer_oid) DO UPDATE SET + revoked_at = excluded.revoked_at, + revoked_pubkey = excluded.revoked_pubkey, + reason = excluded.reason, + attested_by_pubkey = excluded.attested_by_pubkey, + revocation_row_hash = excluded.revocation_row_hash", + params![ + producer_oid, + revoked_at, + identity.ed25519_pubkey, + reason, + attested_by_pubkey, + envelope.row_hash, + ], + )?; + + Ok(RevocationRecord { + producer_oid: producer_oid.to_string(), + revoked_at, + revoked_pubkey: identity.ed25519_pubkey, + reason: reason.to_string(), + attested_by_pubkey, + revocation_row_hash: envelope.row_hash, + }) + } + + pub fn lookup_revocation(&self, producer_oid: &str) -> SqliteResult> { + let row = self + .conn + .query_row( + "SELECT producer_oid, revoked_at, revoked_pubkey, reason, + attested_by_pubkey, revocation_row_hash + FROM producer_revocation WHERE producer_oid = ?", + params![producer_oid], + |r| { + Ok(RevocationRecord { + producer_oid: r.get(0)?, + revoked_at: r.get(1)?, + revoked_pubkey: r.get(2)?, + reason: r.get(3)?, + attested_by_pubkey: r.get(4)?, + revocation_row_hash: r.get(5)?, + }) + }, + ) + .optional()?; + Ok(row) + } + + pub fn checkpoint( + &mut self, + entity_id: &str, + attested_by: &str, + signer: &dyn Signer, + ) -> SqliteResult { + let tip = self.read_tip(entity_id)?.ok_or_else(|| { + SqliteError::NotFound(format!("checkpoint: no tip for entity {}", entity_id)) + })?; + let last = self.read_last(entity_id, &tip.chain_id)?.ok_or_else(|| { + SqliteError::NotFound(format!("checkpoint: chain_id {} has no rows", tip.chain_id)) + })?; + let row_hash = last.row_hash.clone(); + let sequence_int = last.sequence_int; + + let to_sign = checkpoint_signing_bytes(&tip.chain_id, sequence_int, &row_hash); + let sig = signer.sign(&to_sign); + let checkpoint_at = now_iso(); + + // SQLite supports RETURNING since 3.35; rusqlite handles it. + let id: i64 = self.conn.query_row( + "INSERT INTO chain_checkpoint + (chain_id, sequence_int, row_hash, checkpoint_at, attested_by, attestation_sig) + VALUES (?, ?, ?, ?, ?, ?) + RETURNING id", + params![ + tip.chain_id, + sequence_int, + row_hash, + checkpoint_at, + attested_by, + sig, + ], + |r| r.get(0), + )?; + + Ok(CheckpointRow { + id, + chain_id: tip.chain_id, + sequence_int, + row_hash, + checkpoint_at, + attested_by: attested_by.to_string(), + attestation_sig: sig, + }) + } + + pub fn latest_checkpoint(&self, chain_id: &str) -> SqliteResult> { + let row = self + .conn + .query_row( + "SELECT id, chain_id, sequence_int, row_hash, checkpoint_at, + attested_by, attestation_sig + FROM chain_checkpoint WHERE chain_id = ? + ORDER BY id DESC LIMIT 1", + params![chain_id], + |r| { + Ok(CheckpointRow { + id: r.get(0)?, + chain_id: r.get(1)?, + sequence_int: r.get(2)?, + row_hash: r.get(3)?, + checkpoint_at: r.get(4)?, + attested_by: r.get(5)?, + attestation_sig: r.get(6)?, + }) + }, + ) + .optional()?; + Ok(row) + } + + /// Window-pruned verify with checkpoint authentication. See the + /// trusty-postgres twin for full semantics. Same authentication + /// rules: checkpoint's attested_by must be in producer_identity; + /// attestation_sig must verify; if revoked AND checkpoint_at >= + /// revoked_at, reject. + pub fn verify_since_checkpoint(&self, entity_id: &str) -> SqliteResult { + let tip = match self.read_tip(entity_id)? { + Some(t) => t, + None => return self.verify(entity_id), + }; + let ckpt = match self.latest_checkpoint(&tip.chain_id)? { + Some(c) => c, + None => return self.verify(entity_id), + }; + + let identity = match self.lookup_producer(&ckpt.attested_by)? { + Some(id) => id, + None => { + return Ok(VerifyReport { + entity_id: entity_id.to_string(), + rows_valid: false, + // The checkpoint was refused, so no rows were walked + // and continuity is UNESTABLISHED rather than proved. + // False, with no breaks listed: fail closed, because + // absent evidence is not evidence of absence. + continuous: false, + // Nothing was walked, so no ceiling was reached either. + limit_reached: false, + breaks: Vec::new(), + row_count: 0, + errors: vec![VerifyError { + sequence: ckpt.sequence_int, + chain_id: ckpt.chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "checkpoint attested_by={} not in producer_identity", + ckpt.attested_by + ), + }, + }], + }); + } + }; + + let signing_view = + checkpoint_signing_bytes(&ckpt.chain_id, ckpt.sequence_int, &ckpt.row_hash); + if Ed25519Verifier::verify( + &identity.ed25519_pubkey, + &signing_view, + &ckpt.attestation_sig, + ) + .is_err() + { + return Ok(VerifyReport { + entity_id: entity_id.to_string(), + rows_valid: false, + // Refused, not walked -- see the note on the sibling + // early return above. Continuity is unestablished. + continuous: false, + // Nothing was walked, so no ceiling was reached either. + limit_reached: false, + breaks: Vec::new(), + row_count: 0, + errors: vec![VerifyError { + sequence: ckpt.sequence_int, + chain_id: ckpt.chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "checkpoint attestation_sig invalid for attested_by={}", + ckpt.attested_by + ), + }, + }], + }); + } + + if let Some(rev) = self.lookup_revocation(&ckpt.attested_by)? { + if ckpt.checkpoint_at.as_str() >= rev.revoked_at.as_str() { + return Ok(VerifyReport { + entity_id: entity_id.to_string(), + rows_valid: false, + // The checkpoint was refused, so no rows were walked + // and continuity is UNESTABLISHED rather than proved. + // False, with no breaks listed: fail closed, because + // absent evidence is not evidence of absence. + continuous: false, + // Nothing was walked, so no ceiling was reached either. + limit_reached: false, + breaks: Vec::new(), + row_count: 0, + errors: vec![VerifyError { + sequence: ckpt.sequence_int, + chain_id: ckpt.chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "checkpoint attested_by={} revoked at {} (checkpoint_at={})", + ckpt.attested_by, rev.revoked_at, ckpt.checkpoint_at + ), + }, + }], + }); + } + } + + // The checkpoint's signature vouches for the head at ckpt.sequence_int, + // so only the rows STRICTLY AFTER it need re-walking -- and that window + // begins chaining from the checkpoint's row_hash, not from genesis. (Was + // `>= ckpt.sequence_int` seeded at genesis, which made every checkpoint + // past sequence 0 fail on a prev-hash mismatch. S126 fix.) + let mut stmt = self.conn.prepare( + "SELECT chain_id, event_type, sequence_int, prev_hash, + row_hash, anchor_hash, raw_payload + FROM provenance_envelope + WHERE entity_id = ? AND sequence_int > ? + ORDER BY id ASC", + )?; + let rows = stmt.query_map(params![entity_id, ckpt.sequence_int], |r| { + Ok(ChainLink { + entity_id: entity_id.to_string(), + chain_id: r.get(0)?, + event_type: r.get(1)?, + sequence: r.get(2)?, + prev_hash: r.get(3)?, + row_hash: r.get(4)?, + anchor_hash: r.get(5)?, + raw_payload: r.get(6)?, + }) + })?; + let mut links: Vec = Vec::new(); + for link in rows { + links.push(link?); + } + Ok(verify_links_from( + entity_id, + &links, + &self.key, + &ckpt.row_hash, + )) + } + + /// Row-level auth-gated verify. See the trusty-postgres twin. + pub fn verify_with_producer( + &self, + entity_id: &str, + expected_oid: &str, + ) -> SqliteResult { + let identity = self.lookup_producer(expected_oid)?; + // Refuse before doing any signature work. A COSE credential + // signs `authenticatorData || SHA-256(clientDataJSON)`, not the + // message, so running it through the bare-signature path + // produces a failure that looks like a wrong key -- and invites + // someone to loosen the check until it passes. + if let Some(id) = &identity { + if let Err(e) = trusty_core::require_bare_signature(id.key_format, expected_oid) { + return Err(SqliteError::UnsupportedCredential(e.to_string())); + } + } + let revocation = self.lookup_revocation(expected_oid)?; + + let mut stmt = self.conn.prepare( + "SELECT chain_id, event_type, sequence_int, prev_hash, row_hash, + anchor_hash, raw_payload, detected_at + FROM provenance_envelope WHERE entity_id = ? ORDER BY id ASC", + )?; + let rows = stmt.query_map(params![entity_id], |r| { + Ok(( + r.get::<_, String>(0)?, // chain_id + r.get::<_, String>(1)?, // event_type + r.get::<_, i64>(2)?, // sequence_int + r.get::<_, String>(3)?, // prev_hash + r.get::<_, Option>(4)?, // row_hash + r.get::<_, Option>(5)?, // anchor_hash + r.get::<_, Option>(6)?, // raw_payload + r.get::<_, String>(7)?, // detected_at + )) + })?; + + let mut links: Vec = Vec::new(); + let mut auth_errors: Vec = Vec::new(); + + for row in rows { + let ( + chain_id, + event_type, + sequence, + prev_hash, + row_hash, + anchor_hash, + raw_payload, + detected_at, + ) = row?; + + if let Some(raw) = raw_payload.as_deref() { + if let Some(sig_pks) = parse_signature_pubkeys(raw) { + for pk_hex in sig_pks { + let pk_bytes = match hex::decode(&pk_hex) { + Ok(b) => b, + Err(e) => { + auth_errors.push(VerifyError { + sequence, + chain_id: chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!("row signature pk hex malformed: {}", e), + }, + }); + continue; + } + }; + match &identity { + None => auth_errors.push(VerifyError { + sequence, + chain_id: chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "expected_oid={} not in producer_identity", + expected_oid + ), + }, + }), + Some(id) if pk_bytes != id.ed25519_pubkey => { + auth_errors.push(VerifyError { + sequence, + chain_id: chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "row signed by unauthorized pubkey (expected_oid={}); pk_hex={}", + expected_oid, pk_hex + ), + }, + }); + } + _ => {} + } + if let Some(rev) = &revocation { + if detected_at.as_str() >= rev.revoked_at.as_str() { + auth_errors.push(VerifyError { + sequence, + chain_id: chain_id.clone(), + kind: VerifyErrorType::SignatureInvalid { + detail: format!( + "row signed AT or AFTER revocation (expected_oid={} revoked_at={} detected_at={})", + expected_oid, rev.revoked_at, detected_at + ), + }, + }); + } + } + } + } + } + + links.push(ChainLink { + entity_id: entity_id.to_string(), + chain_id, + event_type, + sequence, + prev_hash, + row_hash, + anchor_hash, + raw_payload, + }); + } + + let mut report = verify_links(entity_id, &links, &self.key); + if !auth_errors.is_empty() { + report.rows_valid = false; + report.errors.extend(auth_errors); + } + Ok(report) + } + + fn read_tip(&self, entity_id: &str) -> SqliteResult> { + let row = self + .conn + .query_row( + "SELECT chain_id, current_tip_hash FROM chain_tips WHERE entity_id = ?", + params![entity_id], + |r| { + Ok(TipRow { + chain_id: r.get::<_, Option>(0)?.unwrap_or_default(), + current_tip_hash: r.get(1)?, + }) + }, + ) + .optional()?; + Ok(row) + } + + fn read_last(&self, entity_id: &str, chain_id: &str) -> SqliteResult> { + let row = self + .conn + .query_row( + "SELECT row_hash, sequence_int FROM provenance_envelope + WHERE entity_id = ? AND chain_id = ? + AND event_type != 'BREAK' AND row_hash IS NOT NULL + ORDER BY id DESC LIMIT 1", + params![entity_id, chain_id], + |r| { + Ok(LastRow { + row_hash: r.get(0)?, + sequence_int: r.get(1)?, + }) + }, + ) + .optional()?; + Ok(row) + } + + #[allow(clippy::too_many_arguments)] + fn write_envelope( + &self, + chain_id: &str, + entity_id: &str, + event_type: &str, + field_name: Option<&str>, + old_value: Option<&str>, + new_value: Option<&str>, + source_url: &str, + detected_at: &str, + extraction_method: &str, + prev_hash: &str, + row_hash: Option<&str>, + anchor_hash: Option<&str>, + sequence_int: i64, + raw_payload: Option<&str>, + ) -> SqliteResult { + self.conn.execute( + "INSERT INTO provenance_envelope ( + chain_id, entity_id, event_type, + field_name, old_value, new_value, + source_url, detected_at, extraction_method, + prev_hash, row_hash, anchor_hash, sequence_int, raw_payload + ) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)", + params![ + chain_id, + entity_id, + event_type, + field_name, + old_value, + new_value, + source_url, + detected_at, + extraction_method, + prev_hash, + row_hash, + anchor_hash, + sequence_int, + raw_payload, + ], + )?; + Ok(self.conn.last_insert_rowid()) + } + + fn append_hash_index( + &self, + row_hash: &str, + chain_id: &str, + sequence_int: i64, + entity_id: &str, + ) -> SqliteResult<()> { + let recorded_at = now_iso(); + self.conn.execute( + "INSERT OR IGNORE INTO hash_index + (row_hash, chain_id, sequence_int, entity_id, recorded_at) + VALUES (?, ?, ?, ?, ?)", + params![row_hash, chain_id, sequence_int, entity_id, recorded_at], + )?; + Ok(()) + } + + fn set_tip(&self, entity_id: &str, chain_id: &str, tip_hash: Option<&str>) -> SqliteResult<()> { + let updated_at = now_iso(); + self.conn.execute( + "INSERT INTO chain_tips (entity_id, current_tip_hash, chain_id, updated_at) + VALUES (?, ?, ?, ?) + ON CONFLICT(entity_id) DO UPDATE SET + current_tip_hash = excluded.current_tip_hash, + chain_id = excluded.chain_id, + updated_at = excluded.updated_at", + params![entity_id, tip_hash, chain_id, updated_at], + )?; + Ok(()) + } +} + +fn now_iso() -> String { + chrono::Utc::now().to_rfc3339_opts(SecondsFormat::Secs, true) +} + +/// Pull the `signatures` array out of a v2/v3 raw_payload and return +/// the hex-encoded pubkey from each entry. Returns None for v1 rows +/// or unparseable payloads. Mirrors the helper in trusty-postgres. +fn parse_signature_pubkeys(raw_payload: &str) -> Option> { + let v: serde_json::Value = serde_json::from_str(raw_payload).ok()?; + let arr = v.get("signatures")?.as_array()?; + if arr.is_empty() { + return None; + } + let pks: Vec = arr + .iter() + .filter_map(|s| s.get("pk")?.as_str().map(|s| s.to_string())) + .collect(); + if pks.is_empty() { + None + } else { + Some(pks) + } +} + +fn now_iso_micros() -> String { + chrono::Utc::now().to_rfc3339_opts(SecondsFormat::Micros, true) +} + +fn parse_generation(chain_id: &str) -> SqliteResult { + core_parse_generation(chain_id) + .ok_or_else(|| SqliteError::MalformedChainId(chain_id.to_string())) +} + +fn reason_code(reason: &trusty_core::BreakReason) -> String { + // `code()` (not `as_code()`) so a PropertyViolated break records WHICH rule + // broke (`PROPERTY_VIOLATED:`), not just the category. + reason.code() +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn test_key() -> SecretKey { + SecretKey::new(*b"0123456789abcdef") + } + + fn open() -> Trusty { + Trusty::open_in_memory(test_key()).expect("open in-memory db") + } + + #[test] + fn append_first_is_genesis() { + let mut t = open(); + let r = t + .append("acme", &json!({"name": "first"}), "src://a") + .expect("append ok"); + assert!(r.is_genesis); + assert_eq!(r.event_type, "GENESIS"); + assert_eq!(r.chain_id, "chain_acme_000001"); + assert_eq!(r.sequence, 0); + assert!(!r.chain_break); + } + + #[test] + fn second_append_is_mutation() { + let mut t = open(); + let first = t.append("acme", &json!({"v": 1}), "s").expect("ok1"); + let second = t.append("acme", &json!({"v": 2}), "s").expect("ok2"); + assert!(!second.is_genesis); + assert_eq!(second.event_type, "MUTATION"); + assert_eq!(second.sequence, 1); + assert_eq!(second.chain_id, "chain_acme_000001"); + assert_ne!(first.row_hash, second.row_hash); + } + + #[test] + fn verify_fresh_chain_ok() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("ok"); + t.append("acme", &json!({"v": 2}), "s").expect("ok"); + t.append("acme", &json!({"v": 3}), "s").expect("ok"); + let report = t.verify("acme").expect("verify"); + assert!(report.rows_valid, "fresh chain must verify: {:?}", report); + assert_eq!(report.row_count, 3); + } + + #[test] + fn verify_detects_tampered_row_hash() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("ok"); + t.conn + .execute( + "UPDATE provenance_envelope SET row_hash = 'deadbeefdeadbeef' \ + WHERE entity_id = 'acme' AND event_type = 'GENESIS'", + [], + ) + .expect("tamper update"); + let report = t.verify("acme").expect("verify"); + assert!(!report.rows_valid, "tampered chain must not verify"); + let has_mismatch = report + .errors + .iter() + .any(|e| matches!(e.kind, trusty_core::VerifyErrorType::RowHashMismatch { .. })); + assert!( + has_mismatch, + "expected RowHashMismatch in {:?}", + report.errors + ); + } + + #[test] + fn anchor_returns_tip() { + let mut t = open(); + let r = t.append("acme", &json!({"v": 1}), "s").expect("ok"); + let a = t.anchor("acme").expect("anchor").expect("some"); + assert_eq!(a.tip_hash.as_deref(), Some(r.row_hash.as_str())); + assert_eq!(a.chain_id, r.chain_id); + } + + #[test] + fn history_excludes_break_rows() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("ok"); + t.conn + .execute( + "INSERT INTO provenance_envelope ( + chain_id, entity_id, event_type, field_name, old_value, new_value, + source_url, detected_at, extraction_method, prev_hash, + row_hash, anchor_hash, sequence_int, raw_payload + ) VALUES ('chain_acme_000001','acme','BREAK','__chain_break__','x','y','','now','trusty','x',NULL,NULL,-1,NULL)", + [], + ) + .expect("manual break"); + let h = t.history("acme", 100).expect("history"); + assert_eq!(h.len(), 1); + assert_eq!(h[0].event_type, "GENESIS"); + } + + #[test] + fn integrity_ok_after_writes() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("ok"); + t.append("acme", &json!({"v": 2}), "s").expect("ok"); + let r = t.integrity().expect("integrity"); + assert!(r.ok, "integrity must be ok: {:?}", r); + assert_eq!(r.entities_checked, 1); + } + + #[test] + fn integrity_detects_kv_mismatch() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("ok"); + t.conn + .execute( + "UPDATE chain_tips SET current_tip_hash = 'fakehashfakehash' WHERE entity_id = 'acme'", + [], + ) + .expect("tamper"); + let r = t.integrity().expect("integrity"); + assert!(!r.ok); + assert!(!r.mismatches.is_empty()); + } + + #[test] + fn break_seals_and_creates_new_genesis() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("ok"); + t.conn + .execute( + "UPDATE chain_tips SET current_tip_hash = 'tamperedhashvalue' WHERE entity_id = 'acme'", + [], + ) + .expect("tamper tip"); + let r = t + .append("acme", &json!({"v": 2}), "s") + .expect("break append"); + assert!(r.chain_break); + assert_eq!(r.sealed_chain.as_deref(), Some("chain_acme_000001")); + assert_eq!(r.chain_id, "chain_acme_000002"); + assert_eq!(r.event_type, "BREAK"); + } + + #[test] + fn two_entities_independent_chains() { + let mut t = open(); + let a = t.append("ent_a", &json!({"v": 1}), "s").expect("ok"); + let b = t.append("ent_b", &json!({"v": 1}), "s").expect("ok"); + assert_eq!(a.chain_id, "chain_ent_a_000001"); + assert_eq!(b.chain_id, "chain_ent_b_000001"); + let a2 = t.append("ent_a", &json!({"v": 2}), "s").expect("ok"); + assert_eq!(a2.sequence, 1); + let b_hist = t.history("ent_b", 100).expect("hist"); + assert_eq!(b_hist.len(), 1); + } + + fn test_signer() -> trusty_core::Ed25519Signer { + trusty_core::Ed25519Signer::from_secret_bytes(&[13u8; 32]).unwrap() + } + + #[test] + fn append_signed_writes_v2_payload() { + let mut t = open(); + let s = test_signer(); + let r = t + .append_signed("acme", &json!({"v": 1}), "s", &s) + .expect("append_signed ok"); + assert_eq!(r.event_type, "GENESIS"); + let hist = t.history("acme", 10).expect("hist"); + let raw = hist[0].raw_payload.as_ref().expect("raw"); + assert!(raw.contains("\"payload_version\":2"), "got {}", raw); + assert!(raw.contains("\"alg\":\"ed25519\""), "got {}", raw); + } + + #[test] + fn verify_passes_on_mixed_v1_and_v2_chain() { + // Genesis unsigned (v1), mutation signed (v2), mutation unsigned + // (v1 again). verify_links must accept the mix -- signature check + // only activates per-row for v2 rows. + let mut t = open(); + let s = test_signer(); + t.append("acme", &json!({"v": 1}), "s").expect("g"); + t.append_signed("acme", &json!({"v": 2}), "s", &s) + .expect("m-signed"); + t.append("acme", &json!({"v": 3}), "s").expect("m"); + let report = t.verify("acme").expect("verify"); + assert!(report.rows_valid, "mixed chain must verify: {:?}", report); + assert_eq!(report.row_count, 3); + } + + #[test] + fn append_batch_empty_is_ok() { + let mut t = open(); + let r = t.append_batch(&[]).expect("empty ok"); + assert!(r.is_empty()); + } + + #[test] + fn append_batch_matches_loop_structure() { + // Append the same sequence via loop and via batch into two + // separate stores. Row hashes WILL differ (timestamps are + // part of the keyed payload, and the two runs happen at + // different moments), but structural properties -- event + // type, sequence, chain_id, break/genesis flags -- must be + // identical. + let mut loop_store = open(); + let mut batch_store = open(); + + let rows = [json!({"v": 1}), json!({"v": 2}), json!({"v": 3})]; + let mut loop_results = Vec::new(); + for r in &rows { + loop_results.push(loop_store.append("acme", r, "s").expect("loop append ok")); + } + + let specs: Vec = rows + .iter() + .map(|r| RowSpec { + entity_id: "acme", + data: r, + source_url: "s", + }) + .collect(); + let batch_results = batch_store.append_batch(&specs).expect("batch ok"); + + assert_eq!(loop_results.len(), batch_results.len()); + for (a, b) in loop_results.iter().zip(batch_results.iter()) { + assert_eq!(a.event_type, b.event_type); + assert_eq!(a.sequence, b.sequence); + assert_eq!(a.chain_id, b.chain_id); + assert_eq!(a.is_genesis, b.is_genesis); + assert_eq!(a.chain_break, b.chain_break); + } + } + + #[test] + fn append_batch_verifies_as_chain() { + let mut t = open(); + let rows = [json!({"v": 1}), json!({"v": 2}), json!({"v": 3})]; + let specs: Vec = rows + .iter() + .map(|r| RowSpec { + entity_id: "acme", + data: r, + source_url: "s", + }) + .collect(); + t.append_batch(&specs).expect("batch ok"); + + let report = t.verify("acme").expect("verify"); + assert!( + report.rows_valid, + "batch-appended chain must verify: {:?}", + report + ); + assert_eq!(report.row_count, 3); + } + + #[test] + fn append_batch_multi_entity() { + let mut t = open(); + let a_data = json!({"src": "a"}); + let b_data = json!({"src": "b"}); + let specs = [ + RowSpec { + entity_id: "ent_a", + data: &a_data, + source_url: "s", + }, + RowSpec { + entity_id: "ent_b", + data: &b_data, + source_url: "s", + }, + RowSpec { + entity_id: "ent_a", + data: &a_data, + source_url: "s", + }, + ]; + let results = t.append_batch(&specs).expect("batch ok"); + assert_eq!(results[0].entity_id, "ent_a"); + assert_eq!(results[0].sequence, 0); + assert_eq!(results[1].entity_id, "ent_b"); + assert_eq!(results[1].sequence, 0); + assert_eq!(results[2].entity_id, "ent_a"); + assert_eq!(results[2].sequence, 1); + } + + #[test] + fn breaks_since_none_when_clean() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("g"); + t.append("acme", &json!({"v": 2}), "s").expect("m"); + let events = t.breaks_since("acme", None).expect("breaks"); + assert!(events.is_empty()); + } + + #[test] + fn breaks_since_surfaces_tamper_break() { + let mut t = open(); + let r1 = t.append("acme", &json!({"v": 1}), "s").expect("g"); + t.conn + .execute( + "UPDATE chain_tips SET current_tip_hash = 'tamperedhashvalue' \ + WHERE entity_id = 'acme'", + [], + ) + .expect("tamper tip"); + t.append("acme", &json!({"v": 2}), "s") + .expect("break append"); + + let events = t.breaks_since("acme", None).expect("breaks"); + assert_eq!(events.len(), 1); + let e = &events[0]; + assert_eq!(e.entity_id, "acme"); + assert_eq!(e.chain_id, "chain_acme_000001"); + assert_eq!(e.sealed_at_hash, r1.row_hash); + assert!(!e.break_reason.is_empty(), "reason code must be populated"); + } + + #[test] + fn export_bundle_roundtrip_verifies() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("g"); + t.append("acme", &json!({"v": 2}), "s").expect("m1"); + t.append("acme", &json!({"v": 3}), "s").expect("m2"); + + let bundle = t.export_bundle("acme", None, None).expect("bundle"); + assert_eq!(bundle.entity_id, "acme"); + assert_eq!(bundle.row_count, 3); + assert_eq!(bundle.rows.len(), 3); + assert_eq!(bundle.anchors.len(), 3); + assert_eq!(bundle.sequence_range, (Some(0), Some(2))); + + let report = bundle.verify(&test_key()); + assert!(report.rows_valid, "bundle verify must pass: {:?}", report); + assert_eq!(report.row_count, 3); + } + + #[test] + fn export_bundle_json_roundtrips_and_verifies() { + // Serialize to JSON on the producer side, deserialize on the + // verifier side, then verify. Exercises the wire path. + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("g"); + t.append("acme", &json!({"v": 2}), "s").expect("m"); + let bundle = t.export_bundle("acme", None, None).expect("bundle"); + + let wire = serde_json::to_string(&bundle).expect("serialize"); + let decoded: trusty_core::ProofBundle = serde_json::from_str(&wire).expect("deserialize"); + assert_eq!(decoded.row_count, 2); + + let report = decoded.verify(&test_key()); + assert!( + report.rows_valid, + "decoded bundle must verify: {:?}", + report + ); + } + + #[test] + fn export_bundle_tampered_row_hash_is_rejected() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("g"); + t.append("acme", &json!({"v": 2}), "s").expect("m"); + let mut bundle = t.export_bundle("acme", None, None).expect("bundle"); + + // Flip one char of the genesis row_hash inside the bundle. + // Since the verifier recomputes SipHash from raw_payload, the + // mutated row_hash must fail RowHashMismatch. + let original = bundle.rows[0] + .row_hash + .as_ref() + .expect("row_hash present") + .clone(); + let (head, tail) = original.split_at(original.len() - 1); + let flipped = if tail == "0" { "1" } else { "0" }; + bundle.rows[0].row_hash = Some(format!("{}{}", head, flipped)); + + let report = bundle.verify(&test_key()); + assert!(!report.rows_valid, "tampered bundle must not verify"); + assert!( + report + .errors + .iter() + .any(|e| matches!(e.kind, trusty_core::VerifyErrorType::RowHashMismatch { .. })), + "expected RowHashMismatch, got {:?}", + report.errors + ); + } + + #[test] + fn anchors_since_none_returns_every_anchor() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("g"); + t.append("acme", &json!({"v": 2}), "s").expect("m1"); + t.append("acme", &json!({"v": 3}), "s").expect("m2"); + + let anchors = t.anchors_since("acme", None).expect("anchors"); + assert_eq!(anchors.len(), 3); + assert_eq!(anchors[0].sequence, 0); + assert_eq!(anchors[1].sequence, 1); + assert_eq!(anchors[2].sequence, 2); + // anchor_hash is a hex string, non-empty, distinct per row. + assert!(!anchors[0].anchor_hash.is_empty()); + assert_ne!(anchors[0].anchor_hash, anchors[1].anchor_hash); + } + + #[test] + fn anchors_since_skips_break_rows() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("g"); + t.conn + .execute( + "UPDATE chain_tips SET current_tip_hash = 'tamperedhashvalue' \ + WHERE entity_id = 'acme'", + [], + ) + .expect("tamper tip"); + let _break = t + .append("acme", &json!({"v": 2}), "s") + .expect("break append"); + + let anchors = t.anchors_since("acme", None).expect("anchors"); + // Genesis of chain 1 + new genesis of chain 2; the BREAK row + // between them carries no anchor and must be excluded. + assert_eq!(anchors.len(), 2, "got: {:?}", anchors); + } + + #[test] + fn export_bundle_includes_break_rows_for_chain_break_proof() { + let mut t = open(); + t.append("acme", &json!({"v": 1}), "s").expect("g"); + t.conn + .execute( + "UPDATE chain_tips SET current_tip_hash = 'tamperedhashvalue' \ + WHERE entity_id = 'acme'", + [], + ) + .expect("tamper tip"); + t.append("acme", &json!({"v": 2}), "s").expect("break"); + + let bundle = t.export_bundle("acme", None, None).expect("bundle"); + let has_break = bundle.rows.iter().any(|r| r.event_type == "BREAK"); + assert!( + has_break, + "bundle must carry BREAK rows for chain-break-as-event proof" + ); + + let report = bundle.verify(&test_key()); + assert!( + report.rows_valid, + "bundle with break must still verify: {:?}", + report + ); + } + + #[test] + fn verify_detects_tampered_v2_signature() { + let mut t = open(); + let s = test_signer(); + t.append_signed("acme", &json!({"v": 1}), "s", &s) + .expect("g"); + + // Flip the last hex char of the stored signature, then recompute + // row_hash+anchor_hash over the mutated bytes so the row_hash + // check passes and the signature check is what fails. + let hist = t.history("acme", 10).expect("hist"); + let raw = hist[0].raw_payload.clone().expect("raw"); + let mut fields = trusty_core::chain::parse_raw_payload(&raw).unwrap(); + let sig_obj = fields.signatures[0].as_object_mut().unwrap(); + let sig_hex = sig_obj["sig"].as_str().unwrap().to_string(); + let (head, tail) = sig_hex.split_at(sig_hex.len() - 1); + let flipped_last = if tail == "0" { "1" } else { "0" }; + sig_obj.insert("sig".into(), json!(format!("{}{}", head, flipped_last))); + let mutated = trusty_core::build_payload(&fields).unwrap(); + let new_rh = trusty_core::row_hash(&test_key(), mutated.as_bytes()); + let new_ah = trusty_core::anchor_hash(&new_rh, "acme", "chain_acme_000001"); + t.conn + .execute( + "UPDATE provenance_envelope SET raw_payload = ?, row_hash = ?, anchor_hash = ? \ + WHERE entity_id = 'acme' AND event_type = 'GENESIS'", + params![mutated, new_rh, new_ah], + ) + .expect("tamper"); + + let report = t.verify("acme").expect("verify"); + assert!(!report.rows_valid); + assert!(report.errors.iter().any(|e| matches!( + e.kind, + trusty_core::VerifyErrorType::SignatureInvalid { .. } + ))); + } + + #[test] + fn liveness_check_clean_is_empty_for_zero_gap() { + let mut t = open(); + t.append("sensor-a", &json!({"v": 1}), "s").expect("append"); + // Just appended -- gap is ~0 seconds, threshold is 9999. Must be silent. + let silent = t.liveness_check(None, 9999).expect("liveness"); + assert!( + silent.is_empty(), + "freshly-appended entity must not be silent" + ); + } + + #[test] + fn liveness_check_surfaces_stale_entity() { + let mut t = open(); + t.append("sensor-a", &json!({"v": 1}), "s").expect("append"); + // Force detected_at to a timestamp 100 seconds in the past via direct SQL. + t.conn + .execute( + "UPDATE provenance_envelope SET detected_at = datetime('now', '-100 seconds') \ + WHERE entity_id = 'sensor-a'", + [], + ) + .expect("backdate"); + let silent = t.liveness_check(None, 10).expect("liveness"); + assert_eq!(silent.len(), 1); + assert_eq!(silent[0].entity_id, "sensor-a"); + assert!(silent[0].gap_secs > 10.0); + } + + #[test] + fn liveness_check_filtered_by_entity_id() { + let mut t = open(); + t.append("sensor-a", &json!({"v": 1}), "s").expect("a"); + t.append("sensor-b", &json!({"v": 1}), "s").expect("b"); + t.conn + .execute( + "UPDATE provenance_envelope SET detected_at = datetime('now', '-100 seconds') \ + WHERE entity_id = 'sensor-a'", + [], + ) + .expect("backdate a"); + // Only check sensor-b -- must be empty even though sensor-a is stale. + let silent = t.liveness_check(Some("sensor-b"), 10).expect("liveness"); + assert!(silent.is_empty()); + // Check sensor-a directly -- must surface. + let silent = t.liveness_check(Some("sensor-a"), 10).expect("liveness"); + assert_eq!(silent.len(), 1); + } + + #[test] + fn liveness_check_ignores_break_rows() { + let mut t = open(); + t.append("sensor-a", &json!({"v": 1}), "s").expect("append"); + // Tamper chain to produce a BREAK row, then backdate all rows. + t.conn + .execute( + "UPDATE chain_tips SET current_tip_hash = 'fakehash' WHERE entity_id = 'sensor-a'", + [], + ) + .expect("tamper"); + t.append("sensor-a", &json!({"v": 2}), "s") + .expect("break append"); + // Backdate only the BREAK row -- the real row stays fresh. + t.conn + .execute( + "UPDATE provenance_envelope SET detected_at = datetime('now', '-100 seconds') \ + WHERE event_type = 'BREAK'", + [], + ) + .expect("backdate break"); + // BREAK rows must not count toward liveness -- entity has a fresh non-BREAK row. + let silent = t.liveness_check(None, 10).expect("liveness"); + assert!( + silent.is_empty(), + "BREAK rows must not affect liveness check" + ); + } + + // ================================================================ + // Federation primitives (S17 cycle 4) -- producer_identity, + // producer_revocation, chain_checkpoint, verify_since_checkpoint, + // verify_with_producer. Mirrors trusty-postgres integration tests + // but in-process so no env-var gating needed. + // ================================================================ + + fn fed_signer() -> trusty_core::Ed25519Signer { + trusty_core::Ed25519Signer::from_secret_bytes(&[31u8; 32]).unwrap() + } + + #[test] + fn fed_register_lookup_producer_round_trip() { + let mut t = open(); + let s = fed_signer(); + t.register_producer("oid:1.2.3", &s.public_key_bytes(), b"sip-1", Some("a")) + .unwrap(); + + let got = t.lookup_producer("oid:1.2.3").unwrap().unwrap(); + assert_eq!(got.producer_oid, "oid:1.2.3"); + assert_eq!(got.ed25519_pubkey, s.public_key_bytes()); + assert_eq!(got.siphash_key, b"sip-1"); + assert_eq!(got.display_name.as_deref(), Some("a")); + + // UPSERT. + let s2 = trusty_core::Ed25519Signer::from_secret_bytes(&[5u8; 32]).unwrap(); + t.register_producer("oid:1.2.3", &s2.public_key_bytes(), b"sip-2", None) + .unwrap(); + let updated = t.lookup_producer("oid:1.2.3").unwrap().unwrap(); + assert_eq!(updated.ed25519_pubkey, s2.public_key_bytes()); + assert_eq!(updated.siphash_key, b"sip-2"); + assert!(updated.display_name.is_none()); + + assert!(t.lookup_producer("oid:nope").unwrap().is_none()); + } + + #[test] + fn fed_revoke_producer_round_trip_and_lookup() { + let mut t = open(); + let s = fed_signer(); + t.register_producer("oid:test", &s.public_key_bytes(), b"sip", None) + .unwrap(); + + assert!(t.lookup_revocation("oid:test").unwrap().is_none()); + + let rec = t.revoke_producer("oid:test", "key compromise", &s).unwrap(); + assert_eq!(rec.producer_oid, "oid:test"); + assert_eq!(rec.reason, "key compromise"); + assert!(!rec.revocation_row_hash.is_empty()); + + let got = t.lookup_revocation("oid:test").unwrap().unwrap(); + assert_eq!(got.revocation_row_hash, rec.revocation_row_hash); + + // The REVOCATION envelope verifies on the 'revocations' chain. + let report = t.verify("revocations").unwrap(); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 1); + + // Re-revoke -> two envelopes, one index row. + t.revoke_producer("oid:test", "second", &s).unwrap(); + let report2 = t.verify("revocations").unwrap(); + assert_eq!(report2.row_count, 2); + } + + #[test] + fn fed_revoke_producer_unknown_oid_errors() { + let mut t = open(); + let s = fed_signer(); + let err = t.revoke_producer("oid:ghost", "missing", &s); + assert!(matches!(err, Err(SqliteError::NotFound(_)))); + } + + #[test] + fn fed_checkpoint_round_trip() { + let mut t = open(); + let s = fed_signer(); + for i in 0..3 { + t.append("dev1", &json!({"i": i}), "src").unwrap(); + } + let ckpt = t.checkpoint("dev1", "oid:test", &s).unwrap(); + assert_eq!(ckpt.attested_by, "oid:test"); + assert_eq!(ckpt.sequence_int, 2); + assert!(!ckpt.attestation_sig.is_empty()); + + let latest = t.latest_checkpoint(&ckpt.chain_id).unwrap().unwrap(); + assert_eq!(latest.id, ckpt.id); + assert_eq!(latest.row_hash, ckpt.row_hash); + } + + #[test] + fn fed_verify_since_checkpoint_falls_back_when_no_checkpoint() { + let mut t = open(); + for i in 0..2 { + t.append("dev2", &json!({"i": i}), "src").unwrap(); + } + let full = t.verify("dev2").unwrap(); + let pruned = t.verify_since_checkpoint("dev2").unwrap(); + assert_eq!(full.rows_valid, pruned.rows_valid); + assert_eq!(full.row_count, pruned.row_count); + } + + #[test] + fn fed_verify_since_checkpoint_authenticates_and_rejects_unknown_attestor() { + let mut t = open(); + let s = fed_signer(); + for i in 0..2 { + t.append("dev3", &json!({"i": i}), "src").unwrap(); + } + // checkpoint by unregistered attestor -> verify rejects. + t.checkpoint("dev3", "oid:ghost", &s).unwrap(); + let report = t.verify_since_checkpoint("dev3").unwrap(); + assert!(!report.rows_valid); + let detail = match &report.errors[0].kind { + VerifyErrorType::SignatureInvalid { detail } => detail.clone(), + other => panic!("expected SignatureInvalid, got {:?}", other), + }; + assert!( + detail.contains("not in producer_identity"), + "detail={}", + detail + ); + } + + #[test] + fn fed_verify_since_checkpoint_rejects_post_revocation() { + let mut t = open(); + let s = fed_signer(); + t.register_producer("oid:test", &s.public_key_bytes(), b"sip", None) + .unwrap(); + for i in 0..3 { + t.append("dev4", &json!({"i": i}), "src").unwrap(); + } + // revoke first, then checkpoint -> rejected + t.revoke_producer("oid:test", "compromise", &s).unwrap(); + std::thread::sleep(std::time::Duration::from_millis(1100)); // RFC3339 secs precision + t.checkpoint("dev4", "oid:test", &s).unwrap(); + + let report = t.verify_since_checkpoint("dev4").unwrap(); + assert!(!report.rows_valid); + let detail = match &report.errors[0].kind { + VerifyErrorType::SignatureInvalid { detail } => detail.clone(), + other => panic!("expected SignatureInvalid, got {:?}", other), + }; + assert!(detail.contains("revoked at"), "detail={}", detail); + } + + #[test] + fn fed_verify_with_producer_accepts_authorized_rows() { + let mut t = open(); + let s = fed_signer(); + t.register_producer("oid:test", &s.public_key_bytes(), b"sip", None) + .unwrap(); + for i in 0..3 { + t.append_signed("dev5", &json!({"i": i}), "src", &s) + .unwrap(); + } + let report = t.verify_with_producer("dev5", "oid:test").unwrap(); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 3); + } + + #[test] + fn fed_verify_with_producer_rejects_unauthorized_pubkey() { + let mut t = open(); + let registered = fed_signer(); + let imposter = trusty_core::Ed25519Signer::from_secret_bytes(&[77u8; 32]).unwrap(); + t.register_producer("oid:test", ®istered.public_key_bytes(), b"sip", None) + .unwrap(); + t.append_signed("dev6", &json!({"i": 0}), "src", ®istered) + .unwrap(); + t.append_signed("dev6", &json!({"i": 1}), "src", &imposter) + .unwrap(); + + let report = t.verify_with_producer("dev6", "oid:test").unwrap(); + assert!(!report.rows_valid); + let bad = report.errors.iter().find(|e| match &e.kind { + VerifyErrorType::SignatureInvalid { detail } => detail.contains("unauthorized pubkey"), + _ => false, + }); + assert!( + bad.is_some(), + "expected unauthorized pubkey error, got {:?}", + report.errors + ); + } + + #[test] + fn fed_verify_with_producer_pre_revoke_rows_remain_valid() { + let mut t = open(); + let s = fed_signer(); + t.register_producer("oid:test", &s.public_key_bytes(), b"sip", None) + .unwrap(); + for i in 0..2 { + t.append_signed("dev7", &json!({"i": i}), "src", &s) + .unwrap(); + } + std::thread::sleep(std::time::Duration::from_millis(1100)); + t.revoke_producer("oid:test", "later", &s).unwrap(); + let report = t.verify_with_producer("dev7", "oid:test").unwrap(); + assert!( + report.rows_valid, + "expected pre-revocation rows to remain valid; errors: {:?}", + report.errors + ); + assert_eq!(report.row_count, 2); + } +} + +#[cfg(test)] +mod signed_only_tests { + use super::*; + use trusty_core::Ed25519Signer; + + fn store() -> Trusty { + Trusty::open_in_memory(SecretKey::new([7u8; 16])).unwrap() + } + + #[test] + fn off_by_default_so_existing_callers_are_unaffected() { + let mut t = store(); + assert!(!t.is_signed_only()); + t.append("acme", &serde_json::json!({"v": 1}), "s").unwrap(); + } + + #[test] + fn signed_only_refuses_an_unsigned_append() { + let mut t = store(); + t.set_signed_only(true); + let err = t + .append("acme", &serde_json::json!({"v": 1}), "s") + .unwrap_err(); + assert!( + matches!(err, SqliteError::UnsignedAppendRefused(ref e) if e == "acme"), + "got {err:?}" + ); + } + + #[test] + fn signed_only_still_accepts_a_signed_append() { + let mut t = store(); + t.set_signed_only(true); + let signer = Ed25519Signer::generate(); + t.append_signed("acme", &serde_json::json!({"v": 1}), "s", &signer) + .unwrap(); + } + + #[test] + fn genesis_is_covered_too() { + // The first row of a chain takes a different internal path; the check + // sits at the shared entry point precisely so genesis cannot slip past. + let mut t = store(); + t.set_signed_only(true); + assert!(t + .append("brand-new-entity", &serde_json::json!({}), "s") + .is_err()); + } + + #[test] + fn append_with_method_is_covered_too() { + let mut t = store(); + t.set_signed_only(true); + assert!( + t.append_with_method("acme", &serde_json::json!({"v": 1}), "s", "manual") + .is_err(), + "every unsigned entry point must be refused, not just append()" + ); + } + + #[test] + fn existing_unsigned_rows_are_left_alone() { + // Turning the flag on must not rewrite history -- that is the one thing + // this crate exists to prevent. The chain keeps its unsigned rows and + // fully_signed reports them. + let mut t = store(); + t.append("acme", &serde_json::json!({"v": 1}), "s").unwrap(); + t.set_signed_only(true); + let signer = Ed25519Signer::generate(); + t.append_signed("acme", &serde_json::json!({"v": 2}), "s", &signer) + .unwrap(); + + let bundle = t.export_bundle("acme", None, None).unwrap(); + let report = bundle.verify_public(); + assert_eq!(report.row_count, 2); + assert!( + !report.fully_signed, + "the pre-existing unsigned row remains" + ); + assert!(report.rows_valid, "{:?}", report.errors); + } +} diff --git a/crates/adapters/trusty-sqlite/tests/annotation.rs b/crates/adapters/trusty-sqlite/tests/annotation.rs new file mode 100644 index 0000000..880c37b --- /dev/null +++ b/crates/adapters/trusty-sqlite/tests/annotation.rs @@ -0,0 +1,569 @@ +//! Custody annotations across real, separately-owned chains. +//! +//! Each node here is a genuinely separate store with its own signing key +//! and its own chain. Nothing writes to anybody else's, which is the +//! property the design rests on: a node cannot forge an annotation it did not +//! perform, because it can only sign its own. + +use trusty_core::{ + Layer, + entity_id_for, verify_annotations, Ed25519Signer, Annotation, SecretKey, SourceRef, + AnnotationProblem, +}; +use trusty_sqlite::Trusty; + +const TAG: &str = "sha256:d41d8cd9"; + +/// A node: its own store, its own key, its own chain. +struct Node { + name: String, + store: Trusty, + signer: Ed25519Signer, +} + +impl Node { + fn new(name: &str, seed: u8) -> Self { + Self { + name: name.to_string(), + store: Trusty::open_in_memory(SecretKey::new([seed; 16])).expect("store"), + signer: Ed25519Signer::generate(), + } + } + + /// Record that this node holds the data, and say where it came from. + /// Returns the reference a downstream node will cite. + fn record(&mut self, from: Option, note: &str) -> SourceRef { + let mut annotation = Annotation::custody(TAG, &self.name).because(note); + annotation.received_from = from; + let entity = entity_id_for(TAG, &self.name); + let appended = self + .store + .append_signed( + &entity, + &annotation.to_data(), + "custody://test", + &self.signer, + ) + .expect("append"); + let bundle = self.store.export_bundle(&entity, None, None).expect("export"); + let row = bundle + .rows + .iter() + .find(|r| r.row_hash.as_deref() == Some(appended.row_hash.as_str())) + .expect("the row just written must be in the bundle"); + SourceRef { + chain_id: row.chain_id.clone(), + sequence: row.sequence, + row_hash: appended.row_hash.clone(), + } + } + + fn bundle(&self) -> trusty_core::ProofBundle { + self.store + .export_bundle(&entity_id_for(TAG, &self.name), None, None) + .expect("export") + } +} + +#[test] +fn a_path_across_three_nodes_reads_in_order() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + let mut sink = Node::new("sink", 3); + + let a = origin.record(None, "produced here"); + let b = relay.record(Some(a), "forwarded"); + sink.record(Some(b), "delivered"); + + let report = verify_annotations(&[origin.bundle(), relay.bundle(), sink.bundle()], TAG); + + assert!( + report.intact(), + "three honest nodes must produce a clean path; problems: {:?}", + report.problems + ); + assert_eq!( + report.custody_line(), + vec!["origin", "relay", "sink"], + "the order is reconstructed from the links, not from the order the \ + bundles happened to be passed in" + ); + assert_eq!(report.found, 3); +} + +#[test] +fn the_order_does_not_depend_on_the_order_bundles_arrive_in() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + let mut sink = Node::new("sink", 3); + let a = origin.record(None, "produced here"); + let b = relay.record(Some(a), "forwarded"); + sink.record(Some(b), "delivered"); + + let shuffled = verify_annotations(&[sink.bundle(), origin.bundle(), relay.bundle()], TAG); + + assert_eq!( + shuffled.custody_line(), + vec!["origin", "relay", "sink"], + "a merge that depended on argument order would be reporting the \ + caller's filing habits rather than the history" + ); +} + +#[test] +fn a_missing_middle_node_is_reported_not_papered_over() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + let mut sink = Node::new("sink", 3); + + let a = origin.record(None, "produced here"); + let b = relay.record(Some(a), "forwarded"); + sink.record(Some(b), "delivered"); + + // The relay's bundle is withheld -- exactly what a relay that would + // rather not be in the record would do. + let report = verify_annotations(&[origin.bundle(), sink.bundle()], TAG); + + assert!( + !report.intact(), + "a gap must not read as a clean history; problems: {:?}", + report.problems + ); + assert!( + report + .problems + .iter() + .any(|p| matches!(p, AnnotationProblem::DanglingPredecessor { node, .. } if node == "sink")), + "the sink cites a row nobody can produce, and that is the finding; \ + got {:?}", + report.problems + ); +} + +#[test] +fn a_hop_citing_a_row_that_was_altered_does_not_link() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + + let a = origin.record(None, "produced here"); + // The relay cites the right position but the wrong content -- what a + // forged provenance claim looks like when the citation is guessed. + relay.record( + Some(SourceRef { + row_hash: "00".repeat(32), + ..a + }), + "claims to have had it from origin", + ); + + let report = verify_annotations(&[origin.bundle(), relay.bundle()], TAG); + + assert!( + report + .problems + .iter() + .any(|p| matches!(p, AnnotationProblem::DanglingPredecessor { .. })), + "naming a real position with the wrong hash must not link; the hash \ + is what makes the citation a citation; got {:?}", + report.problems + ); +} + +#[test] +fn two_nodes_receiving_from_the_same_hop_is_reported_as_a_fork() { + let mut origin = Node::new("origin", 1); + let mut left = Node::new("left", 2); + let mut right = Node::new("right", 3); + + let a = origin.record(None, "produced here"); + left.record(Some(a.clone()), "one copy"); + right.record(Some(a), "another copy"); + + let report = verify_annotations(&[origin.bundle(), left.bundle(), right.bundle()], TAG); + + assert!( + report + .problems + .iter() + .any(|p| matches!(p, AnnotationProblem::Fork { .. })), + "fanning out is legitimate, but a caller expecting a line must be \ + told it got a tree; got {:?}", + report.problems + ); + assert!( + !report.intact(), + "intact() is the strict reading; a caller who accepts forks reads \ + problems and decides" + ); +} + +#[test] +fn two_claimed_origins_are_reported() { + let mut one = Node::new("one", 1); + let mut two = Node::new("two", 2); + one.record(None, "I made this"); + two.record(None, "no, I made this"); + + let report = verify_annotations(&[one.bundle(), two.bundle()], TAG); + + assert!( + report + .problems + .iter() + .any(|p| matches!(p, AnnotationProblem::MultipleOrigins { .. })), + "the same data cannot originate twice, and which claim is false is \ + not something the log can settle -- but it can refuse to pick; got {:?}", + report.problems + ); + assert!(report.hops.is_empty(), "with no single origin there is no order"); +} + +#[test] +fn traffic_about_other_data_is_ignored() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + + // The relay's chain also carries an annotation about something else. + let other = Annotation::custody("sha256:something-else", &relay.name); + relay + .store + .append_signed( + &entity_id_for(TAG, &relay.name), + &other.to_data(), + "custody://test", + &relay.signer, + ) + .expect("append other"); + + let a = origin.record(None, "produced here"); + relay.record(Some(a), "forwarded"); + + let report = verify_annotations(&[origin.bundle(), relay.bundle()], TAG); + + assert!( + report.intact(), + "a bundle may carry unrelated traffic; problems: {:?}", + report.problems + ); + assert_eq!( + report.found, 2, + "only annotations about the tag asked for count, or a busy node would \ + appear to have handled everything" + ); +} + +#[test] +fn nothing_recorded_is_not_a_clean_history() { + let origin = Node::new("origin", 1); + let report = verify_annotations(&[origin.bundle()], TAG); + + assert!( + !report.intact(), + "no records at all must not read as an unbroken chain of custody; \ + the difference matters most exactly when someone is asking" + ); + assert_eq!(report.found, 0); +} + +#[test] +fn a_tampered_bundle_takes_no_part_in_the_path() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + let a = origin.record(None, "produced here"); + relay.record(Some(a), "forwarded"); + + // Rewrite what the relay claims it did, after export. The signature + // no longer covers it. + let mut forged = relay.bundle(); + for row in forged.rows.iter_mut() { + if let Some(payload) = row.raw_payload.as_mut() { + *payload = payload.replace("forwarded", "never touched it"); + } + } + + let report = verify_annotations(&[origin.bundle(), forged], TAG); + + assert!( + report + .problems + .iter() + .any(|p| matches!(p, AnnotationProblem::BundleUnverified { .. })), + "a bundle whose rows no longer match their signatures must be \ + reported as such; got {:?}", + report.problems + ); + assert_eq!( + report.found, 1, + "and its rows must not contribute hops -- a tidy custody report \ + built over forged rows is worse than no report at all" + ); + assert!(!report.intact()); +} + +#[test] +fn an_honest_bundle_beside_a_forged_one_still_counts() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + let a = origin.record(None, "produced here"); + relay.record(Some(a), "forwarded"); + + let mut forged = relay.bundle(); + for row in forged.rows.iter_mut() { + if let Some(payload) = row.raw_payload.as_mut() { + *payload = payload.replace("forwarded", "never touched it"); + } + } + let report = verify_annotations(&[origin.bundle(), forged], TAG); + + // The origin is untouched and still says what it said. One party + // lying must not erase another party's honest record. + assert_eq!( + report.custody_line(), + vec!["origin"], + "the surviving path is what the verified bundles support, no more \ + and no less; got {:?}", + report.custody_line() + ); +} + +// --------------------------------------------------------------------- +// Kinds beyond custody. +// --------------------------------------------------------------------- + +impl Node { + /// Record that this node ran a check, and what it concluded. + fn checked(&mut self, ok: bool, note: &str) { + let annotation = Annotation::integrity(TAG, &self.name, ok) + .at_layer(Layer::Device) + .because(note); + self.store + .append_signed( + &entity_id_for(TAG, &self.name), + &annotation.to_data(), + "custody://test", + &self.signer, + ) + .expect("append"); + } +} + +#[test] +fn a_failed_check_becomes_an_actionable_marker() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + + let a = origin.record(None, "produced here"); + relay.record(Some(a), "forwarded"); + relay.checked(false, "checksum mismatch on receipt"); + + let report = verify_annotations(&[origin.bundle(), relay.bundle()], TAG); + + assert_eq!( + report.failed_checks().len(), + 1, + "a node that ran a check and found a problem must surface it \ + separately from structural findings, so a peer can decide to \ + refuse, quarantine or inspect; got {:?}", + report.problems + ); + assert!( + !report.intact(), + "a datum somebody found wrong must not read as clean" + ); + assert_eq!( + report.custody_line(), + vec!["origin", "relay"], + "the custody path is still readable; a failed check is a finding \ + about the datum, not a reason to lose the history" + ); +} + +#[test] +fn a_passed_check_is_recorded_without_becoming_a_problem() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + let a = origin.record(None, "produced here"); + relay.record(Some(a), "forwarded"); + relay.checked(true, "checksum matched"); + + let report = verify_annotations(&[origin.bundle(), relay.bundle()], TAG); + + assert!( + report.intact(), + "a check that passed is evidence in favour, not a finding; \ + problems: {:?}", + report.problems + ); + assert_eq!( + report.found, 3, + "the passing check must still be counted -- knowing a check RAN is \ + the difference between verified and merely unexamined" + ); +} + +#[test] +fn a_custody_claim_is_not_read_as_a_check_that_passed() { + let mut origin = Node::new("origin", 1); + origin.record(None, "produced here"); + + let report = verify_annotations(&[origin.bundle()], TAG); + + assert!( + report.failed_checks().is_empty(), + "holding data is not a claim about its correctness" + ); + assert_eq!( + report.found, 1, + "and it is still an annotation, counted as one" + ); +} + +#[test] +fn an_annotation_of_an_unknown_kind_is_reported_not_ignored() { + let mut origin = Node::new("origin", 1); + origin.record(None, "produced here"); + + let mut odd = Annotation::custody(TAG, &origin.name); + odd.kind = trusty_core::AnnotationKind::Other("future.kind.v9".into()); + origin + .store + .append_signed( + &entity_id_for(TAG, &origin.name), + &odd.to_data(), + "custody://test", + &origin.signer, + ) + .expect("append"); + + let report = verify_annotations(&[origin.bundle()], TAG); + + assert!( + report.problems.iter().any(|p| matches!( + p, + AnnotationProblem::UnrecognisedKind { kind, .. } if kind == "future.kind.v9" + )), + "silence would let a future kind carrying a failed check pass an old \ + verifier unnoticed; got {:?}", + report.problems + ); +} + +#[test] +fn a_forged_bundle_cannot_erase_a_failed_check() { + let mut origin = Node::new("origin", 1); + let mut relay = Node::new("relay", 2); + let a = origin.record(None, "produced here"); + relay.record(Some(a), "forwarded"); + relay.checked(false, "checksum mismatch on receipt"); + + // The relay rewrites its own bundle to say the check passed. + let mut forged = relay.bundle(); + for row in forged.rows.iter_mut() { + if let Some(payload) = row.raw_payload.as_mut() { + *payload = payload.replace("\"satisfied\":false", "\"satisfied\":true"); + } + } + + let report = verify_annotations(&[origin.bundle(), forged], TAG); + + assert!( + report.problems.iter().any(|p| matches!( + p, + AnnotationProblem::BundleUnverified { .. } + )), + "editing a finding out of your own bundle must break its signature, \ + not quietly succeed; got {:?}", + report.problems + ); + assert!( + !report.intact(), + "and the result must not read as clean just because the evidence was \ + removed" + ); +} + +// --------------------------------------------------------------------- +// Credential formats. A WebAuthn credential must never be checked as a +// bare signature. +// --------------------------------------------------------------------- + +#[test] +fn a_cose_producer_is_refused_by_the_bare_signature_path() { + use trusty_core::KeyFormat; + + let mut trusty = Trusty::open_in_memory(SecretKey::new([4u8; 16])).expect("store"); + let signer = Ed25519Signer::generate(); + trusty + .append_signed("doc", &serde_json::json!({ "v": 1 }), "t://", &signer) + .expect("append"); + + // A user enrolled through WebAuthn: the credential is a COSE key, + // and its signatures cover authenticatorData || + // SHA-256(clientDataJSON) rather than the message. + trusty + .register_producer_with_format("user:01H", &[0u8; 32], &[0u8; 16], None, KeyFormat::Cose) + .expect("enrol"); + + let err = trusty + .verify_with_producer("doc", "user:01H") + .expect_err("must refuse"); + let msg = err.to_string(); + + assert!( + msg.contains("clientDataJSON"), + "the refusal must explain WHY, or the next person reads it as a wrong key and loosens the check until it passes: {msg}" + ); + assert!( + !msg.contains("verification failed"), + "and it must not look like a failed verification -- that confusion is what the change exists to remove: {msg}" + ); +} + +#[test] +fn a_raw_producer_is_still_verified_normally() { + use trusty_core::KeyFormat; + + let mut trusty = Trusty::open_in_memory(SecretKey::new([6u8; 16])).expect("store"); + let signer = Ed25519Signer::generate(); + trusty + .append_signed("doc", &serde_json::json!({ "v": 1 }), "t://", &signer) + .expect("append"); + trusty + .register_producer_with_format( + "agent:01J", + &[0u8; 32], + &[0u8; 16], + None, + KeyFormat::Ed25519Raw, + ) + .expect("enrol"); + + // May or may not verify -- the key is a placeholder. What matters is + // that it is not REFUSED: the raw path must be untouched, or this + // change would have broken every existing producer. + match trusty.verify_with_producer("doc", "agent:01J") { + Ok(_) => {} + Err(e) => assert!( + !e.to_string().contains("unsupported credential"), + "a raw Ed25519 producer must reach the signature check, not be turned away before it: {e}" + ), + } +} + +#[test] +fn a_producer_enrolled_before_the_column_existed_reads_as_raw() { + let mut trusty = Trusty::open_in_memory(SecretKey::new([5u8; 16])).expect("store"); + trusty + .register_producer("agent:old", &[0u8; 32], &[0u8; 16], None) + .expect("enrol"); + + let found = trusty + .lookup_producer("agent:old") + .expect("lookup") + .expect("row"); + assert_eq!( + found.key_format, + trusty_core::KeyFormat::Ed25519Raw, + "every credential enrolled before the column existed WAS a raw Ed25519 key, so the schema default is correct -- applied at write time, never as a parse fallback" + ); +} diff --git a/crates/adapters/trusty-sqlite/tests/conformance.rs b/crates/adapters/trusty-sqlite/tests/conformance.rs new file mode 100644 index 0000000..f7fd563 --- /dev/null +++ b/crates/adapters/trusty-sqlite/tests/conformance.rs @@ -0,0 +1,8 @@ +use trusty_sqlite::SqliteStore; +use trusty_store::testing::run_conformance; + +#[test] +fn sqlite_store_passes_conformance() { + let mut store = SqliteStore::open_in_memory().expect("open in-memory sqlite store"); + run_conformance(&mut store); +} diff --git a/crates/adapters/trusty-tokio/Cargo.toml b/crates/adapters/trusty-tokio/Cargo.toml new file mode 100644 index 0000000..63b1306 --- /dev/null +++ b/crates/adapters/trusty-tokio/Cargo.toml @@ -0,0 +1,16 @@ +[package] +name = "trusty-tokio" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "Runs a blocking trusty Store on a tokio runtime's blocking pool, so an async caller does not park a worker thread. An adapter, not a core concern: a runtime is a substrate." + +[dependencies] +trusty-store = { workspace = true } +tokio = { workspace = true, features = ["rt", "sync"] } + +[dev-dependencies] +tokio = { workspace = true, features = ["rt", "rt-multi-thread", "macros", "time", "sync"] } diff --git a/crates/adapters/trusty-tokio/src/lib.rs b/crates/adapters/trusty-tokio/src/lib.rs new file mode 100644 index 0000000..b04c981 --- /dev/null +++ b/crates/adapters/trusty-tokio/src/lib.rs @@ -0,0 +1,264 @@ +#![deny(clippy::all)] + +//! An async face for a blocking [`Store`]. +//! +//! # Why this is an adapter and not a core concern +//! +//! A runtime is a substrate. `trusty-core` touches no filesystem, clock, +//! thread, socket or process API, and `trusty-store` is four pure +//! dependencies; putting an optional tokio in either would give a crate +//! designed to have no platform a platform -- even switched off, it is +//! in the manifest and in the lockfile. +//! +//! So this lives beside `trusty-sqlite` and `trusty-postgres`, named for +//! its substrate the same way they are. A consumer on async-std, smol, +//! or no runtime at all pays nothing and is not told tokio is the +//! answer. +//! +//! # What this is, and what it is not +//! +//! This is async **ergonomics**, not async **I/O**. Every call still runs +//! the same blocking code on the same backend; it simply runs it on a +//! blocking pool instead of on the async runtime's worker threads. +//! SQLite writes still serialise, one at a time. A genuinely concurrent +//! backend would be a different adapter, not this one. +//! +//! Saying so plainly matters, because the failure this prevents and the +//! failure it does not prevent look alike from outside. +//! +//! # The problem it does solve +//! +//! An async runtime has a small pool of worker threads running every +//! task. A blocking call inside a task **parks a whole worker** for the +//! duration of a write and its fsync, during which that thread runs +//! nothing else. Enough concurrent appends and the runtime starves: +//! tasks with nothing to do with trusty simply stop making progress. +//! +//! Distant Star is the consumer this was waiting for -- it is tokio +//! throughout, and a relay writing a custody annotation does so inside +//! an async task while it is also moving packets. +//! +//! # Two things deliberately do not cross +//! +//! [`Store::iter_chain`] returns an iterator borrowing the store, and +//! [`Store::with_txn`] takes a closure that is not `'static`. Neither can +//! be sent to another thread, so neither is mirrored here: +//! +//! - **`chain_rows`** materialises the chain into a `Vec` instead of +//! streaming it. That is a real difference and not a free one -- the +//! result is bounded by memory rather than by patience -- so +//! [`AsyncStore::chain_rows_bounded`] exists for callers who would +//! rather cap it than find out. +//! - **`with_txn_owned`** takes an owned `FnOnce` that can be moved to +//! the blocking pool. Callers return values out of it directly rather +//! than capturing outer state by reference, because a reference to +//! outer state is exactly what cannot travel. + +pub mod pool; +pub use pool::BlockingPool; + +use std::sync::{Arc, Mutex}; + +use trusty_store::{ + KvCrossCheckReport, LastRow, RowInsert, Store, StoreError, StoreResult, StoredRow, TipRow, + TipUpsert, +}; + +/// The async surface. Mirrors [`Store`] where mirroring is possible, and +/// says so where it is not. +/// +/// Not dyn-compatible: `async fn` in a trait cannot be. Consumers that +/// need dynamic dispatch should hold [`BlockingStore`] concretely, or +/// wrap it themselves. +#[allow(async_fn_in_trait)] +pub trait AsyncStore { + async fn last_row(&self, entity_id: &str) -> StoreResult>; + async fn insert_row(&self, row: RowInsert) -> StoreResult; + async fn get_tip(&self, chain_id: &str) -> StoreResult>; + async fn get_tip_for_entity(&self, entity_id: &str) -> StoreResult>; + async fn set_tip(&self, tip: TipUpsert) -> StoreResult<()>; + async fn kv_cross_check(&self) -> StoreResult; + + /// The whole chain, materialised. + /// + /// Unlike [`Store::iter_chain`] this cannot stream: the iterator + /// borrows the store, and a borrow cannot be sent to a blocking + /// thread. Prefer [`AsyncStore::chain_rows_bounded`] on any chain + /// whose length is not known. + async fn chain_rows(&self, entity_id: &str) -> StoreResult>; + + /// At most `limit` rows, and whether more were left behind. + /// + /// Returning the "there is more" flag rather than silently stopping + /// is the whole point: a truncated history that looks complete is + /// how a verifier concludes a chain is intact when it has only seen + /// the part that is. + async fn chain_rows_bounded( + &self, + entity_id: &str, + limit: usize, + ) -> StoreResult<(Vec, bool)>; +} + +/// Runs a blocking [`Store`] on the runtime's blocking pool. +/// +/// Cloneable and shareable: every clone talks to the same store behind +/// the same lock, so concurrent callers serialise rather than race. That +/// is not a limitation this type imposes -- it is the backend's, made +/// visible instead of hidden. +pub struct BlockingStore { + inner: Arc>, +} + +impl Clone for BlockingStore { + fn clone(&self) -> Self { + Self { + inner: Arc::clone(&self.inner), + } + } +} + +impl BlockingStore { + pub fn new(store: S) -> Self { + Self { + inner: Arc::new(Mutex::new(store)), + } + } + + /// Take the store back. Fails if any clone is still alive. + pub fn into_inner(self) -> Result { + match Arc::try_unwrap(self.inner) { + Ok(mutex) => Ok(mutex.into_inner().unwrap_or_else(|e| e.into_inner())), + Err(inner) => Err(Self { inner }), + } + } + + /// Run an owned closure against the store on the blocking pool. + /// + /// The escape hatch for everything this facade does not mirror, + /// including transactions. The closure must be `'static` and `Send`, + /// which is not a formality: a closure borrowing local state is + /// precisely what cannot be moved to another thread. + pub async fn with_store(&self, f: F) -> StoreResult + where + F: FnOnce(&mut S) -> StoreResult + Send + 'static, + T: Send + 'static, + { + let inner = Arc::clone(&self.inner); + run_blocking(move || { + let mut guard = inner.lock().unwrap_or_else(|e| e.into_inner()); + f(&mut guard) + }) + .await + } + + /// A transaction, with its result carried out by value. + /// + /// [`Store::with_txn`] captures results through outer state because + /// its closure cannot return one. Here the closure is owned, so it + /// can -- which removes the need for the `Option` dance the blocking + /// API requires, and with it the chance of reading the captured slot + /// on a path where the transaction rolled back. + pub async fn with_txn_owned(&self, f: F) -> StoreResult + where + F: FnOnce(&mut dyn Store) -> StoreResult + Send + 'static, + T: Send + 'static, + { + self.with_store(move |store| { + let mut captured: Option = None; + let mut once = Some(f); + store.with_txn(&mut |txn: &mut dyn Store| { + let f = once.take().ok_or_else(|| { + StoreError::Backend( + "the backend invoked the transaction body more than once, which \ + this facade cannot support: an owned closure can only run once" + .to_string(), + ) + })?; + captured = Some(f(txn)?); + Ok(()) + })?; + captured.ok_or_else(|| { + StoreError::Backend( + "the transaction committed without the body having run".to_string(), + ) + }) + }) + .await + } +} + +/// Hand work to the blocking pool, and turn a panic there into an error +/// rather than letting it take the caller down. +/// +/// A panic inside a store operation is a bug, but a bug that unwinds +/// into an unrelated task is a much harder bug to find than one that +/// arrives as a `Backend` error at the call site that caused it. +async fn run_blocking(f: F) -> StoreResult +where + F: FnOnce() -> StoreResult + Send + 'static, + T: Send + 'static, +{ + match tokio::task::spawn_blocking(f).await { + Ok(result) => result, + Err(join) => Err(StoreError::Backend(format!( + "the store operation did not finish: {join}" + ))), + } +} + +impl AsyncStore for BlockingStore { + async fn last_row(&self, entity_id: &str) -> StoreResult> { + let entity_id = entity_id.to_string(); + self.with_store(move |s| s.last_row(&entity_id)).await + } + + async fn insert_row(&self, row: RowInsert) -> StoreResult { + self.with_store(move |s| s.insert_row(&row)).await + } + + async fn get_tip(&self, chain_id: &str) -> StoreResult> { + let chain_id = chain_id.to_string(); + self.with_store(move |s| s.get_tip(&chain_id)).await + } + + async fn get_tip_for_entity(&self, entity_id: &str) -> StoreResult> { + let entity_id = entity_id.to_string(); + self.with_store(move |s| s.get_tip_for_entity(&entity_id)) + .await + } + + async fn set_tip(&self, tip: TipUpsert) -> StoreResult<()> { + self.with_store(move |s| s.set_tip(&tip)).await + } + + async fn kv_cross_check(&self) -> StoreResult { + self.with_store(|s| s.kv_cross_check()).await + } + + async fn chain_rows(&self, entity_id: &str) -> StoreResult> { + let entity_id = entity_id.to_string(); + self.with_store(move |s| s.iter_chain(&entity_id)?.collect()) + .await + } + + async fn chain_rows_bounded( + &self, + entity_id: &str, + limit: usize, + ) -> StoreResult<(Vec, bool)> { + let entity_id = entity_id.to_string(); + self.with_store(move |s| { + let mut iter = s.iter_chain(&entity_id)?; + let mut rows = Vec::new(); + for row in iter.by_ref().take(limit) { + rows.push(row?); + } + // Asking for one more is how "there is more" is learned + // without walking the rest of a chain nobody asked for. + let more = iter.next().is_some(); + Ok((rows, more)) + }) + .await + } +} diff --git a/crates/adapters/trusty-tokio/src/pool.rs b/crates/adapters/trusty-tokio/src/pool.rs new file mode 100644 index 0000000..a2d5998 --- /dev/null +++ b/crates/adapters/trusty-tokio/src/pool.rs @@ -0,0 +1,282 @@ +//! A pool of stores, so concurrent callers are not queued behind one. +//! +//! # Why a pool rather than removing the lock +//! +//! [`crate::BlockingStore`] holds one store behind one mutex, so every +//! caller serialises. That looks like needless queuing to delete, and it +//! is not: `Store::insert_row(&mut self)` requires exclusive access by +//! its signature. The mutex is the trait's demand, not this type's +//! choice. +//! +//! Concurrency therefore needs **more stores**, not fewer locks. Each +//! holds its own connection, so two calls genuinely run at once. +//! +//! # The correctness this depends on +//! +//! Two connections appending to the same entity would race the chain +//! tip: appending is read-tip, compute, insert, and those are not +//! atomic. `trusty-postgres` holds a per-entity advisory lock across +//! that sequence, so same-entity writers serialise in the database while +//! different entities proceed in parallel. +//! +//! **That lock is a precondition of this type, not a detail.** Before it +//! existed, the single mutex was the only thing making concurrent +//! appends safe, and pooling without it would have converted a slowness +//! into a lost update -- a trade that reads as a performance win in a +//! diff. +//! +//! A backend with no such lock, or one writer per file like SQLite, +//! should keep using [`crate::BlockingStore`]. + +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::{Arc, Mutex}; + +use tokio::sync::Semaphore; +use trusty_store::{Store, StoreError, StoreResult}; + +/// Builds a fresh store, each with its own connection. +pub type Factory = Arc StoreResult + Send + Sync>; + +/// Runs blocking store work across several connections. +pub struct BlockingPool { + idle: Arc>>, + make: Factory, + /// Bounds concurrency to the connection budget. Without it, load + /// would open connections until the database refused them, and the + /// refusal would arrive as a failed append rather than as waiting. + permits: Arc, + live: Arc, + max: usize, +} + +impl Clone for BlockingPool { + fn clone(&self) -> Self { + Self { + idle: Arc::clone(&self.idle), + make: Arc::clone(&self.make), + permits: Arc::clone(&self.permits), + live: Arc::clone(&self.live), + max: self.max, + } + } +} + +impl BlockingPool { + /// A pool of at most `max` connections, opened on demand. + /// + /// Connections are created lazily rather than up front: a pool that + /// opens its full budget at startup turns a misconfigured connection + /// string into a failure at boot for capacity nobody has asked for + /// yet. + pub fn new(max: usize, make: impl Fn() -> StoreResult + Send + Sync + 'static) -> Self { + let max = max.max(1); + Self { + idle: Arc::new(Mutex::new(Vec::new())), + make: Arc::new(make), + permits: Arc::new(Semaphore::new(max)), + live: Arc::new(AtomicUsize::new(0)), + max, + } + } + + pub fn capacity(&self) -> usize { + self.max + } + + /// Connections currently open. Rises to `capacity` under load and + /// does not fall; a pool that closed idle connections would pay + /// reconnection cost exactly when traffic returned. + pub fn open_connections(&self) -> usize { + self.live.load(Ordering::SeqCst) + } + + /// Run `f` against a store from the pool. + /// + /// The store is moved into the blocking closure and handed back with + /// the result, so ownership never has to be shared and no lock is + /// held while the work runs. + /// + /// A store whose work returned an error is still returned to the + /// pool: a failed append usually means a rejected row, not a broken + /// connection, and discarding the connection would turn every + /// validation error into a reconnect. + pub async fn with(&self, f: F) -> StoreResult + where + F: FnOnce(&mut S) -> StoreResult + Send + 'static, + T: Send + 'static, + { + let _permit = self + .permits + .acquire() + .await + .map_err(|_| StoreError::Backend("the store pool has been closed".into()))?; + + let mut store = match self.idle.lock().unwrap_or_else(|e| e.into_inner()).pop() { + Some(s) => s, + None => { + let s = (self.make)()?; + self.live.fetch_add(1, Ordering::SeqCst); + s + } + }; + + let joined = tokio::task::spawn_blocking(move || { + let out = f(&mut store); + (store, out) + }) + .await; + + match joined { + Ok((store, out)) => { + self.idle + .lock() + .unwrap_or_else(|e| e.into_inner()) + .push(store); + out + } + Err(join) => { + // The store went down with the panicking task, so the + // connection is gone rather than merely idle. + self.live.fetch_sub(1, Ordering::SeqCst); + Err(StoreError::Backend(format!( + "the store operation did not finish and its connection was lost: {join}" + ))) + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::time::Duration; + use trusty_store::{KvCrossCheckReport, LastRow, RowInsert, StoredRow, TipRow, TipUpsert}; + + struct Slow { + id: usize, + delay: Duration, + } + + impl Store for Slow { + fn last_row(&self, _e: &str) -> StoreResult> { + std::thread::sleep(self.delay); + Ok(None) + } + fn insert_row(&mut self, _r: &RowInsert) -> StoreResult { + std::thread::sleep(self.delay); + Ok(self.id as i64) + } + fn iter_chain<'a>( + &'a self, + _e: &str, + ) -> StoreResult> + 'a>> { + Ok(Box::new(std::iter::empty())) + } + fn get_tip(&self, _c: &str) -> StoreResult> { + Ok(None) + } + fn get_tip_for_entity(&self, _e: &str) -> StoreResult> { + Ok(None) + } + fn set_tip(&mut self, _t: &TipUpsert) -> StoreResult<()> { + Ok(()) + } + fn kv_cross_check(&self) -> StoreResult { + Err(StoreError::Backend("not implemented".into())) + } + fn with_txn( + &mut self, + f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>, + ) -> StoreResult<()> { + let mut scratch = Slow { + id: self.id, + delay: Duration::ZERO, + }; + f(&mut scratch) + } + } + + fn pool(max: usize, delay_ms: u64) -> BlockingPool { + let next = Arc::new(AtomicUsize::new(0)); + BlockingPool::new(max, move || { + Ok(Slow { + id: next.fetch_add(1, Ordering::SeqCst), + delay: Duration::from_millis(delay_ms), + }) + }) + } + + /// **The claim.** Four slow calls across four connections finish in + /// far less than four times one call. + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn calls_run_concurrently_rather_than_queueing() { + let p = pool(4, 250); + let start = std::time::Instant::now(); + let mut hs = Vec::new(); + for _ in 0..4 { + let p = p.clone(); + hs.push(tokio::spawn(async move { p.with(|s| s.last_row("e")).await })); + } + for h in hs { + h.await.expect("join").expect("call"); + } + let elapsed = start.elapsed(); + assert!( + elapsed < Duration::from_millis(700), + "four 250ms calls on four connections took {elapsed:?}. Serialised \ + they would take about a second, which is what a single store \ + behind one mutex does and what this type exists to stop" + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn concurrency_is_bounded_by_the_connection_budget() { + let p = pool(2, 60); + let mut hs = Vec::new(); + for _ in 0..8 { + let p = p.clone(); + hs.push(tokio::spawn(async move { p.with(|s| s.last_row("e")).await })); + } + for h in hs { + h.await.expect("join").expect("call"); + } + assert!( + p.open_connections() <= 2, + "eight callers opened {} connections against a budget of 2. Without \ + the bound, load opens connections until the database refuses, and \ + the refusal arrives as a failed append rather than as waiting", + p.open_connections() + ); + } + + #[tokio::test] + async fn connections_are_reused_not_opened_per_call() { + let p = pool(4, 0); + for _ in 0..20 { + p.with(|s| s.last_row("e")).await.expect("call"); + } + assert_eq!( + p.open_connections(), + 1, + "sequential calls must reuse one connection; opening one per call \ + would make the pool slower than the mutex it replaces" + ); + } + + #[tokio::test] + async fn a_failed_call_returns_its_connection_to_the_pool() { + let p = pool(2, 0); + let _ = p + .with(|s| s.kv_cross_check().map(|_| ())) + .await + .expect_err("this store returns an error for that call"); + p.with(|s| s.last_row("e")).await.expect("still usable"); + assert_eq!( + p.open_connections(), + 1, + "a failed append usually means a rejected row, not a broken \ + connection; discarding it would turn every validation error into \ + a reconnect" + ); + } +} diff --git a/crates/adapters/trusty-tokio/tests/runtime.rs b/crates/adapters/trusty-tokio/tests/runtime.rs new file mode 100644 index 0000000..d167d79 --- /dev/null +++ b/crates/adapters/trusty-tokio/tests/runtime.rs @@ -0,0 +1,280 @@ +//! Does the async facade actually keep the runtime free? +//! +//! That is the only claim it makes, so it is the one worth testing. +//! Everything else here exists because it is a way the facade could be +//! quietly wrong: a lost value, a swallowed panic, a truncation that +//! reads as a complete chain. + +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::Arc; +use std::time::Duration; + +use trusty_store::{ + KvCrossCheckReport, LastRow, RowInsert, Store, StoreError, StoreResult, StoredRow, TipRow, + TipUpsert, +}; +use trusty_tokio::{AsyncStore, BlockingStore}; + +/// A store that is deliberately slow, so blocking is observable. +struct SlowStore { + delay: Duration, + rows: usize, + inserts: Arc, + panic_on_insert: bool, +} + +impl SlowStore { + fn new(delay_ms: u64) -> Self { + Self { + delay: Duration::from_millis(delay_ms), + rows: 0, + inserts: Arc::new(AtomicUsize::new(0)), + panic_on_insert: false, + } + } + + fn with_rows(mut self, rows: usize) -> Self { + self.rows = rows; + self + } +} + +fn stored_row(sequence: i64) -> StoredRow { + StoredRow { + id: sequence + 1, + entity_id: "e".into(), + chain_id: "c".into(), + event_type: "MUTATION".into(), + sequence, + prev_hash: String::new(), + row_hash: Some(format!("{sequence:064}")), + anchor_hash: None, + raw_payload: None, + detected_at: String::new(), + received_at: String::new(), + source_url: String::new(), + extraction_method: "trusty".into(), + } +} + +impl Store for SlowStore { + fn last_row(&self, _entity_id: &str) -> StoreResult> { + std::thread::sleep(self.delay); + Ok(Some(LastRow { + row_hash: "aa".into(), + sequence: 7, + })) + } + + fn insert_row(&mut self, _row: &RowInsert) -> StoreResult { + std::thread::sleep(self.delay); + if self.panic_on_insert { + panic!("a bug inside a store operation"); + } + Ok(self.inserts.fetch_add(1, Ordering::SeqCst) as i64 + 1) + } + + fn iter_chain<'a>( + &'a self, + _entity_id: &str, + ) -> StoreResult> + 'a>> { + Ok(Box::new((0..self.rows as i64).map(|i| Ok(stored_row(i))))) + } + + fn get_tip(&self, _chain_id: &str) -> StoreResult> { + Ok(None) + } + + fn get_tip_for_entity(&self, _entity_id: &str) -> StoreResult> { + Ok(None) + } + + fn set_tip(&mut self, _tip: &TipUpsert) -> StoreResult<()> { + Ok(()) + } + + fn kv_cross_check(&self) -> StoreResult { + Err(StoreError::Backend("not implemented for this test".into())) + } + + fn with_txn( + &mut self, + f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>, + ) -> StoreResult<()> { + let mut scratch = SlowStore::new(0); + f(&mut scratch) + } +} + +fn row_insert() -> RowInsert { + RowInsert { + entity_id: "e".into(), + chain_id: "c".into(), + event_type: "MUTATION".into(), + sequence: 0, + prev_hash: String::new(), + row_hash: None, + anchor_hash: None, + raw_payload: None, + detected_at: String::new(), + received_at: String::new(), + source_url: String::new(), + extraction_method: "trusty".into(), + } +} + +/// **The claim.** A single-worker runtime, one slow store call, and an +/// unrelated task that must keep running while it is in flight. +/// +/// Single-threaded on purpose: with more than one worker the test would +/// pass even if the call blocked, because another worker would pick the +/// second task up. One worker is the only configuration where blocking +/// is fatal and therefore the only one that proves anything. +#[tokio::test(flavor = "current_thread")] +async fn a_slow_store_call_does_not_park_the_runtime() { + let store = BlockingStore::new(SlowStore::new(300)); + let ticks = Arc::new(AtomicUsize::new(0)); + + let counter = Arc::clone(&ticks); + let ticker = tokio::spawn(async move { + for _ in 0..20 { + tokio::time::sleep(Duration::from_millis(10)).await; + counter.fetch_add(1, Ordering::SeqCst); + } + }); + + store.insert_row(row_insert()).await.expect("insert"); + let progressed = ticks.load(Ordering::SeqCst); + ticker.abort(); + + assert!( + progressed > 5, + "the other task must have kept running while the store was busy. \ + It ticked {progressed} times; blocking the worker would have left \ + it at or near zero, which is exactly the runtime starvation this \ + facade exists to prevent" + ); +} + +#[tokio::test(flavor = "current_thread")] +async fn concurrent_callers_serialise_rather_than_race() { + let store = BlockingStore::new(SlowStore::new(20)); + let mut handles = Vec::new(); + for _ in 0..8 { + let s = store.clone(); + handles.push(tokio::spawn(async move { s.insert_row(row_insert()).await })); + } + let mut ids = Vec::new(); + for h in handles { + ids.push(h.await.expect("join").expect("insert")); + } + ids.sort_unstable(); + + assert_eq!( + ids, + (1..=8).collect::>(), + "every caller must get its own row and none may be lost. The \ + backend serialises writes; that is its property, made visible \ + here rather than hidden" + ); +} + +#[tokio::test] +async fn a_panic_in_the_store_becomes_an_error_not_a_dead_task() { + let mut slow = SlowStore::new(0); + slow.panic_on_insert = true; + let store = BlockingStore::new(slow); + + let result = store.insert_row(row_insert()).await; + + assert!( + matches!(result, Err(StoreError::Backend(_))), + "a panic inside a store operation is a bug, but one that unwinds \ + into an unrelated task is far harder to find than one arriving as \ + an error at the call site that caused it" + ); +} + +#[tokio::test] +async fn a_bounded_read_says_when_it_left_something_behind() { + let store = BlockingStore::new(SlowStore::new(0).with_rows(10)); + + let (rows, more) = store.chain_rows_bounded("e", 4).await.expect("bounded"); + assert_eq!(rows.len(), 4); + assert!( + more, + "a truncated history that looks complete is how a verifier \ + concludes a chain is intact having seen only the part that is" + ); + + let (all, more) = store.chain_rows_bounded("e", 50).await.expect("bounded"); + assert_eq!(all.len(), 10); + assert!(!more, "with nothing left behind it must not claim there is"); +} + +#[tokio::test] +async fn the_whole_chain_can_still_be_read_when_that_is_wanted() { + let store = BlockingStore::new(SlowStore::new(0).with_rows(5)); + let rows = store.chain_rows("e").await.expect("chain"); + assert_eq!( + rows.len(), + 5, + "iter_chain cannot cross a thread boundary, so this materialises \ + instead of streaming -- but it must still return everything" + ); +} + +#[tokio::test] +async fn a_transaction_carries_its_result_out_by_value() { + let store = BlockingStore::new(SlowStore::new(0)); + + let answer = store + .with_txn_owned(|txn| { + txn.insert_row(&row_insert())?; + Ok("committed".to_string()) + }) + .await + .expect("txn"); + + assert_eq!( + answer, "committed", + "the blocking with_txn captures results through outer state \ + because its closure cannot return one; an owned closure can, \ + which removes the chance of reading a captured slot on a path \ + where the transaction rolled back" + ); +} + +#[tokio::test] +async fn a_failing_transaction_reports_the_failure_not_a_missing_value() { + let store = BlockingStore::new(SlowStore::new(0)); + + let result: StoreResult = store + .with_txn_owned(|_txn| Err(StoreError::Backend("deliberate".into()))) + .await; + + match result { + Err(StoreError::Backend(msg)) => assert_eq!( + msg, "deliberate", + "the body's own error must survive, not be replaced by a \ + complaint that no value was produced" + ), + other => panic!("expected the body's error, got {other:?}"), + } +} + +#[tokio::test] +async fn clones_share_one_store() { + let store = BlockingStore::new(SlowStore::new(0)); + let clone = store.clone(); + + store.insert_row(row_insert()).await.expect("first"); + let second = clone.insert_row(row_insert()).await.expect("second"); + + assert_eq!( + second, 2, + "a clone must talk to the same store, or two halves of an \ + application would be appending to different chains while \ + believing they shared one" + ); +} diff --git a/crates/core/trusty-core/Cargo.toml b/crates/core/trusty-core/Cargo.toml new file mode 100644 index 0000000..f57cfdb --- /dev/null +++ b/crates/core/trusty-core/Cargo.toml @@ -0,0 +1,39 @@ +[package] +name = "trusty-core" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "Tamper-evident data provenance chain primitives. Dual hash (SipHash-2-4 + SHA-256), chain-break-as-event. Framework neutral, no storage dependency." + +[features] +# On by default, so every existing consumer is unaffected and `rand` resolves +# exactly as before. Verified across MITCH-stack, AI-Playbooks, kirbo and +# good-ol-googz: nothing sets `default-features = false` on trusty-core, so +# this gate is invisible to all of them. +default = ["keygen"] + +# Generating a fresh Ed25519 key pair. This is the ONLY item in trusty-core +# that needs an operating-system RNG, and therefore the only thing standing +# between this crate and a `wasm32-unknown-unknown` build -- the rest of the +# crate uses no filesystem, clock, thread, socket, or process API at all. +# +# Verification needs no RNG, which is exactly what a browser peer wants. Build +# with `default-features = false` to get hashing, canonicalisation, chain +# construction, signature verification, `from_secret_bytes`, and ProofBundle, +# leaving key generation to the host (Web Crypto, a hardware token, or a key +# supplied from elsewhere). +keygen = ["dep:rand"] + +[dependencies] +serde = { workspace = true } +serde_json = { workspace = true } +sha2 = { workspace = true } +siphasher = { workspace = true } +zeroize = { workspace = true } +hex = { workspace = true } +thiserror = { workspace = true } +ed25519-dalek = { workspace = true } +rand = { workspace = true, optional = true } diff --git a/crates/core/trusty-core/src/annotation.rs b/crates/core/trusty-core/src/annotation.rs new file mode 100644 index 0000000..aac20fc --- /dev/null +++ b/crates/core/trusty-core/src/annotation.rs @@ -0,0 +1,696 @@ +//! Signed claims a node makes about a piece of data. +//! +//! # What this is for +//! +//! Encryption in Distant Star is per hop, not end to end -- a relaying +//! peer decrypts a chunk and re-encrypts it onward, so it holds +//! plaintext. Data crosses machines nobody controls, and without a +//! record there is no answer to "which systems touched this, and what +//! did they find?". That is the first question after a spill, and the +//! one DFARS 252.204-7012 and NIST SP 800-171 require an answer to for +//! CUI. +//! +//! # Kinds +//! +//! A node may attest **several different properties** about the same +//! datum, each checkable on its own: +//! +//! - [`AnnotationKind::Custody`] -- I held a copy, and had it from there. +//! - [`AnnotationKind::Integrity`] -- I ran a check; here is the result. +//! - [`AnnotationKind::Derivation`] -- I produced this from that. +//! - [`AnnotationKind::Source`] -- I attest where this originated. +//! +//! The shape follows Project Alvarium's annotation model (`kind`, +//! `is_satisfied`, `layer`). Alvarium's confidence **score** is +//! deliberately not followed: collapsing heterogeneous evidence into one +//! number says how much to worry but not *what failed*, and weighting a +//! TPM attestation against a checksum is a judgement dressed as +//! arithmetic. A consumer wanting a routing signal can score these; the +//! core reports per-property results. +//! +//! What is kept that Alvarium does not have: a **backward link**. +//! Alvarium annotations are independent facts whose ordering comes from +//! a ledger underneath. These carry [`SourceRef`], and trusty's +//! signature covers `prev_hash`, `sequence`, `entity_id` and `chain_id` +//! -- which is why a **gap** is detectable here, and why an annotation +//! cannot be replayed at a different position. +//! +//! # Annotation, not ledger +//! +//! Each node annotates its own copy on a chain of its own, +//! `entity_id = ":"`. Nobody writes to anybody else's, so a +//! node cannot forge an annotation it did not write. Recovering the +//! picture is a merge across bundles. +//! +//! A custody *chain* in the evidentiary sense presumes exclusivity -- +//! the bag is in exactly one place. Copies destroy that: when a node +//! passes data on, the sender still has it. So fan-out is the ordinary +//! case and the shape is a tree rooted at the origin. An annotation is a +//! claim by one party about itself, which fans out without +//! contradiction. (Unrelated to CUSTODY the containment framework, which +//! is not used here.) +//! +//! # Evidence, not enforcement +//! +//! Nothing here stops a node copying data, and a node that annotates +//! nothing is invisible -- no append-only log can record what was never +//! written. What an outside party holding only bundles does get: +//! annotations cannot be **forged**, gaps are **visible**, and history +//! cannot be **rewritten**. + +use std::collections::{HashMap, HashSet}; + +use serde::{Deserialize, Serialize}; +use serde_json::{json, Value}; + +use crate::bundle::{BundleRow, ProofBundle, SourceRef}; + +/// Discriminator written into `data.kind_of`. +/// +/// Versioned in the string: a reader must be able to tell which shape it +/// is holding before it trusts the fields. +pub const ANNOTATION_KIND: &str = "annotation.v1"; + +/// What property an annotation is about. +/// +/// `Other` exists rather than a parse failure so an annotation of a kind +/// this build does not know is **visible** instead of silently dropped. +/// A future kind carrying a failed check must not vanish because an old +/// verifier did not recognise the word. +#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum AnnotationKind { + /// This node held a copy, and had it from `received_from`. + Custody, + /// This node ran an integrity check; `satisfied` carries the result. + Integrity, + /// This node produced the datum from the cited source. + Derivation, + /// This node attests where the datum originated. + Source, + /// A kind this build does not know. Reported, never assumed benign. + #[serde(untagged)] + Other(String), +} + +impl AnnotationKind { + pub fn as_str(&self) -> &str { + match self { + Self::Custody => "custody", + Self::Integrity => "integrity", + Self::Derivation => "derivation", + Self::Source => "source", + Self::Other(s) => s, + } + } + + /// Whether this build understands the kind well enough to act on it. + pub fn is_known(&self) -> bool { + !matches!(self, Self::Other(_)) + } +} + +/// Where in the stack the claim was inserted. +/// +/// Tells a verifier what an annotation is worth: a check run in the +/// application is a weaker statement than the same check run in +/// hardware, and the two should not read alike. +#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum Layer { + Device, + Host, + Os, + App, + Network, + #[serde(untagged)] + Other(String), +} + +impl Layer { + pub fn as_str(&self) -> &str { + match self { + Self::Device => "device", + Self::Host => "host", + Self::Os => "os", + Self::App => "app", + Self::Network => "network", + Self::Other(s) => s, + } + } +} + +impl Default for Layer { + fn default() -> Self { + Self::App + } +} + +/// One node's signed claim about one datum. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct Annotation { + /// What property this claims. See [`AnnotationKind`]. + pub kind: AnnotationKind, + /// What the claim is about. Opaque; a content hash is the obvious + /// choice. + pub content_tag: String, + /// Who is claiming. The owner of the chain this row is on. + pub node: String, + /// Where in the stack the claim was inserted. + #[serde(default)] + pub layer: Layer, + /// Whether a check ran, and what it concluded. + /// + /// `None` means **no check was performed**, which is not the same as + /// a check that failed. A custody claim asserts holding, not + /// correctness, and must not read as a passed check. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub satisfied: Option, + /// The upstream row this claim refers to. `None` at an origin. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub received_from: Option, + /// Free text, in the `_why` tradition. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub note: Option, +} + +impl Annotation { + /// A custody claim: this node held a copy. + pub fn custody(content_tag: impl Into, node: impl Into) -> Self { + Self { + kind: AnnotationKind::Custody, + content_tag: content_tag.into(), + node: node.into(), + layer: Layer::default(), + satisfied: None, + received_from: None, + note: None, + } + } + + /// An integrity claim: this node ran a check and it came out `ok`. + pub fn integrity( + content_tag: impl Into, + node: impl Into, + ok: bool, + ) -> Self { + Self { + kind: AnnotationKind::Integrity, + satisfied: Some(ok), + ..Self::custody(content_tag, node) + } + } + + pub fn from(mut self, source: SourceRef) -> Self { + self.received_from = Some(source); + self + } + + pub fn at_layer(mut self, layer: Layer) -> Self { + self.layer = layer; + self + } + + pub fn because(mut self, note: impl Into) -> Self { + self.note = Some(note.into()); + self + } + + /// The value to write into the row's `data` field. + pub fn to_data(&self) -> Value { + let mut v = json!({ + "kind_of": ANNOTATION_KIND, + "kind": self.kind.as_str(), + "content_tag": self.content_tag, + "node": self.node, + "layer": self.layer.as_str(), + }); + if let Some(satisfied) = self.satisfied { + v["satisfied"] = Value::Bool(satisfied); + } + if let Some(from) = &self.received_from { + v["received_from"] = serde_json::to_value(from).unwrap_or(Value::Null); + } + if let Some(note) = &self.note { + v["note"] = Value::String(note.clone()); + } + v + } + + /// Recover an annotation from a bundle row, if it is one. + /// + /// `None` for any row that is not a well-formed annotation of a + /// version this code knows. A row that is *nearly* an annotation is + /// not treated as one: a partially understood claim is worse than an + /// unread one, because it looks like evidence. + /// + /// An unrecognised **kind** is different, and is kept as + /// [`AnnotationKind::Other`] so it can be reported. + pub fn from_row(row: &BundleRow) -> Option { + let payload: Value = serde_json::from_str(row.raw_payload.as_deref()?).ok()?; + let data = payload.get("data")?; + if data.get("kind_of").and_then(Value::as_str)? != ANNOTATION_KIND { + return None; + } + let kind = match data.get("kind").and_then(Value::as_str)? { + "custody" => AnnotationKind::Custody, + "integrity" => AnnotationKind::Integrity, + "derivation" => AnnotationKind::Derivation, + "source" => AnnotationKind::Source, + other => AnnotationKind::Other(other.to_string()), + }; + let layer = match data.get("layer").and_then(Value::as_str).unwrap_or("app") { + "device" => Layer::Device, + "host" => Layer::Host, + "os" => Layer::Os, + "app" => Layer::App, + "network" => Layer::Network, + other => Layer::Other(other.to_string()), + }; + Some(Self { + kind, + content_tag: data.get("content_tag")?.as_str()?.to_string(), + node: data.get("node")?.as_str()?.to_string(), + layer, + satisfied: data.get("satisfied").and_then(Value::as_bool), + received_from: data + .get("received_from") + .and_then(|v| serde_json::from_value(v.clone()).ok()), + note: data.get("note").and_then(Value::as_str).map(str::to_string), + }) + } +} + +/// The `entity_id` a node uses for its own claims about one datum. +/// +/// Both halves matter: without the tag a node would keep one chain for +/// everything it ever handled, and a bundle exported for one recipient +/// would carry every other recipient's traffic with it. +pub fn entity_id_for(content_tag: &str, node: &str) -> String { + format!("{content_tag}:{node}") +} + +/// One custody hop on a reconstructed path. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct CustodyHop { + pub node: String, + pub chain_id: String, + pub sequence: i64, + pub row_hash: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub received_from: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub note: Option, +} + +/// Something wrong, named precisely enough to act on. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "problem", rename_all = "snake_case")] +pub enum AnnotationProblem { + /// A bundle failed its own verification, so nothing it says counts. + BundleUnverified { entity_id: String, detail: String }, + /// **A node ran a check and it did not pass.** The actionable + /// marker: whoever reads this can refuse, quarantine, or inspect. + CheckFailed { + node: String, + kind: String, + layer: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + note: Option, + }, + /// An annotation of a kind this build does not understand. Reported + /// rather than ignored: silence would let a future kind carrying a + /// failed check pass an old verifier unnoticed. + UnrecognisedKind { node: String, kind: String }, + /// A claim naming an upstream row absent from every bundle given, or + /// present with a different `row_hash`. + DanglingPredecessor { node: String, source_ref: SourceRef }, + /// Two nodes claim to have received the datum from the same row. Not + /// wrong -- data fans out -- but the path is a tree, not a line. + Fork { + source_ref: SourceRef, + claimants: Vec, + }, + /// No custody claim is an origin: a cycle, or a missing first claim. + NoOrigin, + /// More than one node claims to be the origin of the same datum. + MultipleOrigins { nodes: Vec }, + /// A hop not reachable from the origin by following links. + Unreachable { node: String }, +} + +/// What the bundles, taken together, say about a datum. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AnnotationReport { + pub content_tag: String, + /// Custody hops reachable from the origin, in order. Empty when + /// there is no single origin to order them from. + pub hops: Vec, + /// Every annotation found, of every kind. + pub found: usize, + pub problems: Vec, +} + +impl AnnotationReport { + /// One origin, every link resolving, nothing unreachable, no fork, + /// no failed check, and every bundle verifying on its own. + /// + /// A method rather than a stored field: a caller who accepts forks + /// can read `problems` and decide, and a stored boolean invites + /// reading the verdict without the reasons. + pub fn intact(&self) -> bool { + self.problems.is_empty() && !self.hops.is_empty() + } + + /// The nodes that held the datum, in order. + pub fn custody_line(&self) -> Vec<&str> { + self.hops.iter().map(|h| h.node.as_str()).collect() + } + + /// Every check that a node reported as failed. + /// + /// The marker another node acts on: refuse to serve, quarantine, or + /// route for inspection. + pub fn failed_checks(&self) -> Vec<&AnnotationProblem> { + self.problems + .iter() + .filter(|p| matches!(p, AnnotationProblem::CheckFailed { .. })) + .collect() + } +} + +/// Read every annotation about a datum, and check what they say together. +/// +/// **Verifies each bundle first.** A structural finding drawn from rows +/// nobody checked is not a finding, and a tidy report over forged rows +/// is worse than no report. A bundle that fails is named and its rows +/// take no further part -- but one party lying does not erase another's +/// honest record. +pub fn verify_annotations(bundles: &[ProofBundle], content_tag: &str) -> AnnotationReport { + let mut problems = Vec::new(); + let mut hops: Vec = Vec::new(); + let mut rows_by_key: HashMap<(String, i64), String> = HashMap::new(); + let mut found = 0usize; + + for bundle in bundles { + let report = bundle.verify_public(); + if !report.intact() { + problems.push(AnnotationProblem::BundleUnverified { + entity_id: bundle.entity_id.clone(), + detail: format!( + "rows_valid={}, fully_signed={}, continuous={}", + report.rows_valid, report.fully_signed, report.continuous + ), + }); + continue; + } + + for row in &bundle.rows { + if let Some(hash) = &row.row_hash { + rows_by_key.insert((row.chain_id.clone(), row.sequence), hash.clone()); + } + let Some(annotation) = Annotation::from_row(row) else { + continue; + }; + if annotation.content_tag != content_tag { + continue; + } + found += 1; + + if annotation.satisfied == Some(false) { + problems.push(AnnotationProblem::CheckFailed { + node: annotation.node.clone(), + kind: annotation.kind.as_str().to_string(), + layer: annotation.layer.as_str().to_string(), + note: annotation.note.clone(), + }); + } + if !annotation.kind.is_known() { + problems.push(AnnotationProblem::UnrecognisedKind { + node: annotation.node.clone(), + kind: annotation.kind.as_str().to_string(), + }); + } + if annotation.kind == AnnotationKind::Custody { + hops.push(CustodyHop { + node: annotation.node, + chain_id: row.chain_id.clone(), + sequence: row.sequence, + row_hash: row.row_hash.clone(), + received_from: annotation.received_from, + note: annotation.note, + }); + } + } + } + + // A claimed predecessor must exist, at that position, with that hash. + for hop in &hops { + if let Some(sr) = &hop.received_from { + match rows_by_key.get(&(sr.chain_id.clone(), sr.sequence)) { + Some(hash) if *hash == sr.row_hash => {} + _ => problems.push(AnnotationProblem::DanglingPredecessor { + node: hop.node.clone(), + source_ref: sr.clone(), + }), + } + } + } + + // Fan-out: two hops naming the same upstream row. + let mut by_predecessor: HashMap<(String, i64), Vec> = HashMap::new(); + for hop in &hops { + if let Some(sr) = &hop.received_from { + by_predecessor + .entry((sr.chain_id.clone(), sr.sequence)) + .or_default() + .push(hop.node.clone()); + } + } + for ((chain_id, sequence), claimants) in &by_predecessor { + if claimants.len() > 1 { + problems.push(AnnotationProblem::Fork { + source_ref: SourceRef { + chain_id: chain_id.clone(), + sequence: *sequence, + row_hash: rows_by_key + .get(&(chain_id.clone(), *sequence)) + .cloned() + .unwrap_or_default(), + }, + claimants: claimants.clone(), + }); + } + } + + let origins: Vec<&CustodyHop> = hops.iter().filter(|h| h.received_from.is_none()).collect(); + match origins.len() { + 0 => { + if !hops.is_empty() { + problems.push(AnnotationProblem::NoOrigin); + } + } + 1 => {} + _ => problems.push(AnnotationProblem::MultipleOrigins { + nodes: origins.iter().map(|h| h.node.clone()).collect(), + }), + } + + let ordered = if origins.len() == 1 { + let mut successors: HashMap<(String, i64), Vec> = HashMap::new(); + for hop in &hops { + if let Some(sr) = &hop.received_from { + successors + .entry((sr.chain_id.clone(), sr.sequence)) + .or_default() + .push(hop.clone()); + } + } + let mut ordered = vec![origins[0].clone()]; + let mut seen: HashSet<(String, i64)> = HashSet::new(); + let mut cursor = (origins[0].chain_id.clone(), origins[0].sequence); + // Follows one line. A fork is already reported; taking the first + // branch leaves the caller a readable prefix rather than nothing. + while seen.insert(cursor.clone()) { + let Some(next) = successors.get(&cursor).and_then(|v| v.first()) else { + break; + }; + ordered.push(next.clone()); + cursor = (next.chain_id.clone(), next.sequence); + } + ordered + } else { + Vec::new() + }; + + let reached: HashSet<(String, i64)> = ordered + .iter() + .map(|h| (h.chain_id.clone(), h.sequence)) + .collect(); + if !ordered.is_empty() { + for hop in &hops { + if !reached.contains(&(hop.chain_id.clone(), hop.sequence)) { + problems.push(AnnotationProblem::Unreachable { + node: hop.node.clone(), + }); + } + } + } + + AnnotationReport { + content_tag: content_tag.to_string(), + hops: ordered, + found, + problems, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn row(chain: &str, seq: i64, hash: &str, a: &Annotation) -> BundleRow { + BundleRow { + entity_id: entity_id_for(&a.content_tag, &a.node), + chain_id: chain.to_string(), + event_type: "MUTATION".to_string(), + sequence: seq, + prev_hash: String::new(), + row_hash: Some(hash.to_string()), + anchor_hash: None, + raw_payload: Some( + serde_json::to_string(&json!({ "data": a.to_data() })).expect("payload"), + ), + break_reason: None, + } + } + + #[test] + fn an_entity_id_carries_both_the_datum_and_the_node() { + assert_eq!(entity_id_for("sha256:abc", "node-1"), "sha256:abc:node-1"); + assert_ne!( + entity_id_for("sha256:abc", "node-1"), + entity_id_for("sha256:abc", "node-2"), + "one chain per (data, node) is what lets a node sign only its own \ + claims; sharing a chain would need a writer they both trust" + ); + } + + #[test] + fn every_field_round_trips_through_the_data_field() { + let a = Annotation::integrity("sha256:abc", "node-2", false) + .at_layer(Layer::Device) + .because("checksum mismatch on receipt") + .from(SourceRef { + chain_id: "c1".into(), + sequence: 4, + row_hash: "aa".into(), + }); + let parsed = Annotation::from_row(&row("c2", 1, "bb", &a)).expect("parse"); + assert_eq!(parsed, a, "what is written must be what is read back"); + } + + #[test] + fn no_check_is_not_the_same_as_a_passed_check() { + let custody = Annotation::custody("sha256:abc", "n"); + assert_eq!( + custody.satisfied, None, + "a custody claim asserts holding, not correctness, and must not \ + read as a check that passed" + ); + assert_eq!(Annotation::integrity("sha256:abc", "n", true).satisfied, Some(true)); + assert_eq!(Annotation::integrity("sha256:abc", "n", false).satisfied, Some(false)); + } + + #[test] + fn an_unknown_kind_survives_parsing_so_it_can_be_reported() { + let mut a = Annotation::custody("sha256:abc", "n"); + a.kind = AnnotationKind::Other("future.kind".into()); + let parsed = Annotation::from_row(&row("c", 0, "aa", &a)).expect("parse"); + assert_eq!(parsed.kind, AnnotationKind::Other("future.kind".into())); + assert!( + !parsed.kind.is_known(), + "a kind an old verifier does not recognise must be visible, or a \ + future kind carrying a failed check would pass unnoticed" + ); + } + + #[test] + fn an_unknown_layer_is_kept_rather_than_flattened_to_the_default() { + let a = Annotation::custody("sha256:abc", "n").at_layer(Layer::Other("enclave".into())); + let parsed = Annotation::from_row(&row("c", 0, "aa", &a)).expect("parse"); + assert_eq!( + parsed.layer, + Layer::Other("enclave".into()), + "silently reporting an unknown layer as `app` would understate \ + where the claim came from, and layer exists precisely to say \ + what a claim is worth" + ); + } + + #[test] + fn a_row_that_is_not_an_annotation_is_not_read_as_one() { + let a = Annotation::custody("sha256:abc", "n"); + for payload in [ + r#"{"data":{"kind_of":"something.else"}}"#, + r#"{"data":{"kind_of":"annotation.v1"}}"#, + "not json at all", + ] { + let mut r = row("c", 0, "aa", &a); + r.raw_payload = Some(payload.to_string()); + assert!( + Annotation::from_row(&r).is_none(), + "a partially understood claim is worse than an unread one, \ + because it looks like evidence: {payload}" + ); + } + } + + #[test] + fn the_version_is_inside_the_discriminator() { + assert!( + ANNOTATION_KIND.ends_with(".v1"), + "a reader must be able to tell which shape it is holding before \ + it trusts the fields" + ); + } + + #[test] + fn an_empty_report_is_not_an_intact_one() { + let report = AnnotationReport { + content_tag: "sha256:abc".into(), + hops: vec![], + found: 0, + problems: vec![], + }; + assert!( + !report.intact(), + "no records at all is not the same as a clean history, and the \ + difference matters most exactly when someone is asking" + ); + } + + #[test] + fn failed_checks_are_singled_out_for_acting_on() { + let report = AnnotationReport { + content_tag: "t".into(), + hops: vec![], + found: 2, + problems: vec![ + AnnotationProblem::NoOrigin, + AnnotationProblem::CheckFailed { + node: "relay".into(), + kind: "integrity".into(), + layer: "device".into(), + note: None, + }, + ], + }; + assert_eq!( + report.failed_checks().len(), + 1, + "a caller deciding whether to refuse, quarantine or inspect needs \ + the failed checks separated from the structural findings" + ); + } +} diff --git a/crates/core/trusty-core/src/bundle.rs b/crates/core/trusty-core/src/bundle.rs new file mode 100644 index 0000000..a9b83e9 --- /dev/null +++ b/crates/core/trusty-core/src/bundle.rs @@ -0,0 +1,799 @@ +// ============================================================ +// MODULE: bundle +// PURPOSE: Self-contained proof packages for external verification. +// A ProofBundle lets a party without database access +// (auditor, insurer, carbon-credit buyer) reverify a +// slice of a chain, given the operator's SipHash key +// (shared out-of-band with trusted verifiers) and/or +// the Ed25519 public keys embedded inline in v2 rows. +// +// DESIGN NOTES: +// - Wire format is JSON via serde. The bundle itself is NOT +// hashed as a whole; integrity comes from the row_hash + +// anchor_hash + signature fields of each contained row. +// - ChainLink stays internal (no serde derives) to keep the +// chain primitive untouched. BundleRow mirrors its shape and +// carries the serde derives. Conversion is free. +// - SipHash key is NEVER embedded. Key distribution is the +// operator's concern. +// ============================================================ +// +// v1 SHAPE (this module is additive, does not alter payload bytes): +// AnchorEntry { sequence, anchor_hash, detected_at } +// BundleRow { entity_id, chain_id, event_type, sequence, +// prev_hash, row_hash, anchor_hash, raw_payload } +// ProofBundle { entity_id, generated_at, row_count, +// sequence_range, rows, anchors } +// +// ============================================================ + +use serde::{Deserialize, Serialize}; +use serde_json::Value; + +use crate::chain::{ + verify_links, verify_links_public, verify_row_signatures, ChainLink, PublicVerifyReport, + VerifyReport, +}; +use crate::hash::SecretKey; + +/// One per-row anchor commitment, as emitted by +/// `Trusty::anchors_since`. `anchor_hash` is the SHA-256 commitment +/// written at append time; `detected_at` is the ISO-8601 timestamp +/// the operator recorded. Anchor entries are what an operator +/// publishes (or pushes to a buyer / insurer) for external spot +/// checks without exposing the row contents themselves. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +pub struct AnchorEntry { + pub sequence: i64, + pub anchor_hash: String, + pub detected_at: String, +} + +/// Wire-format row carried inside a `ProofBundle`. Mirrors the +/// in-memory `ChainLink` exactly; the only reason for a separate +/// type is to carry serde derives without touching the chain +/// primitive. Round-trip: `From<&ChainLink>` to serialize, then +/// `to_chain_link` on the verifier side to feed `verify_links`. +#[non_exhaustive] +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +pub struct BundleRow { + pub entity_id: String, + pub chain_id: String, + pub event_type: String, + pub sequence: i64, + pub prev_hash: String, + pub row_hash: Option, + pub anchor_hash: Option, + pub raw_payload: Option, + /// Why a BREAK row exists, for BREAK rows only. + /// + /// Additive, and `None` everywhere else. Without it a reader holding + /// only a bundle learns THAT a chain broke and not WHICH rule broke + /// -- the reason lives in a column the bundle did not carry, so it + /// stopped at the machine that recorded it. A marker nobody else can + /// read is not a marker anybody can act on. + /// + /// Safe to add: BREAK rows are skipped in hash verification, so this + /// was never part of any hashed payload, and `skip_serializing_if` + /// keeps bundles without one byte-identical to before. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub break_reason: Option, +} + + +impl BundleRow { + /// Build a row from the fields every backend has. + /// + /// `BundleRow` is `#[non_exhaustive]`, so it cannot be built with a + /// struct literal outside this crate. That is deliberate and it is + /// the lesson of 2026-08-25: adding `break_reason` broke a consumer + /// that constructs one, while this workspace's own tests stayed + /// green, because the consumer depends on trusty by PATH and nothing + /// pins a version to warn anybody. + /// + /// Adding a field is additive on the wire -- serde skips what is + /// absent -- and NOT additive for struct literals. Those are + /// different compatibility questions, and conflating them is what + /// made the break silent. A constructor plus setters means the next + /// field costs downstream nothing. + pub fn new( + entity_id: impl Into, + chain_id: impl Into, + event_type: impl Into, + sequence: i64, + prev_hash: impl Into, + ) -> Self { + Self { + entity_id: entity_id.into(), + chain_id: chain_id.into(), + event_type: event_type.into(), + sequence, + prev_hash: prev_hash.into(), + row_hash: None, + anchor_hash: None, + raw_payload: None, + break_reason: None, + } + } + + pub fn with_hashes(mut self, row_hash: Option, anchor_hash: Option) -> Self { + self.row_hash = row_hash; + self.anchor_hash = anchor_hash; + self + } + + pub fn with_payload(mut self, raw_payload: Option) -> Self { + self.raw_payload = raw_payload; + self + } + + /// Why a BREAK row exists. Ignored for any other event type, so a + /// caller threading it through generically cannot mislabel a row. + pub fn with_break_reason(mut self, reason: Option) -> Self { + if self.event_type == "BREAK" { + self.break_reason = reason; + } + self + } +} + +impl BundleRow { + pub fn to_chain_link(&self) -> ChainLink { + ChainLink { + entity_id: self.entity_id.clone(), + chain_id: self.chain_id.clone(), + event_type: self.event_type.clone(), + sequence: self.sequence, + prev_hash: self.prev_hash.clone(), + row_hash: self.row_hash.clone(), + anchor_hash: self.anchor_hash.clone(), + raw_payload: self.raw_payload.clone(), + } + } +} + +impl From<&ChainLink> for BundleRow { + fn from(link: &ChainLink) -> Self { + Self { + entity_id: link.entity_id.clone(), + chain_id: link.chain_id.clone(), + event_type: link.event_type.clone(), + sequence: link.sequence, + prev_hash: link.prev_hash.clone(), + row_hash: link.row_hash.clone(), + anchor_hash: link.anchor_hash.clone(), + raw_payload: link.raw_payload.clone(), + // A ChainLink has no break reason to give; only a store that + // read the row from its columns can supply one. + break_reason: None, + } + } +} + +/// Self-contained proof package. Packages a slice of a chain plus +/// its anchor commitments into a single serializable blob. The +/// bundle alone is not cryptographically self-verifying -- the +/// caller must supply the operator's SipHash key to reproduce the +/// keyed row_hash values. v2 rows additionally carry their own +/// Ed25519 public keys inline, so their signatures are verifiable +/// with no external input. +/// +/// `sequence_range` reports the min/max `sequence` seen across the +/// non-BREAK rows in the bundle. BREAK rows carry sequence -1 and +/// are excluded from the range but included in the bundle so +/// chain-break-as-event proof survives export. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct ProofBundle { + pub entity_id: String, + pub generated_at: String, + pub row_count: usize, + pub sequence_range: (Option, Option), + pub rows: Vec, + pub anchors: Vec, +} + +/// One `source_refs` entry embedded in a twin computation row's data field. +/// Carries enough info to locate and validate the referenced sensor row +/// in a separate `ProofBundle` without database access. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +pub struct SourceRef { + pub chain_id: String, + pub sequence: i64, + pub row_hash: String, +} + +/// One failed cross-chain reference found by `ProofBundle::verify_source_refs`. +/// `reason` is either `"not_found"` (no sensor row at that sequence) or +/// `"hash_mismatch"` (row found but row_hash differs). +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +pub struct SourceRefMismatch { + pub twin_sequence: i64, + pub source_ref: SourceRef, + pub reason: String, +} + +/// Result of `ProofBundle::verify_source_refs`. `ok` is `true` when every +/// source_ref in the twin bundle resolves to a matching row in the sensor bundle. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct SourceRefReport { + pub checked: usize, + pub matched: usize, + pub mismatches: Vec, + pub ok: bool, +} + +/// Proof that one producer signed two different rows at the same position. +/// +/// Both rows carry a valid signature from the same key over the same +/// `(chain_id, sequence)`, and their `row_hash` values differ. That cannot +/// happen by accident: the signature commits to `prev_hash`, `sequence`, +/// `entity_id` and `chain_id`, so producing two is a deliberate act of showing +/// different histories to different parties. +/// +/// This is a stronger result than "the chain looks wrong". A broken chain might +/// be corruption; an equivocation is attributable to a public key, and the +/// evidence is self-contained -- a third party can re-verify both signatures +/// without trusting whoever reported it. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +pub struct EquivocationProof { + pub chain_id: String, + pub sequence: i64, + /// The public key, hex-encoded, that signed both rows. + pub producer_pk: String, + pub row_a: BundleRow, + pub row_b: BundleRow, +} + +/// Compare two bundles for the same entity and return proof of every position +/// where the producer signed two different rows. +/// +/// **Signature validity is required on both sides.** An unsigned or +/// invalidly-signed row proves nothing about its author, and admitting one +/// would let anyone manufacture a "proof" against a producer by fabricating the +/// second half. Rows that do not verify are skipped rather than reported. +/// +/// Detects nothing when the two bundles are the same history, which is the +/// common case; the cost is one signature verification per row per side. +pub fn detect_equivocation(a: &ProofBundle, b: &ProofBundle) -> Vec { + use std::collections::HashMap; + + // Index one side by position. BREAK rows carry no row_hash and sequence -1, + // so they cannot equivocate and are excluded. + let mut left: HashMap<(&str, i64), &BundleRow> = HashMap::new(); + for row in &a.rows { + if row.event_type == "BREAK" || row.row_hash.is_none() { + continue; + } + left.insert((row.chain_id.as_str(), row.sequence), row); + } + + let mut out = Vec::new(); + for right in &b.rows { + if right.event_type == "BREAK" || right.row_hash.is_none() { + continue; + } + let Some(&leftr) = left.get(&(right.chain_id.as_str(), right.sequence)) else { + continue; + }; + if leftr.row_hash == right.row_hash { + continue; // same row; nothing to see + } + + // Both must be validly signed BY THE SAME KEY, or this proves nothing. + let (Some(raw_a), Some(raw_b)) = (&leftr.raw_payload, &right.raw_payload) else { + continue; + }; + if !verify_row_signatures(raw_a).is_empty() || !verify_row_signatures(raw_b).is_empty() { + continue; + } + let (Some(pk_a), Some(pk_b)) = (first_signature_pk(raw_a), first_signature_pk(raw_b)) + else { + continue; + }; + if pk_a != pk_b { + continue; // two producers disagreeing is not one producer equivocating + } + + out.push(EquivocationProof { + chain_id: right.chain_id.clone(), + sequence: right.sequence, + producer_pk: pk_a, + row_a: (*leftr).clone(), + row_b: right.clone(), + }); + } + out +} + +// Hex public key of the first signature entry in a raw payload, if any. +fn first_signature_pk(raw: &str) -> Option { + let parsed = crate::chain::parse_raw_payload(raw).ok()?; + parsed + .signatures + .first()? + .as_object()? + .get("pk")? + .as_str() + .map(str::to_string) +} + +impl ProofBundle { + /// Reverify the bundle's chain slice. Runs the same + /// `verify_links` pipeline that `Trusty::verify` runs on a live + /// database. + /// + /// CHECK TWO FIELDS, NOT ONE. `report.rows_valid` says the rows hashed and + /// signed correctly; `report.continuous` says no break was recorded + /// in the range, with `report.breaks` giving the positions. A bundle + /// can be `ok` and NOT continuous -- the segments either side of a + /// break are well-formed, and a break is written precisely because a + /// monitored condition failed at append time. + /// + /// This doc comment used to say "inspect `report.rows_valid` for the + /// summary", which is how a third party ended up able to export a + /// bundle, verify it, read `ok: true` and conclude a chain was + /// unbroken when it carried the system's own record of detected + /// tampering. `report.errors` still holds the detail when `ok` is + /// false. + pub fn verify(&self, key: &SecretKey) -> VerifyReport { + let links: Vec = self.rows.iter().map(BundleRow::to_chain_link).collect(); + verify_links(&self.entity_id, &links, key) + } + + /// Reverify using only what travels inside the bundle -- no secret key. + /// + /// This is the path this module header promises to a party holding a + /// bundle and nothing else: Ed25519 signatures (public keys are inline in + /// v2+ rows), plus linkage and anchor consistency. It previously had no + /// public entry point, so a recipient without the operator SipHash key + /// could not verify a bundle at all -- which is the audience a bundle + /// exists for. + /// + /// Read `ok` for whether everything is accounted for and `continuous` + /// for whether a break was recorded, exactly as with [`Self::verify`]. + /// Note `ok` is false when any row is unsigned: without the key such a + /// row is unattributable. + pub fn verify_public(&self) -> PublicVerifyReport { + let links: Vec = self.rows.iter().map(BundleRow::to_chain_link).collect(); + verify_links_public(&self.entity_id, &links) + } + + /// Cross-chain reference verification. `self` is the twin bundle; + /// `sensor_bundle` is the bundle that should contain the referenced rows. + /// + /// Walks every twin row whose `raw_payload` contains a `data.source_refs` + /// array (the twin coupling convention from docs/twin-coupling.md). For each + /// `{chain_id, sequence, row_hash}` ref, confirms a matching sensor row + /// exists in `sensor_bundle` at that sequence with an identical `row_hash`. + /// + /// Returns a `SourceRefReport`. `ok` is `true` when all refs resolve. + /// Unresolved refs appear in `mismatches` with reason `"not_found"` or + /// `"hash_mismatch"`. + pub fn verify_source_refs(&self, sensor_bundle: &ProofBundle) -> SourceRefReport { + // Index sensor rows by sequence for O(1) lookup. + let sensor_index: std::collections::HashMap = sensor_bundle + .rows + .iter() + .filter(|r| r.sequence >= 0) + .map(|r| (r.sequence, r)) + .collect(); + + let mut checked = 0usize; + let mut matched = 0usize; + let mut mismatches = Vec::new(); + + for twin_row in self.rows.iter().filter(|r| r.sequence >= 0) { + let payload: Value = match twin_row + .raw_payload + .as_deref() + .and_then(|p| serde_json::from_str(p).ok()) + { + Some(v) => v, + None => continue, + }; + let refs = match payload + .get("data") + .and_then(|d| d.get("source_refs")) + .and_then(|v| v.as_array()) + { + Some(r) => r, + None => continue, + }; + for sr in refs { + let chain_id = match sr.get("chain_id").and_then(|v| v.as_str()) { + Some(s) => s.to_string(), + None => continue, + }; + let sequence = match sr.get("sequence").and_then(|v| v.as_i64()) { + Some(s) => s, + None => continue, + }; + let row_hash = match sr.get("row_hash").and_then(|v| v.as_str()) { + Some(s) => s.to_string(), + None => continue, + }; + checked += 1; + let source_ref = SourceRef { + chain_id, + sequence, + row_hash: row_hash.clone(), + }; + match sensor_index.get(&sequence) { + None => mismatches.push(SourceRefMismatch { + twin_sequence: twin_row.sequence, + source_ref, + reason: "not_found".into(), + }), + Some(sensor_row) => { + if sensor_row.row_hash.as_deref() == Some(&row_hash) { + matched += 1; + } else { + mismatches.push(SourceRefMismatch { + twin_sequence: twin_row.sequence, + source_ref, + reason: "hash_mismatch".into(), + }); + } + } + } + } + } + + let ok = mismatches.is_empty(); + SourceRefReport { + checked, + matched, + mismatches, + ok, + } + } + + /// Convenience constructor. Computes `row_count` and + /// `sequence_range` from `rows`; callers do not need to + /// pre-compute them. Non-BREAK rows (sequence >= 0) contribute + /// to the range. + pub fn new( + entity_id: String, + generated_at: String, + rows: Vec, + anchors: Vec, + ) -> Self { + let row_count = rows.len(); + let (mut lo, mut hi): (Option, Option) = (None, None); + for r in rows.iter().filter(|r| r.sequence >= 0) { + lo = Some(lo.map_or(r.sequence, |x| x.min(r.sequence))); + hi = Some(hi.map_or(r.sequence, |x| x.max(r.sequence))); + } + Self { + entity_id, + generated_at, + row_count, + sequence_range: (lo, hi), + rows, + anchors, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn make_row(sequence: i64, row_hash: &str, raw_payload: Option<&str>) -> BundleRow { + BundleRow { + entity_id: "test".into(), + chain_id: "chain_test_000001".into(), + event_type: if sequence == 0 { + "GENESIS".into() + } else { + "MUTATION".into() + }, + sequence, + prev_hash: "0".into(), + row_hash: Some(row_hash.into()), + anchor_hash: None, + raw_payload: raw_payload.map(|s| s.into()), + break_reason: None, + } + } + + fn sensor_bundle(rows: Vec) -> ProofBundle { + ProofBundle::new("sensor".into(), "2026-01-01T00:00:00Z".into(), rows, vec![]) + } + + fn twin_bundle(rows: Vec) -> ProofBundle { + ProofBundle::new("twin".into(), "2026-01-01T00:00:00Z".into(), rows, vec![]) + } + + #[test] + fn verify_source_refs_all_match() { + let s0 = make_row(0, "hash_aaa", None); + let s1 = make_row(1, "hash_bbb", None); + let sensor = sensor_bundle(vec![s0.clone(), s1.clone()]); + + let twin_payload = serde_json::json!({ + "data": { + "source_refs": [ + {"chain_id": "chain_test_000001", "sequence": 0, "row_hash": "hash_aaa"}, + {"chain_id": "chain_test_000001", "sequence": 1, "row_hash": "hash_bbb"}, + ] + } + }) + .to_string(); + let twin_row = make_row(0, "hash_twin", Some(&twin_payload)); + let twin = twin_bundle(vec![twin_row]); + + let report = twin.verify_source_refs(&sensor); + assert!(report.ok); + assert_eq!(report.checked, 2); + assert_eq!(report.matched, 2); + assert!(report.mismatches.is_empty()); + } + + #[test] + fn verify_source_refs_missing_row() { + let sensor = sensor_bundle(vec![make_row(0, "hash_aaa", None)]); + + let twin_payload = serde_json::json!({ + "data": { + "source_refs": [ + {"chain_id": "chain_test_000001", "sequence": 99, "row_hash": "hash_zzz"}, + ] + } + }) + .to_string(); + let twin = twin_bundle(vec![make_row(0, "hash_twin", Some(&twin_payload))]); + + let report = twin.verify_source_refs(&sensor); + assert!(!report.ok); + assert_eq!(report.checked, 1); + assert_eq!(report.matched, 0); + assert_eq!(report.mismatches[0].reason, "not_found"); + } + + #[test] + fn verify_source_refs_hash_mismatch() { + let sensor = sensor_bundle(vec![make_row(0, "hash_aaa", None)]); + + let twin_payload = serde_json::json!({ + "data": { + "source_refs": [ + {"chain_id": "chain_test_000001", "sequence": 0, "row_hash": "hash_WRONG"}, + ] + } + }) + .to_string(); + let twin = twin_bundle(vec![make_row(0, "hash_twin", Some(&twin_payload))]); + + let report = twin.verify_source_refs(&sensor); + assert!(!report.ok); + assert_eq!(report.mismatches[0].reason, "hash_mismatch"); + } + + #[test] + fn verify_source_refs_no_refs_in_twin_is_ok() { + let sensor = sensor_bundle(vec![make_row(0, "hash_aaa", None)]); + let twin = twin_bundle(vec![make_row(0, "hash_twin", None)]); + let report = twin.verify_source_refs(&sensor); + assert!(report.ok); + assert_eq!(report.checked, 0); + } +} + +#[cfg(all(test, feature = "keygen"))] +mod equivocation_tests { + use super::*; + use crate::chain::{compute_record_signed, make_chain_id, ComputedRecord, EventType}; + use crate::crypto::{Ed25519Signer, Signer}; + use crate::hash::SecretKey; + use crate::payload::{PayloadFields, PAYLOAD_VERSION_V2}; + use serde_json::json; + + fn key() -> SecretKey { + SecretKey::from_bytes(b"0123456789abcdef").unwrap() + } + + /// One signed row at `sequence`, whose contents depend on `marker` -- two + /// different markers at the same sequence is the equivocation. + fn signed(signer: &Ed25519Signer, sequence: i64, marker: i64) -> BundleRow { + let chain_id = make_chain_id("acme", 1); + let mut f = PayloadFields::new_v1( + "acme", + chain_id.clone(), + json!({ "v": marker }), + "s", + "2026-03-08T00:00:00Z", + "2026-03-08T00:00:00.123456Z", + "trusty", + "genesis", + sequence, + ); + f.payload_version = PAYLOAD_VERSION_V2; + let rec: ComputedRecord = compute_record_signed(&f, &key(), signer).unwrap(); + BundleRow { + entity_id: "acme".into(), + chain_id, + event_type: EventType::Mutation.as_str().into(), + sequence, + prev_hash: "genesis".into(), + row_hash: Some(rec.row_hash), + anchor_hash: Some(rec.anchor_hash), + raw_payload: Some(rec.raw_payload), + break_reason: None, + } + } + + fn bundle(rows: Vec) -> ProofBundle { + ProofBundle::new("acme".into(), "2026-01-01T00:00:00Z".into(), rows, vec![]) + } + + #[test] + fn two_signed_rows_at_one_sequence_are_proof() { + let signer = Ed25519Signer::generate(); + let a = bundle(vec![signed(&signer, 1, 100)]); + let b = bundle(vec![signed(&signer, 1, 999)]); + + let proofs = detect_equivocation(&a, &b); + assert_eq!(proofs.len(), 1, "one producer, one position, two histories"); + assert_eq!(proofs[0].sequence, 1); + assert_ne!(proofs[0].row_a.row_hash, proofs[0].row_b.row_hash); + assert_eq!( + proofs[0].producer_pk, + hex::encode(signer.public_key_bytes()), + "the proof must name the key that signed both" + ); + } + + #[test] + fn the_same_history_twice_proves_nothing() { + let signer = Ed25519Signer::generate(); + let row = signed(&signer, 1, 100); + let a = bundle(vec![row.clone()]); + let b = bundle(vec![row]); + assert!(detect_equivocation(&a, &b).is_empty()); + } + + #[test] + fn two_different_producers_are_not_equivocating() { + // Two keys signing different content at the same sequence is a + // disagreement between parties, not one party showing two faces. + let one = Ed25519Signer::generate(); + let two = Ed25519Signer::generate(); + let a = bundle(vec![signed(&one, 1, 100)]); + let b = bundle(vec![signed(&two, 1, 999)]); + assert!( + detect_equivocation(&a, &b).is_empty(), + "differing keys must never be reported as equivocation" + ); + } + + #[test] + fn a_forged_second_row_cannot_manufacture_a_proof() { + // The attack this guards: fabricate a row at someone's sequence to + // "prove" they equivocated. Without a valid signature it proves nothing, + // so it must be skipped rather than reported. + let signer = Ed25519Signer::generate(); + let a = bundle(vec![signed(&signer, 1, 100)]); + + let mut forged = signed(&signer, 1, 999); + // Break the signature by rewriting the payload after signing. + let raw = forged.raw_payload.clone().unwrap(); + forged.raw_payload = Some(raw.replace("\"v\":999", "\"v\":1234")); + let b = bundle(vec![forged]); + + assert!( + detect_equivocation(&a, &b).is_empty(), + "an unsigned or tampered row must never be admitted as proof" + ); + } + + #[test] + fn break_rows_cannot_equivocate() { + let signer = Ed25519Signer::generate(); + let mut brk = signed(&signer, -1, 1); + brk.event_type = EventType::Break.as_str().into(); + brk.row_hash = None; + let a = bundle(vec![brk.clone()]); + let b = bundle(vec![brk]); + assert!(detect_equivocation(&a, &b).is_empty()); + } +} + +/// A recorded rule violation, read off a bundle. +/// +/// Self-contained on purpose: everything needed to say what broke, and +/// where, without going back to the machine that recorded it. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct Violation { + pub entity_id: String, + pub chain_id: String, + /// The rule's name, when the break named one. + /// + /// `None` for a break with a category but no rule -- tamper + /// detection, a sequence gap. Those are real breaks and are still + /// reported; they simply have no invariant to name. + pub rule: Option, + /// The reason exactly as recorded. + pub reason: String, +} + +/// What a bundle says about whether its data broke a rule. +/// +/// Three answers, not two. **Absent evidence must never read as clean** +/// -- the same rule as "absent policy is not permission" -- so a bundle +/// that cannot be verified reports [`Stance::Unverifiable`] rather than +/// quietly counting zero violations and looking spotless. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "stance", rename_all = "snake_case")] +pub enum Stance { + /// Verified, and records no break. + Clean, + /// Verified, and records at least one break. + Violated { violations: Vec }, + /// Could not be verified, so it says nothing either way. Treating + /// this as clean is the mistake this variant exists to prevent. + Unverifiable { detail: String }, +} + +impl Stance { + /// Whether it is safe to proceed on this bundle alone. + /// + /// Only [`Stance::Clean`]. An unverifiable bundle is not a permit. + pub fn is_clean(&self) -> bool { + matches!(self, Stance::Clean) + } +} + +impl ProofBundle { + /// Every rule violation this bundle records. + /// + /// Reads the BREAK rows only. Says nothing about whether the bundle + /// verifies -- use [`ProofBundle::stance`] for the question a caller + /// deciding whether to act should actually be asking. + pub fn violations(&self) -> Vec { + self.rows + .iter() + .filter(|r| r.event_type == "BREAK") + .map(|r| { + let reason = r.break_reason.clone().unwrap_or_default(); + Violation { + entity_id: r.entity_id.clone(), + chain_id: r.chain_id.clone(), + rule: reason + .strip_prefix("PROPERTY_VIOLATED:") + .map(str::to_string), + reason, + } + }) + .collect() + } + + /// Verify, then say whether anything broke. + /// + /// The question a peer asks before serving, quarantining, or + /// inspecting: *is this datum carrying a signed violation?* Needs no + /// secret key, so any party holding the bundle can ask it. + pub fn stance(&self) -> Stance { + let report = self.verify_public(); + if !report.rows_valid { + return Stance::Unverifiable { + detail: "a signature did not verify or linkage did not hold, so \ + nothing this bundle says can be relied on" + .to_string(), + }; + } + if !report.fully_signed { + return Stance::Unverifiable { + detail: "a row carries no signature, so it is not merely \ + unchecked but unattributable" + .to_string(), + }; + } + let violations = self.violations(); + if violations.is_empty() { + Stance::Clean + } else { + Stance::Violated { violations } + } + } +} diff --git a/crates/core/trusty-core/src/chain.rs b/crates/core/trusty-core/src/chain.rs new file mode 100644 index 0000000..ccfcd8b --- /dev/null +++ b/crates/core/trusty-core/src/chain.rs @@ -0,0 +1,1742 @@ +// Chain primitives. +// +// Pure functions over the dual-hash + canonical-payload layer. +// Storage lives elsewhere — this module only knows about records as +// in-memory structs. The Python reference is +// trusty_core_watermark_proxy/core/trusty/core/chain.py +// specifically _compute_hashes, _write_record, and verify_chain. + +use serde::Serialize; +use serde_json::{json, Value}; + +use crate::crypto::{Ed25519Verifier, Signer, Verifier}; +use crate::error::{TrustyError, TrustyResult}; +#[cfg_attr(not(test), allow(unused_imports))] +use crate::hash::{ + anchor_hash, anchor_hash_for_version, row_hash, row_hash_for_version, SecretKey, +}; +use crate::payload::{ + build_payload, signing_view_bytes, PayloadFields, PAYLOAD_VERSION_V1, PAYLOAD_VERSION_V2, + PAYLOAD_VERSION_V3, +}; + +pub const GENESIS_SENTINEL: &str = "genesis"; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum EventType { + Genesis, + Mutation, + Break, + Rollback, +} + +impl EventType { + pub fn as_str(&self) -> &'static str { + match self { + EventType::Genesis => "GENESIS", + EventType::Mutation => "MUTATION", + EventType::Break => "BREAK", + EventType::Rollback => "ROLLBACK", + } + } +} + +#[derive(Debug, Clone)] +pub struct ComputedRecord { + pub row_hash: String, + pub anchor_hash: String, + pub raw_payload: String, +} + +pub fn compute_record(fields: &PayloadFields, key: &SecretKey) -> TrustyResult { + let raw_payload = build_payload(fields)?; + let rh = row_hash_for_version(fields.payload_version, key, raw_payload.as_bytes())?; + let ah = anchor_hash_for_version( + fields.payload_version, + &rh, + &fields.entity_id, + &fields.chain_id, + )?; + Ok(ComputedRecord { + row_hash: rh, + anchor_hash: ah, + raw_payload, + }) +} + +/// Signed-record path: sign the canonical signing-view bytes, splice the +/// signature object into `signatures`, then compute row_hash over the +/// final bytes. row_hash therefore commits to the signature as well as +/// the data. +/// +/// Version dispatch: +/// - v1 callers are upgraded to v2 (existing behavior since S5; v1 has +/// no signatures field semantics). +/// - v2 callers stay on v2 (legacy hashes, no domain tag). +/// - v3 callers get the v3 signed path (domain-tagged row_hash and +/// anchor_hash via `row_hash_for_version` / `anchor_hash_for_version`). +/// +/// Resolves S14 OQ #17 / s15 HARD BLOCKER. Multi-sig (compute_record_signed_multi) +/// is a separate follow-on tied to Gitea #16. +pub fn compute_record_signed( + fields: &PayloadFields, + key: &SecretKey, + signer: &dyn Signer, +) -> TrustyResult { + let effective_version = match fields.payload_version { + PAYLOAD_VERSION_V1 | PAYLOAD_VERSION_V2 => PAYLOAD_VERSION_V2, + PAYLOAD_VERSION_V3 => PAYLOAD_VERSION_V3, + other => { + return Err(TrustyError::PayloadSerialization(format!( + "compute_record_signed: unsupported payload_version {}", + other + ))); + } + }; + + let mut signed = fields.clone(); + signed.payload_version = effective_version; + signed.signatures = Vec::new(); + + let signing_view = signing_view_bytes(&signed)?; + let sig_bytes = signer.sign(signing_view.as_bytes()); + let pk_bytes = signer.public_key_bytes(); + + signed.signatures = vec![json!({ + "alg": "ed25519", + "pk": hex::encode(&pk_bytes), + "sig": hex::encode(&sig_bytes), + })]; + + let raw_payload = build_payload(&signed)?; + let rh = row_hash_for_version(effective_version, key, raw_payload.as_bytes())?; + let ah = anchor_hash_for_version(effective_version, &rh, &signed.entity_id, &signed.chain_id)?; + Ok(ComputedRecord { + row_hash: rh, + anchor_hash: ah, + raw_payload, + }) +} + +pub fn make_chain_id(entity_id: &str, generation: u32) -> String { + format!("chain_{}_{:06}", entity_id, generation) +} + +/// Canonical byte layout an attestor signs to produce a checkpoint +/// signature. Domain-tagged so the signature is non-replayable as a +/// chain row signature or anchor commitment. Used by both writer +/// (Trusty::checkpoint) and verifier (Trusty::verify_since_checkpoint). +/// +/// Layout: `DOMAIN_TAG_CHECKPOINT_V1 || chain_id || 0x00 || +/// sequence_int (8-byte big-endian i64) || 0x00 || row_hash_hex`. +/// Uses big-endian i64 (matches our hex encoding convention from +/// project_be_encoding_decision -- big-endian is canonical here, not +/// the threadkeeper LE legacy). +pub fn checkpoint_signing_bytes(chain_id: &str, sequence_int: i64, row_hash: &str) -> Vec { + use crate::payload::DOMAIN_TAG_CHECKPOINT_V1; + let mut out = Vec::with_capacity( + DOMAIN_TAG_CHECKPOINT_V1.len() + chain_id.len() + 1 + 8 + 1 + row_hash.len(), + ); + out.extend_from_slice(DOMAIN_TAG_CHECKPOINT_V1); + out.extend_from_slice(chain_id.as_bytes()); + out.push(0x00); + out.extend_from_slice(&sequence_int.to_be_bytes()); + out.push(0x00); + out.extend_from_slice(row_hash.as_bytes()); + out +} + +/// Inverse of `make_chain_id`. Parses the trailing `_` segment +/// from a chain_id string, tolerating leading zeros (output of `:06`) and +/// mixed-arity producers. Returns `None` if the suffix is not a u32 or +/// the chain_id has no underscore. +/// +/// Promoted from `trusty-sqlite::store::parse_generation` (S16, resolves +/// §12 OQ #16 / s14_insight item 3) so every backend uses the same +/// parser. Pure string operation; no byte-shape involvement. +pub fn parse_generation(chain_id: &str) -> Option { + let gen_segment = chain_id.rsplit('_').next()?; + let trimmed = gen_segment.trim_start_matches('0'); + let parse_target = if trimmed.is_empty() { "0" } else { trimmed }; + parse_target.parse::().ok() +} + +#[derive(Debug, Clone)] +pub struct ChainLink { + pub entity_id: String, + pub chain_id: String, + pub event_type: String, + pub sequence: i64, + pub prev_hash: String, + pub row_hash: Option, + pub anchor_hash: Option, + pub raw_payload: Option, +} + +/// Where a chain was severed, and why. +/// +/// A BREAK row is written by `handle_break` on `ConditionOutcome::Fail` -- +/// that is, it exists BECAUSE a monitored condition (hash integrity, +/// sequence continuity, a declared run rule) failed at append time. It is +/// the system's own record that something went wrong, not an ordinary +/// event, and a verifier that does not mention it is answering a +/// different question from the one the caller asked. +#[derive(Debug, Clone, Serialize)] +pub struct BreakPoint { + /// The chain generation that ENDED here. The rows that follow belong + /// to generation n+1 and hash from a fresh genesis. + pub chain_id: String, + /// Position in the verified row order, so a caller can say what came + /// before the break without re-walking the chain. + pub index: usize, +} + +/// Resource ceilings for verifying a chain supplied by someone else. +/// +/// `verify_links` and friends walk a caller-supplied slice, and +/// `ProofBundle` deserialises an unbounded `rows` vector. A bundle is by +/// definition an artifact handed over by another party -- often the party whose +/// behaviour is in question -- so an unbounded walk over it is a denial of +/// service the verifier volunteers for. `MAX_DATA_BYTES` bounds one row's +/// `data`; nothing bounded the collection. +/// +/// Defaults are generous enough that an honest chain never trips them and +/// finite enough that a hostile one cannot exhaust the verifier. +#[derive(Debug, Clone, Copy)] +pub struct VerifyLimits { + /// Maximum rows examined, BREAK markers included. + pub max_rows: usize, + /// Maximum total `raw_payload` bytes examined across all rows. + pub max_total_bytes: usize, +} + +impl Default for VerifyLimits { + fn default() -> Self { + Self { + max_rows: 1_000_000, + max_total_bytes: 512 * 1024 * 1024, + } + } +} + +impl VerifyLimits { + /// Ceilings for verifying something received over a network, where the + /// sender chose the size. Deliberately much tighter than `Default`. + pub fn untrusted() -> Self { + Self { + max_rows: 10_000, + max_total_bytes: 64 * 1024 * 1024, + } + } +} + +/// The result of walking a chain. +/// +/// READ `ok` NARROWLY. It means "every row this walk checked hashed and +/// signed correctly" -- nothing more. It is deliberately NOT affected by +/// breaks: a break is a recorded event, present by design, and the +/// segments either side of one can be perfectly well-formed. `ok` is +/// about rows. +/// +/// This split was found by review on 2026-08-20 and it had a sharp edge. +/// `verify_links` skips BREAK rows and resets `prev` to the genesis +/// sentinel, so a chain containing the system's own record of DETECTED +/// TAMPERING returned `ok: true` with nothing in the report to say so. +/// The rows survived export -- `ProofBundle` keeps them deliberately -- +/// but a third party doing the obvious thing (export, `verify()`, read +/// `ok`) concluded continuity. The verdict was narrower than its name. +/// +/// So: ask `continuous` when the question is "was this chain unbroken", +/// and `ok` when the question is "are these rows intact". Most callers +/// who think they are asking the second are actually asking the first. +#[derive(Debug, Clone, Serialize)] +pub struct VerifyReport { + pub entity_id: String, + /// Every row checked hashed and signed correctly. Says nothing about + /// breaks -- see `continuous`, or `intact()` for both together. + pub rows_valid: bool, + /// No break was recorded anywhere in the verified range. When this is + /// false, a monitored condition failed at some point and `breaks` + /// says where. + pub continuous: bool, + /// Every break in the verified range, in order. + pub breaks: Vec, + pub row_count: usize, + /// A `VerifyLimits` ceiling was hit and the walk stopped early, so this + /// verdict covers a PREFIX of what was supplied. + /// + /// Forces `intact()` false. A verifier that quietly checks the first N rows + /// and answers "fine" is a worse failure than refusing outright, and is the + /// same class of mistake as reporting `ok: true` over a recorded break. + pub limit_reached: bool, + pub errors: Vec, +} + +impl VerifyReport { + /// Nothing wrong with any row, and no break recorded. The question + /// "is this chain fine?" -- answered correctly. + /// + /// Reach for this rather than `rows_valid` alone. That field was called + /// `ok`, and its doc said to read it narrowly; five call sites across two + /// repositories collapsed it to a trust decision anyway, each reporting a + /// chain that had recorded its own tamper detection as fine. A name at the + /// call site does work a doc comment on the field cannot. + pub fn intact(&self) -> bool { + self.rows_valid && self.continuous && !self.limit_reached + } +} + +#[derive(Debug, Clone, Serialize)] +pub struct VerifyError { + pub sequence: i64, + pub chain_id: String, + pub kind: VerifyErrorType, +} + +#[derive(Debug, Clone, Serialize)] +#[serde(tag = "error")] +pub enum VerifyErrorType { + #[serde(rename = "prev_hash_mismatch")] + PrevHashMismatch { expected: String, actual: String }, + #[serde(rename = "missing_raw_payload")] + MissingRawPayload, + #[serde(rename = "invalid_raw_payload")] + InvalidRawPayload { detail: String }, + #[serde(rename = "row_hash_mismatch")] + RowHashMismatch { expected: String, actual: String }, + #[serde(rename = "anchor_hash_mismatch")] + AnchorHashMismatch { expected: String, actual: String }, + #[serde(rename = "missing_row_hash")] + MissingRowHash, + #[serde(rename = "signature_invalid")] + SignatureInvalid { detail: String }, + #[serde(rename = "signature_malformed")] + SignatureMalformed { detail: String }, + #[serde(rename = "unknown_signature_alg")] + UnknownSignatureAlg { alg: String }, +} + +pub fn verify_links(entity_id: &str, links: &[ChainLink], key: &SecretKey) -> VerifyReport { + // A full chain starts at genesis. + verify_links_from(entity_id, links, key, GENESIS_SENTINEL) +} + +/// Like [`verify_links`], but seeds the expected `prev_hash` of the FIRST link +/// with `start_prev` instead of the genesis sentinel. This is what a windowed +/// (checkpoint-pruned) verify needs: the window does not begin at genesis, it +/// begins right after a trusted checkpoint whose `row_hash` is the seed. Passing +/// `GENESIS_SENTINEL` recovers the full-chain behaviour. +pub fn verify_links_from( + entity_id: &str, + links: &[ChainLink], + key: &SecretKey, + start_prev: &str, +) -> VerifyReport { + verify_links_from_bounded(entity_id, links, key, start_prev, &VerifyLimits::default()) +} + +/// [`verify_links_from`] with explicit resource ceilings. +/// +/// Use [`VerifyLimits::untrusted`] for anything whose size was chosen by the +/// party being verified. When a ceiling is hit the walk stops and +/// `limit_reached` is set, which forces `intact()` false -- the verdict covers +/// a prefix and says so rather than implying the whole chain was checked. +pub fn verify_links_from_bounded( + entity_id: &str, + links: &[ChainLink], + key: &SecretKey, + start_prev: &str, + limits: &VerifyLimits, +) -> VerifyReport { + let mut errors: Vec = Vec::new(); + let mut breaks: Vec = Vec::new(); + let mut prev = start_prev.to_string(); + let mut non_break = 0usize; + let mut budget_bytes = limits.max_total_bytes; + let mut limit_reached = false; + + for (index, link) in links.iter().enumerate() { + if index >= limits.max_rows { + limit_reached = true; + break; + } + let row_bytes = link.raw_payload.as_ref().map_or(0, |r| r.len()); + match budget_bytes.checked_sub(row_bytes) { + Some(remaining) => budget_bytes = remaining, + None => { + limit_reached = true; + break; + } + } + + if link.event_type == "BREAK" { + // Break markers carry no hashes; they reset the prev chain + // for the next segment. A new genesis follows them. + // + // RECORDED, not merely skipped. A break exists because a + // monitored condition FAILED at append time, so skipping it + // silently made `ok: true` mean less than every caller read + // it to mean. Resetting `prev` is still correct -- the next + // segment genuinely does start from a fresh genesis -- but + // the reset is now visible in the report. + breaks.push(BreakPoint { + chain_id: link.chain_id.clone(), + index, + }); + prev = GENESIS_SENTINEL.to_string(); + continue; + } + + non_break += 1; + + if link.prev_hash != prev { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::PrevHashMismatch { + expected: prev.clone(), + actual: link.prev_hash.clone(), + }, + }); + } + + let raw = match &link.raw_payload { + Some(r) => r, + None => { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::MissingRawPayload, + }); + if let Some(h) = &link.row_hash { + prev = h.clone(); + } + continue; + } + }; + + let stored_row_hash = match &link.row_hash { + Some(h) => h.clone(), + None => { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::MissingRowHash, + }); + continue; + } + }; + + let pv = match extract_payload_version(raw) { + Ok(v) => v, + Err(e) => { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::InvalidRawPayload { + detail: e.to_string(), + }, + }); + prev = stored_row_hash; + continue; + } + }; + + let computed_row = match row_hash_for_version(pv, key, raw.as_bytes()) { + Ok(h) => h, + Err(e) => { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::InvalidRawPayload { + detail: e.to_string(), + }, + }); + prev = stored_row_hash; + continue; + } + }; + if computed_row != stored_row_hash { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::RowHashMismatch { + expected: computed_row.clone(), + actual: stored_row_hash.clone(), + }, + }); + } + + let computed_anchor = + match anchor_hash_for_version(pv, &computed_row, &link.entity_id, &link.chain_id) { + Ok(h) => h, + Err(_) => anchor_hash(&computed_row, &link.entity_id, &link.chain_id), + }; + if let Some(stored_anchor) = &link.anchor_hash { + if &computed_anchor != stored_anchor { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::AnchorHashMismatch { + expected: computed_anchor, + actual: stored_anchor.clone(), + }, + }); + } + } + + for kind in verify_row_signatures(raw) { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind, + }); + } + + prev = stored_row_hash; + } + + VerifyReport { + entity_id: entity_id.to_string(), + rows_valid: errors.is_empty(), + continuous: breaks.is_empty(), + breaks, + row_count: non_break, + limit_reached, + errors, + } +} + +/// The result of walking a chain **without** the operator's secret key. +/// +/// Mirrors `VerifyReport`, including the `ok` / `continuous` split, and adds +/// the two counts that only matter when no key is present. See +/// [`verify_links_public`]. +#[derive(Debug, Clone, Serialize)] +pub struct PublicVerifyReport { + pub entity_id: String, + /// Every signature present verified and linkage held. Says nothing about + /// unsigned rows or breaks -- see `fully_signed`, `continuous`, or + /// `intact()` for all three together. + pub rows_valid: bool, + /// Every non-BREAK row carried a signature. Without the operator's key an + /// unsigned row is not merely unchecked, it is unattributable, so this is + /// a separate question from whether the rows that ARE signed hold up. + pub fully_signed: bool, + /// No break was recorded anywhere in the verified range. Same meaning as + /// `VerifyReport::continuous`, and carried for the same reason: a break + /// exists because a monitored condition failed, and a verdict that does + /// not mention it answers a different question from the one asked. + pub continuous: bool, + /// Every break in the verified range, in order. + pub breaks: Vec, + /// Non-BREAK rows examined. + pub row_count: usize, + /// Rows carrying at least one signature, all of which verified. + pub signed_row_count: usize, + /// Rows carrying no signature at all -- v1 rows, or v2+ with + /// `signatures: []`. Nothing in the rows establishes their authorship or + /// their contents. + pub unsigned_row_count: usize, + /// A `VerifyLimits` ceiling was hit and the walk stopped early, so this + /// verdict covers a PREFIX of what was supplied. Forces `intact()` false. + pub limit_reached: bool, + pub errors: Vec, +} + +/// Walk a chain using only what travels with it -- no secret key. +/// +/// [`verify_links`] needs the operator's SipHash key, because `row_hash` is a +/// keyed MAC. That makes it unusable for the audience a [`ProofBundle`] is +/// built for: a party with no database access and no shared secret. This is +/// the path for that party. +/// +/// [`ProofBundle`]: crate::bundle::ProofBundle +/// +/// # What this establishes +/// +/// - **Ed25519 signatures**, against the reconstructed signing view. This +/// covers more than it appears to: the signed payload includes `prev_hash`, +/// `sequence`, `entity_id` and `chain_id`, so a signature commits to the +/// row's position in the chain and to its backward link. Altering a row's +/// contents, reordering rows, or splicing them between chains each +/// invalidate a signature. +/// - **Linkage**: each row's `prev_hash` equals the previous row's stored +/// `row_hash`. With the above, this closes removal of interior rows. +/// - **Anchor consistency**: `anchor_hash` equals the unkeyed SHA-256 over the +/// stored `row_hash`, `entity_id` and `chain_id`. +/// - **Breaks**, recorded exactly as [`verify_links`] records them. +/// +/// # What it does not +/// +/// - That `row_hash` is the correct MAC of `raw_payload`. That is keyed. For a +/// **signed** row this matters little, since the signature independently +/// commits to the payload; for an **unsigned** row nothing here attests to +/// its contents, which is why such rows force `ok` to false. +/// - Truncation at the tip. No append-only log detects its own tail being +/// trimmed without an external commitment; publish anchors for that. +impl PublicVerifyReport { + /// Nothing wrong with any row, every row attributable, and no break + /// recorded. The question "is this chain fine?" -- answered correctly. + /// + /// Reach for this rather than composing the three fields by hand. The + /// field this replaced was called `ok`, and every caller that collapsed it + /// to a trust decision silently ignored `continuous`. + pub fn intact(&self) -> bool { + self.rows_valid && self.fully_signed && self.continuous && !self.limit_reached + } +} + +pub fn verify_links_public(entity_id: &str, links: &[ChainLink]) -> PublicVerifyReport { + verify_links_public_from(entity_id, links, GENESIS_SENTINEL) +} + +/// [`verify_links_public`], seeded from a known-good `prev` instead of genesis. +/// +/// The keyless twin of [`verify_links_from`]: verify a window of a long chain +/// from a trusted checkpoint, without replaying from the beginning and without +/// holding the secret key. +pub fn verify_links_public_from( + entity_id: &str, + links: &[ChainLink], + start_prev: &str, +) -> PublicVerifyReport { + verify_links_public_from_bounded(entity_id, links, start_prev, &VerifyLimits::default()) +} + +/// [`verify_links_public_from`] with explicit resource ceilings. +/// +/// This is the one most likely to face a hostile input: a keyless verifier is +/// by definition working from a bundle someone else produced, so reach for +/// [`VerifyLimits::untrusted`] here by default rather than as a special case. +pub fn verify_links_public_from_bounded( + entity_id: &str, + links: &[ChainLink], + start_prev: &str, + limits: &VerifyLimits, +) -> PublicVerifyReport { + let mut errors: Vec = Vec::new(); + let mut breaks: Vec = Vec::new(); + let mut prev = start_prev.to_string(); + let mut row_count = 0usize; + let mut signed_row_count = 0usize; + let mut unsigned_row_count = 0usize; + let mut budget_bytes = limits.max_total_bytes; + let mut limit_reached = false; + + for (index, link) in links.iter().enumerate() { + if index >= limits.max_rows { + limit_reached = true; + break; + } + let row_bytes = link.raw_payload.as_ref().map_or(0, |r| r.len()); + match budget_bytes.checked_sub(row_bytes) { + Some(remaining) => budget_bytes = remaining, + None => { + limit_reached = true; + break; + } + } + + if link.event_type == "BREAK" { + // Recorded, not skipped -- for the reason spelled out in + // verify_links_from. Resetting prev stays correct; the reset is + // simply visible now. + breaks.push(BreakPoint { + chain_id: link.chain_id.clone(), + index, + }); + prev = GENESIS_SENTINEL.to_string(); + continue; + } + + row_count += 1; + + if link.prev_hash != prev { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::PrevHashMismatch { + expected: prev.clone(), + actual: link.prev_hash.clone(), + }, + }); + } + + let stored_row_hash = match &link.row_hash { + Some(h) => h.clone(), + None => { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::MissingRowHash, + }); + continue; + } + }; + + let raw = match &link.raw_payload { + Some(r) => r, + None => { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::MissingRawPayload, + }); + prev = stored_row_hash; + continue; + } + }; + + // Computable without the key: an unkeyed SHA-256 over the STORED + // row_hash. Proves the anchor matches the row_hash on record, not that + // either matches the payload. + if let Some(stored_anchor) = &link.anchor_hash { + let pv = extract_payload_version(raw).unwrap_or(PAYLOAD_VERSION_V1); + let computed_anchor = + anchor_hash_for_version(pv, &stored_row_hash, &link.entity_id, &link.chain_id) + .unwrap_or_else(|_| { + anchor_hash(&stored_row_hash, &link.entity_id, &link.chain_id) + }); + if &computed_anchor != stored_anchor { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind: VerifyErrorType::AnchorHashMismatch { + expected: computed_anchor, + actual: stored_anchor.clone(), + }, + }); + } + } + + let sig_errors = verify_row_signatures(raw); + let had_sig_errors = !sig_errors.is_empty(); + for kind in sig_errors { + errors.push(VerifyError { + sequence: link.sequence, + chain_id: link.chain_id.clone(), + kind, + }); + } + + if row_is_signed(raw) { + if !had_sig_errors { + signed_row_count += 1; + } + } else { + unsigned_row_count += 1; + } + + prev = stored_row_hash; + } + + PublicVerifyReport { + entity_id: entity_id.to_string(), + rows_valid: errors.is_empty(), + fully_signed: unsigned_row_count == 0, + continuous: breaks.is_empty(), + breaks, + row_count, + signed_row_count, + unsigned_row_count, + limit_reached, + errors, + } +} + +// True when `raw` parses and carries at least one signature entry. +fn row_is_signed(raw: &str) -> bool { + parse_raw_payload(raw) + .map(|p| !p.signatures.is_empty()) + .unwrap_or(false) +} + +/// Verify the Ed25519 signatures embedded in a raw payload. +/// +/// Returns one [`VerifyErrorType`] per problem found; an empty vector means +/// "either unsigned (no-op) or every signature valid". Row-hash integrity is a +/// separate, keyed concern -- this checks only the signatures, against the +/// reconstructed signing view. +/// +/// Public because it is the only verification here that needs no shared +/// secret, and [`crate::bundle::ProofBundle`] documents that path as available +/// to a bundle holder. It previously had no public entry point. +pub fn verify_row_signatures(raw: &str) -> Vec { + let parsed = match parse_raw_payload(raw) { + Ok(p) => p, + Err(e) => { + return vec![VerifyErrorType::InvalidRawPayload { + detail: e.to_string(), + }]; + } + }; + + if parsed.payload_version < PAYLOAD_VERSION_V2 || parsed.signatures.is_empty() { + return Vec::new(); + } + + let signing_view = match signing_view_bytes(&parsed) { + Ok(s) => s, + Err(e) => { + return vec![VerifyErrorType::InvalidRawPayload { + detail: format!("signing-view rebuild: {}", e), + }]; + } + }; + + let mut out = Vec::new(); + for sig_val in &parsed.signatures { + let obj = match sig_val.as_object() { + Some(o) => o, + None => { + out.push(VerifyErrorType::SignatureMalformed { + detail: "signature entry not a JSON object".into(), + }); + continue; + } + }; + let alg = obj.get("alg").and_then(Value::as_str).unwrap_or(""); + let pk_hex = obj.get("pk").and_then(Value::as_str).unwrap_or(""); + let sig_hex = obj.get("sig").and_then(Value::as_str).unwrap_or(""); + + if alg != "ed25519" { + out.push(VerifyErrorType::UnknownSignatureAlg { + alg: alg.to_string(), + }); + continue; + } + + let pk = match hex::decode(pk_hex) { + Ok(b) => b, + Err(e) => { + out.push(VerifyErrorType::SignatureMalformed { + detail: format!("pk hex decode: {}", e), + }); + continue; + } + }; + let sig = match hex::decode(sig_hex) { + Ok(b) => b, + Err(e) => { + out.push(VerifyErrorType::SignatureMalformed { + detail: format!("sig hex decode: {}", e), + }); + continue; + } + }; + + if let Err(e) = Ed25519Verifier::verify(&pk, signing_view.as_bytes(), &sig) { + out.push(VerifyErrorType::SignatureInvalid { + detail: e.to_string(), + }); + } + } + out +} + +// Fast-path extraction of just `payload_version` from a raw payload. +// Avoids building a full PayloadFields when the verify loop only needs +// to pick the correct hash primitive. +fn extract_payload_version(raw: &str) -> TrustyResult { + let v: serde_json::Value = + serde_json::from_str(raw).map_err(|e| TrustyError::InvalidPayloadJson(e.to_string()))?; + v.as_object() + .and_then(|o| o.get("payload_version")) + .and_then(Value::as_u64) + .map(|n| n as u32) + .ok_or_else(|| TrustyError::InvalidPayloadJson("missing payload_version".into())) +} + +// Parse an incoming raw_payload string back into a PayloadFields for +// callers that want to recompute without holding the original struct. +// Rarely needed in trusty-core itself; exposed for ergonomics. +pub fn parse_raw_payload(raw: &str) -> TrustyResult { + let v: serde_json::Value = + serde_json::from_str(raw).map_err(|e| TrustyError::InvalidPayloadJson(e.to_string()))?; + let obj = v + .as_object() + .ok_or_else(|| TrustyError::InvalidPayloadJson("not a JSON object".into()))?; + + let string_field = |name: &str| -> TrustyResult { + obj.get(name) + .and_then(|v| v.as_str()) + .map(|s| s.to_string()) + .ok_or_else(|| TrustyError::InvalidPayloadJson(format!("missing {}", name))) + }; + let data = obj + .get("data") + .cloned() + .ok_or_else(|| TrustyError::InvalidPayloadJson("missing data".into()))?; + let sequence = obj + .get("sequence") + .and_then(|v| v.as_i64()) + .ok_or_else(|| TrustyError::InvalidPayloadJson("missing sequence".into()))?; + let payload_version = obj + .get("payload_version") + .and_then(|v| v.as_u64()) + .ok_or_else(|| TrustyError::InvalidPayloadJson("missing payload_version".into()))? + as u32; + let signatures = obj + .get("signatures") + .and_then(|v| v.as_array()) + .cloned() + .ok_or_else(|| TrustyError::InvalidPayloadJson("missing signatures".into()))?; + + Ok(PayloadFields { + entity_id: string_field("entity_id")?, + chain_id: string_field("chain_id")?, + data, + source_url: string_field("source_url")?, + detected_at: string_field("detected_at")?, + received_at: string_field("received_at")?, + extraction_method: string_field("extraction_method")?, + prev_hash: string_field("prev_hash")?, + sequence, + hash_alg: string_field("hash_alg")?, + payload_version, + signatures, + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn key() -> SecretKey { + SecretKey::from_bytes(b"0123456789abcdef").unwrap() + } + + fn sample_fields(seq: i64, prev: &str) -> PayloadFields { + PayloadFields::new_v1( + "acme", + make_chain_id("acme", 1), + json!({"v": seq}), + "s", + "2026-03-08T00:00:00Z", + "2026-03-08T00:00:00.123456Z", + "trusty", + prev, + seq, + ) + } + + // -------------------------------------------------------------- + // Resource ceilings. A bundle is an artifact handed over by another + // party -- often the one whose behaviour is in question -- so an + // unbounded walk over it is a DoS the verifier volunteers for. + // -------------------------------------------------------------- + + #[cfg(feature = "keygen")] + #[test] + fn row_ceiling_stops_the_walk_and_says_so() { + let links = signed_chain(6); + let limits = VerifyLimits { + max_rows: 3, + ..VerifyLimits::default() + }; + let report = verify_links_public_from_bounded("acme", &links, GENESIS_SENTINEL, &limits); + + assert!(report.limit_reached, "hitting the ceiling must be reported"); + assert_eq!(report.row_count, 3, "walk stopped at the ceiling"); + assert!( + !report.intact(), + "a verdict over a PREFIX must not read as intact" + ); + // The rows it did examine were fine -- the failure is coverage, not validity. + assert!(report.rows_valid, "{:?}", report.errors); + } + + #[cfg(feature = "keygen")] + #[test] + fn byte_ceiling_stops_the_walk() { + let links = signed_chain(6); + let limits = VerifyLimits { + max_total_bytes: 100, + ..VerifyLimits::default() + }; + let report = verify_links_public_from_bounded("acme", &links, GENESIS_SENTINEL, &limits); + assert!(report.limit_reached); + assert!(!report.intact()); + } + + #[cfg(feature = "keygen")] + #[test] + fn an_honest_chain_does_not_trip_the_defaults() { + let report = verify_links_public("acme", &signed_chain(4)); + assert!(!report.limit_reached); + assert!(report.intact(), "{:?}", report.errors); + } + + #[cfg(feature = "keygen")] + #[test] + fn untrusted_limits_are_tighter_than_default() { + assert!(VerifyLimits::untrusted().max_rows < VerifyLimits::default().max_rows); + assert!( + VerifyLimits::untrusted().max_total_bytes < VerifyLimits::default().max_total_bytes + ); + } + + // -------------------------------------------------------------- + // Keyless verification. None of these constructs a SecretKey for the + // public path -- that is the point: a ProofBundle holder has none. + // -------------------------------------------------------------- + + #[cfg(feature = "keygen")] + fn signed_chain(len: i64) -> Vec { + let signer = crate::crypto::Ed25519Signer::generate(); + let k = key(); + let chain_id = make_chain_id("acme", 1); + let mut links = Vec::new(); + let mut prev = GENESIS_SENTINEL.to_string(); + + for seq in 0..len { + let mut fields = sample_fields(seq, &prev); + fields.payload_version = PAYLOAD_VERSION_V2; + let rec = compute_record_signed(&fields, &k, &signer).unwrap(); + let event = if seq == 0 { + EventType::Genesis + } else { + EventType::Mutation + }; + links.push(link_from_record("acme", &chain_id, event, seq, &prev, &rec)); + prev = rec.row_hash.clone(); + } + links + } + + #[cfg(feature = "keygen")] + #[test] + fn signed_chain_verifies_with_no_secret_key() { + let report = verify_links_public("acme", &signed_chain(4)); + assert!( + report.intact(), + "expected clean verify, got {:?}", + report.errors + ); + assert!(report.rows_valid); + assert!(report.fully_signed); + assert!(report.continuous); + assert!(report.breaks.is_empty()); + assert_eq!(report.row_count, 4); + assert_eq!(report.signed_row_count, 4); + assert_eq!(report.unsigned_row_count, 0); + } + + #[cfg(feature = "keygen")] + #[test] + fn tampering_with_a_signed_row_is_caught_without_the_key() { + let mut links = signed_chain(3); + let raw = links[1].raw_payload.clone().unwrap(); + links[1].raw_payload = Some(raw.replace("\"v\":1", "\"v\":99")); + + let report = verify_links_public("acme", &links); + assert!(!report.rows_valid); + assert!( + report.errors.iter().any(|e| matches!( + e.kind, + VerifyErrorType::SignatureInvalid { .. } + | VerifyErrorType::InvalidRawPayload { .. } + )), + "expected a signature failure, got {:?}", + report.errors + ); + } + + #[cfg(feature = "keygen")] + #[test] + fn reordering_a_signed_chain_is_caught_without_the_key() { + let mut links = signed_chain(3); + links.swap(1, 2); + let report = verify_links_public("acme", &links); + assert!(!report.rows_valid); + assert!(report + .errors + .iter() + .any(|e| matches!(e.kind, VerifyErrorType::PrevHashMismatch { .. }))); + } + + #[cfg(feature = "keygen")] + #[test] + fn removing_an_interior_row_is_caught_without_the_key() { + let mut links = signed_chain(4); + links.remove(2); + let report = verify_links_public("acme", &links); + assert!(!report.rows_valid); + assert!(report + .errors + .iter() + .any(|e| matches!(e.kind, VerifyErrorType::PrevHashMismatch { .. }))); + } + + #[test] + fn unsigned_rows_are_counted_and_block_intact() { + // A v1 chain is well formed but carries no signatures, so nothing in + // it attests to authorship or contents. The rows are valid; the chain + // is not intact, and the two are now separate questions. + let k = key(); + let chain_id = make_chain_id("acme", 1); + let mut links = Vec::new(); + let mut prev = GENESIS_SENTINEL.to_string(); + + for seq in 0..3 { + let fields = sample_fields(seq, &prev); + let rec = compute_record(&fields, &k).unwrap(); + links.push(link_from_record( + "acme", + &chain_id, + EventType::Mutation, + seq, + &prev, + &rec, + )); + prev = rec.row_hash.clone(); + } + + let report = verify_links_public("acme", &links); + assert!(report.rows_valid, "structure is sound: {:?}", report.errors); + assert!(!report.fully_signed, "nothing attests to these rows"); + assert!( + !report.intact(), + "an unsigned chain must not verify publicly" + ); + assert_eq!(report.unsigned_row_count, 3); + assert_eq!(report.signed_row_count, 0); + // Structure is sound; the failure is lack of attestation. + assert!(report.errors.is_empty(), "{:?}", report.errors); + assert!(report.continuous); + } + + #[cfg(feature = "keygen")] + #[test] + fn a_recorded_break_is_reported_by_the_keyless_path_too() { + // The defect this mirrors: verify() used to return ok:true on a chain + // carrying its own tamper detection. The keyless path must not + // reintroduce it -- rows either side of a break are well formed, so + // `ok` stays true, and `continuous` is what goes false. + let mut links = signed_chain(2); + links.push(ChainLink { + entity_id: "acme".into(), + chain_id: make_chain_id("acme", 1), + event_type: EventType::Break.as_str().to_string(), + sequence: 2, + prev_hash: GENESIS_SENTINEL.into(), + row_hash: None, + anchor_hash: None, + raw_payload: None, + }); + + let report = verify_links_public("acme", &links); + assert_eq!(report.row_count, 2, "a BREAK is not a row"); + assert!(report.errors.is_empty(), "{:?}", report.errors); + assert!( + !report.continuous, + "a recorded break must show in the verdict" + ); + assert_eq!(report.breaks.len(), 1); + assert_eq!(report.breaks[0].index, 2); + } + + #[cfg(feature = "keygen")] + #[test] + fn keyless_windowed_verification_starts_from_a_checkpoint() { + // The keyless twin of verify_links_from: verify a tail without + // replaying the head, and without a secret key. + let links = signed_chain(4); + let checkpoint_prev = links[1].row_hash.clone().unwrap(); + + let report = verify_links_public_from("acme", &links[2..], &checkpoint_prev); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 2); + } + + fn link_from_record( + entity_id: &str, + chain_id: &str, + event: EventType, + seq: i64, + prev: &str, + rec: &ComputedRecord, + ) -> ChainLink { + ChainLink { + entity_id: entity_id.into(), + chain_id: chain_id.into(), + event_type: event.as_str().to_string(), + sequence: seq, + prev_hash: prev.into(), + row_hash: Some(rec.row_hash.clone()), + anchor_hash: Some(rec.anchor_hash.clone()), + raw_payload: Some(rec.raw_payload.clone()), + } + } + + #[test] + fn checkpoint_signing_bytes_layout_is_canonical() { + // Lock the byte layout: domain || chain_id || 0x00 || + // BE i64 sequence || 0x00 || row_hash_hex. + let out = checkpoint_signing_bytes("chain_acme_000001", 0x0102030405060708, "deadbeef"); + let mut expected = Vec::new(); + expected.extend_from_slice(b"trusty.checkpoint.v1\x00"); + expected.extend_from_slice(b"chain_acme_000001"); + expected.push(0x00); + expected.extend_from_slice(&[0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08]); + expected.push(0x00); + expected.extend_from_slice(b"deadbeef"); + assert_eq!(out, expected); + } + + #[test] + fn checkpoint_signing_bytes_distinguishes_negative_sequences() { + // i64 BE encoding: -1 is 0xFF..FF; 1 is 0x00..01. Adjacent + // sequences must produce distinct bytes (no truncation). + let a = checkpoint_signing_bytes("c", -1, "h"); + let b = checkpoint_signing_bytes("c", 1, "h"); + assert_ne!(a, b); + } + + #[test] + fn make_chain_id_zero_pads() { + assert_eq!(make_chain_id("acme", 1), "chain_acme_000001"); + assert_eq!(make_chain_id("acme", 42), "chain_acme_000042"); + assert_eq!(make_chain_id("acme", 999), "chain_acme_000999"); + } + + #[test] + fn compute_and_verify_single_row() { + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record(&f0, &k).unwrap(); + let link = link_from_record( + "acme", + &f0.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r0, + ); + let report = verify_links("acme", &[link], &k); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 1); + } + + #[test] + fn verify_links_detects_tampered_row_hash() { + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record(&f0, &k).unwrap(); + let mut link = link_from_record( + "acme", + &f0.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r0, + ); + link.row_hash = Some("deadbeefdeadbeef".into()); + let report = verify_links("acme", &[link], &k); + assert!(!report.rows_valid); + assert!(report + .errors + .iter() + .any(|e| matches!(e.kind, VerifyErrorType::RowHashMismatch { .. }))); + } + + #[test] + fn verify_links_detects_prev_hash_mismatch() { + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record(&f0, &k).unwrap(); + let l0 = link_from_record( + "acme", + &f0.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r0, + ); + + let f1 = sample_fields(1, "not-the-real-prev"); + let r1 = compute_record(&f1, &k).unwrap(); + let l1 = link_from_record( + "acme", + &f1.chain_id, + EventType::Mutation, + 1, + "not-the-real-prev", + &r1, + ); + + let report = verify_links("acme", &[l0, l1], &k); + assert!(!report.rows_valid); + assert!(report + .errors + .iter() + .any(|e| matches!(e.kind, VerifyErrorType::PrevHashMismatch { .. }))); + } + + #[test] + fn break_marker_resets_prev() { + let k = key(); + // seg1 + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record(&f0, &k).unwrap(); + let l0 = link_from_record( + "acme", + &f0.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r0, + ); + + // break marker + let brk = ChainLink { + entity_id: "acme".into(), + chain_id: f0.chain_id.clone(), + event_type: "BREAK".to_string(), + sequence: -1, + prev_hash: r0.row_hash.clone(), + row_hash: None, + anchor_hash: None, + raw_payload: None, + }; + + // seg2 starts fresh from genesis sentinel under new chain id + let mut f2 = sample_fields(0, GENESIS_SENTINEL); + f2.chain_id = make_chain_id("acme", 2); + let r2 = compute_record(&f2, &k).unwrap(); + let l2 = link_from_record( + "acme", + &f2.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r2, + ); + + let report = verify_links("acme", &[l0, brk, l2], &k); + // `ok` stays true, and that is CORRECT: both segments are + // well-formed. It is also exactly why it was misleading on its + // own -- a caller reading only this field learns nothing about + // the break sitting between them. + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 2); + + // The report now says what `ok` never could. + assert!(!report.continuous, "a chain with a BREAK is not continuous"); + assert_eq!(report.breaks.len(), 1); + assert_eq!( + report.breaks[0].index, 1, + "the break sat between the two segments" + ); + assert_eq!( + report.breaks[0].chain_id, f0.chain_id, + "the generation that ENDED" + ); + } + + /// The control for the assertions above. Without it, `continuous` + /// hardcoded to `false` would satisfy every break test in this file + /// and nothing would notice -- a field that is always alarming is as + /// useless as one that is never alarming. + #[test] + fn an_unbroken_chain_reports_continuous() { + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record(&f0, &k).unwrap(); + let l0 = link_from_record( + "acme", + &f0.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r0, + ); + + let f1 = sample_fields(1, &r0.row_hash); + let r1 = compute_record(&f1, &k).unwrap(); + let l1 = link_from_record( + "acme", + &f1.chain_id, + EventType::Mutation, + 1, + &r0.row_hash, + &r1, + ); + + let report = verify_links("acme", &[l0, l1], &k); + assert!(report.rows_valid, "{:?}", report.errors); + assert!(report.continuous); + assert!(report.breaks.is_empty()); + } + + /// The two fields are INDEPENDENT, and this is the case that proves + /// it: a chain can be broken AND have a tampered row. Reporting one + /// must not mask the other. + #[test] + fn a_broken_chain_with_a_bad_row_reports_both() { + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record(&f0, &k).unwrap(); + let mut l0 = link_from_record( + "acme", + &f0.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r0, + ); + l0.row_hash = Some("deadbeefdeadbeef".into()); + + let brk = ChainLink { + entity_id: "acme".into(), + chain_id: f0.chain_id.clone(), + event_type: "BREAK".to_string(), + sequence: -1, + prev_hash: r0.row_hash.clone(), + row_hash: None, + anchor_hash: None, + raw_payload: None, + }; + + let report = verify_links("acme", &[l0, brk], &k); + assert!( + !report.rows_valid, + "the tampered row must still fail verification" + ); + assert!(!report.continuous, "and the break must still be reported"); + assert_eq!(report.breaks.len(), 1); + } + + fn signer() -> crate::crypto::Ed25519Signer { + let seed = [7u8; 32]; + crate::crypto::Ed25519Signer::from_secret_bytes(&seed).unwrap() + } + + #[test] + fn v2_signed_record_roundtrips_through_verify_links() { + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record_signed(&f0, &k, &signer()).unwrap(); + assert!(r0.raw_payload.contains("\"payload_version\":2")); + assert!(r0.raw_payload.contains("\"alg\":\"ed25519\"")); + + let link = link_from_record( + "acme", + &f0.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r0, + ); + let report = verify_links("acme", &[link], &k); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 1); + } + + fn mutate_sig_bytes(raw: &str) -> String { + let mut parsed = parse_raw_payload(raw).unwrap(); + let obj = parsed.signatures[0].as_object_mut().unwrap(); + let sig_hex = obj.get("sig").unwrap().as_str().unwrap().to_string(); + let mut bytes = hex::decode(&sig_hex).unwrap(); + bytes[0] ^= 0xff; + obj.insert("sig".into(), json!(hex::encode(&bytes))); + build_payload(&parsed).unwrap() + } + + #[test] + fn v2_tampered_signature_fails_verify() { + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record_signed(&f0, &k, &signer()).unwrap(); + + // Corrupt the sig bytes but recompute row_hash so only the + // signature check (not row_hash) fires the failure. + let mutated = mutate_sig_bytes(&r0.raw_payload); + assert_ne!(mutated, r0.raw_payload, "mutation setup failed"); + let new_rh = row_hash(&k, mutated.as_bytes()); + let new_ah = anchor_hash(&new_rh, &f0.entity_id, &f0.chain_id); + + let link = ChainLink { + entity_id: f0.entity_id.clone(), + chain_id: f0.chain_id.clone(), + event_type: EventType::Genesis.as_str().to_string(), + sequence: 0, + prev_hash: GENESIS_SENTINEL.into(), + row_hash: Some(new_rh), + anchor_hash: Some(new_ah), + raw_payload: Some(mutated), + }; + let report = verify_links("acme", &[link], &k); + assert!(!report.rows_valid); + assert!(report + .errors + .iter() + .any(|e| matches!(e.kind, VerifyErrorType::SignatureInvalid { .. }))); + } + + #[test] + fn v2_unknown_alg_flagged() { + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record_signed(&f0, &k, &signer()).unwrap(); + let mutated = r0.raw_payload.replace("\"ed25519\"", "\"rsa4096\""); + let new_rh = row_hash(&k, mutated.as_bytes()); + let new_ah = anchor_hash(&new_rh, &f0.entity_id, &f0.chain_id); + let link = ChainLink { + entity_id: f0.entity_id.clone(), + chain_id: f0.chain_id.clone(), + event_type: EventType::Genesis.as_str().to_string(), + sequence: 0, + prev_hash: GENESIS_SENTINEL.into(), + row_hash: Some(new_rh), + anchor_hash: Some(new_ah), + raw_payload: Some(mutated), + }; + let report = verify_links("acme", &[link], &k); + assert!(!report.rows_valid); + assert!(report + .errors + .iter() + .any(|e| matches!(e.kind, VerifyErrorType::UnknownSignatureAlg { .. }))); + } + + #[test] + fn v2_signatures_field_affects_row_hash() { + // row_hash must commit to the signature bytes: if you swap the + // sig value and keep row_hash stable, RowHashMismatch fires. + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record_signed(&f0, &k, &signer()).unwrap(); + let mutated = mutate_sig_bytes(&r0.raw_payload); + let link = ChainLink { + entity_id: f0.entity_id.clone(), + chain_id: f0.chain_id.clone(), + event_type: EventType::Genesis.as_str().to_string(), + sequence: 0, + prev_hash: GENESIS_SENTINEL.into(), + row_hash: Some(r0.row_hash.clone()), + anchor_hash: Some(r0.anchor_hash.clone()), + raw_payload: Some(mutated), + }; + let report = verify_links("acme", &[link], &k); + assert!(!report.rows_valid); + assert!(report + .errors + .iter() + .any(|e| matches!(e.kind, VerifyErrorType::RowHashMismatch { .. }))); + } + + fn sample_fields_v3(seq: i64, prev: &str) -> PayloadFields { + PayloadFields::new_v3( + "acme", + make_chain_id("acme", 1), + json!({"v": seq}), + "s", + "2026-03-08T00:00:00Z", + "2026-03-08T00:00:00.123456Z", + "trusty", + prev, + seq, + ) + } + + #[test] + fn v3_row_hash_prefixed_with_domain_tag() { + let k = key(); + let f1 = sample_fields(0, GENESIS_SENTINEL); + let mut f3 = sample_fields_v3(0, GENESIS_SENTINEL); + // Make the build_payload output for v3 equal to v1 except for + // payload_version, which is the only field that should differ. + // row_hash must still differ because of the domain prefix. + f3.entity_id = f1.entity_id.clone(); + f3.chain_id = f1.chain_id.clone(); + let r1 = compute_record(&f1, &k).unwrap(); + let r3 = compute_record(&f3, &k).unwrap(); + assert_ne!(r1.row_hash, r3.row_hash, "v3 must use domain-tagged input"); + assert!(r3.raw_payload.contains("\"payload_version\":3")); + } + + #[test] + fn v3_anchor_hash_prefixed_with_domain_tag() { + let k = key(); + let f3 = sample_fields_v3(0, GENESIS_SENTINEL); + let r3 = compute_record(&f3, &k).unwrap(); + // anchor_hash (v1/v2) over the v3 row_hash would produce a + // different digest than the v3 anchor_hash. Assert the stored + // anchor matches the v3 variant, not the untagged one. + let untagged = anchor_hash(&r3.row_hash, &f3.entity_id, &f3.chain_id); + assert_ne!(r3.anchor_hash, untagged); + } + + #[test] + fn v3_signed_record_verifies() { + // v3 signed path (S16, resolves S14 OQ #17 / s15 HARD BLOCKER): + // compute_record_signed dispatches v3 to the domain-tagged + // hash funcs while keeping v2 callers byte-identical. + let k = key(); + let f3 = sample_fields_v3(0, GENESIS_SENTINEL); + let r3 = compute_record_signed(&f3, &k, &signer()).unwrap(); + assert!(r3.raw_payload.contains("\"payload_version\":3")); + assert!(r3.raw_payload.contains("\"alg\":\"ed25519\"")); + + // v3 anchor must differ from the untagged anchor over the same + // row_hash -- proves the v3 path actually applied the domain tag. + let untagged_anchor = anchor_hash(&r3.row_hash, &f3.entity_id, &f3.chain_id); + assert_ne!( + r3.anchor_hash, untagged_anchor, + "v3 signed path must use anchor_hash_v3" + ); + + let link = link_from_record( + "acme", + &f3.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r3, + ); + let report = verify_links("acme", &[link], &k); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 1); + } + + #[test] + fn v3_tampered_signature_fails_verify() { + // Same shape as v2 tampered test but on the v3 signed path. + let k = key(); + let f3 = sample_fields_v3(0, GENESIS_SENTINEL); + let r3 = compute_record_signed(&f3, &k, &signer()).unwrap(); + + let mutated = mutate_sig_bytes(&r3.raw_payload); + assert_ne!(mutated, r3.raw_payload, "mutation setup failed"); + // Recompute v3 hashes so row_hash and anchor_hash both pass and + // only the signature check fires the failure. + let new_rh = row_hash_for_version(3, &k, mutated.as_bytes()).unwrap(); + let new_ah = anchor_hash_for_version(3, &new_rh, &f3.entity_id, &f3.chain_id).unwrap(); + + let link = ChainLink { + entity_id: f3.entity_id.clone(), + chain_id: f3.chain_id.clone(), + event_type: EventType::Genesis.as_str().to_string(), + sequence: 0, + prev_hash: GENESIS_SENTINEL.into(), + row_hash: Some(new_rh), + anchor_hash: Some(new_ah), + raw_payload: Some(mutated), + }; + + let report = verify_links("acme", &[link], &k); + assert!(!report.rows_valid); + assert!( + report + .errors + .iter() + .any(|e| matches!(e.kind, VerifyErrorType::SignatureInvalid { .. })), + "expected SignatureInvalid, got {:?}", + report.errors + ); + } + + #[test] + fn compute_record_signed_v1_field_upgrades_to_v2() { + // Backward-compat: callers that pass v1 PayloadFields to + // compute_record_signed get v2 output (existing behavior). + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + assert_eq!(f0.payload_version, 1); + let r0 = compute_record_signed(&f0, &k, &signer()).unwrap(); + assert!(r0.raw_payload.contains("\"payload_version\":2")); + } + + #[test] + fn verify_links_dispatches_by_payload_version() { + // Mixed chain: row 0 is v1, row 1 is v3 (second chain segment). + // Both must verify with the same key, using their own domain. + let k = key(); + + let f1 = sample_fields(0, GENESIS_SENTINEL); + let r1 = compute_record(&f1, &k).unwrap(); + let l1 = link_from_record( + "acme", + &f1.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r1, + ); + + let mut f3 = sample_fields_v3(1, &r1.row_hash); + f3.chain_id = f1.chain_id.clone(); + let r3 = compute_record(&f3, &k).unwrap(); + let l3 = link_from_record( + "acme", + &f3.chain_id, + EventType::Mutation, + 1, + &r1.row_hash, + &r3, + ); + + let report = verify_links("acme", &[l1, l3], &k); + assert!(report.rows_valid, "{:?}", report.errors); + assert_eq!(report.row_count, 2); + } + + #[test] + fn unknown_payload_version_rejected_by_compute() { + let k = key(); + let mut f = sample_fields(0, GENESIS_SENTINEL); + f.payload_version = 99; + let err = compute_record(&f, &k).unwrap_err(); + assert!(matches!(err, TrustyError::UnsupportedPayloadVersion(99))); + } + + #[test] + fn unknown_payload_version_rejected_by_verify() { + // Craft a row whose raw_payload carries payload_version:99. + // verify_links must flag it via InvalidRawPayload rather than + // silently hashing with v1 primitives. + let k = key(); + let f = sample_fields(0, GENESIS_SENTINEL); + let raw = build_payload(&f) + .unwrap() + .replace("\"payload_version\":1", "\"payload_version\":99"); + let link = ChainLink { + entity_id: f.entity_id.clone(), + chain_id: f.chain_id.clone(), + event_type: EventType::Genesis.as_str().to_string(), + sequence: 0, + prev_hash: GENESIS_SENTINEL.into(), + row_hash: Some("0000000000000000".into()), + anchor_hash: Some("0".repeat(64)), + raw_payload: Some(raw), + }; + let report = verify_links("acme", &[link], &k); + assert!(!report.rows_valid); + assert!(report.errors.iter().any(|e| matches!( + &e.kind, + VerifyErrorType::InvalidRawPayload { detail } if detail.contains("unsupported payload_version") + ))); + } + + #[test] + fn v1_record_ignored_by_signature_verify() { + let k = key(); + let f0 = sample_fields(0, GENESIS_SENTINEL); + let r0 = compute_record(&f0, &k).unwrap(); + assert!(r0.raw_payload.contains("\"payload_version\":1")); + assert!(r0.raw_payload.contains("\"signatures\":[]")); + let link = link_from_record( + "acme", + &f0.chain_id, + EventType::Genesis, + 0, + GENESIS_SENTINEL, + &r0, + ); + let report = verify_links("acme", &[link], &k); + assert!(report.rows_valid, "{:?}", report.errors); + } +} diff --git a/crates/core/trusty-core/src/conditions.rs b/crates/core/trusty-core/src/conditions.rs new file mode 100644 index 0000000..6ed6039 --- /dev/null +++ b/crates/core/trusty-core/src/conditions.rs @@ -0,0 +1,334 @@ +// Pluggable break conditions. +// +// A condition evaluates the state at append time and votes Pass or +// Fail(reason). First failure wins — subsequent conditions do not +// run. Default set: HashIntegrity, SequenceContinuity. SourceDomain +// exists but is opt-in (requires a domain list). +// +// Byte-identical vote outcomes to the Python reference at +// trusty_core_watermark_proxy/core/trusty/conditions/defaults.py + +#[derive(Debug, Clone)] +pub struct ConditionContext { + pub entity_id: String, + pub chain_id: String, + pub sequence: i64, + pub stored_tip_hash: Option, + pub computed_hash: Option, + pub prev_hash: Option, + pub last_sequence: Option, + pub source_url: String, + pub extraction_method: String, + /// The MEASURED DATA being appended (the payload). S126 gap B: this is what + /// lets a condition monitor the DATA PLANE, not just chain metadata -- e.g. a + /// threshold over a measured metric (`latency_ms > 500`) evaluated at append + /// time, recording a break that names the variable and the observed value. + /// `Value::Null` when the backend supplies no data (older callers, tests). + pub data: serde_json::Value, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum BreakReason { + TamperDetected, + SequenceGap, + SourceChange, + ManualReset, + SchemaMigration, + ConditionEvaluatorError, + /// A declared RUN RULE / invariant was violated at append time -- carries the + /// rule's name so the recorded break says WHICH invariant broke, instead of + /// being mislabelled `SequenceGap` (the S126 gap A). ADDITIVE and safe: BREAK + /// rows are skipped in hash verification (`chain::verify_links`), so the break + /// reason was never part of the hashed payload -- existing chains are + /// unaffected. The rule name travels via `code()`, never via the frozen + /// `as_code()`. + PropertyViolated(String), +} + +impl BreakReason { + /// The FROZEN category code (`&'static str`). Unchanged for existing variants + /// so any historical use is byte-stable; `PropertyViolated` reports its + /// category here and its rule name via [`code`](Self::code). + pub fn as_code(&self) -> &'static str { + match self { + BreakReason::TamperDetected => "TAMPER_DETECTED", + BreakReason::SequenceGap => "SEQUENCE_GAP", + BreakReason::SourceChange => "SOURCE_CHANGE", + BreakReason::ManualReset => "MANUAL_RESET", + BreakReason::SchemaMigration => "SCHEMA_MIGRATION", + BreakReason::ConditionEvaluatorError => "CONDITION_EVALUATOR_ERROR", + BreakReason::PropertyViolated(_) => "PROPERTY_VIOLATED", + } + } + + /// The recorded reason string: the category, plus the rule name for a + /// `PropertyViolated` (`PROPERTY_VIOLATED:`). This is what a break row + /// stores, so the answer to "which invariant broke?" travels on the chain. + pub fn code(&self) -> String { + match self { + BreakReason::PropertyViolated(rule) => format!("PROPERTY_VIOLATED:{rule}"), + other => other.as_code().to_string(), + } + } + + /// The rule name, when this break is a `PropertyViolated`. `None` otherwise. + pub fn rule(&self) -> Option<&str> { + match self { + BreakReason::PropertyViolated(rule) => Some(rule), + _ => None, + } + } +} + +#[derive(Debug, Clone)] +pub enum ConditionOutcome { + Pass, + Fail(BreakReason), +} + +pub trait Condition: Send + Sync { + fn name(&self) -> &'static str; + fn evaluate(&self, ctx: &ConditionContext) -> ConditionOutcome; +} + +pub struct HashIntegrity; + +impl Condition for HashIntegrity { + fn name(&self) -> &'static str { + "hash_integrity" + } + + fn evaluate(&self, ctx: &ConditionContext) -> ConditionOutcome { + let stored = &ctx.stored_tip_hash; + let computed = &ctx.computed_hash; + + // Genesis — first ingestion for this entity. + if stored.is_none() && ctx.sequence == 0 { + return ConditionOutcome::Pass; + } + + // Stored matches computed — chain intact. + if let (Some(s), Some(c)) = (stored.as_ref(), computed.as_ref()) { + if s == c { + return ConditionOutcome::Pass; + } + return ConditionOutcome::Fail(BreakReason::TamperDetected); + } + + // stored present, computed missing — cannot compare, treat as fail. + if stored.is_some() && computed.is_none() { + return ConditionOutcome::Fail(BreakReason::TamperDetected); + } + + // stored=None, sequence>0 — unusual but recoverable. + ConditionOutcome::Pass + } +} + +pub struct SequenceContinuity; + +impl Condition for SequenceContinuity { + fn name(&self) -> &'static str { + "sequence_continuity" + } + + fn evaluate(&self, ctx: &ConditionContext) -> ConditionOutcome { + match ctx.last_sequence { + None => ConditionOutcome::Pass, + Some(last) => { + if ctx.sequence == last + 1 { + ConditionOutcome::Pass + } else { + ConditionOutcome::Fail(BreakReason::SequenceGap) + } + } + } + } +} + +pub struct SourceDomain { + pub trusted_domains: Vec, + pub hard: bool, +} + +impl SourceDomain { + pub fn new(trusted_domains: Vec, hard: bool) -> Self { + Self { + trusted_domains, + hard, + } + } +} + +impl Condition for SourceDomain { + fn name(&self) -> &'static str { + "source_domain" + } + + fn evaluate(&self, ctx: &ConditionContext) -> ConditionOutcome { + if ctx.source_url.is_empty() || self.trusted_domains.is_empty() { + return ConditionOutcome::Pass; + } + let ok = self + .trusted_domains + .iter() + .any(|d| ctx.source_url.contains(d)); + if ok { + ConditionOutcome::Pass + } else if self.hard { + ConditionOutcome::Fail(BreakReason::SourceChange) + } else { + ConditionOutcome::Pass + } + } +} + +pub fn default_conditions() -> Vec> { + vec![Box::new(HashIntegrity), Box::new(SequenceContinuity)] +} + +pub fn evaluate(conditions: &[Box], ctx: &ConditionContext) -> ConditionOutcome { + for c in conditions { + match c.evaluate(ctx) { + ConditionOutcome::Pass => continue, + ConditionOutcome::Fail(reason) => return ConditionOutcome::Fail(reason), + } + } + ConditionOutcome::Pass +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn property_violated_carries_the_rule_name_via_code_not_the_frozen_as_code() { + let r = BreakReason::PropertyViolated("anchor-before-commit".into()); + assert_eq!(r.as_code(), "PROPERTY_VIOLATED", "frozen category"); + assert_eq!( + r.code(), + "PROPERTY_VIOLATED:anchor-before-commit", + "recorded reason names the rule" + ); + assert_eq!(r.rule(), Some("anchor-before-commit")); + // Existing variants are byte-stable: code() == as_code() for them, so no + // historical break row changes meaning. + assert_eq!(BreakReason::SequenceGap.code(), "SEQUENCE_GAP"); + assert_eq!(BreakReason::TamperDetected.rule(), None); + } + + fn ctx_genesis() -> ConditionContext { + ConditionContext { + data: serde_json::Value::Null, + entity_id: "acme".into(), + chain_id: "chain_acme_001".into(), + sequence: 0, + stored_tip_hash: None, + computed_hash: None, + prev_hash: None, + last_sequence: None, + source_url: "s".into(), + extraction_method: "trusty".into(), + } + } + + #[test] + fn hash_integrity_passes_on_genesis() { + let c = HashIntegrity; + matches!(c.evaluate(&ctx_genesis()), ConditionOutcome::Pass); + } + + #[test] + fn hash_integrity_fails_on_mismatch() { + let mut ctx = ctx_genesis(); + ctx.sequence = 1; + ctx.stored_tip_hash = Some("abc".into()); + ctx.computed_hash = Some("def".into()); + ctx.last_sequence = Some(0); + let out = HashIntegrity.evaluate(&ctx); + assert!(matches!( + out, + ConditionOutcome::Fail(BreakReason::TamperDetected) + )); + } + + #[test] + fn hash_integrity_passes_when_match() { + let mut ctx = ctx_genesis(); + ctx.sequence = 1; + ctx.stored_tip_hash = Some("same".into()); + ctx.computed_hash = Some("same".into()); + ctx.last_sequence = Some(0); + assert!(matches!( + HashIntegrity.evaluate(&ctx), + ConditionOutcome::Pass + )); + } + + #[test] + fn sequence_continuity_passes_first_row() { + assert!(matches!( + SequenceContinuity.evaluate(&ctx_genesis()), + ConditionOutcome::Pass + )); + } + + #[test] + fn sequence_continuity_fails_on_gap() { + let mut ctx = ctx_genesis(); + ctx.last_sequence = Some(5); + ctx.sequence = 7; + let out = SequenceContinuity.evaluate(&ctx); + assert!(matches!( + out, + ConditionOutcome::Fail(BreakReason::SequenceGap) + )); + } + + #[test] + fn sequence_continuity_passes_on_next() { + let mut ctx = ctx_genesis(); + ctx.last_sequence = Some(5); + ctx.sequence = 6; + assert!(matches!( + SequenceContinuity.evaluate(&ctx), + ConditionOutcome::Pass + )); + } + + #[test] + fn evaluate_returns_first_failure() { + let conditions: Vec> = + vec![Box::new(HashIntegrity), Box::new(SequenceContinuity)]; + let mut ctx = ctx_genesis(); + ctx.sequence = 7; + ctx.last_sequence = Some(5); + ctx.stored_tip_hash = Some("abc".into()); + ctx.computed_hash = Some("def".into()); + // Both conditions fail — HashIntegrity runs first, wins. + let out = evaluate(&conditions, &ctx); + assert!(matches!( + out, + ConditionOutcome::Fail(BreakReason::TamperDetected) + )); + } + + #[test] + fn source_domain_soft_does_not_break() { + let cond = SourceDomain::new(vec!["sec.gov".into()], false); + let mut ctx = ctx_genesis(); + ctx.source_url = "http://evil.example/data".into(); + assert!(matches!(cond.evaluate(&ctx), ConditionOutcome::Pass)); + } + + #[test] + fn source_domain_hard_breaks_on_mismatch() { + let cond = SourceDomain::new(vec!["sec.gov".into()], true); + let mut ctx = ctx_genesis(); + ctx.source_url = "http://evil.example/data".into(); + assert!(matches!( + cond.evaluate(&ctx), + ConditionOutcome::Fail(BreakReason::SourceChange) + )); + } +} diff --git a/crates/core/trusty-core/src/credential.rs b/crates/core/trusty-core/src/credential.rs new file mode 100644 index 0000000..85958ed --- /dev/null +++ b/crates/core/trusty-core/src/credential.rs @@ -0,0 +1,160 @@ +//! What kind of key a producer enrolled, and what may be done with it. +//! +//! # The failure this prevents +//! +//! A `user` principal authenticates with WebAuthn. The credential is a +//! **COSE key**, and the authenticator does not sign the message a +//! relying party cares about -- it signs +//! `authenticatorData ‖ SHA-256(clientDataJSON)`, with the challenge +//! buried inside the client data. +//! +//! An `agent` or `program` holds a raw Ed25519 key and produces a bare +//! signature over the message itself. +//! +//! These are not interchangeable, and the dangerous part is how they +//! fail together. Handing a COSE key and a WebAuthn assertion to a bare +//! Ed25519 verifier does not produce a clean error -- it produces a +//! verification failure that looks like a wrong key, which invites +//! somebody to "fix" it by loosening the check. A registry with one +//! column and one verifier makes that mistake available; this makes it +//! unavailable. +//! +//! # Where the boundary is +//! +//! Trusty stores the format and **refuses** to verify what it cannot +//! verify. It does not verify WebAuthn assertions, deliberately: doing +//! that requires the relying party's RP ID, origin, and challenge +//! policy, none of which trusty has any business knowing. A half-built +//! WebAuthn verifier is worse than none, because it looks like one. +//! +//! The consumer that *is* the relying party verifies the assertion and +//! tells trusty the outcome. + +use serde::{Deserialize, Serialize}; + +use crate::crypto::CryptoError; + +/// How a producer's enrolled key material is encoded, and therefore how +/// a signature from it must be checked. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize, Default)] +#[serde(rename_all = "kebab-case")] +pub enum KeyFormat { + /// A raw 32-byte Ed25519 public key. The signature covers the + /// message directly. This is what an agent or program enrols. + #[default] + Ed25519Raw, + /// A COSE_Key from a WebAuthn authenticator. The signature covers + /// `authenticatorData ‖ SHA-256(clientDataJSON)`, not the message, + /// so it cannot be checked without the assertion that produced it. + /// This is what a user enrols. + Cose, +} + +impl KeyFormat { + pub fn as_str(self) -> &'static str { + match self { + Self::Ed25519Raw => "ed25519-raw", + Self::Cose => "cose", + } + } + + /// Parse a stored format string. + /// + /// An unrecognised value is an error, not a default. Defaulting to + /// `Ed25519Raw` would silently route a credential of unknown shape + /// into the bare-signature path -- the exact failure this type + /// exists to prevent. + pub fn parse(s: &str) -> Result { + match s { + "ed25519-raw" => Ok(Self::Ed25519Raw), + "cose" => Ok(Self::Cose), + other => Err(CryptoError::UnsupportedCredential(format!( + "unknown key format {other:?}; refusing rather than guessing, \ + because guessing wrong means checking a WebAuthn credential \ + as a bare signature" + ))), + } + } + + /// Whether a bare signature over the message is the right check. + pub fn is_bare_signature(self) -> bool { + matches!(self, Self::Ed25519Raw) + } +} + +/// Refuse to verify a credential whose format is not bare-signature. +/// +/// Call this before any `Verifier::verify` that uses a key from the +/// producer registry. Returns the error a caller should surface rather +/// than working around. +pub fn require_bare_signature(format: KeyFormat, producer_oid: &str) -> Result<(), CryptoError> { + if format.is_bare_signature() { + return Ok(()); + } + Err(CryptoError::UnsupportedCredential(format!( + "producer {producer_oid:?} enrolled a {} credential, which signs \ + `authenticatorData || SHA-256(clientDataJSON)` rather than the message. \ + It cannot be checked as a bare signature. Verify the WebAuthn assertion \ + at the relying party -- which holds the RP ID, origin and challenge -- \ + and record the outcome; trusty does not verify assertions and will not \ + pretend to.", + format.as_str() + ))) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_cose_credential_cannot_be_checked_as_a_bare_signature() { + let err = require_bare_signature(KeyFormat::Cose, "user:01H") + .expect_err("must refuse"); + let msg = err.to_string(); + assert!( + msg.contains("clientDataJSON"), + "the refusal must say WHY, or the next person will assume a wrong \ + key and loosen the check to make it pass: {msg}" + ); + assert!( + msg.contains("relying party"), + "and it must say who should verify instead, or the reader is left \ + with a refusal and nowhere to go: {msg}" + ); + } + + #[test] + fn a_raw_ed25519_credential_is_allowed_through() { + assert!(require_bare_signature(KeyFormat::Ed25519Raw, "agent:01J").is_ok()); + } + + #[test] + fn an_unknown_format_refuses_rather_than_defaulting() { + assert!( + KeyFormat::parse("something-new").is_err(), + "defaulting an unrecognised format to raw Ed25519 would route a \ + credential of unknown shape into the bare-signature path, which \ + is the whole failure this type prevents" + ); + // Default is still Ed25519Raw for a row written before the + // column existed -- but that is a schema default applied at + // write time, never a parse fallback. + assert_eq!(KeyFormat::default(), KeyFormat::Ed25519Raw); + } + + #[test] + fn the_stored_strings_round_trip() { + for f in [KeyFormat::Ed25519Raw, KeyFormat::Cose] { + assert_eq!(KeyFormat::parse(f.as_str()).expect("round trip"), f); + } + } + + #[test] + fn the_two_formats_are_distinguishable_in_storage() { + assert_ne!( + KeyFormat::Ed25519Raw.as_str(), + KeyFormat::Cose.as_str(), + "two formats sharing a stored value would make the column useless" + ); + } +} diff --git a/crates/core/trusty-core/src/crypto.rs b/crates/core/trusty-core/src/crypto.rs new file mode 100644 index 0000000..752fab4 --- /dev/null +++ b/crates/core/trusty-core/src/crypto.rs @@ -0,0 +1,198 @@ +// ============================================================ +// MODULE: trusty_core::crypto +// PURPOSE: Signer/Verifier trait surface + Ed25519 default impl. +// Reserved for payload signatures (v2 payload.signatures field). +// STATUS: Wired. `chain::compute_record_signed` embeds a signature (with the +// public key inline) into `payload.signatures`, and +// `chain::verify_links` verifies it via `verify_row_signatures`. +// v1 payloads carry `signatures = []` by definition, so verification +// is a no-op for them; v2 and v3 are signed and checked. +// +// This read "Scaffold. Chain-payload integration deferred to a later +// session." -- true when written, no longer. Corrected because a +// reader who trusts the header concludes signatures do not work, and +// they are the one part of a chain that a party WITHOUT the SipHash +// key can verify. +// +// DESIGN: +// - Trait surface is byte-oriented (Vec / &[u8]) so chain +// code can remain free of signature-crate type leakage. +// - Ed25519Signer wraps ed25519-dalek's SigningKey. Generation +// uses OsRng (rand crate). Deterministic reconstruction via +// from_secret_bytes for test vectors and key-escrow flows. +// - Ed25519Verifier is stateless (verify is a pure function of +// pk, msg, sig bytes). Kept as a unit struct to carry the +// Verifier trait impl. +// ============================================================ + +use ed25519_dalek::{ + Signature as DalekSignature, Signer as DalekSigner, SigningKey, Verifier as DalekVerifier, + VerifyingKey, PUBLIC_KEY_LENGTH, SECRET_KEY_LENGTH, SIGNATURE_LENGTH, +}; +#[cfg(feature = "keygen")] +use rand::rngs::OsRng; + +#[derive(Debug, thiserror::Error)] +#[non_exhaustive] +pub enum CryptoError { + #[error("invalid secret key length (expected 32, got {0})")] + InvalidSecretLen(usize), + #[error("invalid public key: {0}")] + InvalidPublicKey(String), + #[error("invalid signature length (expected 64, got {0})")] + InvalidSignatureLen(usize), + #[error("signature verification failed")] + VerifyFailed, + /// The credential is well-formed but cannot be checked this way. + /// + /// Distinct from `VerifyFailed` on purpose: a failed check invites + /// "wrong key, try another", while this says the check itself is the + /// wrong one. Conflating them is how a WebAuthn credential ends up + /// being verified as a bare signature. + #[error("unsupported credential: {0}")] + UnsupportedCredential(String), +} + +pub trait Signer { + fn sign(&self, msg: &[u8]) -> Vec; + fn public_key_bytes(&self) -> Vec; +} + +pub trait Verifier { + fn verify(pk_bytes: &[u8], msg: &[u8], sig_bytes: &[u8]) -> Result<(), CryptoError>; +} + +pub struct Ed25519Signer { + key: SigningKey, +} + +impl Ed25519Signer { + /// Generate a fresh key pair from the operating-system RNG. + /// + /// Requires the `keygen` feature (on by default). It is the only + /// RNG-dependent item in this crate; a `default-features = false` build + /// keeps every verification and reconstruction path and drops just this, + /// which is what makes the crate usable on `wasm32`. + #[cfg(feature = "keygen")] + pub fn generate() -> Self { + Self { + key: SigningKey::generate(&mut OsRng), + } + } + + pub fn from_secret_bytes(bytes: &[u8]) -> Result { + if bytes.len() != SECRET_KEY_LENGTH { + return Err(CryptoError::InvalidSecretLen(bytes.len())); + } + let mut arr = [0u8; SECRET_KEY_LENGTH]; + arr.copy_from_slice(bytes); + Ok(Self { + key: SigningKey::from_bytes(&arr), + }) + } + + pub fn secret_bytes(&self) -> [u8; SECRET_KEY_LENGTH] { + self.key.to_bytes() + } +} + +impl Signer for Ed25519Signer { + fn sign(&self, msg: &[u8]) -> Vec { + DalekSigner::sign(&self.key, msg).to_bytes().to_vec() + } + + fn public_key_bytes(&self) -> Vec { + self.key.verifying_key().to_bytes().to_vec() + } +} + +pub struct Ed25519Verifier; + +impl Verifier for Ed25519Verifier { + fn verify(pk_bytes: &[u8], msg: &[u8], sig_bytes: &[u8]) -> Result<(), CryptoError> { + if pk_bytes.len() != PUBLIC_KEY_LENGTH { + return Err(CryptoError::InvalidPublicKey(format!( + "expected {} bytes, got {}", + PUBLIC_KEY_LENGTH, + pk_bytes.len() + ))); + } + if sig_bytes.len() != SIGNATURE_LENGTH { + return Err(CryptoError::InvalidSignatureLen(sig_bytes.len())); + } + let mut pk_arr = [0u8; PUBLIC_KEY_LENGTH]; + pk_arr.copy_from_slice(pk_bytes); + let vk = VerifyingKey::from_bytes(&pk_arr) + .map_err(|e| CryptoError::InvalidPublicKey(e.to_string()))?; + let mut sig_arr = [0u8; SIGNATURE_LENGTH]; + sig_arr.copy_from_slice(sig_bytes); + let sig = DalekSignature::from_bytes(&sig_arr); + vk.verify(msg, &sig).map_err(|_| CryptoError::VerifyFailed) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[cfg(feature = "keygen")] + #[test] + fn generated_key_roundtrip() { + let signer = Ed25519Signer::generate(); + let msg = b"trusty chain payload v2"; + let sig = signer.sign(msg); + let pk = signer.public_key_bytes(); + assert_eq!(sig.len(), SIGNATURE_LENGTH); + assert_eq!(pk.len(), PUBLIC_KEY_LENGTH); + assert!(Ed25519Verifier::verify(&pk, msg, &sig).is_ok()); + } + + #[test] + fn deterministic_signature_from_fixed_seed() { + // Ed25519 is deterministic: same key + same msg -> same signature. + // This invariant underwrites reproducible chain verification. + let seed = [7u8; 32]; + let signer = Ed25519Signer::from_secret_bytes(&seed).unwrap(); + assert_eq!(signer.secret_bytes(), seed); + let msg = b"deterministic"; + let sig1 = signer.sign(msg); + let sig2 = signer.sign(msg); + assert_eq!(sig1, sig2); + assert!(Ed25519Verifier::verify(&signer.public_key_bytes(), msg, &sig1).is_ok()); + } + + #[test] + fn invalid_secret_len_rejected() { + assert!(matches!( + Ed25519Signer::from_secret_bytes(&[0u8; 16]), + Err(CryptoError::InvalidSecretLen(16)) + )); + assert!(matches!( + Ed25519Signer::from_secret_bytes(&[0u8; 64]), + Err(CryptoError::InvalidSecretLen(64)) + )); + } + + #[test] + fn tampered_message_fails_verify() { + let signer = Ed25519Signer::from_secret_bytes(&[9u8; 32]).unwrap(); + let sig = signer.sign(b"hello"); + let pk = signer.public_key_bytes(); + assert!(matches!( + Ed25519Verifier::verify(&pk, b"hello!", &sig), + Err(CryptoError::VerifyFailed) + )); + } + + #[cfg(feature = "keygen")] + #[test] + fn malformed_signature_rejected() { + let signer = Ed25519Signer::generate(); + let pk = signer.public_key_bytes(); + let short_sig = vec![0u8; 32]; + assert!(matches!( + Ed25519Verifier::verify(&pk, b"x", &short_sig), + Err(CryptoError::InvalidSignatureLen(32)) + )); + } +} diff --git a/crates/core/trusty-core/src/error.rs b/crates/core/trusty-core/src/error.rs new file mode 100644 index 0000000..22d4b22 --- /dev/null +++ b/crates/core/trusty-core/src/error.rs @@ -0,0 +1,27 @@ +use thiserror::Error; + +#[derive(Debug, Error)] +pub enum TrustyError { + #[error("secret key must be exactly 16 bytes, got {0}")] + InvalidKeyLength(usize), + + #[error("payload serialization failed: {0}")] + PayloadSerialization(String), + + #[error("payload is not valid JSON: {0}")] + InvalidPayloadJson(String), + + #[error("raw_payload missing for row at sequence {0}")] + MissingRawPayload(i64), + + #[error("chain id malformed: {0}")] + MalformedChainId(String), + + #[error("payload data too large: {size} bytes exceeds max {max}")] + DataTooLarge { size: usize, max: usize }, + + #[error("unsupported payload_version: {0}")] + UnsupportedPayloadVersion(u32), +} + +pub type TrustyResult = Result; diff --git a/crates/core/trusty-core/src/hash.rs b/crates/core/trusty-core/src/hash.rs new file mode 100644 index 0000000..7eb6468 --- /dev/null +++ b/crates/core/trusty-core/src/hash.rs @@ -0,0 +1,253 @@ +// Dual-hash primitives. +// +// row_hash — SipHash-2-4 over canonical payload bytes, 16-byte secret key, +// rendered as lowercase hex of the 64-bit digest (16 chars). +// anchor_hash — SHA-256 over row_hash || 0x00 || entity_id || 0x00 || chain_id, +// rendered as lowercase hex (64 chars). +// +// Semantics are byte-identical to the Python reference at +// trusty_core_watermark_proxy/core/trusty/hashing/{siphash,sha256}.py. + +use sha2::{Digest, Sha256}; +use siphasher::sip::SipHasher24; +use std::hash::Hasher; + +use zeroize::{Zeroize, ZeroizeOnDrop}; + +use crate::error::{TrustyError, TrustyResult}; + +pub const SECRET_KEY_LEN: usize = 16; + +/// A SipHash key. +/// +/// Wiped on drop. Key material that merely goes out of scope stays in +/// freed memory until something reuses the page, and from there reaches +/// a core dump, a swap file, or a hibernation image -- none of which the +/// process controls, and all of which outlive it. +/// +/// `ZeroizeOnDrop` rather than a hand-written `Drop`: the compiler is +/// free to elide a plain `write` to memory nobody reads again, and the +/// crate's whole purpose is to use the primitives that are not elided. +/// +/// Not `Copy`, deliberately. A `Copy` key produces silent duplicates +/// that no `Drop` ever runs for, which would make the wipe decorative. +/// It stays `Clone`, because a store legitimately holds one, but each +/// clone now cleans up after itself. +#[derive(Clone, Zeroize, ZeroizeOnDrop)] +pub struct SecretKey([u8; SECRET_KEY_LEN]); + +impl std::fmt::Debug for SecretKey { + /// Never print key material. + /// + /// A derived `Debug` puts the key in any log line, panic message or + /// error report that formats a struct containing one -- the most + /// common way a secret escapes a process that never intended to + /// disclose it. + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("SecretKey()") + } +} + +impl SecretKey { + pub fn new(bytes: [u8; SECRET_KEY_LEN]) -> Self { + SecretKey(bytes) + } + + pub fn from_bytes(bytes: &[u8]) -> TrustyResult { + if bytes.len() != SECRET_KEY_LEN { + return Err(TrustyError::InvalidKeyLength(bytes.len())); + } + let mut out = [0u8; SECRET_KEY_LEN]; + out.copy_from_slice(bytes); + Ok(SecretKey(out)) + } + + pub fn as_bytes(&self) -> &[u8; SECRET_KEY_LEN] { + &self.0 + } + + fn k0(&self) -> u64 { + u64::from_le_bytes(self.0[0..8].try_into().unwrap()) + } + + fn k1(&self) -> u64 { + u64::from_le_bytes(self.0[8..16].try_into().unwrap()) + } +} + +pub trait RowHasher { + fn hash(&self, key: &SecretKey, data: &[u8]) -> String; +} + +pub trait AnchorHasher { + fn hash(&self, row_hash: &str, entity_id: &str, chain_id: &str) -> String; +} + +pub struct SipHash24Hasher; + +impl RowHasher for SipHash24Hasher { + fn hash(&self, key: &SecretKey, data: &[u8]) -> String { + let mut hasher = SipHasher24::new_with_keys(key.k0(), key.k1()); + hasher.write(data); + format!("{:016x}", hasher.finish()) + } +} + +pub struct Sha256Hasher; + +impl AnchorHasher for Sha256Hasher { + fn hash(&self, row_hash: &str, entity_id: &str, chain_id: &str) -> String { + let mut h = Sha256::new(); + h.update(row_hash.as_bytes()); + h.update(b"\x00"); + h.update(entity_id.as_bytes()); + h.update(b"\x00"); + h.update(chain_id.as_bytes()); + hex::encode(h.finalize()) + } +} + +pub fn row_hash(key: &SecretKey, data: &[u8]) -> String { + SipHash24Hasher.hash(key, data) +} + +pub fn anchor_hash(row_hash: &str, entity_id: &str, chain_id: &str) -> String { + Sha256Hasher.hash(row_hash, entity_id, chain_id) +} + +// v3 hashes: identical algorithms, domain-tagged inputs. + +use crate::payload::{DOMAIN_TAG_ANCHOR_V3, DOMAIN_TAG_ROW_V3}; + +pub fn row_hash_v3(key: &SecretKey, data: &[u8]) -> String { + let mut buf = Vec::with_capacity(DOMAIN_TAG_ROW_V3.len() + data.len()); + buf.extend_from_slice(DOMAIN_TAG_ROW_V3); + buf.extend_from_slice(data); + SipHash24Hasher.hash(key, &buf) +} + +pub fn anchor_hash_v3(row_hash: &str, entity_id: &str, chain_id: &str) -> String { + let mut h = Sha256::new(); + h.update(DOMAIN_TAG_ANCHOR_V3); + h.update(row_hash.as_bytes()); + h.update(b"\x00"); + h.update(entity_id.as_bytes()); + h.update(b"\x00"); + h.update(chain_id.as_bytes()); + hex::encode(h.finalize()) +} + +pub fn row_hash_for_version(version: u32, key: &SecretKey, data: &[u8]) -> TrustyResult { + match version { + 1 | 2 => Ok(row_hash(key, data)), + 3 => Ok(row_hash_v3(key, data)), + other => Err(TrustyError::UnsupportedPayloadVersion(other)), + } +} + +pub fn anchor_hash_for_version( + version: u32, + row_hash_hex: &str, + entity_id: &str, + chain_id: &str, +) -> TrustyResult { + match version { + 1 | 2 => Ok(anchor_hash(row_hash_hex, entity_id, chain_id)), + 3 => Ok(anchor_hash_v3(row_hash_hex, entity_id, chain_id)), + other => Err(TrustyError::UnsupportedPayloadVersion(other)), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn test_key() -> SecretKey { + SecretKey::from_bytes(b"0123456789abcdef").unwrap() + } + + #[test] + fn row_hash_is_16_hex_chars() { + let h = row_hash(&test_key(), b"hello world"); + assert_eq!(h.len(), 16); + assert!(h.chars().all(|c| c.is_ascii_hexdigit())); + } + + #[test] + fn row_hash_is_deterministic() { + let k = test_key(); + assert_eq!(row_hash(&k, b"abc"), row_hash(&k, b"abc")); + } + + #[test] + fn row_hash_differs_with_key() { + let k1 = SecretKey::from_bytes(b"0123456789abcdef").unwrap(); + let k2 = SecretKey::from_bytes(b"fedcba9876543210").unwrap(); + assert_ne!(row_hash(&k1, b"data"), row_hash(&k2, b"data")); + } + + #[test] + fn anchor_hash_is_64_hex_chars() { + let h = anchor_hash("0123456789abcdef", "acme", "chain_acme_001"); + assert_eq!(h.len(), 64); + assert!(h.chars().all(|c| c.is_ascii_hexdigit())); + } + + #[test] + fn anchor_hash_depends_on_row_hash() { + let a = anchor_hash("0000000000000000", "acme", "chain_acme_001"); + let b = anchor_hash("1111111111111111", "acme", "chain_acme_001"); + assert_ne!(a, b); + } + + #[test] + fn anchor_hash_depends_on_entity() { + let a = anchor_hash("0000000000000000", "acme", "chain_acme_001"); + let b = anchor_hash("0000000000000000", "widgets", "chain_acme_001"); + assert_ne!(a, b); + } + + #[test] + fn bad_key_length_rejected() { + assert!(matches!( + SecretKey::from_bytes(b"short"), + Err(TrustyError::InvalidKeyLength(5)) + )); + } +} + +#[cfg(test)] +mod secret_key_tests { + use super::*; + + #[test] + fn a_key_is_never_printed() { + let k = SecretKey::new([0xAB; SECRET_KEY_LEN]); + let shown = format!("{k:?}"); + assert!( + !shown.contains("ab") && !shown.contains("171"), + "a derived Debug puts key material into any log line or panic message that formats a struct containing one, which is the most common way a secret leaves a process that never meant to disclose it: {shown}" + ); + assert_eq!(shown, "SecretKey()"); + } + + #[test] + fn a_key_is_not_copy() { + // Structural: if SecretKey were Copy, every use would leave a + // silent duplicate that no Drop runs for, and the wipe would be + // decorative. This asserts the property by construction -- the + // function below only compiles for non-Copy types being moved. + fn assert_not_copy(_: &T) {} + let k = SecretKey::new([1u8; SECRET_KEY_LEN]); + assert_not_copy(&k); + let moved = k; + assert_eq!(moved.as_bytes()[0], 1); + } + + #[test] + fn a_key_still_clones_because_a_store_holds_one() { + let k = SecretKey::new([7u8; SECRET_KEY_LEN]); + let c = k.clone(); + assert_eq!(k.as_bytes(), c.as_bytes(), "cloning must preserve the key"); + } +} diff --git a/crates/core/trusty-core/src/lib.rs b/crates/core/trusty-core/src/lib.rs new file mode 100644 index 0000000..245e9bc --- /dev/null +++ b/crates/core/trusty-core/src/lib.rs @@ -0,0 +1,75 @@ +//! Format-agnostic tamper-evidence primitives for the Trusty chain. +//! +//! Dual hash: SipHash-2-4 keyed fingerprint (`row_hash`) + SHA-256 cryptographic +//! anchor (`anchor_hash`). Chain-break-as-event primitive. No storage dependency. +//! +//! # Design invariants (ERC002) +//! 1. `raw_payload` is stored verbatim -- verification recomputes from it. +//! 2. The chain is append-only. +//! 3. `prev_hash` of row N equals `row_hash` of row N-1 on the same chain. +//! +//! # Payload format (v1 -- locked) +//! Canonical JSON with sorted keys and compact separators. Fields in alphabetical +//! order: `chain_id`, `data`, `detected_at`, `entity_id`, `extraction_method`, +//! `hash_alg`, `payload_version`, `prev_hash`, `received_at`, `sequence`, +//! `signatures`, `source_url`. v2 is additive only -- v1 byte shape is never +//! altered. See `docs/payload-v2.md`. + +#![deny(clippy::all)] + +pub mod bundle; +pub mod chain; +pub mod conditions; +pub mod credential; +pub mod crypto; +pub mod error; +pub mod hash; +pub mod payload; +pub mod annotation; + +pub use bundle::{ + Stance, Violation, + AnchorEntry, BundleRow, ProofBundle, SourceRef, SourceRefMismatch, SourceRefReport, +}; +pub use chain::{ + checkpoint_signing_bytes, + compute_record, + compute_record_signed, + make_chain_id, + parse_generation, + parse_raw_payload, + verify_links, + verify_links_from, + // Verification WITHOUT the operator secret key: the path available to a + // holder of a ProofBundle. See `chain::verify_links_public`. + verify_links_public, + verify_links_public_from, + verify_row_signatures, + BreakPoint, + ChainLink, + EventType, + PublicVerifyReport, + VerifyError, + VerifyErrorType, + VerifyReport, + GENESIS_SENTINEL, +}; +pub use conditions::{ + default_conditions, evaluate, BreakReason, Condition, ConditionContext, ConditionOutcome, + HashIntegrity, SequenceContinuity, +}; +pub use credential::{require_bare_signature, KeyFormat}; +pub use crypto::{CryptoError, Ed25519Signer, Ed25519Verifier, Signer, Verifier}; +pub use error::{TrustyError, TrustyResult}; +pub use hash::{ + anchor_hash, row_hash, AnchorHasher, RowHasher, SecretKey, Sha256Hasher, SipHash24Hasher, +}; +pub use payload::{ + build_payload, signing_view_bytes, PayloadFields, DOMAIN_TAG_CHECKPOINT_V1, PAYLOAD_VERSION, + PAYLOAD_VERSION_V1, PAYLOAD_VERSION_V2, +}; + +pub use annotation::{ + entity_id_for, verify_annotations, Annotation, AnnotationKind, AnnotationProblem, + AnnotationReport, CustodyHop, Layer, ANNOTATION_KIND, +}; diff --git a/crates/core/trusty-core/src/payload.rs b/crates/core/trusty-core/src/payload.rs new file mode 100644 index 0000000..3e11b1a --- /dev/null +++ b/crates/core/trusty-core/src/payload.rs @@ -0,0 +1,337 @@ +// Canonical payload serializer — PAYLOAD VERSIONS 1 and 2. +// +// The canonical form is JSON, sorted keys recursively, no whitespace. +// The exact byte sequence emitted here is what row_hash (SipHash-2-4) +// is computed over, so this file is consensus-critical: any change +// to the v1 byte-shape is a breaking format change. +// +// Fields are shared across versions (same schema + ordering); only the +// `payload_version` discriminator and the population of `signatures` +// differ. v1 leaves `signatures` empty; v2 emits one object per signer +// with shape `{alg, pk, sig}`, hex-encoded, keys alphabetical. +// +// Fields in canonical (alphabetical) order: +// chain_id, data, detected_at, entity_id, extraction_method, +// hash_alg, payload_version, prev_hash, received_at, sequence, +// signatures, source_url. + +use serde_json::{Map, Value}; +use std::collections::BTreeMap; + +use crate::error::{TrustyError, TrustyResult}; + +/// Legacy constant preserved for existing call sites. Prefer +/// `PAYLOAD_VERSION_V1` in new code. +pub const PAYLOAD_VERSION: u32 = 1; + +/// v1 payload format (signatures reserved []). Locked byte-shape. +pub const PAYLOAD_VERSION_V1: u32 = 1; + +/// v2 payload format (signatures populated with one `{alg,pk,sig}` +/// per signer). Emitted only when a caller opts in via +/// `chain::compute_record_signed`. +pub const PAYLOAD_VERSION_V2: u32 = 2; + +/// v3 payload format. Identical on-wire byte-shape to v1/v2 (same +/// canonical JSON serializer, same 12 fields in alphabetical order), +/// but hash inputs are prefixed with domain tags to eliminate +/// cross-protocol collision risk. Dispatched by payload_version at +/// the hash step, not at the payload step. +pub const PAYLOAD_VERSION_V3: u32 = 3; + +/// Domain tag prefixed to the canonical payload bytes before SipHash +/// when payload_version == 3. +pub const DOMAIN_TAG_ROW_V3: &[u8] = b"trusty.chain.v3\x00"; + +/// Domain tag prefixed to the SHA-256 input when payload_version == 3. +pub const DOMAIN_TAG_ANCHOR_V3: &[u8] = b"trusty.anchor.v3\x00"; + +/// Domain tag prefixed to the canonical bytes a checkpoint attestor +/// signs. Distinct from the chain row and anchor domains so a +/// checkpoint signature cannot be replayed as a row or anchor +/// signature. v1 because the checkpoint envelope is verification- +/// layer only and lives outside the v1/v2/v3 payload byte-shape lock +/// (memo §5: "MUST NOT change v1 byte-shape"). +pub const DOMAIN_TAG_CHECKPOINT_V1: &[u8] = b"trusty.checkpoint.v1\x00"; + +/// Hash algorithm dispatch string. Verifiers read this before +/// recomputing row_hash so future algorithm agility is trivial. +pub const HASH_ALG: &str = "siphash24+sha256"; + +/// Hard cap on the canonicalized `data` field (bytes). Blobs belong +/// in object storage; put a hash + URI in `data`, not the blob +/// itself. 64 KiB is generous for metadata and keeps parse/verify +/// cost bounded per row. +pub const MAX_DATA_BYTES: usize = 65_536; + +#[derive(Debug, Clone)] +pub struct PayloadFields { + pub entity_id: String, + pub chain_id: String, + pub data: Value, + pub source_url: String, + pub detected_at: String, + pub received_at: String, + pub extraction_method: String, + pub prev_hash: String, + pub sequence: i64, + pub hash_alg: String, + pub payload_version: u32, + pub signatures: Vec, +} + +impl PayloadFields { + /// Construct a v1 `PayloadFields` with `hash_alg`, `payload_version`, + /// and `signatures` set to their v1 defaults. + #[allow(clippy::too_many_arguments)] + pub fn new_v1( + entity_id: impl Into, + chain_id: impl Into, + data: Value, + source_url: impl Into, + detected_at: impl Into, + received_at: impl Into, + extraction_method: impl Into, + prev_hash: impl Into, + sequence: i64, + ) -> Self { + Self { + entity_id: entity_id.into(), + chain_id: chain_id.into(), + data, + source_url: source_url.into(), + detected_at: detected_at.into(), + received_at: received_at.into(), + extraction_method: extraction_method.into(), + prev_hash: prev_hash.into(), + sequence, + hash_alg: HASH_ALG.to_string(), + payload_version: PAYLOAD_VERSION, + signatures: Vec::new(), + } + } + + /// Construct a v3 `PayloadFields`. Same shape as `new_v1`; only + /// `payload_version` differs. The hash step (not this builder) + /// is what applies the v3 domain tag. + #[allow(clippy::too_many_arguments)] + pub fn new_v3( + entity_id: impl Into, + chain_id: impl Into, + data: Value, + source_url: impl Into, + detected_at: impl Into, + received_at: impl Into, + extraction_method: impl Into, + prev_hash: impl Into, + sequence: i64, + ) -> Self { + Self { + entity_id: entity_id.into(), + chain_id: chain_id.into(), + data, + source_url: source_url.into(), + detected_at: detected_at.into(), + received_at: received_at.into(), + extraction_method: extraction_method.into(), + prev_hash: prev_hash.into(), + sequence, + hash_alg: HASH_ALG.to_string(), + payload_version: PAYLOAD_VERSION_V3, + signatures: Vec::new(), + } + } +} + +/// Canonical bytes of a v2 payload with `signatures` forced empty -- +/// this is the "signing view" a Signer signs over. The final emitted +/// payload replaces `signatures` with the signer output and is then +/// re-canonicalized; row_hash commits to that final form. Verification +/// reconstructs this view by stripping `signatures` back to `[]`. +pub fn signing_view_bytes(fields: &PayloadFields) -> TrustyResult { + let mut view = fields.clone(); + view.signatures = Vec::new(); + build_payload(&view) +} + +pub fn build_payload(fields: &PayloadFields) -> TrustyResult { + let data_canonical = canonicalize(&fields.data); + let data_bytes = serde_json::to_string(&data_canonical) + .map_err(|e| TrustyError::PayloadSerialization(e.to_string()))?; + if data_bytes.len() > MAX_DATA_BYTES { + return Err(TrustyError::DataTooLarge { + size: data_bytes.len(), + max: MAX_DATA_BYTES, + }); + } + + let mut top: BTreeMap<&'static str, Value> = BTreeMap::new(); + top.insert("chain_id", Value::String(fields.chain_id.clone())); + top.insert("data", data_canonical); + top.insert("detected_at", Value::String(fields.detected_at.clone())); + top.insert("entity_id", Value::String(fields.entity_id.clone())); + top.insert( + "extraction_method", + Value::String(fields.extraction_method.clone()), + ); + top.insert("hash_alg", Value::String(fields.hash_alg.clone())); + top.insert( + "payload_version", + Value::Number(fields.payload_version.into()), + ); + top.insert("prev_hash", Value::String(fields.prev_hash.clone())); + top.insert("received_at", Value::String(fields.received_at.clone())); + top.insert("sequence", Value::Number(fields.sequence.into())); + // Canonicalize nested signature objects so v2 byte-shape is + // independent of how the caller built the JSON values. v1 always + // passes [], so this is a no-op for v1. + top.insert( + "signatures", + canonicalize(&Value::Array(fields.signatures.clone())), + ); + top.insert("source_url", Value::String(fields.source_url.clone())); + + let mut sorted = Map::new(); + for (k, v) in top.into_iter() { + sorted.insert(k.to_string(), v); + } + serde_json::to_string(&Value::Object(sorted)) + .map_err(|e| TrustyError::PayloadSerialization(e.to_string())) +} + +fn canonicalize(value: &Value) -> Value { + match value { + Value::Object(map) => { + let mut sorted: BTreeMap = BTreeMap::new(); + for (k, v) in map.iter() { + sorted.insert(k.clone(), canonicalize(v)); + } + let mut out = Map::new(); + for (k, v) in sorted.into_iter() { + out.insert(k, v); + } + Value::Object(out) + } + Value::Array(arr) => Value::Array(arr.iter().map(canonicalize).collect()), + other => other.clone(), + } +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn sample() -> PayloadFields { + PayloadFields::new_v1( + "acme", + "chain_acme_000001", + json!({"name": "widget", "count": 3}), + "src://x", + "2026-03-08T00:00:00Z", + "2026-03-08T00:00:00.123456Z", + "trusty", + "genesis", + 0, + ) + } + + #[test] + fn keys_are_sorted_alphabetically() { + let out = build_payload(&sample()).unwrap(); + let positions = [ + "chain_id", + "data", + "detected_at", + "entity_id", + "extraction_method", + "hash_alg", + "payload_version", + "prev_hash", + "received_at", + "sequence", + "signatures", + "source_url", + ]; + let mut last: usize = 0; + for key in positions { + let needle = format!("\"{}\":", key); + let idx = out + .find(&needle) + .unwrap_or_else(|| panic!("missing {}", key)); + assert!(idx >= last, "field {} out of canonical order", key); + last = idx; + } + } + + #[test] + fn no_whitespace_in_output() { + let out = build_payload(&sample()).unwrap(); + assert!(!out.contains(' ')); + assert!(!out.contains('\n')); + } + + #[test] + fn nested_data_keys_are_sorted() { + let fields = PayloadFields { + data: json!({"z": 1, "a": 2, "m": 3}), + ..sample() + }; + let out = build_payload(&fields).unwrap(); + // The data object is the only place keys "a","m","z" appear. + let a = out.find("\"a\":").unwrap(); + let m = out.find("\"m\":").unwrap(); + let z = out.find("\"z\":").unwrap(); + assert!(a < m); + assert!(m < z); + } + + #[test] + fn deterministic_across_input_order() { + let fields_a = PayloadFields { + data: json!({"a": 1, "b": 2}), + ..sample() + }; + let fields_b = PayloadFields { + data: json!({"b": 2, "a": 1}), + ..sample() + }; + assert_eq!( + build_payload(&fields_a).unwrap(), + build_payload(&fields_b).unwrap() + ); + } + + #[test] + fn v1_constant_fields_present() { + let out = build_payload(&sample()).unwrap(); + assert!(out.contains("\"hash_alg\":\"siphash24+sha256\"")); + assert!(out.contains("\"payload_version\":1")); + assert!(out.contains("\"received_at\":")); + assert!(out.contains("\"signatures\":[]")); + } + + #[test] + fn data_size_cap_enforced() { + let huge = "x".repeat(MAX_DATA_BYTES + 1); + let fields = PayloadFields { + data: json!({ "blob": huge }), + ..sample() + }; + let err = build_payload(&fields).unwrap_err(); + assert!(matches!(err, TrustyError::DataTooLarge { .. })); + } + + #[test] + fn detected_and_received_are_distinct_fields() { + // Setting received_at far from detected_at must reach the + // canonical bytes verbatim; this guards against code that + // accidentally collapses the two values. + let mut f = sample(); + f.detected_at = "2024-01-01T00:00:00Z".to_string(); + f.received_at = "2026-12-31T23:59:59.999999Z".to_string(); + let out = build_payload(&f).unwrap(); + assert!(out.contains("\"detected_at\":\"2024-01-01T00:00:00Z\"")); + assert!(out.contains("\"received_at\":\"2026-12-31T23:59:59.999999Z\"")); + } +} diff --git a/crates/core/trusty-policy/Cargo.toml b/crates/core/trusty-policy/Cargo.toml new file mode 100644 index 0000000..873104e --- /dev/null +++ b/crates/core/trusty-policy/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "trusty-policy" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "Scoped FSM + policy engine. Constrains agent / tool-chain action surfaces; emits a structured audit event per evaluation. Substrate-independent (no trusty-core dep) -- consumers wire a PolicyAuditor sink (trusty for production, no-op for tests, file logger for dev)." + +[dependencies] +serde = { workspace = true } +serde_json = { workspace = true } +thiserror = { workspace = true } + +[features] +default = [] diff --git a/crates/core/trusty-policy/examples/claude_session.json b/crates/core/trusty-policy/examples/claude_session.json new file mode 100644 index 0000000..27b61ee --- /dev/null +++ b/crates/core/trusty-policy/examples/claude_session.json @@ -0,0 +1,171 @@ +{ + "transitions": [ + { + "from": "idle", + "action": "boot", + "to": "booted" + }, + { + "from": "booted", + "action": "present_plan", + "to": "planned" + }, + { + "from": "planned", + "action": "approved", + "to": "working" + }, + { + "from": "working", + "action": "read_file", + "to": "working" + }, + { + "from": "working", + "action": "edit_file", + "to": "working" + }, + { + "from": "working", + "action": "write_new_file", + "to": "working" + }, + { + "from": "working", + "action": "bash_safe", + "to": "working" + }, + { + "from": "working", + "action": "delegate_agent", + "to": "working" + }, + { + "from": "working", + "action": "hit_edit_limit", + "to": "cycle_pause" + }, + { + "from": "cycle_pause", + "action": "approved", + "to": "working" + }, + { + "from": "cycle_pause", + "action": "rollback_cycle", + "to": "working" + }, + { + "from": "working", + "action": "run_tests", + "to": "testing" + }, + { + "from": "testing", + "action": "tests_green", + "to": "working" + }, + { + "from": "testing", + "action": "tests_red", + "to": "working" + }, + { + "from": "working", + "action": "begin_close", + "to": "closing" + }, + { + "from": "closing", + "action": "write_delta_log", + "to": "closing" + }, + { + "from": "closing", + "action": "write_tag", + "to": "closing" + }, + { + "from": "closing", + "action": "write_handoff", + "to": "closing" + }, + { + "from": "closing", + "action": "update_session_gate", + "to": "closing" + }, + { + "from": "closing", + "action": "run_close_check", + "to": "closing" + }, + { + "from": "closing", + "action": "commit_branch", + "to": "closed" + }, + { + "from": "working", + "action": "halt_destructive_op", + "to": "await_human" + }, + { + "from": "working", + "action": "halt_main_branch_write", + "to": "await_human" + }, + { + "from": "working", + "action": "halt_skip_hooks", + "to": "await_human" + }, + { + "from": "working", + "action": "halt_unplanned_file", + "to": "await_human" + }, + { + "from": "working", + "action": "halt_external_publish", + "to": "await_human" + }, + { + "from": "working", + "action": "halt_three_errors", + "to": "await_human" + }, + { + "from": "working", + "action": "halt_secret_in_diff", + "to": "await_human" + }, + { + "from": "working", + "action": "halt_dependency_change", + "to": "await_human" + }, + { + "from": "testing", + "action": "halt_tests_3_attempts", + "to": "await_human" + }, + { + "from": "await_human", + "action": "human_resolves", + "to": "working" + }, + { + "from": "await_human", + "action": "human_aborts", + "to": "idle" + }, + { + "from": "closed", + "action": "next_session", + "to": "idle" + } + ], + "scope": "claude.code.session.v1", + "initial": "idle" +} diff --git a/crates/core/trusty-policy/examples/pipeline.json b/crates/core/trusty-policy/examples/pipeline.json new file mode 100644 index 0000000..f9f5703 --- /dev/null +++ b/crates/core/trusty-policy/examples/pipeline.json @@ -0,0 +1,26 @@ +{ + "transitions": [ + { + "from": "idle", + "action": "claim", + "to": "claimed" + }, + { + "from": "claimed", + "action": "execute", + "to": "running" + }, + { + "from": "running", + "action": "finish", + "to": "done" + }, + { + "from": "running", + "action": "abort", + "to": "aborted" + } + ], + "scope": "agent.executor.v1", + "initial": "idle" +} diff --git a/crates/core/trusty-policy/src/lib.rs b/crates/core/trusty-policy/src/lib.rs new file mode 100644 index 0000000..0be4859 --- /dev/null +++ b/crates/core/trusty-policy/src/lib.rs @@ -0,0 +1,2904 @@ +//! Scoped FSM + policy engine. +//! +//! Forces an action surface to be a finite, declared set of legal +//! transitions. Every evaluation -- allowed or denied -- emits a +//! structured `PolicyEvent` to a pluggable [`PolicyAuditor`] sink so +//! the trail is complete by construction. The library has NO +//! dependency on `trusty-core`; a downstream crate wires a trusty- +//! backed auditor when it wants the cryptographic audit chain. +//! +//! # Why this exists +//! +//! - **AI tool-chain safety**: an agent can only invoke actions that +//! are declared transitions from its current state. Fuzziness gets +//! clipped at the boundary. +//! - **Self-discipline**: a developer's own workflow can ride the +//! same engine ("only release-prep -> test_run -> tag -> publish"), +//! refusing to skip steps. +//! - **Liability flip** (`project_fsm_trusty_syscall_architecture`): +//! when paired with a trusty audit chain, "X is in the chain" / +//! "X is not in the chain" flips burden of proof. +//! +//! # Boundary +//! +//! This crate is the **gatekeeper**. It says yes or no. It does NOT: +//! - run agents (that's `agent-mesh`) +//! - store the chain (that's trusty) +//! - signal between processes (that's Postgres NOTIFY / DS / similar) +//! +//! See the workspace-level architecture pin for how layers fit. +//! +//! # Quickstart +//! +//! ``` +//! use trusty_policy::{Fsm, PolicyEngine, ScopeKey, Transition, NoopAuditor}; +//! +//! let mut fsm: Fsm<&'static str, &'static str> = Fsm::new("idle"); +//! fsm.allow(Transition::new("idle", "claim", "claimed")); +//! fsm.allow(Transition::new("claimed", "execute", "running")); +//! fsm.allow(Transition::new("running", "finish", "done")); +//! +//! let scope = ScopeKey::new("agent.executor.v1"); +//! let mut engine = PolicyEngine::new(NoopAuditor::default()); +//! engine.register(scope.clone(), fsm); +//! +//! let result = engine.evaluate(&scope, &"idle", &"claim", None); +//! assert!(result.is_allow()); +//! +//! let denied = engine.evaluate(&scope, &"idle", &"finish", None); +//! assert!(!denied.is_allow()); +//! ``` +//! +//! # Guards with context (cycle 2) +//! +//! For transitions whose legality depends on runtime data (e.g. +//! "balance > 0", "user is admin", "rate limit not exceeded"), +//! attach a [`Guard`] when registering the transition via +//! [`Fsm::allow_guarded`]. Guards run only when [`PolicyEngine::evaluate_with_context`] +//! is called; the simpler [`PolicyEngine::evaluate`] path skips them +//! (it passes `()` as context, which guards over a non-`()` context +//! cannot match -- a guarded transition under context-less evaluate +//! returns Deny with `GuardRejected { code: "no-context" }`). + +#![deny(clippy::all)] + +use serde::{Deserialize, Serialize}; +use std::collections::HashMap; +use std::fmt::Debug; +use std::hash::Hash; +use thiserror::Error; + +// ------------------------------------------------------------ +// Core types +// ------------------------------------------------------------ + +/// Names a registered FSM. Examples: `"agent.executor.v1"`, +/// `"deploy.pipeline.prod"`, `"legal.case.brief.v3"`. The key is +/// opaque to the engine; the consumer picks a convention. +#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)] +pub struct ScopeKey(String); + +impl ScopeKey { + pub fn new(s: impl Into) -> Self { + Self(s.into()) + } + pub fn as_str(&self) -> &str { + &self.0 + } +} + +impl From<&str> for ScopeKey { + fn from(s: &str) -> Self { + Self(s.to_string()) + } +} + +/// What an action DESTROYS, not what it is called or who is asking. +/// +/// Declared statically on the transition and never supplied by the caller. A +/// caller-supplied tier is a caller-chosen permission: anything able to name +/// its own tier can name the cheapest one. +/// +/// The four values are deliberately the four verbs a relational database +/// already has, so a grant compiles to a row-level-security predicate with no +/// translation layer in between. A translation layer is where the +/// application's idea of a permission and the database's drift apart, and the +/// database wins that argument silently. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum Tier { + /// Destroys nothing. + Read, + /// Adds without disturbing what is there. + Append, + /// Overwrites. Recoverable only if something kept the prior value. + Update, + /// Irreversible. What `irreversible_budget` counts. + Delete, +} + +impl Tier { + /// The SQL verb this tier maps to, for compiling a grant into an RLS + /// policy. One-to-one on purpose -- see the type's documentation. + pub fn sql_verb(self) -> &'static str { + match self { + Tier::Read => "SELECT", + Tier::Append => "INSERT", + Tier::Update => "UPDATE", + Tier::Delete => "DELETE", + } + } + + /// Whether this tier destroys something that cannot be recovered from the + /// row itself. + pub fn is_irreversible(self) -> bool { + matches!(self, Tier::Delete) + } +} + +/// One declared legal transition: from `from`, taking `action`, lands +/// at `to`. Generic over state and action types so callers can use +/// `&'static str`, enums, or richer types. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Transition { + pub from: S, + pub action: A, + pub to: S, + /// What this action destroys. `None` where a consumer has not classified + /// it yet; a grant that checks tiers should refuse an unclassified action + /// rather than assume the cheapest one. + pub tier: Option, +} + +impl Transition { + pub fn new(from: S, action: A, to: S) -> Self { + Self { + from, + action, + to, + tier: None, + } + } + + /// Declare what this action destroys. + pub fn with_tier(mut self, tier: Tier) -> Self { + self.tier = Some(tier); + self + } +} + +/// A predicate over runtime context. Returns `Ok(())` to allow the +/// transition, `Err(code)` to deny with [`DenyReason::GuardRejected { code }`]. +/// `code` is consumer-defined and lands on the `PolicyEvent`. +/// +/// Any `Fn(&C) -> Result<(), String> + Send + Sync` is a valid guard. +pub trait Guard: Send + Sync { + fn check(&self, ctx: &C) -> Result<(), String>; +} + +impl Guard for F +where + F: Fn(&C) -> Result<(), String> + Send + Sync, +{ + fn check(&self, ctx: &C) -> Result<(), String> { + self(ctx) + } +} + +/// A finite-state-machine spec: an initial state plus the set of +/// declared transitions. Lookups are exact-match on `(from, action)`. +/// Multiple transitions out of one state with different actions = a +/// branch. Two transitions with the same `(from, action)` is a +/// programming error and the second registration wins (last writer); +/// callers should detect duplicates if they care. +/// +/// The optional context type parameter `C` (default `()`) lets you +/// attach guards via [`Fsm::allow_guarded`]. Guards run only when +/// [`PolicyEngine::evaluate_with_context`] is called. +pub struct Fsm +where + S: Eq + Hash + Clone, + A: Eq + Hash + Clone, + C: 'static, +{ + initial: S, + transitions: HashMap<(S, A), S>, + guards: HashMap<(S, A), Box>>, + /// Actions refused outright, and the rule name that refused them. + /// `None` in the state position means "in every state". + excludes: HashMap<(Option, A), String>, + /// What each declared transition destroys. Populated from + /// `Transition::tier`; absent where a consumer has not classified it. + tiers: HashMap<(S, A), Tier>, +} + +impl std::fmt::Debug for Fsm +where + S: Eq + Hash + Clone + Debug, + A: Eq + Hash + Clone + Debug, + C: 'static, +{ + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("Fsm") + .field("initial", &self.initial) + .field("transitions", &self.transitions) + .field("guards", &format!("{} guarded", self.guards.len())) + .finish() + } +} + +impl Fsm +where + S: Eq + Hash + Clone, + A: Eq + Hash + Clone, + C: 'static, +{ + pub fn new(initial: S) -> Self { + Self { + initial, + transitions: HashMap::new(), + guards: HashMap::new(), + excludes: HashMap::new(), + tiers: HashMap::new(), + } + } + + pub fn initial(&self) -> &S { + &self.initial + } + + /// Register a legal transition. Idempotent; second call with the + /// same `(from, action)` overwrites the destination. + pub fn allow(&mut self, t: Transition) -> &mut Self { + if let Some(tier) = t.tier { + self.tiers.insert((t.from.clone(), t.action.clone()), tier); + } + self.transitions.insert((t.from, t.action), t.to); + self + } + + /// Register a legal transition gated by a runtime guard. The + /// transition is only allowed if the guard returns `Ok(())` when + /// the engine is called with a context of type `C` via + /// [`PolicyEngine::evaluate_with_context`]. Calling the + /// context-less [`PolicyEngine::evaluate`] on a guarded transition + /// returns Deny with `code = "no-context"` -- guards always + /// require context. + pub fn allow_guarded(&mut self, t: Transition, guard: G) -> &mut Self + where + G: Guard + 'static, + { + if let Some(tier) = t.tier { + self.tiers.insert((t.from.clone(), t.action.clone()), tier); + } + self.transitions + .insert((t.from.clone(), t.action.clone()), t.to); + self.guards.insert((t.from, t.action), Box::new(guard)); + self + } + + /// Forbid `action` in EVERY state, naming the rule that forbids it. + /// + /// Checked before transitions and never overridden by one, so adding a + /// wider grant later cannot re-open it. That ordering is the whole point: + /// an exclusion a broader permission could outvote is not an exclusion. + pub fn exclude(&mut self, action: A, rule: impl Into) -> &mut Self { + self.excludes.insert((None, action), rule.into()); + self + } + + /// Forbid `action` when in `from` specifically. + pub fn exclude_from(&mut self, from: S, action: A, rule: impl Into) -> &mut Self { + self.excludes.insert((Some(from), action), rule.into()); + self + } + + /// What `(from, action)` destroys, if the consumer classified it. + /// + /// `None` means unclassified, NOT harmless. A caller enforcing tiers + /// should refuse an unclassified action rather than assume `Read` -- + /// treating "not stated" as "cheapest" is how an unreviewed action + /// acquires the widest permission. + pub fn tier_of(&self, from: &S, action: &A) -> Option { + self.tiers.get(&(from.clone(), action.clone())).copied() + } + + /// The rule excluding `(from, action)`, if any. State-specific exclusions + /// are checked before blanket ones. + pub fn excluded_by(&self, from: &S, action: &A) -> Option<&str> { + self.excludes + .get(&(Some(from.clone()), action.clone())) + .or_else(|| self.excludes.get(&(None, action.clone()))) + .map(String::as_str) + } + + /// Look up the destination state for `(from, action)`. None if + /// the transition is not declared. + pub fn next_state(&self, from: &S, action: &A) -> Option<&S> { + self.transitions.get(&(from.clone(), action.clone())) + } + + /// `allow_guarded` for a guard already boxed -- used when resolving a + /// guard by name out of a [`GuardRegistry`], where the concrete type is + /// no longer known. + pub fn allow_guarded_boxed( + &mut self, + t: Transition, + guard: Box>, + ) -> &mut Self { + if let Some(tier) = t.tier { + self.tiers.insert((t.from.clone(), t.action.clone()), tier); + } + self.transitions + .insert((t.from.clone(), t.action.clone()), t.to); + self.guards.insert((t.from, t.action), guard); + self + } + + /// Look up the guard, if any, for `(from, action)`. + pub fn guard(&self, from: &S, action: &A) -> Option<&dyn Guard> { + self.guards + .get(&(from.clone(), action.clone())) + .map(|g| g.as_ref()) + } + + /// All declared transitions. Useful for introspection / docs. + pub fn transitions(&self) -> impl Iterator { + self.transitions + .iter() + .map(|((from, action), to)| (from, action, to)) + } +} + +// ------------------------------------------------------------ +// Outcome +// ------------------------------------------------------------ + +/// Outcome of a single policy evaluation. `Deny` carries a +/// machine-readable reason code so audit consumers can aggregate +/// failure modes without parsing free text. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub enum PolicyResult { + Allow { + /// The destination state the FSM moves to on Allow. + next: String, + }, + Deny { + reason: DenyReason, + }, +} + +impl PolicyResult { + pub fn is_allow(&self) -> bool { + matches!(self, Self::Allow { .. }) + } +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub enum DenyReason { + /// The scope key was not registered with the engine. + UnknownScope, + /// The (from, action) pair has no declared transition. + UndeclaredTransition, + /// A guard predicate (registered separately by the consumer) + /// rejected the transition. `code` is consumer-defined. + GuardRejected { code: String }, + /// The rate ceiling for this grant is spent for the current window. + RateExceeded { limit: u32, window_secs: i64 }, + /// No irreversible actions remain on this grant. + /// + /// Counts `Tier::Delete` only. Once spent it does not refill: a budget + /// that replenishes is a rate limit, and the point of this one is that + /// some quantity of destruction is all a grant ever gets. + IrreversibleBudgetExhausted, + /// The subject has not checked in within its dead-man interval, so the + /// grant is treated as revoked. + /// + /// A dead-man switch puts the burden on the SUBJECT: silence revokes. + /// That is the opposite of a liveness probe, where an observer polls and + /// absence of an answer is merely information. + DeadMan { silent_for: i64, limit: i64 }, + /// The grant's wall-clock expiry has passed. + /// + /// Wall clock, deliberately: a task still running past its expiry stops. + /// Expiry tied to completion is not expiry, because the thing that + /// overruns is exactly the thing you wanted bounded. + Expired { expires_at: i64, now: i64 }, + /// An exclusion matched. Exclusions are evaluated BEFORE transitions and + /// cannot be overridden by any grant, however wide -- so this reason is + /// final in a way `UndeclaredTransition` is not: that one means "nothing + /// permitted it", this one means "something forbade it". + Excluded { rule: String }, +} + +// ------------------------------------------------------------ +// Audit sink +// ------------------------------------------------------------ + +/// A structured audit event emitted on every evaluation, allowed or +/// denied. Auditors receive this and persist it however they like +/// (trusty append, file log, OTel, etc.). +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct PolicyEvent { + pub scope: ScopeKey, + /// Stringified form of the from-state for transport-stability; + /// consumers can re-deserialize if they want. Stringification is + /// the consumer's job (caller passes via [`PolicyEngine::evaluate`] + /// which `Debug`-formats both ends). + pub from: String, + pub action: String, + pub result: PolicyResult, + /// Optional caller-supplied context tag (entity_id, agent_oid, + /// task_id, etc.) for downstream filtering. + pub actor: Option, + /// The principal this action was taken ON BEHALF OF, when it was + /// delegated. + /// + /// An audit line reads "the agent, acting for Eric" and never just "the + /// agent". An agent is not a principal in its own right, so a record that + /// names only the agent has lost the part that matters. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub acts_for: Option, +} + +/// Where evaluations get logged. Implement on a trusty-backed +/// adapter, file logger, or anything else. +pub trait PolicyAuditor { + fn audit(&mut self, event: &PolicyEvent); +} + +/// Default no-op auditor. Useful for tests and as a placeholder +/// before a real sink is wired. +#[derive(Debug, Default, Clone)] +pub struct NoopAuditor; + +impl PolicyAuditor for NoopAuditor { + fn audit(&mut self, _event: &PolicyEvent) {} +} + +/// Auditor that captures events into a Vec for inspection. +/// Test-only; not stable API. +#[derive(Debug, Default, Clone)] +pub struct CapturingAuditor { + pub events: Vec, +} + +impl PolicyAuditor for CapturingAuditor { + fn audit(&mut self, event: &PolicyEvent) { + self.events.push(event.clone()); + } +} + +// ------------------------------------------------------------ +// Engine +// ------------------------------------------------------------ + +#[derive(Debug, Error)] +pub enum PolicyError { + #[error("scope already registered: {0}")] + DuplicateScope(String), + #[error("JSON parse error: {0}")] + Parse(String), + #[error("schema error: {0}")] + Schema(String), + #[error( + "transition {from} --{action}--> {to} names guard '{guard}', which is not registered. Register it before building, or remove the name." + )] + UnregisteredGuard { + from: String, + action: String, + to: String, + guard: String, + }, +} + +// ------------------------------------------------------------ +// TOML spec format (cycle 3) +// ------------------------------------------------------------ + +/// Wire format for a declarative FSM spec. One spec = one FSM; a +/// caller building a `PolicyEngine` from a config directory typically +/// loads many specs and registers each under its `scope`. +/// +/// Example: +/// +/// ```toml +/// scope = "agent.executor.v1" +/// initial = "idle" +/// +/// [[transition]] +/// from = "idle" +/// action = "claim" +/// to = "claimed" +/// +/// [[transition]] +/// from = "claimed" +/// action = "execute" +/// to = "running" +/// ``` +/// +/// Guards are NOT expressible in TOML in the cycle-3 surface -- +/// guards live in code and bind by `(scope, from, action)` post-load. +/// A guard-registry pattern is planned for the S20+ consumer wire-up. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct FsmSpec { + pub scope: String, + pub initial: String, + #[serde(default)] + pub transitions: Vec, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct TransitionSpec { + pub from: String, + pub action: String, + pub to: String, + /// Name of a guard in the [`GuardRegistry`], resolved at build time. + /// + /// Naming the guard as DATA is what lets a policy be authored, proved and + /// shipped without a recompile. Previously guards existed only as Rust + /// closures bound after load, so every consumer needed a code change and a + /// policy could not be data end to end. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub guard: Option, + /// What this action destroys: `read`, `append`, `update`, `delete`. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub tier: Option, +} + +/// Guard implementations, by the name a spec refers to them by. +/// +/// The host registers what it can enforce; the spec chooses from that set. A +/// spec naming something absent does not build -- see +/// [`FsmSpec::build_fsm_with_guards`]. +pub struct GuardRegistry { + guards: HashMap>>, +} + +impl Default for GuardRegistry { + fn default() -> Self { + Self::new() + } +} + +impl GuardRegistry { + pub fn new() -> Self { + Self { + guards: HashMap::new(), + } + } + + /// Register `guard` under `name`. Replaces any previous entry. + pub fn insert(&mut self, name: impl Into, guard: G) -> &mut Self + where + G: Guard + 'static, + { + self.guards.insert(name.into(), Box::new(guard)); + self + } + + pub fn contains(&self, name: &str) -> bool { + self.guards.contains_key(name) + } + + fn take(&mut self, name: &str) -> Option>> { + self.guards.remove(name) + } +} + +impl FsmSpec { + /// Parse a spec from TOML text. + pub fn from_json(input: &str) -> Result { + serde_json::from_str::(input).map_err(|e| PolicyError::Parse(e.to_string())) + } + + /// Build a runtime `Fsm` from this spec. Returns + /// the scope key alongside the FSM so callers can register both + /// in one move. + /// + /// Ignores any `guard` names in the spec. Use + /// [`Self::build_fsm_with_guards`] when the spec names guards; this one is + /// for specs that do not. + pub fn build_fsm(&self) -> (ScopeKey, Fsm) { + let mut fsm: Fsm = Fsm::new(self.initial.clone()); + for t in &self.transitions { + let mut transition = Transition::new(t.from.clone(), t.action.clone(), t.to.clone()); + transition.tier = t.tier; + fsm.allow(transition); + } + (ScopeKey::new(&self.scope), fsm) + } + + /// Build an FSM, resolving each transition's `guard` name against + /// `registry`. + /// + /// **Refuses to build when a named guard is absent**, returning + /// [`PolicyError::UnregisteredGuard`]. That refusal is the point of the + /// whole mechanism, not defensive tidiness: a name that silently resolved + /// to nothing would leave the transition UNGUARDED, so a typo in a policy + /// file would quietly remove a restriction rather than fail. The failure + /// has to be loud and at build time, before the gate is ever consulted. + /// + /// Guards are moved out of the registry as they are bound, so registering + /// one name for two transitions is itself an error -- a guard is a single + /// implementation, not a shared handle. + pub fn build_fsm_with_guards( + &self, + registry: &mut GuardRegistry, + ) -> Result<(ScopeKey, Fsm), PolicyError> { + let mut fsm: Fsm = Fsm::new(self.initial.clone()); + for t in &self.transitions { + let mut transition = Transition::new(t.from.clone(), t.action.clone(), t.to.clone()); + transition.tier = t.tier; + match &t.guard { + None => { + fsm.allow(transition); + } + Some(name) => { + let guard = + registry + .take(name) + .ok_or_else(|| PolicyError::UnregisteredGuard { + from: t.from.clone(), + action: t.action.clone(), + to: t.to.clone(), + guard: name.clone(), + })?; + fsm.allow_guarded_boxed(transition, guard); + } + } + } + Ok((ScopeKey::new(&self.scope), fsm)) + } + + /// Round-trip: serialize back to TOML. Useful for tooling that + /// edits the spec graph and writes it back. + /// The canonical byte form, per RFC 8785. + /// + /// A rule set is the thing that decides whether a mutation is legal, + /// so it has to be pinnable: hashed, signed, and cited by the breaks + /// it produces. That needs one byte sequence per document, which is + /// why this exists and why TOML could not be the stored form -- TOML + /// has no canonicalisation spec, the same objection that ruled out + /// YAML. + /// + /// Keys are sorted EXPLICITLY through a `BTreeMap` rather than + /// relying on `serde_json`, because this workspace pins `serde_json` + /// with `preserve_order`: serialising the struct directly would give + /// field-declaration order, stable by accident and silently wrong + /// the day a field is reordered. `Grant::canonical_json` and + /// `trusty-core::build_payload` sort explicitly for the same reason. + /// + /// Transition ORDER is preserved rather than sorted. A rule set is a + /// document an author wrote in an order; reordering it would make + /// two different documents hash alike and lose the reading order for + /// no gain, since the array position is already unambiguous. + pub fn canonical_json(&self) -> Result { + use serde_json::{Map, Value}; + use std::collections::BTreeMap; + + let mut transitions = Vec::with_capacity(self.transitions.len()); + for t in &self.transitions { + let mut e: BTreeMap = BTreeMap::new(); + e.insert("action".into(), Value::String(t.action.clone())); + e.insert("from".into(), Value::String(t.from.clone())); + if let Some(g) = &t.guard { + e.insert("guard".into(), Value::String(g.clone())); + } + if let Some(tier) = t.tier { + e.insert( + "tier".into(), + Value::String( + serde_json::to_value(tier) + .map_err(|e| PolicyError::Schema(e.to_string()))? + .as_str() + .unwrap_or_default() + .to_string(), + ), + ); + } + e.insert("to".into(), Value::String(t.to.clone())); + let mut obj = Map::new(); + for (k, v) in e { + obj.insert(k, v); + } + transitions.push(Value::Object(obj)); + } + + let mut top: BTreeMap = BTreeMap::new(); + top.insert("initial".into(), Value::String(self.initial.clone())); + top.insert("scope".into(), Value::String(self.scope.clone())); + top.insert("transitions".into(), Value::Array(transitions)); + + let mut out = Map::new(); + for (k, v) in top { + out.insert(k, v); + } + serde_json::to_string(&Value::Object(out)) + .map_err(|e| PolicyError::Schema(e.to_string())) + } +} + +/// Resource ceilings on a grant. `None` on a field means unlimited. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct Limits { + /// Actions permitted per 60-second window. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub rate_per_min: Option, + /// How many `Tier::Delete` actions remain, for the life of the grant. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub irreversible_budget: Option, + /// Seconds of subject silence after which the grant is revoked. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub dead_man_secs: Option, +} + +/// Consumption against [`Limits`], advanced only when an action is ALLOWED. +/// +/// A refused action performed no work, so charging it would let one grant's +/// exhaustion be caused by another's rejections. +#[derive(Debug, Clone, Copy)] +struct LimitState { + window_start: i64, + in_window: u32, + irreversible_spent: u32, + last_checkin: Option, +} + +impl LimitState { + fn new() -> Self { + Self { + window_start: 0, + in_window: 0, + irreversible_spent: 0, + last_checkin: None, + } + } +} + +/// A registered scope: its FSM, and the wall-clock instant its grant stops. +struct Registered +where + S: Eq + Hash + Clone, + A: Eq + Hash + Clone, + C: 'static, +{ + fsm: Fsm, + /// Unix seconds. `None` means the grant does not expire. + expires_at: Option, + limits: Limits, + state: LimitState, +} + +pub struct PolicyEngine +where + S: Eq + Hash + Clone + Debug, + A: Eq + Hash + Clone + Debug, + C: 'static, +{ + fsms: HashMap>, + auditor: Box, +} + +impl PolicyEngine +where + S: Eq + Hash + Clone + Debug, + A: Eq + Hash + Clone + Debug, + C: 'static, +{ + pub fn new(auditor: AU) -> Self + where + AU: PolicyAuditor + Send + 'static, + { + Self { + fsms: HashMap::new(), + auditor: Box::new(auditor), + } + } + + pub fn register(&mut self, scope: ScopeKey, fsm: Fsm) -> Result<(), PolicyError> { + if self.fsms.contains_key(&scope) { + return Err(PolicyError::DuplicateScope(scope.0)); + } + self.fsms.insert( + scope, + Registered { + fsm, + expires_at: None, + limits: Limits::default(), + state: LimitState::new(), + }, + ); + Ok(()) + } + + /// Register a grant that stops at `expires_at` (unix seconds). + /// + /// Evaluate it through [`Self::evaluate_at`] or + /// [`Self::evaluate_with_context_at`], which take the current time. The + /// clock-less [`Self::evaluate`] refuses an expiring scope outright with + /// `GuardRejected { code: "no-clock" }` -- mirroring how a guarded + /// transition refuses without context. It cannot establish whether the + /// grant is live, and a permission engine that cannot tell must deny: + /// absent policy is not permission. + /// + /// The engine deliberately owns no clock. One would make it untestable and + /// would put a dependency in a crate whose whole point is having none. + pub fn register_until( + &mut self, + scope: ScopeKey, + fsm: Fsm, + expires_at: i64, + ) -> Result<(), PolicyError> { + if self.fsms.contains_key(&scope) { + return Err(PolicyError::DuplicateScope(scope.0)); + } + self.fsms.insert( + scope, + Registered { + fsm, + expires_at: Some(expires_at), + limits: Limits::default(), + state: LimitState::new(), + }, + ); + Ok(()) + } + + /// Register a grant carrying resource ceilings. + /// + /// Limits are consumed only when an action is ALLOWED, and checked before + /// it is: exhaustion is a refusal, never a slow success. + /// + /// Rate and dead-man need the wall clock, so evaluate through + /// [`Self::evaluate_at`] / [`Self::evaluate_with_context_at`]. The + /// clock-less path refuses a rate- or dead-man-limited grant for the same + /// reason it refuses an expiring one: it cannot tell, so it does not guess. + pub fn register_with_limits( + &mut self, + scope: ScopeKey, + fsm: Fsm, + limits: Limits, + expires_at: Option, + ) -> Result<(), PolicyError> { + if self.fsms.contains_key(&scope) { + return Err(PolicyError::DuplicateScope(scope.0)); + } + self.fsms.insert( + scope, + Registered { + fsm, + expires_at, + limits, + state: LimitState::new(), + }, + ); + Ok(()) + } + + /// Record that the subject of `scope` is still present. + /// + /// The dead-man interval is measured from the most recent check-in, or + /// from the first evaluation if none was ever recorded. + pub fn record_checkin(&mut self, scope: &ScopeKey, now: i64) { + if let Some(reg) = self.fsms.get_mut(scope) { + reg.state.last_checkin = Some(now); + } + } + + /// Irreversible actions still available on `scope`, if it is budgeted. + pub fn irreversible_remaining(&self, scope: &ScopeKey) -> Option { + let reg = self.fsms.get(scope)?; + reg.limits + .irreversible_budget + .map(|b| b.saturating_sub(reg.state.irreversible_spent)) + } + + /// Replace an existing FSM under `scope`. Useful for hot-reload + /// / config updates. Insert if absent. + pub fn upsert(&mut self, scope: ScopeKey, fsm: Fsm) { + self.fsms.insert( + scope, + Registered { + fsm, + expires_at: None, + limits: Limits::default(), + state: LimitState::new(), + }, + ); + } + + pub fn fsm(&self, scope: &ScopeKey) -> Option<&Fsm> { + self.fsms.get(scope).map(|r| &r.fsm) + } + + /// Context-aware gate. Runs the guard (if any) for the requested + /// transition before allowing. Always emits a `PolicyEvent`. + pub fn evaluate_with_context( + &mut self, + scope: &ScopeKey, + from: &S, + action: &A, + ctx: &C, + actor: Option<&str>, + ) -> PolicyResult { + self.eval_inner(scope, from, action, ctx, None, actor) + } + + /// [`Self::evaluate_with_context`] with the current wall-clock time, in + /// unix seconds, so a grant registered via [`Self::register_until`] can be + /// checked for expiry. + pub fn evaluate_with_context_at( + &mut self, + scope: &ScopeKey, + from: &S, + action: &A, + ctx: &C, + now: i64, + actor: Option<&str>, + ) -> PolicyResult { + self.eval_inner(scope, from, action, ctx, Some(now), actor) + } + + /// Would this action fit within `scope`'s ceilings? Read-only. + /// + /// Separate from consumption so a DELEGATED evaluation can test both + /// sides before charging either. Checking and charging in one step would + /// bill an agent for an action its principal then refused. + fn check_limits( + &mut self, + scope: &ScopeKey, + from: &S, + action: &A, + now: i64, + ) -> Option { + let reg = self.fsms.get_mut(scope)?; + + // Rate: a fixed 60-second window, reset when the current one lapses. + // Rolling the window here is idempotent, so a check that never leads + // to a charge leaves nothing inconsistent behind. + if let Some(limit) = reg.limits.rate_per_min { + if now.saturating_sub(reg.state.window_start) >= 60 { + reg.state.window_start = now; + reg.state.in_window = 0; + } + if reg.state.in_window >= limit { + return Some(DenyReason::RateExceeded { + limit, + window_secs: 60, + }); + } + } + + // Irreversible budget: Tier::Delete only, and it does not refill. + if reg + .fsm + .tier_of(from, action) + .is_some_and(Tier::is_irreversible) + { + if let Some(budget) = reg.limits.irreversible_budget { + if reg.state.irreversible_spent >= budget { + return Some(DenyReason::IrreversibleBudgetExhausted); + } + } + } + None + } + + /// Charge `scope`'s ceilings for an action that is going ahead. + fn consume_limits(&mut self, scope: &ScopeKey, from: &S, action: &A) { + let Some(reg) = self.fsms.get_mut(scope) else { + return; + }; + let spends_budget = reg + .fsm + .tier_of(from, action) + .is_some_and(Tier::is_irreversible); + if reg.limits.rate_per_min.is_some() { + reg.state.in_window = reg.state.in_window.saturating_add(1); + } + if spends_budget && reg.limits.irreversible_budget.is_some() { + reg.state.irreversible_spent = reg.state.irreversible_spent.saturating_add(1); + } + } + + /// Evaluate an action an agent takes ON BEHALF OF another principal. + /// + /// ```text + /// effective = grant(agent) ∩ permission(acts_for) + /// ``` + /// + /// An agent is never a principal in its own right. It cannot exceed the + /// party behind it even when its own grant is wider, which is what makes + /// revocation meaningful -- kill the agent's key and the person is + /// untouched -- and what lets a request cross between parties at all. + /// + /// Both sides must allow. A refusal from either is returned as-is, so the + /// audit says WHICH side refused and why: "the agent had no such grant" + /// and "the person it acts for does not have that permission" are + /// different findings. + /// + /// Neither side is charged unless both allow. Checking and charging in one + /// pass would bill an agent for an action its principal then refused. + /// + /// An unknown `acts_for` scope denies, per deny-by-default: a delegation + /// to a principal with no policy is not a delegation to an unrestricted + /// one. + pub fn evaluate_delegated( + &mut self, + agent: &ScopeKey, + acts_for: &ScopeKey, + from: &S, + action: &A, + ctx: &C, + now: Option, + actor: Option<&str>, + ) -> PolicyResult { + // Decide both sides first, WITHOUT charging either. + let agent_side = self.decide(agent, from, action, ctx, now); + let principal_side = self.decide(acts_for, from, action, ctx, now); + + let result = match (&agent_side, &principal_side) { + (PolicyResult::Deny { .. }, _) => agent_side.clone(), + (_, PolicyResult::Deny { .. }) => principal_side.clone(), + _ => { + // Both permit, and both ceilings must have room. + let over = now.and_then(|n| { + self.check_limits(agent, from, action, n) + .or_else(|| self.check_limits(acts_for, from, action, n)) + }); + match over { + Some(reason) => PolicyResult::Deny { reason }, + None => { + if now.is_some() { + // Acting for someone spends both allowances. + self.consume_limits(agent, from, action); + self.consume_limits(acts_for, from, action); + } + agent_side.clone() + } + } + } + }; + + let event = PolicyEvent { + scope: agent.clone(), + from: format!("{:?}", from), + action: format!("{:?}", action), + result: result.clone(), + actor: actor.map(str::to_string), + acts_for: Some(acts_for.as_str().to_string()), + }; + self.auditor.audit(&event); + + result + } + + /// The decision for one scope, with no limit accounting and no audit + /// event. Used by [`Self::evaluate_delegated`], which must weigh two + /// scopes before committing to either. + fn decide( + &self, + scope: &ScopeKey, + from: &S, + action: &A, + ctx: &C, + now: Option, + ) -> PolicyResult { + match self.fsms.get(scope) { + None => PolicyResult::Deny { + reason: DenyReason::UnknownScope, + }, + Some(reg) if reg.expires_at.is_some_and(|e| now.is_some_and(|n| n >= e)) => { + PolicyResult::Deny { + reason: DenyReason::Expired { + expires_at: reg.expires_at.unwrap_or_default(), + now: now.unwrap_or_default(), + }, + } + } + Some(reg) + if now.is_none() + && (reg.expires_at.is_some() + || reg.limits.rate_per_min.is_some() + || reg.limits.dead_man_secs.is_some()) => + { + PolicyResult::Deny { + reason: DenyReason::GuardRejected { + code: "no-clock".into(), + }, + } + } + Some(reg) + if reg.limits.dead_man_secs.is_some_and(|limit| { + now.is_some_and(|n| { + reg.state + .last_checkin + .is_some_and(|seen| n.saturating_sub(seen) > limit) + }) + }) => + { + let limit = reg.limits.dead_man_secs.unwrap_or_default(); + let seen = reg.state.last_checkin.unwrap_or_default(); + PolicyResult::Deny { + reason: DenyReason::DeadMan { + silent_for: now.unwrap_or_default().saturating_sub(seen), + limit, + }, + } + } + Some(reg) if reg.fsm.excluded_by(from, action).is_some() => PolicyResult::Deny { + reason: DenyReason::Excluded { + rule: reg + .fsm + .excluded_by(from, action) + .unwrap_or_default() + .to_string(), + }, + }, + Some(Registered { fsm, .. }) => match fsm.next_state(from, action) { + None => PolicyResult::Deny { + reason: DenyReason::UndeclaredTransition, + }, + Some(next) => match fsm.guard(from, action) { + None => PolicyResult::Allow { + next: format!("{:?}", next), + }, + Some(guard) => match guard.check(ctx) { + Ok(()) => PolicyResult::Allow { + next: format!("{:?}", next), + }, + Err(code) => PolicyResult::Deny { + reason: DenyReason::GuardRejected { code }, + }, + }, + }, + }, + } + } + + fn eval_inner( + &mut self, + scope: &ScopeKey, + from: &S, + action: &A, + ctx: &C, + now: Option, + actor: Option<&str>, + ) -> PolicyResult { + let result = match self.fsms.get(scope) { + None => PolicyResult::Deny { + reason: DenyReason::UnknownScope, + }, + // Expiry before everything. A grant that has stopped permits + // nothing, so there is no point asking what it would have allowed. + Some(reg) if reg.expires_at.is_some_and(|e| now.is_some_and(|n| n >= e)) => { + PolicyResult::Deny { + reason: DenyReason::Expired { + expires_at: reg.expires_at.unwrap_or_default(), + now: now.unwrap_or_default(), + }, + } + } + // An expiring grant evaluated without a clock. Refuse rather than + // guess: this engine cannot establish whether the grant is live, + // and absent policy is not permission. Mirrors how a guarded + // transition refuses without context. + Some(reg) if reg.expires_at.is_some() && now.is_none() => PolicyResult::Deny { + reason: DenyReason::GuardRejected { + code: "no-clock".into(), + }, + }, + // Exclusions next, before the transition is even looked up. A + // check placed after would let a declared transition decide the + // outcome, which is exactly the override this forbids. + Some(reg) if reg.fsm.excluded_by(from, action).is_some() => PolicyResult::Deny { + reason: DenyReason::Excluded { + rule: reg + .fsm + .excluded_by(from, action) + .unwrap_or_default() + .to_string(), + }, + }, + // Dead man: silence revokes. The burden is on the SUBJECT to check + // in, which is what distinguishes this from a liveness probe -- + // there, an observer polls and absence is merely information. + Some(reg) + if reg.limits.dead_man_secs.is_some_and(|limit| { + now.is_some_and(|n| { + reg.state + .last_checkin + .is_some_and(|seen| n.saturating_sub(seen) > limit) + }) + }) => + { + let limit = reg.limits.dead_man_secs.unwrap_or_default(); + let seen = reg.state.last_checkin.unwrap_or_default(); + PolicyResult::Deny { + reason: DenyReason::DeadMan { + silent_for: now.unwrap_or_default().saturating_sub(seen), + limit, + }, + } + } + // A rate- or dead-man-limited grant evaluated without a clock. + // Same refusal as an expiring one: it cannot be established, so it + // is not assumed. + Some(reg) + if now.is_none() + && (reg.limits.rate_per_min.is_some() + || reg.limits.dead_man_secs.is_some()) => + { + PolicyResult::Deny { + reason: DenyReason::GuardRejected { + code: "no-clock".into(), + }, + } + } + Some(Registered { fsm, .. }) => match fsm.next_state(from, action) { + None => PolicyResult::Deny { + reason: DenyReason::UndeclaredTransition, + }, + Some(next) => match fsm.guard(from, action) { + None => PolicyResult::Allow { + next: format!("{:?}", next), + }, + Some(guard) => match guard.check(ctx) { + Ok(()) => PolicyResult::Allow { + next: format!("{:?}", next), + }, + Err(code) => PolicyResult::Deny { + reason: DenyReason::GuardRejected { code }, + }, + }, + }, + }, + }; + + // Limits last among the checks, because irreversible_budget needs the + // tier and the tier needs the transition. Still BEFORE any work: this + // whole function runs before the action does, so exhaustion is a + // refusal rather than a slow success. + // + // Consumed only when the answer is Allow. Charging a refused action + // would let one grant's rejections exhaust it. + let result = match (&result, now) { + (PolicyResult::Allow { .. }, Some(n)) => { + match self.check_limits(scope, from, action, n) { + Some(reason) => PolicyResult::Deny { reason }, + None => { + self.consume_limits(scope, from, action); + result + } + } + } + _ => result, + }; + + let event = PolicyEvent { + scope: scope.clone(), + from: format!("{:?}", from), + action: format!("{:?}", action), + result: result.clone(), + actor: actor.map(str::to_string), + acts_for: None, + }; + self.auditor.audit(&event); + + result + } +} + +// Convenience: when C = (), context-less evaluate routes through +// evaluate_with_context with a `()` context. Guards over `()` see +// the unit context and run; guards over richer types should use +// evaluate_with_context directly. This keeps the existing test +// surface and the cycle-1 trusty-policy-audit adapter unchanged. +impl PolicyEngine +where + S: Eq + Hash + Clone + Debug, + A: Eq + Hash + Clone + Debug, +{ + /// Context-less gate: equivalent to `evaluate_with_context(..., &(), ...)`. + pub fn evaluate( + &mut self, + scope: &ScopeKey, + from: &S, + action: &A, + actor: Option<&str>, + ) -> PolicyResult { + self.evaluate_with_context(scope, from, action, &(), actor) + } + + /// [`Self::evaluate`] with the current wall-clock time in unix seconds. + pub fn evaluate_at( + &mut self, + scope: &ScopeKey, + from: &S, + action: &A, + now: i64, + actor: Option<&str>, + ) -> PolicyResult { + self.evaluate_with_context_at(scope, from, action, &(), now, actor) + } +} + +// ------------------------------------------------------------ +// Tests +// ------------------------------------------------------------ + +#[cfg(test)] +mod tests { + use super::*; + + fn pipeline_fsm() -> Fsm<&'static str, &'static str> { + let mut f = Fsm::new("idle"); + f.allow(Transition::new("idle", "claim", "claimed")) + .allow(Transition::new("claimed", "execute", "running")) + .allow(Transition::new("running", "finish", "done")) + .allow(Transition::new("running", "abort", "aborted")); + f + } + + #[test] + fn legal_transition_allows() { + let scope = ScopeKey::new("agent.executor.v1"); + let mut engine: PolicyEngine<&'static str, &'static str> = + PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), pipeline_fsm()).unwrap(); + + let r = engine.evaluate(&scope, &"idle", &"claim", None); + assert!(r.is_allow(), "{:?}", r); + match r { + PolicyResult::Allow { next } => assert_eq!(next, "\"claimed\""), + _ => panic!(), + } + } + + #[test] + fn undeclared_transition_denies() { + let scope = ScopeKey::new("agent.executor.v1"); + let mut engine: PolicyEngine<&'static str, &'static str> = + PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), pipeline_fsm()).unwrap(); + + let r = engine.evaluate(&scope, &"idle", &"finish", None); + assert!(!r.is_allow()); + assert_eq!( + r, + PolicyResult::Deny { + reason: DenyReason::UndeclaredTransition + } + ); + } + + #[test] + fn unknown_scope_denies() { + let unknown = ScopeKey::new("agent.unknown.v0"); + let mut engine: PolicyEngine<&'static str, &'static str> = + PolicyEngine::new(NoopAuditor::default()); + let r = engine.evaluate(&unknown, &"idle", &"claim", None); + assert!(!r.is_allow()); + assert_eq!( + r, + PolicyResult::Deny { + reason: DenyReason::UnknownScope + } + ); + } + + #[test] + fn auditor_sees_every_evaluation() { + let scope = ScopeKey::new("agent.executor.v1"); + let mut engine: PolicyEngine<&'static str, &'static str> = + PolicyEngine::new(CapturingAuditor::default()); + engine.register(scope.clone(), pipeline_fsm()).unwrap(); + + let _ = engine.evaluate(&scope, &"idle", &"claim", Some("agent_001")); + let _ = engine.evaluate(&scope, &"claimed", &"finish", Some("agent_001")); + let _ = engine.evaluate(&scope, &"running", &"finish", None); + + // We need to recover the auditor to inspect; the engine owns + // it as Box. Simplest path for the test: re-evaluate + // through a fresh engine that holds a direct reference. + // (For now, we just confirm no panic + Allow/Deny are well- + // formed by re-running the same calls and asserting.) + assert!(engine.evaluate(&scope, &"idle", &"claim", None).is_allow()); + assert!(!engine + .evaluate(&scope, &"claimed", &"finish", None) + .is_allow()); + assert!(engine + .evaluate(&scope, &"running", &"finish", None) + .is_allow()); + } + + #[test] + fn duplicate_scope_registration_errors() { + let scope = ScopeKey::new("agent.executor.v1"); + let mut engine: PolicyEngine<&'static str, &'static str> = + PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), pipeline_fsm()).unwrap(); + let err = engine.register(scope, pipeline_fsm()); + assert!(matches!(err, Err(PolicyError::DuplicateScope(_)))); + } + + #[test] + fn upsert_replaces_existing_fsm() { + let scope = ScopeKey::new("agent.executor.v1"); + let mut engine: PolicyEngine<&'static str, &'static str> = + PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), pipeline_fsm()).unwrap(); + + // Replace with a stricter FSM that no longer allows abort. + let mut strict = Fsm::new("idle"); + strict.allow(Transition::new("idle", "claim", "claimed")); + engine.upsert(scope.clone(), strict); + + // execute is no longer declared after upsert. + let r = engine.evaluate(&scope, &"claimed", &"execute", None); + assert!(!r.is_allow()); + } + + #[test] + fn fsm_introspection_returns_all_transitions() { + let f = pipeline_fsm(); + let count = f.transitions().count(); + assert_eq!(count, 4); + } + + // ---- Guard tests (cycle 2) ---- + + #[derive(Debug)] + struct AuthContext { + balance: i64, + is_admin: bool, + } + + #[test] + fn guard_allows_when_predicate_satisfied() { + let scope = ScopeKey::new("agent.admin.v1"); + let mut fsm: Fsm<&'static str, &'static str, AuthContext> = Fsm::new("idle"); + fsm.allow(Transition::new("idle", "claim", "claimed")) + .allow_guarded( + Transition::new("claimed", "withdraw", "withdrawn"), + |ctx: &AuthContext| { + if ctx.balance > 0 { + Ok(()) + } else { + Err("insufficient-balance".to_string()) + } + }, + ); + + let mut engine: PolicyEngine<&'static str, &'static str, AuthContext> = + PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), fsm).unwrap(); + + let ctx = AuthContext { + balance: 100, + is_admin: false, + }; + let r = engine.evaluate_with_context(&scope, &"claimed", &"withdraw", &ctx, None); + assert!(r.is_allow(), "{:?}", r); + } + + #[test] + fn guard_denies_with_code_when_predicate_fails() { + let scope = ScopeKey::new("agent.admin.v1"); + let mut fsm: Fsm<&'static str, &'static str, AuthContext> = Fsm::new("idle"); + fsm.allow_guarded( + Transition::new("idle", "admin_op", "running"), + |ctx: &AuthContext| { + if ctx.is_admin { + Ok(()) + } else { + Err("not-admin".to_string()) + } + }, + ); + + let mut engine: PolicyEngine<&'static str, &'static str, AuthContext> = + PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), fsm).unwrap(); + + let ctx = AuthContext { + balance: 0, + is_admin: false, + }; + let r = engine.evaluate_with_context(&scope, &"idle", &"admin_op", &ctx, None); + assert_eq!( + r, + PolicyResult::Deny { + reason: DenyReason::GuardRejected { + code: "not-admin".into() + } + } + ); + } + + #[test] + fn ungarded_transition_evaluates_without_running_any_guard() { + // Multiple transitions, only some guarded. The unguarded + // ones evaluate exactly as before. + let scope = ScopeKey::new("agent.mixed.v1"); + let mut fsm: Fsm<&'static str, &'static str, AuthContext> = Fsm::new("idle"); + fsm.allow(Transition::new("idle", "noop", "idle")) + .allow_guarded( + Transition::new("idle", "admin_op", "running"), + |ctx: &AuthContext| { + if ctx.is_admin { + Ok(()) + } else { + Err("not-admin".into()) + } + }, + ); + + let mut engine: PolicyEngine<&'static str, &'static str, AuthContext> = + PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), fsm).unwrap(); + + let ctx = AuthContext { + balance: 0, + is_admin: false, + }; + // unguarded transition allowed regardless of context. + assert!(engine + .evaluate_with_context(&scope, &"idle", &"noop", &ctx, None) + .is_allow()); + // guarded transition denied (not admin). + assert!(!engine + .evaluate_with_context(&scope, &"idle", &"admin_op", &ctx, None) + .is_allow()); + } + + #[test] + fn unit_context_evaluate_routes_through_guarded_path_for_default_c() { + // When C = (), the existing context-less `evaluate` must + // continue to work for unguarded transitions (back-compat). + let scope = ScopeKey::new("agent.executor.v1"); + let mut engine: PolicyEngine<&'static str, &'static str> = + PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), pipeline_fsm()).unwrap(); + assert!(engine.evaluate(&scope, &"idle", &"claim", None).is_allow()); + assert!(!engine.evaluate(&scope, &"idle", &"finish", None).is_allow()); + } + + // ---- TOML spec tests (cycle 3) ---- + + const PIPELINE_SPEC: &str = r#" +{ + "transitions": [ + { + "from": "idle", + "action": "claim", + "to": "claimed" + }, + { + "from": "claimed", + "action": "execute", + "to": "running" + }, + { + "from": "running", + "action": "finish", + "to": "done" + } + ], + "scope": "agent.executor.v1", + "initial": "idle" +} +"#; + + #[test] + fn parse_a_spec_and_build_the_fsm() { + let spec = FsmSpec::from_json(PIPELINE_SPEC).unwrap(); + assert_eq!(spec.scope, "agent.executor.v1"); + assert_eq!(spec.initial, "idle"); + assert_eq!(spec.transitions.len(), 3); + + let (scope, fsm): (_, Fsm) = spec.build_fsm(); + assert_eq!(scope.as_str(), "agent.executor.v1"); + assert_eq!(fsm.initial(), "idle"); + assert_eq!(fsm.transitions().count(), 3); + } + + #[test] + fn toml_loaded_fsm_drives_engine() { + let spec = FsmSpec::from_json(PIPELINE_SPEC).unwrap(); + let (scope, fsm) = spec.build_fsm::<()>(); + + let mut engine: PolicyEngine = PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), fsm).unwrap(); + + let r = engine.evaluate(&scope, &"idle".to_string(), &"claim".to_string(), None); + assert!(r.is_allow(), "{:?}", r); + + let r = engine.evaluate(&scope, &"idle".to_string(), &"finish".to_string(), None); + assert!(!r.is_allow()); + } + + #[test] + fn toml_round_trip_preserves_shape() { + let spec = FsmSpec::from_json(PIPELINE_SPEC).unwrap(); + let serialized = spec.canonical_json().unwrap(); + let reparsed = FsmSpec::from_json(&serialized).unwrap(); + assert_eq!(reparsed.scope, spec.scope); + assert_eq!(reparsed.initial, spec.initial); + assert_eq!(reparsed.transitions.len(), spec.transitions.len()); + for (a, b) in reparsed.transitions.iter().zip(spec.transitions.iter()) { + assert_eq!(a.from, b.from); + assert_eq!(a.action, b.action); + assert_eq!(a.to, b.to); + } + } + + #[test] + fn malformed_toml_returns_parse_error() { + let bad = "not valid toml = = ="; + let r = FsmSpec::from_json(bad); + assert!(matches!(r, Err(PolicyError::Parse(_)))); + } + + #[test] + fn a_spec_with_no_transitions_is_allowed() { + // An FSM with only an initial state and no transitions + // is a valid (terminal-only) spec; declared transitions + // can be added later via Fsm::allow. + let spec_str = r#" +{ + "scope": "agent.empty.v1", + "initial": "start" +} +"#; + let spec = FsmSpec::from_json(spec_str).unwrap(); + assert_eq!(spec.transitions.len(), 0); + let (_, fsm): (_, Fsm) = spec.build_fsm(); + assert_eq!(fsm.transitions().count(), 0); + } + + #[test] + fn guard_over_unit_context_runs_under_evaluate() { + // A Guard<()> registered on an FSM<_,_,()> runs on the + // context-less evaluate path. This is rare but supported. + let scope = ScopeKey::new("agent.unit.v1"); + let mut fsm: Fsm<&'static str, &'static str> = Fsm::new("idle"); + fsm.allow_guarded(Transition::new("idle", "go", "running"), |_: &()| { + Err("always-deny".into()) + }); + let mut engine: PolicyEngine<&'static str, &'static str> = + PolicyEngine::new(NoopAuditor::default()); + engine.register(scope.clone(), fsm).unwrap(); + let r = engine.evaluate(&scope, &"idle", &"go", None); + assert_eq!( + r, + PolicyResult::Deny { + reason: DenyReason::GuardRejected { + code: "always-deny".into() + } + } + ); + } +} + +#[cfg(test)] +mod guard_registry_tests { + use super::*; + + const SPEC: &str = r#" +{ + "transitions": [ + { + "from": "idle", + "action": "claim", + "to": "claimed" + }, + { + "from": "claimed", + "action": "spend", + "to": "spent", + "guard": "within_budget" + } + ], + "scope": "agent.executor.v1", + "initial": "idle" +} +"#; + + #[derive(Debug)] + struct Ctx { + budget: i64, + } + + fn within_budget(c: &Ctx) -> Result<(), String> { + if c.budget > 0 { + Ok(()) + } else { + Err("budget-exhausted".into()) + } + } + + #[test] + fn a_named_guard_resolves_and_gates_the_transition() { + let spec = FsmSpec::from_json(SPEC).unwrap(); + let mut reg: GuardRegistry = GuardRegistry::new(); + reg.insert("within_budget", within_budget); + + let (scope, fsm) = spec.build_fsm_with_guards(&mut reg).unwrap(); + let mut engine = PolicyEngine::new(NoopAuditor); + engine.register(scope.clone(), fsm).unwrap(); + + let allowed = engine.evaluate_with_context( + &scope, + &"claimed".to_string(), + &"spend".to_string(), + &Ctx { budget: 5 }, + None, + ); + assert!( + allowed.is_allow(), + "guard passed, transition must be allowed" + ); + + let denied = engine.evaluate_with_context( + &scope, + &"claimed".to_string(), + &"spend".to_string(), + &Ctx { budget: 0 }, + None, + ); + assert!(!denied.is_allow(), "guard rejected, must deny"); + } + + #[test] + fn an_unregistered_guard_refuses_to_build() { + // THE point of the mechanism. A name resolving to nothing would leave + // the transition UNGUARDED, so a typo in a policy file would quietly + // remove a restriction instead of failing. It must fail at build time, + // before the gate is ever consulted. + let spec = FsmSpec::from_json(SPEC).unwrap(); + let mut reg: GuardRegistry = GuardRegistry::new(); + // deliberately empty + + let err = spec.build_fsm_with_guards(&mut reg).unwrap_err(); + match err { + PolicyError::UnregisteredGuard { + ref guard, + ref action, + .. + } => { + assert_eq!(guard, "within_budget"); + assert_eq!(action, "spend", "the error must name WHICH transition"); + } + other => panic!("expected UnregisteredGuard, got {other:?}"), + } + } + + #[test] + fn a_typo_in_the_guard_name_is_caught() { + let spec = FsmSpec::from_json(SPEC).unwrap(); + let mut reg: GuardRegistry = GuardRegistry::new(); + reg.insert("within_budgets", within_budget); // trailing s + + assert!( + spec.build_fsm_with_guards(&mut reg).is_err(), + "a near-miss name must not silently leave the transition open" + ); + } + + #[test] + fn unguarded_transitions_still_build() { + let spec = FsmSpec::from_json( + r#" +{ + "transitions": [ + { + "from": "idle", + "action": "claim", + "to": "claimed" + } + ], + "scope": "s", + "initial": "idle" +} +"#, + ) + .unwrap(); + let mut reg: GuardRegistry = GuardRegistry::new(); + assert!(spec.build_fsm_with_guards(&mut reg).is_ok()); + } + + #[test] + fn guard_names_round_trip_through_toml() { + let spec = FsmSpec::from_json(SPEC).unwrap(); + let out = spec.canonical_json().unwrap(); + assert!( + out.contains("within_budget"), + "guard name must survive:\n{out}" + ); + let back = FsmSpec::from_json(&out).unwrap(); + assert_eq!(back.transitions[1].guard.as_deref(), Some("within_budget")); + } +} + +#[cfg(test)] +mod exclusion_tests { + use super::*; + + fn engine_with( + fsm: Fsm<&'static str, &'static str>, + ) -> (ScopeKey, PolicyEngine<&'static str, &'static str>) { + let scope = ScopeKey::new("s"); + let mut e = PolicyEngine::new(NoopAuditor); + e.register(scope.clone(), fsm).unwrap(); + (scope, e) + } + + #[test] + fn an_exclusion_beats_a_declared_transition() { + // The rule that matters: a grant permitting the action exists, and the + // exclusion still wins. If it did not, an exclusion would be advice. + let mut f = Fsm::new("idle"); + f.allow(Transition::new("idle", "spend", "spent")) + .exclude("spend", "no-spending-without-approval"); + + let (scope, mut e) = engine_with(f); + let r = e.evaluate(&scope, &"idle", &"spend", None); + match r { + PolicyResult::Deny { + reason: DenyReason::Excluded { ref rule }, + } => assert_eq!(rule, "no-spending-without-approval"), + other => panic!("expected Excluded, got {other:?}"), + } + } + + #[test] + fn adding_a_wider_grant_later_does_not_reopen_it() { + let mut f = Fsm::new("idle"); + f.exclude("spend", "hard-stop"); + // A later, wider allow -- the ordering trap this guards against. + f.allow(Transition::new("idle", "spend", "spent")) + .allow(Transition::new("claimed", "spend", "spent")); + + let (scope, mut e) = engine_with(f); + assert!(!e.evaluate(&scope, &"idle", &"spend", None).is_allow()); + assert!(!e.evaluate(&scope, &"claimed", &"spend", None).is_allow()); + } + + #[test] + fn an_exclusion_does_not_leak_to_other_actions() { + let mut f = Fsm::new("idle"); + f.allow(Transition::new("idle", "read", "idle")) + .allow(Transition::new("idle", "spend", "spent")) + .exclude("spend", "hard-stop"); + + let (scope, mut e) = engine_with(f); + assert!(e.evaluate(&scope, &"idle", &"read", None).is_allow()); + assert!(!e.evaluate(&scope, &"idle", &"spend", None).is_allow()); + } + + #[test] + fn a_state_specific_exclusion_binds_only_that_state() { + let mut f = Fsm::new("idle"); + f.allow(Transition::new("idle", "publish", "live")) + .allow(Transition::new("review", "publish", "live")) + .exclude_from("idle", "publish", "review-required"); + + let (scope, mut e) = engine_with(f); + assert!(!e.evaluate(&scope, &"idle", &"publish", None).is_allow()); + assert!(e.evaluate(&scope, &"review", &"publish", None).is_allow()); + } + + #[test] + fn excluded_is_distinguishable_from_merely_undeclared() { + // "nothing permitted it" and "something forbade it" are different + // answers, and an audit reader needs to tell them apart. + let mut f = Fsm::new("idle"); + f.exclude("spend", "hard-stop"); + let (scope, mut e) = engine_with(f); + + let excluded = e.evaluate(&scope, &"idle", &"spend", None); + let undeclared = e.evaluate(&scope, &"idle", &"wander", None); + + assert!(matches!( + excluded, + PolicyResult::Deny { + reason: DenyReason::Excluded { .. } + } + )); + assert!(matches!( + undeclared, + PolicyResult::Deny { + reason: DenyReason::UndeclaredTransition + } + )); + } + + #[test] + fn an_exclusion_outranks_a_guard_that_would_have_passed() { + // Exclusions are checked before the transition, so the guard is never + // consulted -- it cannot vote its way past one. + let mut f: Fsm<&str, &str, i64> = Fsm::new("idle"); + f.allow_guarded(Transition::new("idle", "spend", "spent"), |_c: &i64| Ok(())) + .exclude("spend", "hard-stop"); + + let scope = ScopeKey::new("s"); + let mut e: PolicyEngine<&str, &str, i64> = PolicyEngine::new(NoopAuditor); + e.register(scope.clone(), f).unwrap(); + + let r = e.evaluate_with_context(&scope, &"idle", &"spend", &1i64, None); + assert!(matches!( + r, + PolicyResult::Deny { + reason: DenyReason::Excluded { .. } + } + )); + } +} + +#[cfg(test)] +mod expiry_tests { + use super::*; + + const T: i64 = 1_700_000_000; + + fn spend_fsm() -> Fsm<&'static str, &'static str> { + let mut f = Fsm::new("idle"); + f.allow(Transition::new("idle", "spend", "spent")); + f + } + + fn engine_until(expires_at: i64) -> (ScopeKey, PolicyEngine<&'static str, &'static str>) { + let scope = ScopeKey::new("s"); + let mut e = PolicyEngine::new(NoopAuditor); + e.register_until(scope.clone(), spend_fsm(), expires_at) + .unwrap(); + (scope, e) + } + + #[test] + fn a_live_grant_allows() { + let (scope, mut e) = engine_until(T + 100); + assert!(e.evaluate_at(&scope, &"idle", &"spend", T, None).is_allow()); + } + + #[test] + fn an_expired_grant_denies_even_though_the_transition_is_declared() { + let (scope, mut e) = engine_until(T); + match e.evaluate_at(&scope, &"idle", &"spend", T + 1, None) { + PolicyResult::Deny { + reason: DenyReason::Expired { expires_at, now }, + } => { + assert_eq!(expires_at, T); + assert_eq!(now, T + 1); + } + other => panic!("expected Expired, got {other:?}"), + } + } + + #[test] + fn expiry_is_wall_clock_not_completion() { + // The point of wall clock: work still in flight stops. Nothing here + // signals completion, and the grant ends anyway. + let (scope, mut e) = engine_until(T); + assert!(e + .evaluate_at(&scope, &"idle", &"spend", T - 1, None) + .is_allow()); + assert!(!e.evaluate_at(&scope, &"idle", &"spend", T, None).is_allow()); + } + + #[test] + fn the_boundary_instant_is_expired() { + // now == expires_at is over, not the last permitted moment. + let (scope, mut e) = engine_until(T); + assert!(!e.evaluate_at(&scope, &"idle", &"spend", T, None).is_allow()); + } + + #[test] + fn an_expiring_grant_without_a_clock_refuses() { + // Absent policy is not permission: the engine cannot establish + // liveness, so it denies rather than assuming the grant is live. + let (scope, mut e) = engine_until(T + 100); + match e.evaluate(&scope, &"idle", &"spend", None) { + PolicyResult::Deny { + reason: DenyReason::GuardRejected { ref code }, + } => assert_eq!(code, "no-clock"), + other => panic!("expected no-clock refusal, got {other:?}"), + } + } + + #[test] + fn a_non_expiring_grant_is_unaffected_by_the_clockless_path() { + let scope = ScopeKey::new("s"); + let mut e = PolicyEngine::new(NoopAuditor); + e.register(scope.clone(), spend_fsm()).unwrap(); + assert!(e.evaluate(&scope, &"idle", &"spend", None).is_allow()); + assert!(e.evaluate_at(&scope, &"idle", &"spend", T, None).is_allow()); + } + + #[test] + fn expiry_outranks_a_declared_transition_and_an_exclusion_alike() { + // Ordering: expiry is checked before exclusions, so an expired grant + // reports Expired rather than Excluded. A stopped grant permits + // nothing, and saying WHY it stopped is the more useful answer. + let scope = ScopeKey::new("s"); + let mut f = spend_fsm(); + f.exclude("spend", "hard-stop"); + let mut e = PolicyEngine::new(NoopAuditor); + e.register_until(scope.clone(), f, T).unwrap(); + + assert!(matches!( + e.evaluate_at(&scope, &"idle", &"spend", T + 1, None), + PolicyResult::Deny { + reason: DenyReason::Expired { .. } + } + )); + } +} + +#[cfg(test)] +mod tier_tests { + use super::*; + + #[test] + fn tier_maps_one_to_one_onto_the_sql_verbs() { + // The mapping is the reason there are exactly four. If this ever needs + // a translation layer, that layer is where the application's idea of a + // permission and the database's will drift apart. + assert_eq!(Tier::Read.sql_verb(), "SELECT"); + assert_eq!(Tier::Append.sql_verb(), "INSERT"); + assert_eq!(Tier::Update.sql_verb(), "UPDATE"); + assert_eq!(Tier::Delete.sql_verb(), "DELETE"); + } + + #[test] + fn only_delete_is_irreversible() { + assert!(Tier::Delete.is_irreversible()); + for t in [Tier::Read, Tier::Append, Tier::Update] { + assert!(!t.is_irreversible(), "{t:?} must not count against budget"); + } + } + + #[test] + fn tier_is_declared_on_the_transition_and_read_back() { + let mut f: Fsm<&str, &str> = Fsm::new("idle"); + f.allow(Transition::new("idle", "purge", "purged").with_tier(Tier::Delete)) + .allow(Transition::new("idle", "look", "idle").with_tier(Tier::Read)); + + assert_eq!(f.tier_of(&"idle", &"purge"), Some(Tier::Delete)); + assert_eq!(f.tier_of(&"idle", &"look"), Some(Tier::Read)); + } + + #[test] + fn an_unclassified_action_reports_none_not_read() { + // "Not stated" must never read as "cheapest". That is how an + // unreviewed action acquires the widest permission. + let mut f: Fsm<&str, &str> = Fsm::new("idle"); + f.allow(Transition::new("idle", "mystery", "elsewhere")); + assert_eq!(f.tier_of(&"idle", &"mystery"), None); + } + + #[test] + fn tier_is_ordered_least_to_most_destructive() { + // Ord exists so a grant can express "up to and including update". + assert!(Tier::Read < Tier::Append); + assert!(Tier::Append < Tier::Update); + assert!(Tier::Update < Tier::Delete); + } + + #[test] + fn tier_round_trips_through_a_spec() { + let spec = FsmSpec::from_json( + r#" +{ + "transitions": [ + { + "from": "idle", + "action": "purge", + "to": "purged", + "tier": "delete" + } + ], + "scope": "s", + "initial": "idle" +} +"#, + ) + .unwrap(); + assert_eq!(spec.transitions[0].tier, Some(Tier::Delete)); + + let (_scope, fsm): (_, Fsm) = spec.build_fsm(); + assert_eq!( + fsm.tier_of(&"idle".to_string(), &"purge".to_string()), + Some(Tier::Delete), + "a tier declared in the spec must reach the built FSM" + ); + + let out = spec.canonical_json().unwrap(); + assert!( + out.contains("delete"), + "tier must survive serialisation:\n{out}" + ); + } + + #[test] + fn the_caller_cannot_supply_a_tier() { + // Structural, not a runtime check: no evaluate signature takes a tier. + // A caller able to name its own tier can name the cheapest one, so the + // only safe place for it is the declaration. + let mut f: Fsm<&str, &str> = Fsm::new("idle"); + f.allow(Transition::new("idle", "purge", "purged").with_tier(Tier::Delete)); + let scope = ScopeKey::new("s"); + let mut e = PolicyEngine::new(NoopAuditor); + e.register(scope.clone(), f).unwrap(); + + // The only tier that exists is the declared one. + assert!(e.evaluate(&scope, &"idle", &"purge", None).is_allow()); + assert_eq!( + e.fsm(&scope).unwrap().tier_of(&"idle", &"purge"), + Some(Tier::Delete) + ); + } +} + +#[cfg(test)] +mod limits_tests { + use super::*; + + const T: i64 = 1_700_000_000; + + fn fsm() -> Fsm<&'static str, &'static str> { + let mut f = Fsm::new("idle"); + f.allow(Transition::new("idle", "look", "idle").with_tier(Tier::Read)) + .allow(Transition::new("idle", "purge", "idle").with_tier(Tier::Delete)); + f + } + + fn engine(limits: Limits) -> (ScopeKey, PolicyEngine<&'static str, &'static str>) { + let scope = ScopeKey::new("s"); + let mut e = PolicyEngine::new(NoopAuditor); + e.register_with_limits(scope.clone(), fsm(), limits, None) + .unwrap(); + (scope, e) + } + + #[test] + fn rate_refuses_once_the_window_is_spent() { + let (scope, mut e) = engine(Limits { + rate_per_min: Some(2), + ..Default::default() + }); + assert!(e.evaluate_at(&scope, &"idle", &"look", T, None).is_allow()); + assert!(e.evaluate_at(&scope, &"idle", &"look", T, None).is_allow()); + match e.evaluate_at(&scope, &"idle", &"look", T, None) { + PolicyResult::Deny { + reason: DenyReason::RateExceeded { limit, window_secs }, + } => { + assert_eq!(limit, 2); + assert_eq!(window_secs, 60); + } + other => panic!("expected RateExceeded, got {other:?}"), + } + } + + #[test] + fn the_rate_window_resets() { + let (scope, mut e) = engine(Limits { + rate_per_min: Some(1), + ..Default::default() + }); + assert!(e.evaluate_at(&scope, &"idle", &"look", T, None).is_allow()); + assert!(!e + .evaluate_at(&scope, &"idle", &"look", T + 59, None) + .is_allow()); + assert!(e + .evaluate_at(&scope, &"idle", &"look", T + 60, None) + .is_allow()); + } + + #[test] + fn a_refused_action_does_not_consume_rate() { + // Charging a refusal would let rejections exhaust the grant. + let (scope, mut e) = engine(Limits { + rate_per_min: Some(1), + ..Default::default() + }); + // Undeclared -- denied, and must not count. + assert!(!e + .evaluate_at(&scope, &"idle", &"wander", T, None) + .is_allow()); + assert!( + e.evaluate_at(&scope, &"idle", &"look", T, None).is_allow(), + "the real action must still have its allowance" + ); + } + + #[test] + fn only_delete_tier_spends_the_irreversible_budget() { + let (scope, mut e) = engine(Limits { + irreversible_budget: Some(1), + ..Default::default() + }); + // Reads are free, however many. + for _ in 0..5 { + assert!(e.evaluate_at(&scope, &"idle", &"look", T, None).is_allow()); + } + assert_eq!(e.irreversible_remaining(&scope), Some(1)); + + assert!(e.evaluate_at(&scope, &"idle", &"purge", T, None).is_allow()); + assert_eq!(e.irreversible_remaining(&scope), Some(0)); + + assert!(matches!( + e.evaluate_at(&scope, &"idle", &"purge", T, None), + PolicyResult::Deny { + reason: DenyReason::IrreversibleBudgetExhausted + } + )); + } + + #[test] + fn the_irreversible_budget_does_not_refill_with_time() { + // A budget that replenishes is a rate limit. This one is all the + // destruction the grant ever gets. + let (scope, mut e) = engine(Limits { + irreversible_budget: Some(1), + ..Default::default() + }); + assert!(e.evaluate_at(&scope, &"idle", &"purge", T, None).is_allow()); + assert!(!e + .evaluate_at(&scope, &"idle", &"purge", T + 86_400, None) + .is_allow()); + } + + #[test] + fn dead_man_revokes_on_silence() { + let (scope, mut e) = engine(Limits { + dead_man_secs: Some(30), + ..Default::default() + }); + e.record_checkin(&scope, T); + + assert!(e + .evaluate_at(&scope, &"idle", &"look", T + 30, None) + .is_allow()); + match e.evaluate_at(&scope, &"idle", &"look", T + 31, None) { + PolicyResult::Deny { + reason: DenyReason::DeadMan { silent_for, limit }, + } => { + assert_eq!(limit, 30); + assert_eq!(silent_for, 31); + } + other => panic!("expected DeadMan, got {other:?}"), + } + } + + #[test] + fn checking_in_again_revives_the_grant() { + let (scope, mut e) = engine(Limits { + dead_man_secs: Some(30), + ..Default::default() + }); + e.record_checkin(&scope, T); + assert!(!e + .evaluate_at(&scope, &"idle", &"look", T + 100, None) + .is_allow()); + e.record_checkin(&scope, T + 100); + assert!(e + .evaluate_at(&scope, &"idle", &"look", T + 110, None) + .is_allow()); + } + + #[test] + fn a_clock_dependent_limit_refuses_without_a_clock() { + let (scope, mut e) = engine(Limits { + rate_per_min: Some(5), + ..Default::default() + }); + match e.evaluate(&scope, &"idle", &"look", None) { + PolicyResult::Deny { + reason: DenyReason::GuardRejected { ref code }, + } => assert_eq!(code, "no-clock"), + other => panic!("expected no-clock refusal, got {other:?}"), + } + } + + #[test] + fn an_unlimited_grant_is_unaffected() { + let (scope, mut e) = engine(Limits::default()); + for _ in 0..100 { + assert!(e.evaluate_at(&scope, &"idle", &"purge", T, None).is_allow()); + } + assert_eq!(e.irreversible_remaining(&scope), None); + } +} + +#[cfg(test)] +mod delegation_tests { + use super::*; + + const T: i64 = 1_700_000_000; + + fn agent_key() -> ScopeKey { + ScopeKey::new("agent:googz") + } + fn eric() -> ScopeKey { + ScopeKey::new("user:eric") + } + + /// An FSM permitting exactly the listed actions from "idle". + fn permitting(actions: &[(&'static str, Tier)]) -> Fsm<&'static str, &'static str> { + let mut f = Fsm::new("idle"); + for (a, tier) in actions { + f.allow(Transition::new("idle", *a, "idle").with_tier(*tier)); + } + f + } + + fn engine( + agent_can: &[(&'static str, Tier)], + eric_can: &[(&'static str, Tier)], + ) -> PolicyEngine<&'static str, &'static str> { + let mut e = PolicyEngine::new(NoopAuditor); + e.register(agent_key(), permitting(agent_can)).unwrap(); + e.register(eric(), permitting(eric_can)).unwrap(); + e + } + + #[test] + fn both_permitting_allows() { + let mut e = engine(&[("read", Tier::Read)], &[("read", Tier::Read)]); + assert!(e + .evaluate_delegated(&agent_key(), &eric(), &"idle", &"read", &(), Some(T), None) + .is_allow()); + } + + #[test] + fn an_agent_cannot_exceed_the_principal_it_acts_for() { + // THE rule. The agent's own grant is wider; it does not matter. + let mut e = engine( + &[("read", Tier::Read), ("purge", Tier::Delete)], + &[("read", Tier::Read)], + ); + assert!(e + .evaluate_delegated(&agent_key(), &eric(), &"idle", &"read", &(), Some(T), None) + .is_allow()); + assert!( + !e.evaluate_delegated(&agent_key(), &eric(), &"idle", &"purge", &(), Some(T), None) + .is_allow(), + "an agent must never reach past the party behind it" + ); + } + + #[test] + fn a_wider_principal_does_not_widen_the_agent() { + // The intersection cuts both ways: Eric may purge, his agent may not. + let mut e = engine( + &[("read", Tier::Read)], + &[("read", Tier::Read), ("purge", Tier::Delete)], + ); + assert!(!e + .evaluate_delegated(&agent_key(), &eric(), &"idle", &"purge", &(), Some(T), None) + .is_allow()); + } + + #[test] + fn an_unknown_principal_denies() { + // Deny by default: a delegation to a principal with no policy is not a + // delegation to an unrestricted one. + let mut e = PolicyEngine::new(NoopAuditor); + e.register(agent_key(), permitting(&[("read", Tier::Read)])) + .unwrap(); + assert!(matches!( + e.evaluate_delegated( + &agent_key(), + &ScopeKey::new("user:nobody"), + &"idle", + &"read", + &(), + Some(T), + None + ), + PolicyResult::Deny { + reason: DenyReason::UnknownScope + } + )); + } + + #[test] + fn the_refusing_side_is_the_one_reported() { + // "the agent had no such grant" and "the person does not have that + // permission" are different findings; the audit must tell them apart. + let mut e = engine(&[("read", Tier::Read)], &[("read", Tier::Read)]); + assert!(matches!( + e.evaluate_delegated(&agent_key(), &eric(), &"idle", &"purge", &(), Some(T), None), + PolicyResult::Deny { + reason: DenyReason::UndeclaredTransition + } + )); + } + + #[test] + fn revoking_the_agent_leaves_the_principal_untouched() { + // What the intersection buys operationally. + let mut e = engine(&[("read", Tier::Read)], &[("read", Tier::Read)]); + e.upsert(agent_key(), permitting(&[])); // agent's grant killed + + assert!(!e + .evaluate_delegated(&agent_key(), &eric(), &"idle", &"read", &(), Some(T), None) + .is_allow()); + assert!( + e.evaluate(&eric(), &"idle", &"read", None).is_allow(), + "Eric still has his own permission" + ); + } + + #[test] + fn neither_side_is_charged_when_the_other_refuses() { + // Checking and charging in one pass would bill the agent for an action + // its principal then refused. + let mut e = PolicyEngine::new(NoopAuditor); + e.register_with_limits( + agent_key(), + permitting(&[("purge", Tier::Delete)]), + Limits { + irreversible_budget: Some(1), + ..Default::default() + }, + None, + ) + .unwrap(); + // Eric cannot purge at all. + e.register(eric(), permitting(&[("read", Tier::Read)])) + .unwrap(); + + assert!(!e + .evaluate_delegated(&agent_key(), &eric(), &"idle", &"purge", &(), Some(T), None) + .is_allow()); + assert_eq!( + e.irreversible_remaining(&agent_key()), + Some(1), + "the agent's budget must be untouched by a refusal" + ); + } + + #[test] + fn acting_for_someone_spends_both_allowances() { + let mut e = PolicyEngine::new(NoopAuditor); + let budget = Limits { + irreversible_budget: Some(1), + ..Default::default() + }; + e.register_with_limits( + agent_key(), + permitting(&[("purge", Tier::Delete)]), + budget, + None, + ) + .unwrap(); + e.register_with_limits(eric(), permitting(&[("purge", Tier::Delete)]), budget, None) + .unwrap(); + + assert!(e + .evaluate_delegated(&agent_key(), &eric(), &"idle", &"purge", &(), Some(T), None) + .is_allow()); + assert_eq!(e.irreversible_remaining(&agent_key()), Some(0)); + assert_eq!( + e.irreversible_remaining(&eric()), + Some(0), + "the principal's budget is spent too -- the action was theirs" + ); + } + + #[test] + fn an_exclusion_on_either_side_wins() { + let mut e = PolicyEngine::new(NoopAuditor); + e.register(agent_key(), permitting(&[("spend", Tier::Update)])) + .unwrap(); + let mut erics = permitting(&[("spend", Tier::Update)]); + erics.exclude("spend", "no-spending"); + e.register(eric(), erics).unwrap(); + + assert!(matches!( + e.evaluate_delegated(&agent_key(), &eric(), &"idle", &"spend", &(), Some(T), None), + PolicyResult::Deny { + reason: DenyReason::Excluded { .. } + } + )); + } +} + +// ------------------------------------------------------------ +// Grant: the signable authorisation artifact +// ------------------------------------------------------------ + +/// A single authorisation, in the form that is stored, hashed and signed. +/// +/// Five fields and one ordering rule. A sixth is a conversation, not a default. +/// +/// # Why JSON and not YAML +/// +/// A grant is hashed, so it needs ONE byte sequence per document. YAML has no +/// canonical serialisation -- the same document has many valid encodings, and +/// its parsers turn `no` into `false`. Prose lives in `_why` rather than in +/// comments, which is what YAML was wanted for in the first place. YAML may be +/// accepted at authoring time and converted on save: one canonical form, many +/// front doors. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct Grant { + /// Why this grant exists, in prose. Carried INTO the canonical form and + /// therefore into the hash: the reason a permission was given is part of + /// the permission, and a grant whose justification can be edited without + /// breaking its signature is missing the half an auditor asks about. + #[serde(default, rename = "_why", skip_serializing_if = "Option::is_none")] + pub why: Option, + /// Who holds it. + pub subject: String, + /// The principal this subject acts for, if any. An agent is never a + /// principal in its own right; see [`PolicyEngine::evaluate_delegated`]. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub acts_for: Option, + /// What it is over. + pub scope: String, + /// What the permitted action may destroy. + pub tier: Tier, + /// Resource refs refused outright. Checked FIRST and never overridden by + /// a wider grant. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub excludes: Vec, + #[serde(default)] + pub limits: Limits, + /// Wall clock, unix seconds. Independent of whether work finished. + pub expires_at: i64, +} + +impl Grant { + /// The canonical byte form, per RFC 8785 (JSON Canonicalization Scheme): + /// object keys sorted, no insignificant whitespace, UTF-8. + /// + /// Keys are sorted EXPLICITLY through a `BTreeMap` rather than relying on + /// `serde_json`. This workspace pins `serde_json` with `preserve_order`, + /// so a `Map` keeps insertion order and serialising the struct directly + /// would produce field-declaration order -- stable by accident, and + /// silently wrong the day a field is reordered. `trusty-core::build_payload` + /// sorts explicitly for the same reason. + /// + /// Only integers, strings, booleans and arrays appear here, so JCS number + /// formatting is not exercised. A float field would need the full + /// ECMAScript `Number::toString` treatment before this stays canonical. + pub fn canonical_json(&self) -> Result { + use serde_json::{Map, Value}; + use std::collections::BTreeMap; + + let mut top: BTreeMap = BTreeMap::new(); + if let Some(why) = &self.why { + top.insert("_why".into(), Value::String(why.clone())); + } + if let Some(af) = &self.acts_for { + top.insert("acts_for".into(), Value::String(af.clone())); + } + top.insert( + "excludes".into(), + Value::Array( + self.excludes + .iter() + .map(|e| Value::String(e.clone())) + .collect(), + ), + ); + top.insert("expires_at".into(), Value::Number(self.expires_at.into())); + + let mut limits: BTreeMap = BTreeMap::new(); + if let Some(v) = self.limits.dead_man_secs { + limits.insert("dead_man_secs".into(), Value::Number(v.into())); + } + if let Some(v) = self.limits.irreversible_budget { + limits.insert("irreversible_budget".into(), Value::Number(v.into())); + } + if let Some(v) = self.limits.rate_per_min { + limits.insert("rate_per_min".into(), Value::Number(v.into())); + } + top.insert("limits".into(), sorted_object(limits)); + + top.insert("scope".into(), Value::String(self.scope.clone())); + top.insert("subject".into(), Value::String(self.subject.clone())); + top.insert( + "tier".into(), + Value::String( + serde_json::to_value(self.tier) + .map_err(|e| PolicyError::Schema(e.to_string()))? + .as_str() + .unwrap_or_default() + .to_string(), + ), + ); + + let mut out = Map::new(); + for (k, v) in top { + out.insert(k, v); + } + serde_json::to_string(&Value::Object(out)) + .map_err(|e| PolicyError::Schema(e.to_string())) + } +} + +fn sorted_object(map: std::collections::BTreeMap) -> serde_json::Value { + let mut out = serde_json::Map::new(); + for (k, v) in map { + out.insert(k, v); + } + serde_json::Value::Object(out) +} + +#[cfg(test)] +mod grant_tests { + use super::*; + + fn grant() -> Grant { + Grant { + why: Some("Sam's scheduling agent may propose, never book.".into()), + subject: "agent:01J".into(), + acts_for: Some("user:01H".into()), + scope: "board:9f2/calendar".into(), + tier: Tier::Append, + excludes: vec!["board:9f2/calendar/private".into()], + limits: Limits { + rate_per_min: Some(20), + irreversible_budget: Some(0), + dead_man_secs: Some(1800), + }, + expires_at: 1_756_684_800, + } + } + + #[test] + fn canonical_form_has_sorted_keys_and_no_whitespace() { + let j = grant().canonical_json().unwrap(); + assert!( + !j.contains('\n') && !j.contains(" "), + "must be compact: {j}" + ); + + // Top-level keys in ascending order, per RFC 8785. + let keys = [ + "_why", + "acts_for", + "excludes", + "expires_at", + "limits", + "scope", + "subject", + "tier", + ]; + let mut last = 0usize; + for k in keys { + let at = j + .find(&format!("\"{k}\":")) + .unwrap_or_else(|| panic!("missing {k} in {j}")); + assert!(at > last || last == 0, "key {k} out of order in {j}"); + last = at; + } + } + + #[test] + fn the_same_grant_always_produces_the_same_bytes() { + // The property a hash depends on. One document, one byte sequence. + assert_eq!( + grant().canonical_json().unwrap(), + grant().canonical_json().unwrap() + ); + } + + #[test] + fn field_order_in_the_struct_does_not_leak_into_the_bytes() { + // The trap this guards: serde_json is pinned with preserve_order here, + // so serialising the struct directly would emit DECLARATION order -- + // stable by accident, and silently different the day a field moves. + let j = grant().canonical_json().unwrap(); + assert!( + j.find("\"_why\"").unwrap() < j.find("\"subject\"").unwrap(), + "sorted, not declaration order: {j}" + ); + } + + #[test] + fn why_is_inside_the_hash_not_beside_it() { + // The reason a permission was given is part of the permission. A + // justification editable without breaking the signature is missing the + // half an auditor asks about. + let mut other = grant(); + other.why = Some("a different and less honest reason".into()); + assert_ne!( + grant().canonical_json().unwrap(), + other.canonical_json().unwrap() + ); + } + + #[test] + fn every_field_changes_the_bytes() { + let base = grant().canonical_json().unwrap(); + + let mut t = grant(); + t.tier = Tier::Delete; + assert_ne!(base, t.canonical_json().unwrap(), "tier"); + + let mut e = grant(); + e.expires_at += 1; + assert_ne!(base, e.canonical_json().unwrap(), "expires_at"); + + let mut x = grant(); + x.excludes.push("board:9f2/calendar/secret".into()); + assert_ne!(base, x.canonical_json().unwrap(), "excludes"); + + let mut l = grant(); + l.limits.rate_per_min = Some(21); + assert_ne!(base, l.canonical_json().unwrap(), "limits"); + + let mut a = grant(); + a.acts_for = Some("user:someone-else".into()); + assert_ne!(base, a.canonical_json().unwrap(), "acts_for"); + } + + #[test] + fn tier_serialises_lowercase_as_the_spec_writes_it() { + assert!(grant() + .canonical_json() + .unwrap() + .contains("\"tier\":\"append\"")); + } + + #[test] + fn a_grant_round_trips_through_serde() { + let j = serde_json::to_string(&grant()).unwrap(); + let back: Grant = serde_json::from_str(&j).unwrap(); + assert_eq!(back, grant()); + assert_eq!( + back.canonical_json().unwrap(), + grant().canonical_json().unwrap() + ); + } +} + +#[cfg(test)] +mod spec_canonical_tests { + use super::*; + + fn spec() -> FsmSpec { + FsmSpec { + scope: "doc.review.v1".into(), + initial: "draft".into(), + transitions: vec![ + TransitionSpec { + from: "draft".into(), + action: "submit".into(), + to: "review".into(), + guard: None, + tier: Some(Tier::Append), + }, + TransitionSpec { + from: "review".into(), + action: "approve".into(), + to: "published".into(), + guard: Some("big_enough".into()), + tier: None, + }, + ], + } + } + + #[test] + fn the_canonical_form_is_stable_across_runs() { + assert_eq!( + spec().canonical_json().expect("canonical"), + spec().canonical_json().expect("canonical"), + "a rule set that hashes differently each time cannot be pinned, \ + and pinning is the entire reason this exists" + ); + } + + #[test] + fn keys_are_sorted_and_do_not_follow_field_declaration_order() { + let json = spec().canonical_json().expect("canonical"); + let initial = json.find("\"initial\"").expect("initial"); + let scope = json.find("\"scope\"").expect("scope"); + let transitions = json.find("\"transitions\"").expect("transitions"); + assert!( + initial < scope && scope < transitions, + "this workspace pins serde_json with preserve_order, so serialising \ + the struct directly would give declaration order -- stable by \ + accident and silently wrong the day a field moves: {json}" + ); + let first = json.find("\"action\"").expect("action"); + let from = json.find("\"from\"").expect("from"); + assert!( + first < from, + "transition keys must be sorted too, or a nested object reorders \ + under the same hash: {json}" + ); + } + + #[test] + fn transition_order_is_preserved_rather_than_sorted() { + let mut reordered = spec(); + reordered.transitions.reverse(); + assert_ne!( + spec().canonical_json().expect("a"), + reordered.canonical_json().expect("b"), + "a rule set is a document written in an order. Sorting transitions \ + would make two different documents hash alike and lose the reading \ + order for nothing, since array position is already unambiguous" + ); + } + + #[test] + fn changing_a_guard_name_changes_the_canonical_form() { + let mut tampered = spec(); + tampered.transitions[1].guard = Some("always_true".into()); + assert_ne!( + spec().canonical_json().expect("a"), + tampered.canonical_json().expect("b"), + "swapping the guard is the most valuable edit an attacker could \ + make -- it is how a rule stops being enforced while still looking \ + enforced -- so it must move the hash" + ); + } + + #[test] + fn removing_a_transition_changes_the_canonical_form() { + let mut narrowed = spec(); + narrowed.transitions.pop(); + assert_ne!( + spec().canonical_json().expect("a"), + narrowed.canonical_json().expect("b"), + "loosening or tightening the rule set must be visible; otherwise a \ + violation marker citing this rule says nothing about which rule" + ); + } + + #[test] + fn an_absent_guard_is_absent_rather_than_null() { + let json = spec().canonical_json().expect("canonical"); + assert!( + !json.contains("null"), + "an omitted field and a field explicitly set to null are different \ + byte sequences for the same meaning, which is exactly what a \ + canonical form must not allow: {json}" + ); + } + + #[test] + fn a_spec_round_trips_through_its_canonical_form() { + let json = spec().canonical_json().expect("canonical"); + let back = FsmSpec::from_json(&json).expect("parse"); + assert_eq!( + back.canonical_json().expect("re-canonical"), + json, + "reading a canonical document and writing it again must reproduce \ + it, or the form is not canonical" + ); + } + + #[test] + fn the_spec_and_the_grant_canonicalise_the_same_way() { + // Both sort keys explicitly through a BTreeMap for the same + // reason. If one is ever changed to trust serde ordering, this + // is the test that should be looked at. + let g = Grant { + why: None, + subject: "agent:1".into(), + acts_for: None, + scope: "s".into(), + tier: Tier::Read, + excludes: vec![], + limits: Limits::default(), + expires_at: 1, + }; + let gj = g.canonical_json().expect("grant"); + assert!( + gj.find("\"excludes\"").unwrap() < gj.find("\"expires_at\"").unwrap(), + "grant keys sorted: {gj}" + ); + let sj = spec().canonical_json().expect("spec"); + assert!(sj.starts_with("{\"initial\""), "spec keys sorted: {sj}"); + } +} diff --git a/crates/core/trusty-store/Cargo.toml b/crates/core/trusty-store/Cargo.toml new file mode 100644 index 0000000..389ecec --- /dev/null +++ b/crates/core/trusty-store/Cargo.toml @@ -0,0 +1,20 @@ +[package] +name = "trusty-store" +version.workspace = true +edition.workspace = true +authors.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +description = "Storage trait for Trusty backends. Defines the contract any backend (sqlite, memory, postgres, etc.) must satisfy." + +[features] +default = [] +testing = [] + +[dependencies] +trusty-core = { workspace = true } +serde = { workspace = true } +serde_json = { workspace = true } +thiserror = { workspace = true } + diff --git a/crates/core/trusty-store/src/lib.rs b/crates/core/trusty-store/src/lib.rs new file mode 100644 index 0000000..7a6727f --- /dev/null +++ b/crates/core/trusty-store/src/lib.rs @@ -0,0 +1,821 @@ +//! Backend-agnostic `Store` trait for Trusty. +//! +//! Defines the contract any storage implementation must satisfy. Core logic in +//! `trusty-core` is unchanged regardless of which backend is in use -- callers +//! can swap SQLite for an in-memory store (or a future Postgres backend) without +//! touching hashing, canonicalization, or verification. +//! +//! Implementations: [`trusty-memory`] (ref impl / tests), [`trusty-sqlite`] (production). + +#![deny(clippy::all)] + +use serde::{Deserialize, Serialize}; +use thiserror::Error; + +#[cfg(feature = "testing")] +pub mod testing; + +// ------------------------------------------------------------ +// Trait +// ------------------------------------------------------------ + +pub trait Store: Send { + fn last_row(&self, entity_id: &str) -> StoreResult>; + + fn insert_row(&mut self, row: &RowInsert) -> StoreResult; + + fn iter_chain<'a>( + &'a self, + entity_id: &str, + ) -> StoreResult> + 'a>>; + + fn get_tip(&self, chain_id: &str) -> StoreResult>; + + fn get_tip_for_entity(&self, entity_id: &str) -> StoreResult>; + + fn set_tip(&mut self, tip: &TipUpsert) -> StoreResult<()>; + + fn kv_cross_check(&self) -> StoreResult; + + // Transactional scope. Closure returns `StoreResult<()>`; on Err + // the backend rolls back. Return-value capture uses outer state. + // We use `&mut dyn FnMut` instead of a generic closure because + // trait-object methods cannot be generic. + fn with_txn(&mut self, f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>) + -> StoreResult<()>; +} + +// ------------------------------------------------------------ +// Types +// ------------------------------------------------------------ + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct LastRow { + pub row_hash: String, + pub sequence: i64, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct TipRow { + pub entity_id: String, + pub chain_id: String, + pub current_tip_hash: Option, + pub updated_at: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct StoredRow { + pub id: i64, + pub entity_id: String, + pub chain_id: String, + pub event_type: String, + pub sequence: i64, + pub prev_hash: String, + pub row_hash: Option, + pub anchor_hash: Option, + pub raw_payload: Option, + pub detected_at: String, + pub received_at: String, + pub source_url: String, + pub extraction_method: String, +} + +#[derive(Debug, Clone)] +pub struct RowInsert { + pub entity_id: String, + pub chain_id: String, + pub event_type: String, + pub sequence: i64, + pub prev_hash: String, + pub row_hash: Option, + pub anchor_hash: Option, + pub raw_payload: Option, + pub detected_at: String, + pub received_at: String, + pub source_url: String, + pub extraction_method: String, +} + +#[derive(Debug, Clone)] +pub struct TipUpsert { + pub entity_id: String, + pub chain_id: String, + pub current_tip_hash: Option, +} + +#[derive(Debug, Clone, Serialize)] +pub struct KvCrossCheckReport { + pub ok: bool, + pub kv1_total_hashes: usize, + pub kv2_total_entities: usize, + pub entities_checked: usize, + pub mismatches: Vec, +} + +// ------------------------------------------------------------ +// Higher-level result types +// +// Promoted from `trusty-sqlite` (S16, resolves §12 OQ #16 / +// `s14_insight_general_library_improvements_from_postgres_impl` +// items 1-2). Backend-agnostic shape; every adapter constructs and +// returns these. trusty-sqlite and trusty-postgres re-export them +// for convenience. +// ------------------------------------------------------------ + +/// Result of a single `append` (or `append_signed`). Encodes the +/// chain-break-as-event primitive: when a break is detected, +/// `chain_break = true`, `break_reason` is set, and `sealed_chain` +/// names the chain the break sealed. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AppendResult { + pub entity_id: String, + pub chain_id: String, + pub event_type: String, + pub chain_break: bool, + pub break_reason: Option, + pub sealed_chain: Option, + pub row_hash: String, + pub anchor_hash: String, + pub sequence: i64, + pub is_genesis: bool, +} + +/// Operator-facing view of one BREAK envelope row. Surface for +/// `breaks_since` -- treats chain-break-as-event as a query result, +/// not an exception. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +pub struct BreakEvent { + pub entity_id: String, + pub chain_id: String, + pub sealed_at_hash: String, + pub break_reason: String, + pub detected_at: String, +} + +/// An entity that has not appended a non-BREAK row within +/// `max_gap_secs`. Returned by `liveness_check`. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] +pub struct SilentEntity { + pub entity_id: String, + pub last_seen: String, + pub gap_secs: f64, +} + +/// One row of `history(entity_id, ...)` output. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct HistoryRow { + pub id: i64, + pub chain_id: String, + pub entity_id: String, + pub event_type: String, + pub sequence: i64, + pub detected_at: String, + pub source_url: String, + pub row_hash: Option, + pub anchor_hash: Option, + pub raw_payload: Option, +} + +/// `integrity()` cross-check report. Aggregates KV cross-check +/// outcomes and any envelope/index drifts into one record. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct IntegrityReport { + pub ok: bool, + pub entities_checked: usize, + pub kv1_total_hashes: usize, + pub kv2_total_entities: usize, + pub mismatches: Vec, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct IntegrityMismatch { + pub entity_id: String, + pub reason: String, + pub detail: Option, +} + +/// Borrowed input row for `Trusty::append_batch`. Cheap to construct +/// from caller-owned data; one allocation per call, not per row. +/// For a signed batch, use per-row `append_signed` in a loop until a +/// signed-batch surface lands. Promoted from trusty-sqlite in S16 +/// alongside the other result types. +#[derive(Debug, Clone, Copy)] +pub struct RowSpec<'a> { + pub entity_id: &'a str, + pub data: &'a serde_json::Value, + pub source_url: &'a str, +} + +/// Lightweight tip handle returned by `Trusty::anchor`. Captures the +/// most recent anchor commitment for an entity (chain_id + tip_hash) +/// plus a server-side `anchored_at` timestamp at query time. For the +/// full per-row anchor stream use `anchors_since`. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AnchorInfo { + pub entity_id: String, + pub chain_id: String, + pub tip_hash: Option, + pub anchored_at: String, +} + +/// One row in the `producer_identity` table. OID-anchored producer +/// identity per `s14_insight_oid_anchored_chain_identity`. The OID +/// is the trust spine: a chain's `attested_by` references this row, +/// and the registered `ed25519_pubkey` is the only key trusty trusts +/// for verifying that attestation. `siphash_key` is the producer's +/// per-row SipHash key (see hash.rs::SecretKey); `metadata` is +/// optional caller-supplied JSON. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct ProducerIdentity { + pub producer_oid: String, + /// Key material, encoded per `key_format`. + pub ed25519_pubkey: Vec, + pub siphash_key: Vec, + pub registered_at: String, + pub display_name: Option, + pub metadata: Option, + /// How `ed25519_pubkey` is encoded, and therefore how a signature + /// from it must be checked. Defaults to raw Ed25519 for a row + /// written before the column existed -- a schema default at write + /// time, never a parse fallback. + #[serde(default)] + pub key_format: trusty_core::KeyFormat, +} + +/// One row in the `producer_revocation` table. Forward-only +/// revocation per memo §6 / learn-LCG-ARCH-06: rows signed BEFORE +/// `revoked_at` remain verifiable; rows signed AT or AFTER are +/// rejected by `verify_since_checkpoint`. `revocation_row_hash` +/// references the REVOCATION envelope written to the per-user +/// `revocations` chain at revoke time -- the act of revoking is +/// itself a chained event. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct RevocationRecord { + pub producer_oid: String, + pub revoked_at: String, + pub revoked_pubkey: Vec, + pub reason: String, + pub attested_by_pubkey: Vec, + pub revocation_row_hash: String, +} + +/// One row in the `chain_checkpoint` table. A checkpoint is a +/// verification-layer construct (memo §5, Gitea #13): a signed +/// `(chain_id, sequence_int, row_hash)` tuple that lets verifiers +/// prune their verify window to "from the most recent valid +/// checkpoint forward". `attested_by` names the attestor (typically +/// an OID from `producer_identity`); `attestation_sig` is Ed25519 +/// over `trusty_core::checkpoint_signing_bytes(...)`. `checkpoint_at` +/// is the server-side timestamp at write time. +/// +/// Chain payload bytes are NEVER touched by this primitive -- v1 +/// byte-shape lock preserved. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct CheckpointRow { + pub id: i64, + pub chain_id: String, + pub sequence_int: i64, + pub row_hash: String, + pub checkpoint_at: String, + pub attested_by: String, + pub attestation_sig: Vec, +} + +// ------------------------------------------------------------ +// Error surface +// ------------------------------------------------------------ + +#[derive(Debug, Error)] +pub enum StoreError { + #[error("not found: {0}")] + NotFound(String), + + #[error("io error: {0}")] + Io(String), + + #[error("integrity violation: {0}")] + IntegrityViolation(String), + + #[error("serialization error: {0}")] + Serialization(String), + + #[error("backend error: {0}")] + Backend(String), +} + +pub type StoreResult = Result; + +// ------------------------------------------------------------ +// TrustyExt -- Tier 1 Phase B (S20 cycle 1) +// +// Default-method extension trait for any `T: Store`. Promotes +// higher-level read paths (history, counts, latest hash) into a +// substrate-shared implementation so trusty-sqlite, trusty-postgres, +// trusty-memory, and any future backend gain them automatically by +// implementing only the Store trait. +// +// Cycle-1 scope is read-only and additive: existing backend Trusty +// types keep their direct implementations untouched. A subsequent +// cycle migrates them to delegate to TrustyExt and removes the +// duplication. Methods that depend on break-specific fields not yet +// in StoredRow (breaks_since with sealed_at_hash + break_reason, +// integrity with KV cross-check) follow in S21+ once the StoredRow +// shape is extended OR break-specific Store methods are added. +// ------------------------------------------------------------ + +/// Default-method extension trait for any `T: Store`. Implemented +/// blanket-style; no consumer action required beyond using the +/// methods. +pub trait TrustyExt: Store { + /// History view of an entity's chain: oldest-first, BREAK rows + /// excluded, rows without a `row_hash` excluded. `limit` capped + /// at `i64::MAX` when negative. + /// + /// Mirrors the existing per-backend `history(entity_id, limit)` + /// surface, sourced from `Store::iter_chain` so the result shape + /// is uniform across backends. + fn history(&self, entity_id: &str, limit: i64) -> StoreResult> { + let cap = if limit < 0 { + i64::MAX as usize + } else { + limit as usize + }; + let mut out: Vec = Vec::new(); + for row_res in self.iter_chain(entity_id)? { + let row = row_res?; + if row.event_type == "BREAK" || row.row_hash.is_none() { + continue; + } + out.push(stored_to_history(row)); + if out.len() >= cap { + break; + } + } + Ok(out) + } + + /// Count of rows for an entity (BREAK rows included). Cheaper + /// than collecting `history` when the caller only wants the + /// length. + fn count_rows(&self, entity_id: &str) -> StoreResult { + let mut n: usize = 0; + for row_res in self.iter_chain(entity_id)? { + let _ = row_res?; + n += 1; + } + Ok(n) + } + + /// Latest non-BREAK row hash for an entity. Wraps + /// [`Store::last_row`] for the common single-field-of-interest + /// case. + fn latest_row_hash(&self, entity_id: &str) -> StoreResult> { + Ok(self.last_row(entity_id)?.map(|r| r.row_hash)) + } + + /// Latest non-BREAK sequence number for an entity. Useful for + /// "how far has this chain advanced" reporting without loading + /// full history. + fn latest_sequence(&self, entity_id: &str) -> StoreResult> { + Ok(self.last_row(entity_id)?.map(|r| r.sequence)) + } + + /// Count of BREAK rows for an entity. Health-check primitive: + /// nonzero indicates the chain has been sealed at least once. + fn count_breaks(&self, entity_id: &str) -> StoreResult { + let mut n: usize = 0; + for row_res in self.iter_chain(entity_id)? { + let row = row_res?; + if row.event_type == "BREAK" { + n += 1; + } + } + Ok(n) + } + + /// Whether the entity has at least one row of any kind. Cheaper + /// than counting; short-circuits on first hit. + fn entity_seen(&self, entity_id: &str) -> StoreResult { + // `.next()` rather than a `for` that returns on its first pass: the loop + // form read as an iteration and was not one, which `#![deny(clippy::all)]` + // at the top of this file rejects as `never_loop`. Behaviour is + // unchanged -- first row wins, and its error still propagates. + match self.iter_chain(entity_id)?.next() { + Some(row_res) => { + row_res?; + Ok(true) + } + None => Ok(false), + } + } + + /// Current chain id for an entity, if a tip exists. Wraps + /// [`Store::get_tip_for_entity`] for the common id-only case. + fn chain_id_for(&self, entity_id: &str) -> StoreResult> { + Ok(self.get_tip_for_entity(entity_id)?.map(|t| t.chain_id)) + } + + /// Current tip hash for an entity, if a tip exists and has been + /// hashed. Wraps [`Store::get_tip_for_entity`]. + fn tip_hash(&self, entity_id: &str) -> StoreResult> { + Ok(self + .get_tip_for_entity(entity_id)? + .and_then(|t| t.current_tip_hash)) + } + + /// All BREAK rows for an entity, oldest-first, as raw + /// `StoredRow`s. Consumers parse `raw_payload` for break-specific + /// fields (sealed-at hash, reason, etc.) since those live in the + /// JSON envelope rather than in dedicated columns of every + /// backend. + /// + /// Backend-portable replacement for the per-backend + /// `breaks_since(...)` surface; richer typed views (with + /// `BreakEvent`) stay backend-side until break-specific column + /// access lands on the `Store` trait. + fn iter_breaks(&self, entity_id: &str) -> StoreResult> { + let mut out: Vec = Vec::new(); + for row_res in self.iter_chain(entity_id)? { + let row = row_res?; + if row.event_type == "BREAK" { + out.push(row); + } + } + Ok(out) + } + + /// Most recent BREAK row for an entity, if any. Cheaper than + /// loading the full break list when only the latest seal is of + /// interest (e.g. health checks). + fn latest_break(&self, entity_id: &str) -> StoreResult> { + Ok(self.iter_breaks(entity_id)?.pop()) + } + + /// Count of rows for an entity matching a given `event_type`. + /// Useful for "how many APPEND vs BREAK vs CHECKPOINT" reporting + /// without parsing JSON. `event_type` is case-sensitive, + /// matching the canonical envelope vocabulary. + fn count_events(&self, entity_id: &str, event_type: &str) -> StoreResult { + let mut n: usize = 0; + for row_res in self.iter_chain(entity_id)? { + let row = row_res?; + if row.event_type == event_type { + n += 1; + } + } + Ok(n) + } + + /// All rows for an entity matching a given `event_type`, + /// oldest-first. Generic filter -- callers asking for "EVENT" get + /// the non-BREAK chain; "BREAK" mirrors [`Self::iter_breaks`]; + /// future event types (CHECKPOINT, REVOCATION, etc.) work the + /// same way. + fn iter_by_event_type(&self, entity_id: &str, event_type: &str) -> StoreResult> { + let mut out: Vec = Vec::new(); + for row_res in self.iter_chain(entity_id)? { + let row = row_res?; + if row.event_type == event_type { + out.push(row); + } + } + Ok(out) + } +} + +impl TrustyExt for T {} + +fn stored_to_history(row: StoredRow) -> HistoryRow { + HistoryRow { + id: row.id, + chain_id: row.chain_id, + entity_id: row.entity_id, + event_type: row.event_type, + sequence: row.sequence, + detected_at: row.detected_at, + source_url: row.source_url, + row_hash: row.row_hash, + anchor_hash: row.anchor_hash, + raw_payload: row.raw_payload, + } +} + +#[cfg(test)] +mod trusty_ext_tests { + use super::*; + use std::collections::{BTreeMap, HashMap}; + + /// Minimal in-test Store impl. Mirrors trusty-memory but lives + /// inside trusty-store to avoid a dev-dep cycle (trusty-memory + /// depends on trusty-store). + #[derive(Clone, Default)] + struct TestStore { + rows: BTreeMap, + next_id: i64, + tips_by_chain: HashMap, + tips_by_entity: HashMap, + } + + impl Store for TestStore { + fn last_row(&self, entity_id: &str) -> StoreResult> { + Ok(self + .rows + .values() + .rev() + .find(|r| { + r.entity_id == entity_id && r.event_type != "BREAK" && r.row_hash.is_some() + }) + .map(|r| LastRow { + row_hash: r.row_hash.clone().unwrap(), + sequence: r.sequence, + })) + } + fn insert_row(&mut self, row: &RowInsert) -> StoreResult { + self.next_id += 1; + let id = self.next_id; + self.rows.insert( + id, + StoredRow { + id, + entity_id: row.entity_id.clone(), + chain_id: row.chain_id.clone(), + event_type: row.event_type.clone(), + sequence: row.sequence, + prev_hash: row.prev_hash.clone(), + row_hash: row.row_hash.clone(), + anchor_hash: row.anchor_hash.clone(), + raw_payload: row.raw_payload.clone(), + detected_at: row.detected_at.clone(), + received_at: row.received_at.clone(), + source_url: row.source_url.clone(), + extraction_method: row.extraction_method.clone(), + }, + ); + Ok(id) + } + fn iter_chain<'a>( + &'a self, + entity_id: &str, + ) -> StoreResult> + 'a>> { + let entity = entity_id.to_string(); + Ok(Box::new( + self.rows + .values() + .filter(move |r| r.entity_id == entity) + .cloned() + .map(Ok), + )) + } + fn get_tip(&self, chain_id: &str) -> StoreResult> { + Ok(self.tips_by_chain.get(chain_id).cloned()) + } + fn get_tip_for_entity(&self, entity_id: &str) -> StoreResult> { + Ok(self.tips_by_entity.get(entity_id).cloned()) + } + fn set_tip(&mut self, tip: &TipUpsert) -> StoreResult<()> { + let row = TipRow { + entity_id: tip.entity_id.clone(), + chain_id: tip.chain_id.clone(), + current_tip_hash: tip.current_tip_hash.clone(), + updated_at: String::new(), + }; + self.tips_by_chain.insert(tip.chain_id.clone(), row.clone()); + self.tips_by_entity.insert(tip.entity_id.clone(), row); + Ok(()) + } + fn kv_cross_check(&self) -> StoreResult { + Ok(KvCrossCheckReport { + ok: true, + kv1_total_hashes: self.rows.values().filter(|r| r.row_hash.is_some()).count(), + kv2_total_entities: self.tips_by_entity.len(), + entities_checked: self.tips_by_entity.len(), + mismatches: vec![], + }) + } + fn with_txn( + &mut self, + f: &mut dyn FnMut(&mut dyn Store) -> StoreResult<()>, + ) -> StoreResult<()> { + let snap = self.clone(); + match f(self) { + Ok(()) => Ok(()), + Err(e) => { + *self = snap; + Err(e) + } + } + } + } + + fn insert(store: &mut TestStore, entity: &str, seq: i64, event: &str, hash: Option<&str>) { + let row = RowInsert { + entity_id: entity.into(), + chain_id: format!("chain_{entity}_001"), + event_type: event.into(), + sequence: seq, + prev_hash: "genesis".into(), + row_hash: hash.map(str::to_string), + anchor_hash: hash.map(str::to_string), + raw_payload: Some(format!("{{\"seq\":{}}}", seq)), + detected_at: format!("2026-05-09T00:00:{:02}Z", seq), + received_at: format!("2026-05-09T00:00:{:02}.000Z", seq), + source_url: "test://".into(), + extraction_method: "trusty".into(), + }; + store.insert_row(&row).unwrap(); + store + .set_tip(&TipUpsert { + entity_id: entity.into(), + chain_id: row.chain_id.clone(), + current_tip_hash: hash.map(str::to_string), + }) + .unwrap(); + } + + #[test] + fn history_excludes_break_rows_and_unhashed_rows() { + let mut store = TestStore::default(); + insert(&mut store, "ent_a", 1, "EVENT", Some("h1")); + insert(&mut store, "ent_a", 2, "BREAK", Some("h2")); + insert(&mut store, "ent_a", 3, "EVENT", Some("h3")); + insert(&mut store, "ent_a", 4, "EVENT", None); // unhashed -- excluded + + let hist = store.history("ent_a", 100).unwrap(); + assert_eq!(hist.len(), 2, "BREAK + unhashed excluded: {:?}", hist); + assert_eq!(hist[0].sequence, 1); + assert_eq!(hist[1].sequence, 3); + assert!(hist.iter().all(|h| h.event_type != "BREAK")); + assert!(hist.iter().all(|h| h.row_hash.is_some())); + } + + #[test] + fn history_respects_positive_limit() { + let mut store = TestStore::default(); + for i in 1..=5 { + insert(&mut store, "ent_b", i, "EVENT", Some(&format!("h{i}"))); + } + let hist = store.history("ent_b", 3).unwrap(); + assert_eq!(hist.len(), 3); + assert_eq!(hist[0].sequence, 1); + assert_eq!(hist[2].sequence, 3); + } + + #[test] + fn history_treats_negative_limit_as_unbounded() { + let mut store = TestStore::default(); + for i in 1..=5 { + insert(&mut store, "ent_c", i, "EVENT", Some(&format!("h{i}"))); + } + let hist = store.history("ent_c", -1).unwrap(); + assert_eq!(hist.len(), 5); + } + + #[test] + fn count_rows_includes_break_rows() { + let mut store = TestStore::default(); + insert(&mut store, "ent_d", 1, "EVENT", Some("h1")); + insert(&mut store, "ent_d", 2, "BREAK", Some("h2")); + insert(&mut store, "ent_d", 3, "EVENT", Some("h3")); + let n = store.count_rows("ent_d").unwrap(); + assert_eq!(n, 3); + } + + #[test] + fn latest_row_hash_returns_last_non_break_hash() { + let mut store = TestStore::default(); + insert(&mut store, "ent_e", 1, "EVENT", Some("h1")); + insert(&mut store, "ent_e", 2, "EVENT", Some("h2")); + insert(&mut store, "ent_e", 3, "BREAK", Some("hbreak")); + let h = store.latest_row_hash("ent_e").unwrap(); + // last_row excludes BREAK + unhashed per Store::last_row contract, + // so we expect h2 (the latest non-BREAK). + assert_eq!(h.as_deref(), Some("h2")); + } + + #[test] + fn empty_entity_returns_empty_results() { + let store = TestStore::default(); + assert_eq!(store.history("nope", 10).unwrap().len(), 0); + assert_eq!(store.count_rows("nope").unwrap(), 0); + assert_eq!(store.latest_row_hash("nope").unwrap(), None); + } + + #[test] + fn latest_sequence_returns_last_non_break_seq() { + let mut store = TestStore::default(); + insert(&mut store, "ent_g", 1, "EVENT", Some("h1")); + insert(&mut store, "ent_g", 2, "EVENT", Some("h2")); + insert(&mut store, "ent_g", 3, "BREAK", Some("hb")); + assert_eq!(store.latest_sequence("ent_g").unwrap(), Some(2)); + assert_eq!(store.latest_sequence("nope").unwrap(), None); + } + + #[test] + fn count_breaks_counts_only_break_rows() { + let mut store = TestStore::default(); + insert(&mut store, "ent_h", 1, "EVENT", Some("h1")); + insert(&mut store, "ent_h", 2, "BREAK", Some("hb")); + insert(&mut store, "ent_h", 3, "EVENT", Some("h3")); + insert(&mut store, "ent_h", 4, "BREAK", Some("hb2")); + assert_eq!(store.count_breaks("ent_h").unwrap(), 2); + assert_eq!(store.count_breaks("nope").unwrap(), 0); + } + + #[test] + fn entity_seen_short_circuits() { + let mut store = TestStore::default(); + insert(&mut store, "ent_i", 1, "EVENT", Some("h1")); + assert!(store.entity_seen("ent_i").unwrap()); + assert!(!store.entity_seen("nope").unwrap()); + } + + #[test] + fn chain_id_for_and_tip_hash_track_current_tip() { + let mut store = TestStore::default(); + insert(&mut store, "ent_j", 1, "EVENT", Some("h1")); + insert(&mut store, "ent_j", 2, "EVENT", Some("h2")); + assert_eq!( + store.chain_id_for("ent_j").unwrap().as_deref(), + Some("chain_ent_j_001") + ); + assert_eq!(store.tip_hash("ent_j").unwrap().as_deref(), Some("h2")); + assert_eq!(store.chain_id_for("nope").unwrap(), None); + assert_eq!(store.tip_hash("nope").unwrap(), None); + } + + #[test] + fn iter_breaks_returns_only_break_rows_oldest_first() { + let mut store = TestStore::default(); + insert(&mut store, "ent_k", 1, "EVENT", Some("h1")); + insert(&mut store, "ent_k", 2, "BREAK", Some("hb1")); + insert(&mut store, "ent_k", 3, "EVENT", Some("h3")); + insert(&mut store, "ent_k", 4, "BREAK", Some("hb2")); + + let breaks = store.iter_breaks("ent_k").unwrap(); + assert_eq!(breaks.len(), 2); + assert_eq!(breaks[0].sequence, 2); + assert_eq!(breaks[1].sequence, 4); + assert!(breaks.iter().all(|r| r.event_type == "BREAK")); + } + + #[test] + fn latest_break_returns_most_recent() { + let mut store = TestStore::default(); + insert(&mut store, "ent_l", 1, "BREAK", Some("hb1")); + insert(&mut store, "ent_l", 2, "EVENT", Some("h2")); + insert(&mut store, "ent_l", 3, "BREAK", Some("hb2")); + let latest = store.latest_break("ent_l").unwrap().unwrap(); + assert_eq!(latest.sequence, 3); + assert_eq!(latest.row_hash.as_deref(), Some("hb2")); + + assert!(store.latest_break("nope").unwrap().is_none()); + } + + #[test] + fn count_events_matches_event_type_exactly() { + let mut store = TestStore::default(); + insert(&mut store, "ent_m", 1, "EVENT", Some("h1")); + insert(&mut store, "ent_m", 2, "BREAK", Some("hb")); + insert(&mut store, "ent_m", 3, "CHECKPOINT", Some("hc")); + insert(&mut store, "ent_m", 4, "EVENT", Some("h2")); + + assert_eq!(store.count_events("ent_m", "EVENT").unwrap(), 2); + assert_eq!(store.count_events("ent_m", "BREAK").unwrap(), 1); + assert_eq!(store.count_events("ent_m", "CHECKPOINT").unwrap(), 1); + assert_eq!(store.count_events("ent_m", "UNKNOWN").unwrap(), 0); + } + + #[test] + fn iter_by_event_type_returns_filtered_rows_in_order() { + let mut store = TestStore::default(); + insert(&mut store, "ent_n", 1, "EVENT", Some("h1")); + insert(&mut store, "ent_n", 2, "CHECKPOINT", Some("hc1")); + insert(&mut store, "ent_n", 3, "EVENT", Some("h2")); + insert(&mut store, "ent_n", 4, "CHECKPOINT", Some("hc2")); + + let cps = store.iter_by_event_type("ent_n", "CHECKPOINT").unwrap(); + assert_eq!(cps.len(), 2); + assert_eq!(cps[0].sequence, 2); + assert_eq!(cps[1].sequence, 4); + + let evs = store.iter_by_event_type("ent_n", "EVENT").unwrap(); + assert_eq!(evs.len(), 2); + assert!(evs.iter().all(|r| r.event_type == "EVENT")); + } + + #[test] + fn trusty_ext_is_object_safe_via_dyn_store() { + // The whole point of TrustyExt is that it's blanket-impl'd + // for `T: Store + ?Sized`, including `dyn Store`. Verify by + // calling through a trait object. + let mut store = TestStore::default(); + insert(&mut store, "ent_f", 1, "EVENT", Some("h1")); + let dyn_store: &dyn Store = &store; + assert_eq!(dyn_store.count_rows("ent_f").unwrap(), 1); + assert_eq!( + dyn_store.latest_row_hash("ent_f").unwrap().as_deref(), + Some("h1") + ); + } +} + diff --git a/crates/core/trusty-store/src/testing.rs b/crates/core/trusty-store/src/testing.rs new file mode 100644 index 0000000..efa1b26 --- /dev/null +++ b/crates/core/trusty-store/src/testing.rs @@ -0,0 +1,128 @@ +// Shared conformance harness for Store implementations. +// +// Enabled via the `testing` feature. Each backend crate depends on +// trusty-store with features = ["testing"] in its dev-dependencies +// and calls `run_conformance` from its own integration test. +// +// Adding a new Store backend? Pass its fresh, empty instance here; +// if this function returns without panicking, the backend satisfies +// the basic contract Trusty needs. + +use crate::{RowInsert, Store, StoreError, StoreResult, TipUpsert}; + +pub fn run_conformance(store: &mut S) { + insert_and_read_back(store); + iter_chain_ordered(store); + tip_roundtrip(store); + kv_cross_check_clean(store); + with_txn_commit(store); + with_txn_rollback(store); +} + +fn mk_row(entity: &str, chain: &str, seq: i64, row_hash: &str) -> RowInsert { + RowInsert { + entity_id: entity.to_string(), + chain_id: chain.to_string(), + event_type: if seq == 0 { + "GENESIS".into() + } else { + "MUTATION".into() + }, + sequence: seq, + prev_hash: if seq == 0 { + "genesis".to_string() + } else { + format!("prev_{}", seq - 1) + }, + row_hash: Some(row_hash.to_string()), + anchor_hash: Some("a".repeat(64)), + raw_payload: Some(format!("{{\"v\":{}}}", seq)), + detected_at: "2026-04-15T00:00:00Z".to_string(), + received_at: "2026-04-15T00:00:00.001000Z".to_string(), + source_url: "s://test".to_string(), + extraction_method: "trusty".to_string(), + } +} + +fn insert_and_read_back(store: &mut S) { + let row = mk_row("ent_conf_a", "chain_conf_a", 0, "000000000000000a"); + store.insert_row(&row).expect("insert ok"); + let last = store + .last_row("ent_conf_a") + .expect("last ok") + .expect("some"); + assert_eq!(last.row_hash, "000000000000000a"); + assert_eq!(last.sequence, 0); +} + +fn iter_chain_ordered(store: &mut S) { + for i in 0..3i64 { + let h = format!("{:016x}", 0xB00_u64 + i as u64); + let row = mk_row("ent_conf_b", "chain_conf_b", i, &h); + store.insert_row(&row).expect("insert ok"); + } + let rows: Vec<_> = store + .iter_chain("ent_conf_b") + .expect("iter ok") + .collect::, _>>() + .expect("collect ok"); + assert_eq!(rows.len(), 3, "expected 3 rows, got {:?}", rows.len()); + for (i, row) in rows.iter().enumerate() { + assert_eq!(row.sequence, i as i64, "out of order at index {}", i); + } +} + +fn tip_roundtrip(store: &mut S) { + let row = mk_row("ent_conf_c", "chain_conf_c", 0, "cafebabecafebabe"); + store.insert_row(&row).expect("insert for tip roundtrip"); + let tip = TipUpsert { + entity_id: "ent_conf_c".to_string(), + chain_id: "chain_conf_c".to_string(), + current_tip_hash: Some("cafebabecafebabe".to_string()), + }; + store.set_tip(&tip).expect("set tip"); + let got = store + .get_tip("chain_conf_c") + .expect("get_tip ok") + .expect("some"); + assert_eq!(got.current_tip_hash.as_deref(), Some("cafebabecafebabe")); + let got_ent = store + .get_tip_for_entity("ent_conf_c") + .expect("get_tip_for_entity ok") + .expect("some"); + assert_eq!(got_ent.chain_id, "chain_conf_c"); +} + +fn kv_cross_check_clean(store: &mut S) { + let report = store.kv_cross_check().expect("cross check ok"); + assert!(report.ok, "expected clean store ok: {:?}", report); +} + +fn with_txn_commit(store: &mut S) { + store + .with_txn(&mut |s: &mut dyn Store| { + let row = mk_row("ent_conf_d", "chain_conf_d", 0, "d000000000000001"); + s.insert_row(&row)?; + Ok(()) + }) + .expect("txn commit ok"); + let last = store + .last_row("ent_conf_d") + .expect("last ok") + .expect("committed row visible"); + assert_eq!(last.row_hash, "d000000000000001"); +} + +fn with_txn_rollback(store: &mut S) { + let result = store.with_txn(&mut |s: &mut dyn Store| { + let row = mk_row("ent_conf_e", "chain_conf_e", 0, "e000000000000001"); + s.insert_row(&row)?; + Err(StoreError::Backend("rollback test".into())) + }); + assert!(result.is_err(), "closure returning Err must propagate"); + let last: StoreResult> = store.last_row("ent_conf_e"); + assert!( + last.expect("last ok").is_none(), + "rolled-back row must not be visible after txn rollback" + ); +} diff --git a/crates/extensions/trusty-sqlite-ext/Cargo.lock b/crates/extensions/trusty-sqlite-ext/Cargo.lock new file mode 100644 index 0000000..28c2288 --- /dev/null +++ b/crates/extensions/trusty-sqlite-ext/Cargo.lock @@ -0,0 +1,827 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 3 + +[[package]] +name = "ahash" +version = "0.8.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" +dependencies = [ + "cfg-if", + "once_cell", + "version_check", + "zerocopy", +] + +[[package]] +name = "android_system_properties" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311" +dependencies = [ + "libc", +] + +[[package]] +name = "autocfg" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8" + +[[package]] +name = "base64ct" +version = "1.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" + +[[package]] +name = "bitflags" +version = "2.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3" + +[[package]] +name = "block-buffer" +version = "0.10.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" +dependencies = [ + "generic-array", +] + +[[package]] +name = "bumpalo" +version = "3.20.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d20789868f4b01b2f2caec9f5c4e0213b41e3e5702a50157d699ae31ced2fcb" + +[[package]] +name = "cc" +version = "1.2.60" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43c5703da9466b66a946814e1adf53ea2c90f10063b86290cc9eb67ce3478a20" +dependencies = [ + "find-msvc-tools", + "shlex", +] + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "chrono" +version = "0.4.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c673075a2e0e5f4a1dde27ce9dee1ea4558c7ffe648f576438a20ca1d2acc4b0" +dependencies = [ + "iana-time-zone", + "num-traits", + "serde", + "windows-link", +] + +[[package]] +name = "const-oid" +version = "0.9.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" + +[[package]] +name = "core-foundation-sys" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" + +[[package]] +name = "cpufeatures" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +dependencies = [ + "libc", +] + +[[package]] +name = "crypto-common" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" +dependencies = [ + "generic-array", + "typenum", +] + +[[package]] +name = "curve25519-dalek" +version = "4.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be" +dependencies = [ + "cfg-if", + "cpufeatures", + "curve25519-dalek-derive", + "digest", + "fiat-crypto", + "rustc_version", + "subtle", + "zeroize", +] + +[[package]] +name = "curve25519-dalek-derive" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "der" +version = "0.7.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" +dependencies = [ + "const-oid", + "zeroize", +] + +[[package]] +name = "digest" +version = "0.10.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" +dependencies = [ + "block-buffer", + "crypto-common", +] + +[[package]] +name = "ed25519" +version = "2.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53" +dependencies = [ + "pkcs8", + "signature", +] + +[[package]] +name = "ed25519-dalek" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9" +dependencies = [ + "curve25519-dalek", + "ed25519", + "rand_core", + "serde", + "sha2", + "subtle", + "zeroize", +] + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "fallible-iterator" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2acce4a10f12dc2fb14a218589d4f1f62ef011b2d0cc4b3cb1bba8e94da14649" + +[[package]] +name = "fallible-streaming-iterator" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7360491ce676a36bf9bb3c56c1aa791658183a54d2744120f27285738d90465a" + +[[package]] +name = "fiat-crypto" +version = "0.2.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d" + +[[package]] +name = "find-msvc-tools" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582" + +[[package]] +name = "generic-array" +version = "0.14.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +dependencies = [ + "typenum", + "version_check", +] + +[[package]] +name = "getrandom" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" +dependencies = [ + "cfg-if", + "libc", + "wasi", +] + +[[package]] +name = "hashbrown" +version = "0.14.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" +dependencies = [ + "ahash", +] + +[[package]] +name = "hashbrown" +version = "0.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4f467dd6dccf739c208452f8014c75c18bb8301b050ad1cfb27153803edb0f51" + +[[package]] +name = "hashlink" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ba4ff7128dee98c7dc9794b6a411377e1404dba1c97deb8d1a55297bd25d8af" +dependencies = [ + "hashbrown 0.14.5", +] + +[[package]] +name = "hex" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" + +[[package]] +name = "iana-time-zone" +version = "0.1.65" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470" +dependencies = [ + "android_system_properties", + "core-foundation-sys", + "iana-time-zone-haiku", + "js-sys", + "log", + "wasm-bindgen", + "windows-core", +] + +[[package]] +name = "iana-time-zone-haiku" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f" +dependencies = [ + "cc", +] + +[[package]] +name = "indexmap" +version = "2.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" +dependencies = [ + "equivalent", + "hashbrown 0.17.0", +] + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "js-sys" +version = "0.3.95" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2964e92d1d9dc3364cae4d718d93f227e3abb088e747d92e0395bfdedf1c12ca" +dependencies = [ + "once_cell", + "wasm-bindgen", +] + +[[package]] +name = "libc" +version = "0.2.185" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52ff2c0fe9bc6cb6b14a0592c2ff4fa9ceb83eea9db979b0487cd054946a2b8f" + +[[package]] +name = "libsqlite3-sys" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c10584274047cb335c23d3e61bcef8e323adae7c5c8c760540f73610177fc3f" +dependencies = [ + "cc", + "pkg-config", + "prettyplease", + "quote", + "syn", + "vcpkg", +] + +[[package]] +name = "log" +version = "0.4.29" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897" + +[[package]] +name = "memchr" +version = "2.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79" + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "pkcs8" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7" +dependencies = [ + "der", + "spki", +] + +[[package]] +name = "pkg-config" +version = "0.3.33" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e" + +[[package]] +name = "ppv-lite86" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" +dependencies = [ + "zerocopy", +] + +[[package]] +name = "prettyplease" +version = "0.2.37" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b" +dependencies = [ + "proc-macro2", + "syn", +] + +[[package]] +name = "proc-macro2" +version = "1.0.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "rand" +version = "0.8.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ca0ecfa931c29007047d1bc58e623ab12e5590e8c7cc53200d5202b69266d8a" +dependencies = [ + "libc", + "rand_chacha", + "rand_core", +] + +[[package]] +name = "rand_chacha" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +dependencies = [ + "ppv-lite86", + "rand_core", +] + +[[package]] +name = "rand_core" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" +dependencies = [ + "getrandom", +] + +[[package]] +name = "rusqlite" +version = "0.31.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b838eba278d213a8beaf485bd313fd580ca4505a00d5871caeb1457c55322cae" +dependencies = [ + "bitflags", + "fallible-iterator", + "fallible-streaming-iterator", + "hashlink", + "libsqlite3-sys", + "smallvec", +] + +[[package]] +name = "rustc_version" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92" +dependencies = [ + "semver", +] + +[[package]] +name = "rustversion" +version = "1.0.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d" + +[[package]] +name = "semver" +version = "1.0.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" + +[[package]] +name = "serde" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "serde_json" +version = "1.0.149" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86" +dependencies = [ + "indexmap", + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "sha2" +version = "0.10.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "shlex" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64" + +[[package]] +name = "signature" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de" +dependencies = [ + "rand_core", +] + +[[package]] +name = "siphasher" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2aa850e253778c88a04c3d7323b043aeda9d3e30d5971937c1855769763678e" + +[[package]] +name = "smallvec" +version = "1.15.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03" + +[[package]] +name = "spki" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d" +dependencies = [ + "base64ct", + "der", +] + +[[package]] +name = "subtle" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292" + +[[package]] +name = "syn" +version = "2.0.117" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "thiserror" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "trusty-core" +version = "0.1.0" +dependencies = [ + "ed25519-dalek", + "hex", + "rand", + "serde", + "serde_json", + "sha2", + "siphasher", + "thiserror", +] + +[[package]] +name = "trusty-sqlite" +version = "0.1.0" +dependencies = [ + "chrono", + "hex", + "rusqlite", + "serde", + "serde_json", + "thiserror", + "trusty-core", + "trusty-store", +] + +[[package]] +name = "trusty-sqlite-ext" +version = "0.1.0" +dependencies = [ + "rusqlite", + "serde_json", + "trusty-core", + "trusty-sqlite", +] + +[[package]] +name = "trusty-store" +version = "0.1.0" +dependencies = [ + "serde", + "serde_json", + "thiserror", + "trusty-core", +] + +[[package]] +name = "typenum" +version = "1.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "562d481066bde0658276a35467c4af00bdc6ee726305698a55b86e61d7ad82bb" + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "vcpkg" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426" + +[[package]] +name = "version_check" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "wasm-bindgen" +version = "0.2.118" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0bf938a0bacb0469e83c1e148908bd7d5a6010354cf4fb73279b7447422e3a89" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.118" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eeff24f84126c0ec2db7a449f0c2ec963c6a49efe0698c4242929da037ca28ed" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.118" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d08065faf983b2b80a79fd87d8254c409281cf7de75fc4b773019824196c904" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.118" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5fd04d9e306f1907bd13c6361b5c6bfc7b3b3c095ed3f8a9246390f8dbdee129" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "windows-core" +version = "0.62.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb" +dependencies = [ + "windows-implement", + "windows-interface", + "windows-link", + "windows-result", + "windows-strings", +] + +[[package]] +name = "windows-implement" +version = "0.60.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "windows-interface" +version = "0.59.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-result" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-strings" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091" +dependencies = [ + "windows-link", +] + +[[package]] +name = "zerocopy" +version = "0.8.48" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eed437bf9d6692032087e337407a86f04cd8d6a16a37199ed57949d415bd68e9" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.48" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70e3cd084b1788766f53af483dd21f93881ff30d7320490ec3ef7526d203bad4" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "zeroize" +version = "1.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0" + +[[package]] +name = "zmij" +version = "1.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa" diff --git a/crates/extensions/trusty-sqlite-ext/Cargo.toml b/crates/extensions/trusty-sqlite-ext/Cargo.toml new file mode 100644 index 0000000..58c088c --- /dev/null +++ b/crates/extensions/trusty-sqlite-ext/Cargo.toml @@ -0,0 +1,22 @@ +[package] +name = "trusty-sqlite-ext" +version = "0.1.0" +edition = "2021" +authors = ["MM Advisory LLC / Eric Gottlieb"] +license = "LicenseRef-Proprietary" +repository = "https://github.com/mmadvisory/trusty" +rust-version = "1.75" +description = "Trusty as a SQLite loadable extension. Drop-in tamper-evidence for any SQLite database." +publish = false + +[workspace] + +[lib] +name = "trusty_sqlite_ext" +crate-type = ["cdylib"] + +[dependencies] +trusty-core = { path = "../../core/trusty-core" } +trusty-sqlite = { path = "../../adapters/trusty-sqlite" } +rusqlite = { version = "0.31", features = ["loadable_extension", "functions", "trace"] } +serde_json = { version = "=1.0.149", features = ["preserve_order"] } diff --git a/crates/extensions/trusty-sqlite-ext/src/lib.rs b/crates/extensions/trusty-sqlite-ext/src/lib.rs new file mode 100644 index 0000000..7196900 --- /dev/null +++ b/crates/extensions/trusty-sqlite-ext/src/lib.rs @@ -0,0 +1,222 @@ +//! Trusty as a SQLite loadable extension (`cdylib`). +//! +//! Load into any SQLite database to get tamper-evidence without changing your +//! schema or language stack. Exposes SQL scalar functions backed by a shared, +//! lazily-initialized [`Trusty`](trusty_sqlite::Trusty) instance. +//! +//! # Loading +//! ```sql +//! .load ./trusty_sqlite_ext +//! SELECT trusty_init('/path/to/trusty.db', 'hex-encoded-secret-key'); +//! ``` +//! +//! # SQL functions +//! - `trusty_init(db_path, secret_key_hex)` -- initialize (call once) +//! - `trusty_append(entity_id, data_json, source_url)` -- append a record +//! - `trusty_verify(entity_id)` -- verify chain integrity, returns JSON report +//! - `trusty_anchor(entity_id)` -- write a SHA-256 anchor commitment +//! - `trusty_version()` -- returns crate version string +//! +//! # Safety invariants +//! Never panics across the FFI boundary. All fallible paths use rusqlite error +//! propagation. The only permitted unwraps are `Mutex::lock()` (poisoning is +//! terminal by design) and `serde_json::to_string` of always-serializable types. +// - Global Trusty state guarded by a Mutex; append is mutable. +// - Extension is non-persistent (re-registers per connection). +// ============================================================ + +use std::os::raw::{c_char, c_int}; +use std::sync::Mutex; + +use rusqlite::ffi; +use rusqlite::functions::FunctionFlags; +use rusqlite::{Connection, Error, Result}; + +use trusty_core::SecretKey; +use trusty_sqlite::Trusty; + +// ------------------------------------------------------------ +// Global extension state. +// One Trusty per process. Mutex because SQLite function callbacks +// are invoked in arbitrary threading contexts and append mutates. +// ------------------------------------------------------------ +static STATE: Mutex> = Mutex::new(None); + +// ------------------------------------------------------------ +// Extension entry point. +// +// SQLite's auto-loader for a library named `trusty_sqlite_ext` +// looks for `sqlite3_trustysqliteext_init` (underscores stripped, +// lowercased). We hand-write the entry point rather than rely on +// a macro that varies across rusqlite versions. +// ------------------------------------------------------------ + +/// # Safety +/// Called by sqlite3 at load time with valid, non-null `db` and +/// `p_api` pointers. `_pz_err_msg` is unused — we return +/// SQLITE_ERROR on failure and let sqlite3 surface a generic msg. +#[no_mangle] +pub unsafe extern "C" fn sqlite3_trustysqliteext_init( + db: *mut ffi::sqlite3, + _pz_err_msg: *mut *mut c_char, + p_api: *mut ffi::sqlite3_api_routines, +) -> c_int { + extension_entry(db, p_api) +} + +/// Alias entry point so the extension also loads under the short +/// name `trusty_ext` (e.g. if a user renames the artifact). SQLite +/// would derive `sqlite3_trustyext_init` from `trusty_ext`. +/// +/// # Safety +/// Same contract as [`sqlite3_trustysqliteext_init`]. +#[no_mangle] +pub unsafe extern "C" fn sqlite3_trustyext_init( + db: *mut ffi::sqlite3, + _pz_err_msg: *mut *mut c_char, + p_api: *mut ffi::sqlite3_api_routines, +) -> c_int { + extension_entry(db, p_api) +} + +/// Shared entry-point body. rusqlite 0.31's `extension_init2` +/// installs the SQLITE_EXTENSION_INIT2 api-routines table and +/// returns a non-owning `Connection`. We then register our +/// functions on it. The connection is `forget`-ed because SQLite +/// owns the underlying handle — dropping would double-close. +/// +/// # Safety +/// `db` and `p_api` must be the valid pointers passed by SQLite at +/// extension-load time. +unsafe fn extension_entry( + db: *mut ffi::sqlite3, + p_api: *mut ffi::sqlite3_api_routines, +) -> c_int { + match Connection::extension_init2(db, p_api) { + Ok(conn) => match register_functions(&conn) { + Ok(()) => { + std::mem::forget(conn); + ffi::SQLITE_OK + } + Err(_) => ffi::SQLITE_ERROR, + }, + Err(_) => ffi::SQLITE_ERROR, + } +} + +// ------------------------------------------------------------ +// Function registration. +// ------------------------------------------------------------ +fn register_functions(db: &Connection) -> Result<()> { + // trusty_init(db_path TEXT, secret_key_hex TEXT) -> TEXT + db.create_scalar_function( + "trusty_init", + 2, + FunctionFlags::SQLITE_UTF8, + |ctx| { + let path: String = ctx.get(0)?; + let key_hex: String = ctx.get(1)?; + let key_bytes = hex_decode_16(&key_hex).map_err(sqlite_err)?; + let key = SecretKey::new(key_bytes); + let trusty = Trusty::open(&path, key).map_err(sqlite_err)?; + *STATE.lock().unwrap() = Some(trusty); + Ok("ok".to_string()) + }, + )?; + + // trusty_append(entity_id TEXT, data_json TEXT, source_url TEXT) -> TEXT + // NOT deterministic — each call mutates chain state. + db.create_scalar_function( + "trusty_append", + 3, + FunctionFlags::SQLITE_UTF8, + |ctx| { + let entity_id: String = ctx.get(0)?; + let data_json: String = ctx.get(1)?; + let source_url: String = ctx.get(2)?; + let data: serde_json::Value = + serde_json::from_str(&data_json).map_err(sqlite_err)?; + let mut guard = STATE.lock().unwrap(); + let trusty = guard.as_mut().ok_or_else(not_initialized)?; + let result = trusty + .append(&entity_id, &data, &source_url) + .map_err(sqlite_err)?; + // AppendResult is defined by trusty-sqlite and is always + // serializable; unwrap is documented-safe. + Ok(serde_json::to_string(&result).unwrap()) + }, + )?; + + // trusty_verify(entity_id TEXT) -> TEXT + // Deterministic for a given chain state (read-only). + db.create_scalar_function( + "trusty_verify", + 1, + FunctionFlags::SQLITE_UTF8, + |ctx| { + let entity_id: String = ctx.get(0)?; + let guard = STATE.lock().unwrap(); + let trusty = guard.as_ref().ok_or_else(not_initialized)?; + let report = trusty.verify(&entity_id).map_err(sqlite_err)?; + Ok(serde_json::to_string(&report).unwrap()) + }, + )?; + + // trusty_anchor(entity_id TEXT) -> TEXT + // Returns JSON of AnchorInfo, or the literal string "null" if + // no chain exists for entity_id. + db.create_scalar_function( + "trusty_anchor", + 1, + FunctionFlags::SQLITE_UTF8, + |ctx| { + let entity_id: String = ctx.get(0)?; + let guard = STATE.lock().unwrap(); + let trusty = guard.as_ref().ok_or_else(not_initialized)?; + let info = trusty.anchor(&entity_id).map_err(sqlite_err)?; + Ok(serde_json::to_string(&info).unwrap()) + }, + )?; + + // trusty_version() -> TEXT + db.create_scalar_function( + "trusty_version", + 0, + FunctionFlags::SQLITE_UTF8 | FunctionFlags::SQLITE_DETERMINISTIC, + |_ctx| Ok(env!("CARGO_PKG_VERSION").to_string()), + )?; + + Ok(()) +} + +// ------------------------------------------------------------ +// Helpers. +// ------------------------------------------------------------ + +/// Decode a 32-character hex string into a 16-byte key. +fn hex_decode_16(s: &str) -> std::result::Result<[u8; 16], String> { + if s.len() != 32 { + return Err(format!( + "secret key hex must be 32 chars, got {}", + s.len() + )); + } + let mut out = [0u8; 16]; + for i in 0..16 { + out[i] = u8::from_str_radix(&s[2 * i..2 * i + 2], 16) + .map_err(|e| format!("invalid hex at byte {}: {}", i, e))?; + } + Ok(out) +} + +/// Wrap any Display error into a rusqlite user-function error. +fn sqlite_err(e: E) -> Error { + Error::UserFunctionError(Box::new(std::io::Error::new( + std::io::ErrorKind::Other, + e.to_string(), + ))) +} + +fn not_initialized() -> Error { + sqlite_err("trusty_init must be called before using trusty functions") +} diff --git a/crates/extensions/trusty-sqlite-ext/tests/smoke.rs b/crates/extensions/trusty-sqlite-ext/tests/smoke.rs new file mode 100644 index 0000000..8735dcc --- /dev/null +++ b/crates/extensions/trusty-sqlite-ext/tests/smoke.rs @@ -0,0 +1,18 @@ +// ============================================================ +// Smoke test for trusty-sqlite-ext. +// +// A loadable SQLite extension is only meaningfully exercised by +// actually dlopen'ing the built .dll / .so from a host SQLite +// binary — that is out of scope for `cargo test`. Here we settle +// for a compile-time smoke test: if this file builds, the cdylib +// has linked and exports are resolvable by name. +// ============================================================ + +#[test] +fn extension_builds_as_cdylib() { + // Intentionally trivial. The real verification is: + // cargo build -p trusty-sqlite-ext + // producing target/debug/trusty_sqlite_ext.dll with + // sqlite3_trustysqliteext_init exported. + assert!(true); +} diff --git a/docs/device-signer-provisioning.md b/docs/device-signer-provisioning.md new file mode 100644 index 0000000..c488deb --- /dev/null +++ b/docs/device-signer-provisioning.md @@ -0,0 +1,196 @@ +# Device-Signer Provisioning + +Pattern for enrolling, using, rotating, and revoking Ed25519 signing +keys on devices (IoT sensors, edge nodes, agents) that append data to a +trusty chain. + +--- + +## Problem + +Many trusty deployments have more than one source of writes: + +- A carbon-credit SaaS ingesting readings from hundreds of IoT sensors. +- An AI-agent audit log receiving tool-call records from many running + agents. +- A monitoring/logging API collecting events from thousands of customer + services. + +Each source needs a persistent cryptographic identity. The chain needs +to record those signatures in a way external verifiers can check. The +operator needs a way to recognize an enrolled device, retire a +compromised one, or rotate a scheduled key. + +Trusty keeps chain verification self-contained (signature + embedded +public key per row). Trust *policy* — which devices are still trusted +today — lives in the operator's application layer, not the chain. + +--- + +## Core Idea + +Trusty's v2 payload embeds each row's Ed25519 public key **inline** in +the signature object (see `docs/payload-v2.md` §4). This means: + +- A proof bundle is **cryptographically self-verifying**: a consumer + with the payload bytes can check `verify(pk, payload, sig)` without + any registry lookup. +- Whether a particular pk *should have been trusted* at append time is a + separate question. That question is answered by an application-level + **signer registry**. + +Two layers, cleanly separated: + +``` ++------------------------------------+ +| Chain layer (trusty): | +| raw_payload, row_hash, | +| anchor_hash, signatures[pk,sig] | +| -> verifies math, not policy | ++------------------------------------+ +| App layer (you): | +| signer_registry table | +| device_id, pk, enrolled_at, | +| revoked_at, metadata | +| -> verifies policy (was this | +| device trusted at time T?) | ++------------------------------------+ +``` + +--- + +## Lifecycle + +### 1. Enroll + +On the device: + +```rust +use trusty_core::{Ed25519Signer, Signer}; + +let signer = Ed25519Signer::generate(); // OS RNG +let secret = signer.secret_bytes(); // persist to + // device secure + // storage +let pk_hex = hex::encode(signer.public_key_bytes()); // send to + // operator +``` + +On the operator side (app), insert into a registry: + +```sql +INSERT INTO signer_registry (device_id, public_key, enrolled_at) +VALUES (?, ?, ?); +``` + +The secret never leaves the device. Only the public key is transmitted. + +### 2. Sign and append + +```rust +let signer = Ed25519Signer::from_secret_bytes(&secret)?; +trusty.append_signed("device_42", &data, "sensor://42", &signer)?; +``` + +The resulting row carries the signature with the pk embedded inline. + +### 3. Rotate + +When a device's key reaches its scheduled lifetime (or for routine +hygiene), generate a new keypair on the device and register a new entry +(or update the existing one with a new pk + `rotated_at` timestamp). + +**Old rows remain fully verifiable** — the old pk lives inline in those +rows. Only new rows use the new key. + +### 4. Revoke + +If a device is decommissioned, lost, or suspected of compromise: + +```sql +UPDATE signer_registry SET revoked_at = ? WHERE device_id = ?; +``` + +Revocation is **not a chain event**. Revoked rows still carry valid +signatures (the math is unchanged). The app decides to distrust them: + +- A proof-bundle consumer joins the bundle's rows against the registry + and flags any row where the pk's `revoked_at` precedes the row's + `detected_at`. +- An auditor treats revoked-key rows as quarantined, not missing. + +--- + +## Compromise Scenarios + +Per `docs/threat-model.md` §6 (per-key compromise), trusty's surface +already addresses the crypto side. The registry pattern closes the +policy side. + +| Scenario | Chain behavior | Registry response | +|--------------------------------------- |---------------------------------|---------------------------------------| +| Device secret leaked; attacker signs | Row sig verifies mathematically | Revoke pk; downstream flags the row | +| Old key rotated out | Old-key rows still verify | Mark `rotated_at`; retain the entry | +| Device destroyed | Future writes just stop | Mark `revoked_at` with reason | +| Clone/replay across devices | Chain sequence gate detects it | Registry has no further role | + +--- + +## Registry Schema (Reference, Not Mandatory) + +Trusty does **not** prescribe a registry schema. Apps vary: a carbon- +credit platform cares about `device_type` and `location`; a monitoring +SaaS cares about `customer_id` and `plan`; an agent log cares about +`agent_version` and `operator_account`. + +A minimal schema: + +```sql +CREATE TABLE signer_registry ( + device_id TEXT PRIMARY KEY, + public_key TEXT NOT NULL, -- hex-encoded 32 bytes + enrolled_at TEXT NOT NULL, -- ISO-8601 + rotated_at TEXT, -- NULL until rotated + revoked_at TEXT, -- NULL until revoked + note TEXT +); + +CREATE INDEX idx_sr_pk ON signer_registry(public_key); +``` + +The pk index exists so a verifier can look up registry status given a +pk extracted from a chain row. + +--- + +## Worked Example + +See `crates/trusty-sqlite/examples/device_signer.rs` for a runnable +demo that walks enroll → sign → rotate → revoke end to end on the same +database file trusty uses, using a second `rusqlite::Connection` for +the registry table. + +Run: + +``` +cargo run -p trusty-sqlite --example device_signer +``` + +--- + +## What Trusty Does NOT Do (and Why) + +- **No built-in registry.** Schema needs vary per product; a prescribed + schema would either over-serve (unused fields) or under-serve (missing + fields). Apps roll their own table. +- **No revocation as chain event.** Chain rows are immutable by design. + Introducing a "this pk is revoked" row would conflate cryptographic + truth (the sig is valid) with operational policy (we no longer trust + it). Those belong in different places. +- **No rotation ceremony.** Each v2 row carries its own pk inline; + switching to a new signer at append time is a one-line change on the + device. No chain-side mechanism needed. + +The `Signer` trait accepts any backing implementation (HSM, TPM, cloud +KMS, file-backed). `Ed25519Signer` is the default software impl; +production deployments commonly substitute a hardware-backed signer. diff --git a/docs/payload-v2.md b/docs/payload-v2.md new file mode 100644 index 0000000..f4183fe --- /dev/null +++ b/docs/payload-v2.md @@ -0,0 +1,315 @@ +# Trusty payload format -- v1 + v2 + +Canonical byte-level specification of what goes into `raw_payload`, what +`row_hash` commits to, and how v2 signatures are produced and verified. + +This document is consensus-critical. v1 byte-shape is locked by the +invariant tests in `trusty-core` (see tag `s5_invariant_v1_byte_shape_preserved` +and the `v1_*` tests in `payload.rs` / `chain.rs`). v2 is purely additive +and must preserve v1 bytes exactly when `payload_version=1`. The original +Python port-source is archival only and no longer gates changes here. + +Source of truth in-tree: +- `crates/trusty-core/src/payload.rs` -- canonical serializer, constants. +- `crates/trusty-core/src/chain.rs` -- `compute_record`, `compute_record_signed`, + `verify_links`, `verify_row_signatures`. +- `crates/trusty-core/src/hash.rs` -- `row_hash`, `anchor_hash`. +- `crates/trusty-core/src/crypto.rs` -- `Signer`, `Verifier`, Ed25519 impls. + +--- + +## 1. Canonical form + +Every `raw_payload` is a JSON object. Its canonical byte form is: + +1. **Keys sorted alphabetically**, recursively (every nested object gets the + same treatment). Implemented via `BTreeMap` at the top level and a + recursive `canonicalize` on all nested `serde_json::Value::Object` nodes. +2. **No whitespace** -- `serde_json::to_string` (not `to_string_pretty`). +3. **UTF-8**. No BOM. +4. **Array order preserved**. Only object keys are sorted; arrays keep caller + order (this matters for `signatures`, which is an array -- but each entry + inside is itself an object and gets its keys sorted). + +### 1.1 Field list (alphabetical -- the on-wire order) + +| Field | Type | Notes | +|---------------------|------------|-----------------------------------------------------------------------| +| `chain_id` | string | `chain__<6-digit-generation>` (e.g. `chain_acme_000001`). | +| `data` | any JSON | Caller-provided. Canonicalized recursively. Capped at 65,536 bytes after canonicalization. | +| `detected_at` | string | ISO-8601 UTC. Time trusty observed the event. | +| `entity_id` | string | Opaque caller identifier. | +| `extraction_method` | string | Caller tag for the provenance source (e.g. `trusty`, `scraper-v3`). | +| `hash_alg` | string | Always `"siphash24+sha256"` today. Verifiers read this before recomputing `row_hash`. | +| `payload_version` | number | `1` (v1) or `2` (v2). | +| `prev_hash` | string | Previous row's `row_hash` (16 hex chars), or the literal `"genesis"` on a segment's first row. | +| `received_at` | string | ISO-8601 UTC with sub-second precision. Distinct from `detected_at`; never collapse. | +| `sequence` | number | Per-chain monotonically increasing `i64`. Genesis = 0. | +| `signatures` | array | v1: always `[]`. v2: one object per signer (see Section 3). | +| `source_url` | string | Where the data came from. | + +### 1.2 `data` size cap + +`MAX_DATA_BYTES = 65_536` enforced after canonicalization. Exceeding it +raises `TrustyError::DataTooLarge`. Put blobs in object storage and reference +them by `{uri, sha256}` inside `data` -- do not inline them. + +--- + +## 2. Dual hash + +Each row carries two hashes. Both are computed in `trusty-core/src/hash.rs` +and are byte-identical to the Python reference. + +### 2.1 `row_hash` -- SipHash-2-4 + +- Keyed with the 16-byte `SecretKey` (split into `k0` / `k1` as two + little-endian `u64`s, `bytes[0..8]` and `bytes[8..16]`). +- Input: the exact canonical payload bytes (`raw_payload` above). +- Output: 64-bit digest rendered as **lowercase hex**, **16 chars**, wire + representation derived from `format!("{:016x}", digest)`. This is a + big-endian-style hex rendering of the integer value. **BE hex is + canonical; any legacy LE rendering (e.g. earlier threadkeeper code) is + non-canonical and must be adapter-converted when crossing the boundary.** + See tag `s2_decision_be_encoding_canonical`. + +### 2.2 `anchor_hash` -- SHA-256 + +Input byte layout: +``` +row_hash_bytes || 0x00 || entity_id_bytes || 0x00 || chain_id_bytes +``` + +All three text fields are UTF-8. Output is lowercase hex, **64 chars**. +Null bytes are domain separators between fields -- do not drop them. + +The dual-hash shape (keyed SipHash + keyless SHA-256 with null-byte domain +separation) is locked in `trusty-core/src/hash.rs` and covered by the +`anchor_hash_*` tests there. It is the only hash layout trusty recognises. + +### 2.3 Why two hashes + +SipHash-2-4 is a keyed fingerprint: cheap, fast, tamper-evident against +adversaries who do not hold the key. It is the chain-integrity primitive. + +SHA-256 is the cryptographic anchor: public, collision-resistant, suitable +for external receipts and cross-system reference. The anchor is what you +publish or notarize; the row hash is what you use to link the chain. + +Dual hash is non-negotiable -- do not collapse to one. See +`CLAUDE.md#design-rules`. + +--- + +## 3. v2 signatures + +v2 adds authenticated signatures over the payload. v1 is unchanged. + +### 3.1 Shape of a signature entry + +```json +{"alg":"ed25519","pk":"","sig":""} +``` + +- Keys alphabetical (enforced by recursive canonicalization). +- `alg`: today always `"ed25519"`. Future algorithms add new literal values, + not OIDs. +- `pk`: hex-encoded public key bytes (32 bytes for Ed25519 -> 64 hex chars). +- `sig`: hex-encoded signature bytes (64 bytes for Ed25519 -> 128 hex chars). + +`signatures` is an array so multi-signer is a pure extension -- just append +additional entries. Verifier iterates and validates each independently. + +### 3.2 The signing-view algorithm + +The core problem: if `row_hash` commits to the payload, and the payload +contains the signature, the signature cannot sign the final bytes (it would +have to contain itself). + +Solution -- sign a canonical view with `signatures` forced empty, then +splice the signature in, re-canonicalize, and hash the spliced bytes. Both +sides of the envelope commit to each other: + +**Sign (emission)**: + +1. Start from caller's `PayloadFields`. +2. Force `payload_version = 2`, `signatures = []`. +3. Build the **signing view**: canonical bytes with `signatures=[]`. + `payload::signing_view_bytes(&fields)`. +4. Signer produces `sig = sign(signing_view_bytes)`. +5. Splice `{alg, pk, sig}` (all hex-encoded) into `signatures`. +6. Re-canonicalize -- these are the **final stored bytes** (`raw_payload`). +7. `row_hash = SipHash-2-4(key, raw_payload)`. +8. `anchor_hash = SHA-256(row_hash || 0x00 || entity_id || 0x00 || chain_id)`. + +The row_hash therefore commits to the signature; the signature commits to +everything *except* itself (via the signing view). + +**Verify (consumption)**: + +1. Parse `raw_payload` back into `PayloadFields` (`chain::parse_raw_payload`). +2. If `payload_version < 2` or `signatures` is empty -> skip signature + check. v1 rows and unsigned v2 rows pass the signature stage by design. +3. Rebuild the signing view by replacing `signatures` with `[]` and + re-canonicalizing. +4. For each signature entry: + - Dispatch on `alg`. Unknown -> `UnknownSignatureAlg { alg }`. + - Hex-decode `pk` and `sig`. Decode failure -> `SignatureMalformed`. + - `Ed25519Verifier::verify(&pk, signing_view_bytes, &sig)`. Failure -> + `SignatureInvalid`. +5. `row_hash` is checked **independently** against the full stored bytes. + Tampering with just the sig bytes fires `RowHashMismatch`; recomputing + row_hash to hide the sig-tamper still fires `SignatureInvalid`. Attackers + lose both ways. + +### 3.3 Worked example (conceptual) + +Starting fields (abbreviated): + +```json +{ + "chain_id": "chain_acme_000001", + "data": {"temp_c": 21}, + "entity_id": "acme", + "payload_version": 2, + "prev_hash": "genesis", + "sequence": 0, + "signatures": [], + ... +} +``` + +Signing view = canonical bytes of that object exactly as shown (with all +other canonical fields populated). + +Signer produces `sig` over those bytes. The final stored bytes: + +```json +{ + "chain_id": "chain_acme_000001", + "data": {"temp_c": 21}, + "entity_id": "acme", + "payload_version": 2, + "prev_hash": "genesis", + "sequence": 0, + "signatures": [{"alg":"ed25519","pk":"ab...cd","sig":"12...ef"}], + ... +} +``` + +`row_hash = siphash24(key, )`. This is what lands in the DB. + +### 3.4 Nested object canonicalization (why it matters for v2) + +Signature entries are JSON objects. They MUST be canonicalized (keys sorted) +before serialization, otherwise `serde_json::preserve_order` leaks caller +insertion order into the canonical bytes and different emitters produce +different `row_hash` for the same logical signature. `build_payload` +explicitly canonicalizes the signatures array via the recursive +`canonicalize` helper. See tag +`s5_learn_preserve_order_requires_canonicalize_nested`. + +--- + +## 4. VerifyError taxonomy + +`verify_links` returns `VerifyReport { ok, row_count, errors }`. Errors +carry `{sequence, chain_id, kind}`; the kinds relevant to payload format: + +| Kind | Trigger | +|------------------------|---------------------------------------------------------------------| +| `PrevHashMismatch` | `link.prev_hash` != previous row's `row_hash` (or `genesis` on first row). | +| `MissingRawPayload` | Non-BREAK row with no `raw_payload`. | +| `InvalidRawPayload` | `raw_payload` does not parse as JSON or is missing required fields. | +| `MissingRowHash` | Non-BREAK row with no `row_hash`. | +| `RowHashMismatch` | Recomputed SipHash != stored `row_hash`. | +| `AnchorHashMismatch` | Recomputed SHA-256 != stored `anchor_hash` (when stored). | +| `SignatureInvalid` | v2: Ed25519 verify failed against signing view. | +| `SignatureMalformed` | v2: signature entry not a JSON object, pk/sig hex decode failed. | +| `UnknownSignatureAlg` | v2: `alg` field not recognized (only `ed25519` today). | + +BREAK markers carry no hashes and reset `prev` to `genesis` for the next +segment. They are not counted in `row_count`. + +--- + +## 5. API surface + +### 5.1 `trusty-core` (pure library) + +```rust +use trusty_core::{ + compute_record, compute_record_signed, + Signer, Ed25519Signer, + PayloadFields, SecretKey, + verify_links, +}; + +// v1 path -- unsigned. +let rec = compute_record(&fields, &key)?; +// rec.raw_payload has "payload_version":1, "signatures":[] + +// v2 path -- signed with any Signer impl. +let signer = Ed25519Signer::generate(); +let rec = compute_record_signed(&fields, &key, &signer)?; +// rec.raw_payload has "payload_version":2 and one signatures entry. +// Caller's fields.payload_version and fields.signatures are overridden. + +// Verify (version-agnostic). +let report = verify_links(&entity_id, &links, &key); +assert!(report.ok); +``` + +The `Signer` trait is byte-oriented: + +```rust +pub trait Signer { + fn sign(&self, msg: &[u8]) -> Vec; + fn public_key_bytes(&self) -> Vec; +} +``` + +HSM or remote-signer integrations implement this trait directly; no +ed25519-dalek dependency leaks into chain code. + +### 5.2 `trusty-sqlite` (storage adapter) + +```rust +let store = Trusty::open(path, key)?; + +// v1 append +let receipt = store.append(entity, data, source_url)?; + +// v2 append -- threads Option<&dyn Signer> through the append pipeline +let receipt = store.append_signed(entity, data, source_url, &signer)?; + +// Custom extraction_method +store.append_with_method(entity, data, source_url, method)?; +store.append_with_method_signed(entity, data, source_url, method, &signer)?; +``` + +Mixed v1/v2 chains verify fine; dispatch is per-row on stored +`payload_version`. You can flip signing on/off per-row at will. + +--- + +## 6. Locked invariants + +- **v1 byte-shape is frozen.** Same field set, same canonical ordering, same + `hash_alg` string, `signatures:[]`, `payload_version:1`. Tag: + `s1_invariant_payload_v1_canonical`. +- **v1 is preserved under v2.** v2 was shipped additive-only in S5 -- + calling `compute_record` on a v1 `PayloadFields` emits exactly the same + bytes it did before v2 existed. Tag: `s5_invariant_v1_byte_shape_preserved`. +- **row_hash BE-hex encoding is canonical.** Decision `s2_decision_be_encoding_canonical`. +- **Dual hash inputs are locked.** SipHash-2-4 keyed + SHA-256 unkeyed with + null-byte domain separation. Tag: `s1_invariant_payload_v1_canonical`. +- **Signature algorithm string is a literal.** No OID dance, no algorithm + negotiation. Adding an algorithm = adding a new literal value in verify + dispatch. +- **row_hash commits to the signature bytes** (v2). Anything that would make + the signature not affect `row_hash` is a regression. + +New algorithms, new fields, new versions must be purely additive and must +leave v1 bytes byte-identical. Run `cargo test -p trusty-core` and inspect +the `v1_*` tests before and after any payload-layer change. diff --git a/docs/product-use-cases.md b/docs/product-use-cases.md new file mode 100644 index 0000000..ba89177 --- /dev/null +++ b/docs/product-use-cases.md @@ -0,0 +1,237 @@ +# Trusty — Product Use Cases and Beneficiaries + +A non-technical overview of what Trusty enables, the products it unlocks, and +who benefits. This is the "what it does for the customer" view, not the +"how it works under the hood" view. + +--- + +## What Trusty Is, In One Paragraph + +Trusty is a tamper-evident audit layer. Any data that flows through it — a +sensor reading, a state transition, a decision, an agent action — gets sealed +into a chain that cannot be silently edited after the fact. If anyone changes, +inserts, deletes, or reorders a single record, the chain visibly breaks. That +break is itself an event the system can alert on. It is deliberately small, +cheap to run, and embeddable anywhere data needs to be trustworthy. + +The business model is commodity-priced, high-margin, volume-driven. Trusty is +the door-opener. Every application that sits on top of it — carbon-credit +SaaS, compliance monitoring, AI agent accountability, data marketplaces, and +Threadkeeper — rides the same audit spine. + +--- + +## 1. Carbon Credits You Can Actually Insure + +**The problem.** Carbon credits are only worth what someone will pay for them, +and insurers will not underwrite credits backed by sensor data that could have +been quietly edited after the fact. + +**What Trusty enables.** Every environmental reading from every sensor gets a +tamper-proof timestamp and fingerprint the moment it is recorded. If anyone +changes a single number later, it shows immediately and publicly. Climate +models (via NVIDIA Omniverse / Earth-2) then ingest data that is provably +untampered. + +**Who benefits.** +- Carbon-credit SaaS operators — can now sell credits that insurers will + underwrite. +- Insurers — can price carbon-credit policies with actual evidentiary + foundation. +- Buyers / offset purchasers — know the credit represents real, measured + reduction, not a spreadsheet. +- Regulators — can audit the full chain without trusting the operator. + +--- + +## 2. IoT Sensor Networks That Prove Themselves + +**The problem.** With thousands of sensors in the field — pipelines, factories, +farms, power grids, cold chains, emissions monitors — you cannot watch them +all. A malfunctioning, spoofed, or tampered sensor can go undetected for +months, and the financial, regulatory, and reputational cost is enormous. + +**What Trusty enables.** Every sensor reading is chained to the one before +it. The absence of a valid reading is itself the alarm — you do not need a +separate intrusion detector. Trusty handles fleet-scale workloads +(approximately 19,000 readings per second per host), so thousands of devices +can write into the same audit trail without slowing anything down. + +**Who benefits.** +- Industrial operators (oil and gas, manufacturing, utilities) — catch + tampering and device failure in real time. +- Insurance and reinsurance — underwrite sensor-driven policies with + confidence. +- Supply chain and cold-chain operators — prove goods were handled correctly + end to end. +- Compliance and safety teams — turn "we think the sensors worked" into "we + can prove it." + +--- + +## 3. Active Compliance Monitoring and the Liability Flip + +**The problem.** In regulated industries — pharma, aviation, finance, food +safety, medical devices — disputes come down to "what did your system +record?" Most enterprise logs are editable, which leaves the company +carrying the liability. + +**What Trusty enables.** Pair Trusty with a rule set that defines which +transitions are *legal* ("a dose can only be administered after a +verification step"). Every legitimate transition is sealed into the chain. +Anything outside the rules cannot produce a valid entry. The chain becomes +legal-grade evidence of what actually happened. + +**The liability flip.** "It is in the chain" or "it is not in the chain" is +a stronger claim in a dispute than any editable database log. The burden of +proof moves from the operator (who currently has to defend the integrity of +their logs) onto the claimant. This is the single strongest commercial +story Trusty has. + +**Who benefits.** +- Pharma and medical device manufacturers — audit-ready by default for FDA, + EMA, and equivalent regulators. +- Aviation and aerospace — tamper-evident maintenance and flight-ops records. +- Financial services — provable trade, order, and settlement history. +- Legal teams — reduce discovery cost and win disputes on evidentiary + strength. +- Insurers of the above — lower claim cost when the audit trail is + unambiguous. + +--- + +## 4. Audit-Ready by Default: The Proof Bundle + +**The problem.** Audits, regulatory reviews, insurance claims, and legal +disputes all require handing evidence to someone who does not trust you. +Today that is a scramble of spreadsheets, screenshots, and lawyers, and the +recipient still has to take your word for most of it. + +**What Trusty enables.** Export a single sealed bundle covering any time +window or entity. The recipient — auditor, insurer, regulator, opposing +counsel — verifies it independently on their own equipment, with no access +to your systems and no need to trust the operator. The bundle carries +everything required: the records, the cryptographic anchors, and the +verification keys. + +**Who benefits.** +- Compliance and audit teams — turn compliance from a project into a button. +- Insurers and claims adjusters — verify losses without site visits or + vendor cooperation. +- Regulators — receive evidence in a standard, self-verifying format. +- Law firms — deliver and receive discovery material that is demonstrably + authentic. +- Any business with "prove it" as part of its sales cycle — verifiability + becomes the product. + +--- + +## 5. Data Marketplace Provenance + +**The problem.** In peer-to-peer and decentralized data marketplaces, +publishers can silently swap individual pieces of a dataset even when each +piece is individually signed — because signatures do not prove ordering or +history. + +**What Trusty enables.** Every slice of the dataset is linked to its +neighbors so any substitution, deletion, or reordering is immediately +visible. You can prove not just "this piece is authentic" but "this is the +complete history, in order, unchanged." + +**Who benefits.** +- Distant Star (Eric's decentralized data-network protocol) — gains a + provenance layer it currently lacks. +- Data publishers — can charge a premium for verifiable datasets. +- Data buyers (enterprise, scientific, government) — can finally adopt + decentralized data sources for regulated work. +- Validators and data curators — their attestations now anchor to a chain + state that proves what they reviewed has not changed. + +--- + +## 6. Provable AI Agents and Agent Swarms + +**The problem.** When AI agents make decisions — trading, legal, medical, +autonomous systems, customer-facing automation — regulators, customers, and +insurers increasingly need to prove not just *what the agent did* but *what +it was paying attention to* when it did it. Without that, AI cannot be +adopted at scale in regulated environments. + +**What Trusty enables.** Every agent carries parallel audit chains — one +per dimension being monitored (attention, uncertainty, inter-agent +interaction, input context, output action). Any drift, hallucination, or +off-policy behavior shows up as a chain break on the relevant dimension. +You get an audit trail not just of agent outputs but of agent *state*. + +**Who benefits.** +- AI platform operators — ship AI into regulated markets they currently + cannot enter. +- Enterprises adopting AI — get an accountability layer their compliance + teams will accept. +- Regulators — gain a standard way to inspect agent behavior after the fact. +- AI safety and alignment researchers — have a production-grade substrate + for behavioral monitoring. + +--- + +## 7. Threadkeeper + +**What Threadkeeper is.** Threadkeeper is Eric's desktop knowledge / +conversation-capture application. It records, organizes, and reasons over +long-running threads of work — transcripts, notes, decisions, research +artifacts — across sessions and projects. + +**The problem Threadkeeper has.** A knowledge tool is only as trustworthy as +the history it presents. If a user, a sync bug, or an automated process can +silently rewrite past entries, the tool cannot be used for anything where +"what was said and when" matters — legal matters, investigative work, +research that will later be published, client-facing deliverables, or +regulated professional practice. + +**What Trusty enables for Threadkeeper.** +- Every captured entry is sealed on write. The historical record is + demonstrably the record that was originally created. +- Any external party — a client, a court, a reviewer — can be handed a + Proof Bundle and verify the thread's integrity independently. +- Cross-thread references (entry A cites entry B at a specific point in its + history) are themselves tamper-evident, so citation integrity is + preserved. +- When AI assistance is layered into Threadkeeper (summarization, + reasoning, agentic workflows), the agent's activity rides the same chain + — so "what the assistant did, when, and why" is part of the same audit + trail as the human-authored content. + +**Who benefits inside the Threadkeeper story.** +- Lawyers, consultants, and advisors — deliverables built on Threadkeeper + are audit-ready and defensible by default. +- Investigators and journalists — source material and working notes carry + provable provenance. +- Researchers — lab notebooks, experiment logs, and authorship trails are + tamper-evident. +- Solo professionals and small firms — get enterprise-grade evidentiary + rigor without enterprise-grade tooling cost. +- Anyone handing work product to a skeptical third party — the product + ships with its own proof of integrity. + +**Why this matters commercially.** Threadkeeper plus Trusty is the +reference implementation of "knowledge work that proves itself." It is the +most visible, user-facing demonstration of the Trusty substrate, and it is +the story that translates the technology into something non-technical +buyers intuitively understand. + +--- + +## The Common Thread + +Trusty is deliberately narrow — a commodity audit layer, not a platform. +Low per-unit cost, high margin, easy to embed anywhere trustworthy data +matters. Once embedded, every application layer on top — carbon credits, +compliance monitoring, agent accountability, data marketplaces, +Threadkeeper — rides the same audit spine and inherits the same +evidentiary guarantees. + +The consistent benefit across every use case: **the burden of trust shifts +away from the operator and onto the math.** Customers stop having to +convince counterparties their logs are honest, because the logs prove +themselves. diff --git a/docs/scope-grammar.md b/docs/scope-grammar.md new file mode 100644 index 0000000..9246b76 --- /dev/null +++ b/docs/scope-grammar.md @@ -0,0 +1,96 @@ +# Scope grammar + +The string format `trusty_policy.permits()` matches against. Two repos +depend on it, so it is written down here rather than inferred from the +implementation. + +## The rule + +```sql +CREATE OR REPLACE FUNCTION trusty_policy.covers(scope TEXT, resource TEXT) +RETURNS BOOLEAN +LANGUAGE sql IMMUTABLE PARALLEL SAFE +AS $fn$ + SELECT scope IS NOT NULL + AND resource IS NOT NULL + AND (resource = scope OR starts_with(resource, scope || '/')) +$fn$ +``` + +A scope covers a resource when they are **equal**, or when the resource +begins with the scope **followed by a `/`**. + +That is the whole grammar. There are no wildcards, no globs, and no +pattern syntax — deliberately. + +## Shape + +``` +:[/]* +``` + +| part | example | rule | +|---|---|---| +| class | `board` | opaque to trusty | +| id | `9f2` | opaque to trusty | +| segment | `calendar`, `private` | separated by `/` | + +Only the `/` is load-bearing. `:` is a convention the +consumer chooses; trusty treats everything before the first `/` as one +opaque string. + +## What follows from it + +**`/` is the only separator.** `board:9f2/calendar` covers +`board:9f2/calendar/private`. It does **not** cover +`board:9f2/calendar-private`, because the character after the scope is +`-`, not `/`. + +**A near-miss prefix does not match.** `board:o` does not cover +`board:other/calendar`. Without the separator in the concatenation it +would, which is why the separator is in there and not an afterthought. + +**No wildcards.** A scope of `%` is a scope literally named `%`, and a +scope containing `_` matches only a literal underscore. `starts_with` is +used rather than `LIKE` precisely so an attacker-influenced scope cannot +widen itself into a pattern. + +**Case and Unicode are exact.** `Board:9f2` is a different scope from +`board:9f2`. There is no normalisation, folding, or trimming anywhere in +the path. Emit exactly what you mean to match. + +**No trailing slash.** `board:9f2/calendar/` is a *different* scope from +`board:9f2/calendar`, and covers `board:9f2/calendar//private` rather +than anything useful. Do not emit one. + +**Exclusions use the same rule.** An exclusion of +`board:9f2/calendar/private` shuts that resource and everything beneath +it, checked before coverage and across every live grant of the subject +regardless of tier. + +## What a producer must guarantee + +Anything writing a `resource_scope` column, or issuing a grant, owes the +following. Trusty cannot check them, because the parts are opaque to it: + +1. Segments are emitted in the same shape both sides expect. An + exclusion of `board:9f2/calendar/private` protects nothing if rows + carry `board/9f2/calendar/private`. +2. No segment contains a `/` of its own. A segment that does silently + becomes two levels, and a grant one level up now covers it. +3. Ids are not user-controlled free text. `board:` lets + someone name a board `9f2/calendar` and inherit a scope they were + never granted. + +Point 3 is the one that bites. If ids can contain `/`, the grammar has +no boundary and coverage becomes forgeable. + +## What trusty deliberately does not do + +- Parse, validate or normalise a scope +- Attach meaning to `class` or `id` +- Enforce a maximum depth + +The engine takes two opaque strings and asks whether one contains the +other. Keeping it that narrow is what lets the consumer own the +namespace — and is why the guarantees above are the consumer's to keep. diff --git a/docs/threat-model.md b/docs/threat-model.md new file mode 100644 index 0000000..48fc51e --- /dev/null +++ b/docs/threat-model.md @@ -0,0 +1,296 @@ +# Trusty threat model + +What trusty detects, what it does not detect, what the trust boundaries +are, and how the system behaves under key compromise. Read alongside +`docs/payload-v2.md` for the on-wire format details this doc references. + +This is a living document. If you add a primitive (new condition, new +signature algorithm, new break reason) that changes any property here, this +file gets updated in the same session. + +--- + +## 1. Assets under protection + +| Asset | What it represents | +|-----------------------|------------------------------------------------------------------| +| Chain integrity | The append-only link from genesis through every row's `prev_hash`. | +| Row authenticity (v2) | The claim that a specific signer endorsed a specific row payload. | +| Event history | That every mutation -- benign or adversarial -- was recorded. | +| External anchors | SHA-256 `anchor_hash` values suitable for publication / notary. | + +Trusty is an **integrity and provenance** layer. It is not a confidentiality +layer, not an availability layer, and not an access-control layer. + +--- + +## 2. Adversary model + +We consider an attacker who has one or more of: + +- **Read access to the SQLite file** (or the underlying bytes). +- **Write access to the SQLite file** (can modify rows, re-order, delete, + insert fabricated rows). +- **Control of the upstream source** feeding `data` into `append()`. +- **Control of `detected_at` / `received_at` values** when submitting. +- **Possession of one or both cryptographic keys** -- the 16-byte SipHash + `SecretKey` used for `row_hash`, and/or the Ed25519 signing key used for + v2 signatures. + +We do not consider: + +- Side-channel attacks on SipHash / Ed25519 implementations. The underlying + crates (`siphasher`, `ed25519-dalek`) are treated as correct. +- Compromise of the host OS kernel or memory bus -- below the trust + boundary. +- Quantum-era attacks on Ed25519. Out of scope; algorithm agility is + reserved for a future version bump. + +--- + +## 3. The chain-break-as-event primitive + +Trusty's central design choice differs from most provenance systems: when a +break condition fires, trusty **records the break as a row** (`event_type = +"BREAK"`, `break_reason` set) and **starts a new chain segment** from +genesis for the next append. It does not refuse the write and does not +silently repair. + +Implications for the threat model: + +- Tampering cannot erase its own trace. Even an attacker with write access + who corrupts a row triggers a BREAK at the next honest append, which is + itself an auditable event. +- Verification after the fact surfaces the break: `verify_links` returns + `ok = true` for a chain with BREAK markers (they are valid structural + separators), but the presence of BREAK rows is itself the signal. + Consumers of verify reports MUST inspect BREAK markers and their + `break_reason`, not treat a passing verify as clean history. +- An attacker who controls writes AND holds both keys can forge an entire + new segment that looks benign. BREAK markers detect tampering only when + a subsequent honest append occurs. See Section 6 for the defense. + +--- + +## 4. Detection surfaces + +### 4.1 Chain structural checks (`verify_links`) + +| Surface | Detects | Miss condition | +|-----------------------|----------------------------------------------------------|----------------------------------------------------------------| +| `PrevHashMismatch` | Row reordering, deletion mid-chain, fabricated row with wrong link. | Attacker re-stitches prev_hash and also forges row_hash (requires SecretKey). | +| `RowHashMismatch` | In-place payload mutation without re-hashing. | Attacker recomputes row_hash (requires SecretKey). | +| `AnchorHashMismatch` | Tampering with stored anchor when row_hash/entity/chain is intact. | Attacker recomputes anchor (requires access to entity_id + chain_id only, no key). | +| `MissingRowHash` / `MissingRawPayload` | Rows with structural fields dropped. | None at this layer. | + +### 4.2 Break conditions (at append time, `crates/trusty-core/src/conditions.rs`) + +| Condition | Fires on | `BreakReason` | +|---------------------|--------------------------------------------------------------------------|--------------------------------| +| `HashIntegrity` | Stored tip hash != computed chain-tip hash (pre-append). | `TAMPER_DETECTED` | +| `SequenceContinuity`| `sequence` is not `last_sequence + 1`. | `SEQUENCE_GAP` | +| `SourceDomain` (opt-in) | `source_url` not in configured trusted-domains list, `hard = true`. | `SOURCE_CHANGE` | + +First failing condition wins; remaining conditions do not run. Custom +conditions compose via the `Condition` trait. + +### 4.3 v2 signature checks (`verify_row_signatures`) + +Only applied when `payload_version >= 2` and `signatures` is non-empty. + +| Surface | Detects | +|-------------------------|-------------------------------------------------------------------| +| `SignatureInvalid` | Signature does not verify against the reconstructed signing view. | +| `SignatureMalformed` | Signature entry not a JSON object; pk/sig hex decode failure. | +| `UnknownSignatureAlg` | `alg` field is not `ed25519` (only algorithm supported today). | + +### 4.4 Cross-check KV store (`trusty-sqlite`) + +`trusty-sqlite` mirrors `(anchor_hash -> row_id)` into a `hash_index` table +separate from the main `provenance_envelope` table. A modified anchor on +the primary row is caught when verification walks the cross-check. +Byte-layout matches the Python reference `cross_check_kv_store`. + +--- + +## 5. Ed25519 trust boundary (v2) + +The v2 signing-view algorithm establishes a specific trust relationship: + +- **Signer claims**: "at the time of signing, this `{entity_id, chain_id, + sequence, prev_hash, data, detected_at, received_at, source_url, + extraction_method, hash_alg, payload_version}` tuple was true." The + signature covers the **signing view** (canonical bytes with + `signatures=[]`). +- **Signer does not claim** row_hash correctness. row_hash is produced by + the infrastructure holding the SipHash `SecretKey`, which may be a + different party. +- **Infrastructure claims** (via `SecretKey`): "I received these exact bytes + and they chain from the previous tip." Infrastructure cannot forge the + Ed25519 signature without the signing key. + +This split is deliberate: it lets data-producing entities sign with +hardware-bound Ed25519 keys while the trusty storage layer runs with a +commodity SipHash key. Compromising one key is not enough to silently +rewrite history. See Section 6. + +### 5.1 Row-hash-commits-to-signature property + +`row_hash = SipHash(SecretKey, canonical_bytes_with_signature_spliced_in)`. +Therefore row_hash is a function of the signature bytes themselves. + +Tampering strategies and their failure modes: + +| Attacker action | What fires | +|-----------------------------------------------|---------------------------------------------| +| Change sig bytes, keep row_hash unchanged. | `RowHashMismatch`. | +| Change sig bytes, recompute row_hash. | `SignatureInvalid` (sig no longer verifies over signing view). | +| Change data, recompute sig (no Ed25519 key). | Cannot produce sig -> `SignatureInvalid`. | +| Change data + sig + row_hash (has both keys). | **Passes.** See Section 6. | +| Strip signatures array to empty. | `RowHashMismatch` (canonical bytes changed). Downstream also loses authenticity claim. | +| Downgrade `payload_version` to 1. | `RowHashMismatch`. v1 re-emit is a different byte sequence. | + +--- + +## 6. Key compromise scenarios + +Trusty has two key materials with different blast radii: + +### 6.1 SipHash `SecretKey` compromise (one key, symmetric, per-installation) + +Attacker can: +- Recompute `row_hash` for any fabricated payload. +- Walk the chain forward: craft a new row with correct `prev_hash` and + correct `row_hash`, pass `verify_links`. +- Generate valid `anchor_hash` values (anchor_hash requires no key -- it is + public SHA-256; attacker can always do this step). + +Attacker cannot (in a v2 chain): +- Forge an Ed25519 signature. The `SignatureInvalid` error will fire on any + row they fabricate claiming a signer they do not control. +- **Therefore: v2 signed chains survive SipHash key compromise.** The break + surfaces as `SignatureInvalid`, not structural mismatch, but it is still + detected. + +Mitigation: rotate SipHash key on suspected exposure. Old rows remain +verifiable with the old key; new rows use the new key. Document rotation +boundary in operations. + +### 6.2 Ed25519 signing key compromise (per-signer, asymmetric) + +Attacker can: +- Sign arbitrary data under the compromised public key. Any verifier who + trusts that `pk` will accept the forged signatures. + +Attacker cannot: +- Compute `row_hash` without the SipHash key. +- **Therefore: signed chains survive Ed25519 key compromise.** The break + surfaces as `RowHashMismatch` when the storage-layer key is different + from the attacker's compromised key. + +Mitigation: revoke compromised `pk`; verifiers maintain a revocation list +and reject signatures from revoked keys after the revocation timestamp. +Trusty does not ship a revocation mechanism today -- this is a downstream +consumer concern. + +### 6.3 Dual compromise (both keys) + +Attacker can forge a self-consistent chain end-to-end. This is the only +scenario where trusty's local primitives fail silently. + +Defense-in-depth: + +- **Publish `anchor_hash` externally.** Any row whose anchor was published + (blockchain, transparency log, notary, etc.) before the compromise + window cannot be retroactively rewritten. Post-publication tampering is + caught by comparing local anchor to the external record. +- **Distribute keys across trust boundaries.** SipHash key belongs to the + storage operator; Ed25519 key belongs to the data-producing entity. An + attacker who wants both must compromise both parties. +- **Cross-check with siblings.** Where a chain is mirrored (threadkeeper, + Distant Star), discrepancies between the local chain and the mirror are + evidence of tampering on one side, independent of local key material. + +### 6.4 What dual compromise still cannot do + +Even with both keys, an attacker cannot: +- Forge a row whose `anchor_hash` has already been published externally and + whose copy is archived outside the trusty DB. Anchor publication is the + ultimate integrity witness. +- Hide a historical BREAK marker if any consumer has previously read and + cached the chain (independent copies surface the diff). + +--- + +## 7. Out-of-scope threats + +Trusty does **not** protect against these and callers must layer additional +controls where they matter: + +- **Backdating.** `detected_at` and `received_at` are caller-provided + strings. They are canonical bytes inside the payload and covered by + row_hash + signature, but they are unverifiable claims about when + something happened. Use a trusted time source (TSA, blockchain timestamp) + externally if the timing claim is load-bearing. +- **Data authenticity upstream of ingest.** `data` is whatever the caller + hands `append()`. Trusty certifies that the byte sequence was received + and chained; it does not certify that the byte sequence reflects reality. + IoT sensor integrity, API-to-DB pipelines, etc. are upstream concerns. +- **Availability / DoS.** Trusty makes no liveness guarantees. An attacker + with write access can flood the DB or corrupt the file; trusty detects + tampering on verify but does not prevent service degradation. +- **Confidentiality.** Payloads are stored in cleartext canonical JSON. + Encrypt the DB file (or the `data` field via caller-side envelope + encryption) if confidentiality is required. `row_hash` is computed over + the cleartext. +- **Access control.** Any process with file-system access can call + `append()`. Gate ingestion at a higher layer (API, auth middleware). +- **Replay.** A signed row can be inserted into a different chain if the + attacker also holds the SipHash key for that target chain. Chain-id and + entity-id are in the signing view, so cross-chain replay requires + fabricating anchor_hash with matching entity + chain strings -- detected + by `AnchorHashMismatch` only if the original row has a different anchor. + Defense is at the insertion gate. + +--- + +## 8. Operational guidance for integrators + +Short checklist for consumers (threadkeeper, Distant Star, direct users): + +1. **Verify reports are not pass/fail.** Always inspect `errors` and walk + BREAK markers. A chain with five BREAK events and `ok = true` is an + integrity event, not a clean bill. +2. **Publish anchor_hash early and often.** Anchor publication is the + single strongest defense against dual-key compromise. +3. **Separate key custody.** SipHash key lives with the storage operator; + Ed25519 keys live with signing entities. Do not collocate them in the + same process or HSM slot unless you have a specific reason. +4. **Maintain signer-key registries.** Consumers decide which `pk` values + they trust. Trusty does not; it only verifies signatures against the + pk field in the payload. +5. **Treat `detected_at` / `received_at` as claims, not facts.** If timing + is adversarial, pair with an external TSA. +6. **Do not compare row_hash across installations.** The 16-byte SipHash + key differs per installation, so the same logical payload has different + row_hash values across deployments. Compare `anchor_hash` for + cross-installation equivalence, since that is SHA-256 and keyless. + +--- + +## 9. Related locked surfaces + +Changes to these surfaces require a threat-model review and must leave the +listed invariant tests green: + +- Dual-hash primitive (SipHash-2-4 + SHA-256). Tag + `s1_invariant_payload_v1_canonical`. +- v1 canonical byte shape. Tag `s5_invariant_v1_byte_shape_preserved`. +- row_hash BE-hex encoding. Tag `s2_decision_be_encoding_canonical`. +- Signing-view algorithm (row_hash commits to signature). Tag + `s5_decision_v2_signatures_inline_row_hash_commits`. +- Ed25519 determinism (no randomized-sig variant). Tag + `s4_invariant_ed25519_determinism`. +- Three-table SQLite schema + `hash_index` cross-check. + +Anything that weakens a detection surface is a regression. diff --git a/docs/trusty-postgres-design.md b/docs/trusty-postgres-design.md new file mode 100644 index 0000000..31f723c --- /dev/null +++ b/docs/trusty-postgres-design.md @@ -0,0 +1,1301 @@ +# trusty-postgres -- Design Memo + +Status: DRAFT. Pre-implementation. Author: design pass over the +LCG architecture-antipatterns lessons, the Store trait surface, +and the trusty-sqlite reference implementation. Reader: Eric. + +This memo defines the contract, schema, and architectural decisions +for `trusty-postgres` BEFORE any code is written. The point of writing +it now is the meta-principle in `insight-LCG-ARCH-01`: features that +"we will add in v2" become 5-10x more expensive to retrofit than to +build in from day one. Every section below names the lesson it draws +from so the design's lineage is auditable. + +The reference implementation to mirror is +`crates/adapters/trusty-sqlite/src/store.rs:1-955` and +`crates/adapters/trusty-sqlite/src/schema.rs:1-49`. The trait being +implemented lives at `crates/core/trusty-store/src/lib.rs:22-48`. + +--- + +## 1. Goal and non-goals + +### Goal + +`trusty-postgres` is a sibling adapter crate (likely separate Gitea +repo `Modo-Melius/trusty-postgres`) that implements `trusty_store::Store` +and the higher-level `Trusty` operations against PostgreSQL 14+ with +optional TimescaleDB extension. It targets the deployment tier named in +`s14_insight_store_trait_multi_backend_landscape.toml`: "mid-scale +cloud, time-series query acceleration on received_at via hypertable." + +The product framing comes from +`s14_insight_oid_anchored_chain_identity.toml`: trusty-postgres is the +"OID-anchored TrustyDB" surface for enterprise buyers who already run +Postgres + PKI infrastructure. The pitch is "your existing Postgres, +plus cryptographic proof of stream completeness/authenticity," not +"a new database." This framing forecloses any temptation to bundle +application logic, distributed coordination, or marketplace primitives +into the crate. It is a Store adapter and the higher-level `Trusty` +methods, nothing more. + +### Non-goals + +- Not a replacement for `trusty-sqlite`. SQLite remains the embedded + / edge / single-process reference. Postgres targets multi-writer, + fleet-scale, or organizationally-required-Postgres deployments. +- Not a distributed-coordination layer. Multi-region, multi-master, + or consensus-flavored deployments belong in a hypothetical + `trusty-scylladb` or above. Postgres logical replication is fine + if the operator wants it; we do not invent any cross-cluster chain + reconciliation primitive here. +- Not an analytics surface. DuckDB / columnar read paths are a separate + adapter per the four-tier landscape tag. +- Not a migration of the locked v1 byte-shape. The point of the Store + trait is that `trusty-core` primitives are untouched. Any change that + would shift v1 bytes is out of scope by definition; see + `s5_invariant_v1_byte_shape_preserved` and `CLAUDE.md` "Don't Touch." +- Not a TCB-redefining crate. The trusted computing base added by + this adapter is: postgres + libpq + the chosen Rust driver. Document + this on page one of the README (per `learn-LCG-ARCH-13` / TCB + manifest discipline) and let it sit there as a deliberate cost. + +### Relationship to Distant Star (DS) + +trusty-postgres is orthogonal to the DS network protocol (dspm, dsnl, +dsutils, ds-cli -- the Monorepo-scratch stack). DS is a UDP gossip +transport with X-Wing KEM, exponential-decay hop probability, and +tag-based knowledge-network routing; it transports data, it does not +store it. trusty-postgres stores data behind the Store trait. The two +compose, they do not overlap. + +Concrete composition points (relevant so this crate's API does not +accidentally foreclose them): + +- **trusty-ds-fs (#10)** and **trusty-ds-network (#12)** may use + trusty-postgres as their persistence layer when a DS node is + deployed in the mid-scale cloud tier rather than at the edge. The + Postgres adapter must therefore expose a `stream_anchors_since` and + a ProofBundle export surface that DS handlers can call without + coupling to any internal detail. +- **Advertise-don't-copy pattern from the DS README.** DS nodes + advertise file existence (tag + hash) and serve the file only on + demand. The same pattern applies to chain proofs: a DS-integrated + trusty-postgres node should advertise `(chain_id, tip_hash)` and + serve the ProofBundle on pull, not push full chains through the + gossip layer. This shapes the read API: prefer iterator-based + proof export over "materialize the whole bundle in memory." +- **Key distribution is NOT DS's job** per the S14 SSH-paradigm + decision (`s14_decision_decouple_key_distribution_from_ds_handshake`). + DS session keys (X-Wing) and trusty verification keys (SipHash + + Ed25519 via #17 trusty-keydist) live on separate tracks. trusty-postgres + stores the SipHash keys in the `producer_identity` table (section + 8); it does not participate in any DS handshake. + +Out of scope for this memo but noted: a future `trusty-ds-postgres` +integration crate (separate from trusty-postgres) would implement +the `CapabilityHandler` trait from dspm and wire the Store trait to +DS marketplace messages. That crate is not trusty-postgres. + +--- + +## 2. Scope: trait methods + higher-level Trusty methods + +### Store trait surface + +The 7 methods at `crates/core/trusty-store/src/lib.rs:22-48`: + +- `last_row(entity_id)` +- `insert_row(&RowInsert)` +- `iter_chain(entity_id) -> Iterator>` +- `get_tip(chain_id)` +- `get_tip_for_entity(entity_id)` +- `set_tip(&TipUpsert)` +- `kv_cross_check()` +- `with_txn(&mut dyn FnMut(&mut dyn Store) -> StoreResult<()>)` + +All 7 must be implemented. `iter_chain` is the one with non-trivial +ergonomics in async-Postgres land -- see section 7 (with_txn) and +section 9 (concurrency). + +### Higher-level Trusty methods (NOT in the Store trait today) + +Per `s14_insight_store_trait_multi_backend_landscape.toml`, four +higher-level methods live in `trusty-sqlite::Trusty` rather than +behind the trait. They must be re-implemented in the Postgres +adapter: + +- `append` / `append_signed` / `append_with_method` / `append_with_method_signed` + -- the orchestration around `compute_record` + `write_envelope` + + `append_hash_index` + `set_tip`. Source: `store.rs:152-283`. +- `append_batch(&[RowSpec]) -> Vec` -- single-transaction + amortization. Source: `store.rs:528-553`. +- `verify(entity_id) -> VerifyReport` -- iterate `provenance_envelope` + in id-asc order, reconstruct `ChainLink`s, hand to + `trusty_core::verify_links`. Source: `store.rs:467-500`. +- `breaks_since`, `liveness_check`, `anchors_since`, `export_bundle`, + `history`, `integrity`, `anchor` -- the read/observability surface. + Source: `store.rs:502-836`. + +Estimated 300-400 lines of mechanical work after the trait is +implemented, per the multi-backend tag. + +### Out of scope for v1 of trusty-postgres + +- `tk_*` binaries / APERC tooling -- no adapter-side hooks. +- Schema migrations between trusty-postgres versions. Document + the v1 schema as locked by analogy to the locked v1 byte-shape; + any v2 schema work is a future memo. (See section 11 cross-ref.) +- Connection pooling configuration as a public API. Use the driver's + pool with sane defaults; expose the pool object only if a downstream + user demands it. + +--- + +## 3. Schema design + +### Three-table layout, Postgres dialect + +Mirror `crates/adapters/trusty-sqlite/src/schema.rs:1-49` one-to-one +in structure. The three-table layout is locked per CLAUDE.md "Don't +Touch": `provenance_envelope`, `hash_index`, `chain_tips`. The KV +cross-check (`integrity` method) depends on both `chain_tips` and +`hash_index` agreeing on `(entity_id, chain_id, current_tip_hash)`. + +Postgres dialect translation: + +- `INTEGER PRIMARY KEY AUTOINCREMENT` -> `BIGSERIAL PRIMARY KEY` or + `BIGINT GENERATED BY DEFAULT AS IDENTITY`. Use the latter; identity + columns are SQL-standard and easier to reset for tests. +- `TEXT` -> `TEXT` (Postgres TEXT is unbounded VARCHAR; same semantics). +- `INTEGER` for sequence_int -> `BIGINT`. Sequence numbers are i64 + in the Rust types. +- Indexes: same set as SQLite. Add a partial index + `CREATE INDEX idx_pe_break ON provenance_envelope(entity_id, detected_at) + WHERE event_type = 'BREAK'` to accelerate `breaks_since`. The SQLite + version scans; Postgres can short-circuit with the partial index. +- `raw_payload TEXT` -> `raw_payload TEXT`. Do NOT use `JSONB`. The + payload is a canonical byte-stable JSON string that the verifier + reads back and recomputes hashes over. JSONB normalizes whitespace + and key order, which would silently break v1 byte-shape verification. + This is a v1-byte-shape-preservation invariant; document it inline + in the migration SQL. + +### TimescaleDB hypertable on `received_at` + +Decision: support TimescaleDB as an optional Cargo feature +(`features = ["timescale"]`), default off. When enabled, the migration +runs `SELECT create_hypertable('provenance_envelope', 'received_at', +chunk_time_interval => INTERVAL '7 days')`. + +Reasoning: + +- The IoT/sensor tier (per `project_trusty_role.md` in memory) is the + primary trusty-postgres adopter. Time-series query acceleration on + `received_at` is the right enterprise pitch. +- Hypertables do not change row identity, primary keys, or + read/write semantics from the chain's perspective. The hash chain + is independent of physical chunking. +- Default-off avoids forcing the TimescaleDB dependency on operators + who do not have it. The adapter must work on stock Postgres 14+. +- One constraint: `chain_tips` and `hash_index` are NOT hypertables. + They are small lookup tables; chunking by time would hurt, not help. + +OPEN QUESTION: do we want a separate `received_at` column type +(`TIMESTAMPTZ`) versus the SQLite TEXT-ISO8601 string used today? +Recommendation: yes, store as `TIMESTAMPTZ` for query performance, +but expose as RFC3339 string at the Store trait boundary so the +trait is dialect-neutral. This is purely an internal storage choice +and does not affect chain bytes. Confirm with Eric. + +--- + +## 4. Tri-state null (apply `learn-LCG-ARCH-03`) + +The tag is unambiguous: "wherever absence of a value might be +legitimate, encode three states (PRESENT, ABSENT, UNKNOWN) explicitly." +SQL NULL alone conflates ABSENT and UNKNOWN. + +Audit of nullable columns in the trusty schema: + +| Column | NULL means today | Tri-state interpretation | +|---------------------------------|------------------------------------------------|---------------------------------------------------------------------| +| `provenance_envelope.row_hash` | "BREAK row, no hash applies" | ABSENT (BREAK rows structurally have no hash). Two-state is fine. | +| `provenance_envelope.anchor_hash` | same as row_hash | ABSENT. Two-state is fine. | +| `provenance_envelope.raw_payload` | same as row_hash | ABSENT. Two-state is fine. | +| `provenance_envelope.field_name`/`old_value`/`new_value` | "non-BREAK row, fields don't apply" | ABSENT. Two-state is fine. | +| `chain_tips.current_tip_hash` | TODAY ambiguous | NEEDS TRI-STATE. See below. | +| `chain_tips.chain_id` | "no chain bound to this entity yet" | UNKNOWN/ABSENT ambiguity. See below. | + +The cold-start case is the one that bites: + +> An OID-bound producer is provisioned (its OID is registered, its +> Ed25519 pubkey is in `producer_identity`) but it has not yet +> appended a single row. + +In trusty-sqlite, this state is unrepresentable: `chain_tips` rows +are created on first append, so an unappended producer simply has +no row. That conflates "we know this OID exists and has appended +nothing" with "we have never heard of this OID." For an enterprise +PKI integration that pre-registers devices, this matters. + +Decision: introduce explicit cold-start representation in the +trusty-postgres schema: + +- `chain_tips.current_tip_hash`: encoded as `tip_hash TEXT` paired + with `tip_state TEXT NOT NULL CHECK (tip_state IN ('UNKNOWN', + 'COLD', 'WARM'))`. `UNKNOWN` = pre-provisioned, never appended. + `COLD` = appended once but the chain was sealed by a break and a + new chain is pending. `WARM` = chain is live and tip_hash is the + current valid head. +- `chain_tips.chain_id`: NULL only when `tip_state = 'UNKNOWN'`. + Otherwise NOT NULL. +- New optional column `producer_oid TEXT` (see section 8). + +Per `learn-LCG-ARCH-03`, prefer enum/check-constrained columns over +sentinel-value tricks. The pair `(tip_hash, tip_state)` makes every +`get_tip` result explicit about which of the three states it is in. + +This forces a trait-level decision: should `TipRow` (defined at +`crates/core/trusty-store/src/lib.rs:60-66`) carry a tip_state field? + +Recommendation: YES, additively. Add `tip_state: TipState` to +`TipRow` and `TipUpsert` with a default `WARM` for backward +compatibility with trusty-sqlite. trusty-sqlite would always return +`WARM` (its existing semantics) and trusty-postgres would return the +true state. Verify this does not break any existing trusty-sqlite +test before merging. + +OPEN QUESTION: does this trait-level addition rise to the level of +needing v2 of the Store trait, or can it be additive on v1? Eric +to decide. Recommendation: additive with a default; this matches +the v2-payload-additive precedent in `project_trusty_v2_signing_design.md`. + +--- + +## 5. Compaction strategy (apply `learn-LCG-ARCH-02`) + +The tag is explicit: "chain format supports compaction snapshots +FROM v1. Archive boundaries are themselves attested. No retrofit +possible without breaking historical verifiability." + +The append-only chain has no semantic notion of compaction today. +trusty-sqlite grows unbounded; the IoT-fleet pitch implies hundreds +of devices each appending hundreds of rows per day, which crosses +"too large to verify in a reasonable time" within months. + +### What compaction means for an append-only hash chain + +Three architectural options: + +**Option A: snapshot tables.** Periodically materialize +`(chain_id, sequence_int_high_water, snapshot_root_hash, snapshot_payload)` +into a `chain_snapshot` table. The `snapshot_root_hash` is computed +canonically from the slice it covers (a Merkle root of all row_hashes +in the slice, or simpler: a SipHash over the concatenation, with +domain separation). The snapshot row is itself a valid chain envelope +of `event_type = 'SNAPSHOT'`. After a snapshot is committed, archived +slices can be moved to a cold-storage table or dropped, and verifiers +can resume mid-chain by trusting the snapshot. + +**Option B: TCB manifest checkpoint.** Maintain a +`chain_checkpoint` table with `(chain_id, sequence_int, row_hash, +checkpoint_attestation)`. Verifiers can prune their verification +window to "from the most recent checkpoint forward" and trust earlier +state by checkpoint signature. Lighter weight than snapshots; does +not enable archival, only verification acceleration. + +**Option C: Anchor checkpoints.** Repurpose existing per-row +`anchor_hash` as the compaction primitive. Periodically aggregate +anchors over a window into a single `aggregated_anchor` and publish +it externally (S3, public ledger, whatever). Verifiers spot-check +against the aggregated anchor. This is the lowest-cost option and +mirrors Certificate Transparency Merkle-tree consistency proofs. + +### Decision + +Ship Option B (`CHECKPOINT` envelope, verification-layer only) as +the v1 primitive in trusty-postgres. Reasoning: + +- Issue #13 explicitly says "MUST NOT change v1 byte-shape. + Checkpoints are a verification-layer construct, not a payload + change." Option A (SNAPSHOT) adds a new event_type which, + while additive, is a semantic change to the chain contract and + requires a new trusty-core aggregation primitive -- that is a + larger design pass than v1 of trusty-postgres should attempt. +- Option B keeps the existing chain contract untouched. A + checkpoint is a signed (`chain_id, sequence_int, row_hash, + attestation)` tuple written to a separate `chain_checkpoint` + table. Verifiers can prune their verify window to "from the + most recent valid checkpoint forward." No new envelope type, + no new canonical bytes, no trusty-core change beyond exposing + the checkpointing API in a higher-level module. +- Archival (#5) and row-aggregation compaction (Option A) are + deferred to v2. Both build on top of the checkpoint primitive + once it is proven in production. +- Issues #5 and #13 resolve cleanly: #13 is the v1 feature; #5 + is the v2 feature that uses checkpoints as archive boundaries. +- Terminology lock: the word `compaction` is retired from this + crate's vocabulary in favor of `checkpoint` (for v1) and + `archive` (for v2). No ambiguity. + +Schema: add a `chain_checkpoint` table: + +``` +chain_checkpoint ( + id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, + chain_id TEXT NOT NULL, + sequence_int BIGINT NOT NULL, + row_hash TEXT NOT NULL, + checkpoint_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + attested_by TEXT NOT NULL, -- OID or pubkey fingerprint + attestation_sig BYTEA NOT NULL, -- Ed25519 over (chain_id, sequence_int, row_hash) + UNIQUE (chain_id, sequence_int) +) +``` + +API on `Trusty`: + +- `checkpoint(chain_id, attestation_key) -> CheckpointRow` -- walks + the chain to head, computes a fresh checkpoint at head, writes + the row. +- `verify_since_checkpoint(chain_id) -> VerifyReport` -- reads the + most recent valid checkpoint, iterates forward from there, same + report shape as `verify` today. + +This is all adapter-level; no trusty-core change required for v1. +Archival and row-aggregation compaction become separate design +memos when the archival product case (#5) is ready to ship. + +--- + +## 6. Revocation semantics (apply `learn-LCG-ARCH-06`) + +The tag is blunt: "Every system that said 'we will figure revocation +out in v2' paid for it in v2 with interest." Existing trusty memory +treats revocation as application-layer. Re-examine for OID-anchored +deployment: + +### The case under OID anchoring + +A producer device has OID `1.3.6.1.4.1..` and an Ed25519 +keypair. The chain is signed (v2 payload) per +`project_trusty_v2_signing_design.md`. Mid-stream, the device's key +is revoked -- compromised hardware, retired device, or operator +policy. + +Three sub-cases: + +**Sub-case (a): retroactive revocation of past rows.** Should rows +appended before the revocation timestamp still verify? If yes, the +chain remains valid up to the revocation point and is sealed there. +If no, the entire chain is poisoned by retroactive distrust. + +**Sub-case (b): forward block on new appends.** After revocation, +any append signed by the revoked key must fail. This is structurally +the chain-break primitive: a revoked-key append is a break, sealed +chain ends, no new genesis under the same OID. + +**Sub-case (c): key rotation.** Revoke old key, install new key, +continue chain under new key. Requires a chain-of-custody attestation +linking old-key-tip to new-key-genesis. + +### Recommendation + +**Adopt the Cert-Transparency stance: chains are append-only and +revocation is forward-only.** Sub-case (a) is rejected: past rows +remain verifiable forever. Sub-case (b) is the default behavior +(the existing chain-break primitive handles it for free if the +condition library checks signing-key validity; that condition needs +to exist). Sub-case (c) is a v2 feature involving a `KEY_ROTATION` +envelope and an attestation chain to the new key. + +Schema: add a `producer_revocation` table: + +``` +producer_revocation ( + producer_oid TEXT NOT NULL, + revoked_at TIMESTAMPTZ NOT NULL, + revoked_pubkey BYTEA NOT NULL, + reason TEXT NOT NULL, + attested_by_pubkey BYTEA, -- who authorized the revocation + revocation_envelope_id BIGINT -- FK to a provenance_envelope row +) +``` + +The revocation event is itself a chain envelope of +`event_type = 'REVOCATION'` so the act of revoking is part of the +auditable chain. This satisfies both the append-only invariant and +the auditability requirement. + +Per the LCG-ARCH-06 runtime invariant: "revocation of an upstream +capability invalidates derived attestations within N blocks. Testable +via assert_invariants.py: insert edge, attest, revoke edge, verify +attestation fails on re-check." Translate to a trusty-postgres +integration test: append signed rows, write revocation envelope, +attempt new append signed by revoked key, assert it produces a +chain-break. + +OPEN QUESTION: where does the condition that checks "is this signing +key currently revoked?" live? Options: (i) a new core-level Condition +in `trusty-core`, (ii) a Postgres-adapter-only Condition. Recommendation: +(i), so the semantics are uniform across backends. Schedule as part +of the trusty-core SNAPSHOT-primitive memo (section 5) -- same memo, +same set of additive trusty-core changes. + +--- + +## 7. with_txn decision + +Source of friction: `s14_learn_with_txn_async_friction.toml`. Sync +trait `fn with_txn(&mut self, f: &mut dyn FnMut(&mut dyn Store) -> +StoreResult<()>) -> StoreResult<()>` does not compose with async +Postgres drivers (sqlx, tokio-postgres) because closures cannot be +async without boxing a Future and dealing with lifetime/pinning. + +Three options enumerated in the tag: + +- **(a) `block_on` in adapter.** Keep the sync trait. The Postgres + adapter wraps a `tokio` runtime internally and `block_on`s every + call. Pros: zero changes to trusty-sqlite, zero changes to the + trait. Cons: the adapter's caller cannot be in an async context + without nested runtime issues; or we use the `synchronous-postgres` + driver (libpq directly via `postgres` crate) and skip async entirely. + +- **(b) Boxed async closure.** Add an async variant of with_txn + to the trait: + `async fn with_txn_async(&mut self, f: BoxFuture<...>) -> StoreResult<()>`. + Pros: native async. Cons: trait object methods cannot be generic + AND cannot be async-fn-in-trait without macro support; ergonomics + are awful for callers. + +- **(c) Split into begin / commit / rollback.** Drop the closure + pattern entirely. Trait gains `begin_txn`, `commit_txn`, + `rollback_txn`. Pros: composes with both sync and async. Cons: + callers lose RAII safety (forget to commit and the txn leaks); + trusty-sqlite must be updated too. + +### Recommendation + +**Option (a) -- block_on with the synchronous `postgres` crate (not +sqlx/tokio-postgres) -- for v1 of trusty-postgres.** Reasoning: + +1. The synchronous `postgres` crate (built on `tokio-postgres` but + exposing a sync API) is mature, widely used, and avoids the + nested-runtime problem entirely. No `block_on` needed -- the + driver is sync from the user's perspective. +2. The trait stays unchanged. trusty-sqlite stays unchanged. Other + adapters (DuckDB, ScyllaDB) can make their own sync-vs-async + choice without us locking the trait shape. +3. Async-Postgres consumers who want native `tokio` integration can + wait for a v2 trait or implement Option (c) themselves; this is + not a v1 requirement for the IoT/compliance buyer pitch. + +**However**, document the trait friction explicitly in +trusty-postgres README so the next person who tries to wire it into +an async axum/tonic server is not surprised. Add a worked example +of using `tokio::task::spawn_blocking` to bridge. + +**Defer Option (c) to a future trait-v2 pass.** When we get serious +about ScyllaDB or another genuinely-async backend, revisit. At that +point all adapters get updated together. Eric to decide whether to +formally schedule that pass; not blocking trusty-postgres v1. + +OPEN QUESTION: confirm the `postgres` (sync wrapper) crate is the +right choice over `rust-postgres` direct or `diesel`. Recommendation: +`postgres` crate; it is the de facto standard sync Postgres client +for Rust and has the same maintainer as `tokio-postgres`. + +--- + +## 8. OID + chain_id binding (apply `s14_insight_oid_anchored_chain_identity`) + +The OID-anchored model is the entire commercial reason this crate +exists. Get this right at v1. + +### The binding + +Per the insight tag, the model is: + +- An OID (e.g. `1.3.6.1.4.1..`) identifies + a producer (device, sensor, agent). +- That OID is the canonical chain identity. Either chain_id IS the + OID, or chain_id is derivable from the OID + a generation counter. +- An Ed25519 pubkey is bound to the OID via a known_producers TOFU + registry (per the SSH-paradigm tag). + +The trusty-sqlite chain_id format today is `chain__<6-digit-gen>` +(see `make_chain_id`, used at `store.rs:299` and parsed at `store.rs:965-975`). +Eric: this format is fine for SQLite where entity_id is opaque, but +for OID-anchored Postgres deployments we want OID legibility. + +### Schema + +Add a `producer_identity` table, separate from the three locked tables: + +``` +producer_identity ( + producer_oid TEXT PRIMARY KEY, + ed25519_pubkey BYTEA NOT NULL, + siphash_key BYTEA NOT NULL, -- derived per SSH-paradigm tag + registered_at TIMESTAMPTZ NOT NULL, + display_name TEXT, + metadata JSONB -- ok to use JSONB here: + -- not part of chain bytes +) +``` + +And add a `producer_oid TEXT` column to `chain_tips`, foreign-keyed +to `producer_identity.producer_oid` when present (nullable for backward +compat with non-OID deployments). + +Decision: do NOT change the chain_id format string. The format +`chain__<6-digit-gen>` is locked by trusty-core's +`make_chain_id` (at `crates/core/trusty-core/src/chain.rs:100`). +Note: `parse_generation` currently lives in the SQLite adapter at +`crates/adapters/trusty-sqlite/src/store.rs:965`; it should be +promoted to trusty-core as a pre-requisite of this work so +trusty-postgres does not copy-paste. Additive change; no v1 +byte-shape impact. + +Instead: + +- For OID deployments, set `entity_id` to the OID string. + `make_chain_id` will produce `chain_1.3.6.1.4.1.._000001`, + which is human-readable and OID-legible. +- The `producer_oid` column on `chain_tips` denormalizes the OID + for indexing/joins to `producer_identity`. + +### Reconciliation with Gitea #14 + +Issue #14 proposes `chain_id = H(domain_sep || pubkey || nonce)`. +Taken literally this contradicts the decision above. Reading the +full issue body resolves the conflict: #14 is explicitly additive +-- "Existing v1 chains without attribution remain verifiable." +The two models coexist: + +- **chain_id string** stays as `make_chain_id(entity_id, gen)` for + human legibility, log grep-ability, and OID integration. +- **Cryptographic binding** moves into (a) the v2/v3 genesis-row + signature (the Ed25519 signature over the signing-view commits + to chain_id via the row_hash), plus (b) the + `producer_identity(producer_oid, ed25519_pubkey, ...)` row that + proves which key is allowed to sign for this OID. + +A rival chain with the same chain_id string cannot produce a valid +genesis signature without the private key, so the "malicious party +presents a rival chain" threat in #14 is defended against by the +signature chain, not by the id format. #14 becomes "require genesis +sig + producer_identity lookup before trusting any append," not +"change the chain_id derivation." No trusty-core change to +`make_chain_id`. + +The `producer_identity` table is the local cache equivalent of an +SSH `known_hosts` file (per the SSH-paradigm tag) -- not a CA, just +a TOFU registry. Whether OIDs cross-reference to X.509 cert chains +is application-layer and out of scope here. + +OPEN QUESTION: should the `producer_identity` table be optional +(feature-gated) or always created? Recommendation: always created, +but rows are optional. Adapters that do not use OID-anchoring simply +leave it empty. Cost is one empty table; benefit is one schema for +all deployments. + +OPEN QUESTION: OID validation -- enforce the dotted-numeric format +at the column level (CHECK constraint) or accept arbitrary strings? +Recommendation: accept arbitrary strings. trusty-postgres should not +become an OID validator; that is the producer's responsibility. We +can add a CHECK later if a misuse pattern emerges. + +--- + +## 9. Concurrency model + +### The core difference from SQLite + +trusty-sqlite is single-writer by SQLite's nature (WAL mode plus +`BEGIN IMMEDIATE`, see `store.rs:535`). The chain primitive's +correctness depends on one fundamental invariant: between reading +the tip and writing the next row, no other writer mutates the tip. +SQLite gives us this trivially. + +Postgres allows multi-writer. Two clients calling `append()` for +the same `entity_id` concurrently can both: +1. Read the same tip. +2. Compute next sequence = tip.sequence + 1. +3. Insert two rows with the same sequence and the same prev_hash. + +Result: chain integrity violation. Both rows verify in isolation +(both have correct prev_hash) but the chain has forked. Verification +will eventually catch it (sequence collision in iter_chain) but +silent damage is done. + +### Decision: per-entity row-level lock at append time + +Use Postgres advisory locks keyed on entity_id: + +``` +SELECT pg_advisory_xact_lock(hashtext($1)); -- $1 = entity_id +``` + +Inside the transaction, take an advisory lock derived from +`entity_id` (via `hashtext` or a dedicated 64-bit hash). The lock +is held for the duration of the transaction and released on commit +or rollback. Two appenders for the same entity serialize; appenders +for different entities run concurrently. + +Rationale: + +- Advisory locks are cheap (no row materialization), session-scoped + or transaction-scoped, and Postgres-native. +- They serialize exactly the operations that need serialization: the + read-tip / compute / write-row / update-tip cycle for a single + entity. +- Different entities are independent; no global lock, so multi-entity + fleets scale. + +Alternative considered: `SELECT ... FOR UPDATE` on `chain_tips` rows. +This works but materializes a row lock on every append even when no +contention exists. Advisory locks are more direct. + +Alternative considered: optimistic concurrency with a retry loop on +sequence-collision unique-violation. Possible but complicates the +adapter (retry loops, error mapping). Defer. + +### Test invariants + +Add a concurrent-append test: spawn N threads each appending K rows +to the same entity_id. Assert: total rows = N*K, sequences are +0..N*K-1 with no gaps, `verify` passes, `kv_cross_check` passes. + +OPEN QUESTION: should the advisory lock be transaction-scoped +(`pg_advisory_xact_lock`) or session-scoped +(`pg_advisory_lock` + explicit unlock)? Recommendation: transaction-scoped. +Releases automatically on commit/rollback; no leak risk. + +--- + +## 10. Migration story: trusty-sqlite -> trusty-postgres + +The product use case: a customer runs trusty-sqlite at the edge for a +year, then promotes their archive to trusty-postgres for long-term +audit. This must work without recomputing chain hashes. + +### Byte-shape preservation + +The locked v1 byte-shape (see CLAUDE.md "Don't Touch") guarantees +that `raw_payload` strings are byte-identical regardless of backend. +Migration is a row-by-row copy with no recomputation: + +1. SELECT all rows from sqlite `provenance_envelope` ordered by `id`. +2. INSERT into postgres `provenance_envelope` preserving every column + verbatim. The `id` column re-issues from the postgres identity + sequence; that is fine because chain semantics depend on + sequence_int + prev_hash + chain_id, not on the surrogate id. +3. SELECT all rows from sqlite `hash_index`, INSERT into postgres + `hash_index` verbatim. +4. SELECT all rows from sqlite `chain_tips`, INSERT into postgres + `chain_tips` verbatim. Set `tip_state = 'WARM'` for all migrated + rows; the migration tool cannot reconstruct cold/unknown state. +5. Run `verify` for every entity_id on the postgres side; assert + reports match the sqlite side row-for-row. +6. Run `kv_cross_check` on the postgres side; assert OK. + +### Anchor hashes + +Anchor hashes are NOT recomputed -- they are byte-identical between +backends because they are derived from the locked v1 primitives. +Copy them verbatim. If the post-migration verify pass fails on +anchor mismatch, the bug is in the migration tool, not in the chain. + +### Implementation + +Ship a `trusty-migrate` binary in the trusty-postgres crate: + +``` +trusty-migrate \ + --from sqlite://path/to/file.db \ + --to postgres://user:pass@host/db \ + --verify +``` + +`--verify` runs the full verify pass on both sides post-migration +and asserts byte-identity. Exit non-zero on any mismatch. + +OPEN QUESTION: do we want bidirectional migration (postgres -> +sqlite for offline audit)? Recommendation: yes, but defer to a v2 +of the migration tool. v1 ships sqlite -> postgres only. The +inverse direction has the same byte-shape guarantee so it is +mechanical to add later. + +--- + +## 11. Testing strategy + +The bar to merge trusty-postgres v1: + +### Required test suites + +1. **All trusty-store testing-feature tests pass.** The Store trait + ships a conformance suite under + `crates/core/trusty-store/src/testing.rs` (per the existing + `#[cfg(feature = "testing")]` module, see + `crates/core/trusty-store/src/lib.rs:15-16`). The v1 conformance + suite is 6 tests, enumerated explicitly: + - `insert_and_read_back` -- insert a row; `last_row` returns it. + - `iter_chain_ordered` -- insert 3 rows; iterator yields them + in sequence order with no gaps. + - `tip_roundtrip` -- `set_tip` + `get_tip_for_entity` returns + same row; `get_tip(chain_id)` agrees. + - `kv_cross_check_clean` -- fresh insert produces OK report. + - `with_txn_commit` -- inserts inside `with_txn` closure persist. + - `with_txn_rollback` -- `Err` return from closure rolls back. + + trusty-postgres must pass all 6 identically to trusty-sqlite + and trusty-memory. + +2. **v1 byte-shape conformance.** Add a test that appends N rows + via trusty-postgres, exports a `ProofBundle`, and asserts every + row_hash and anchor_hash matches the same sequence appended via + trusty-sqlite with the same SecretKey, same payload, same + timestamps. This is the runtime version of + `s5_invariant_v1_byte_shape_preserved` for the Postgres adapter. + +3. **Multi-writer concurrency.** Per section 9: N threads, K rows + each, single entity. Assert no sequence gaps, no duplicate + sequences, verify passes. + +4. **Cold-start tri-state.** Provision a producer_identity row with + no chain_tips row; assert `get_tip_for_entity` returns + `tip_state = UNKNOWN`. After first append, assert + `tip_state = WARM`. + +5. **Revocation flow.** Append signed rows, write a REVOCATION + envelope, attempt append signed by revoked key, assert + chain-break. + +6. **Compaction round-trip.** Append N rows, run `compact()` at + high_water = N/2, assert verifier can still verify the full chain + (using the snapshot envelope to skip archived rows). + +7. **Migration tool round-trip.** Append M rows to sqlite, migrate + to postgres, run verify on both; assert reports identical. + +8. **TimescaleDB feature gate.** With `--features timescale`, the + migration runs `create_hypertable` and inserts/queries succeed. + Without the feature, the same tests pass on stock Postgres. + +### Test infrastructure + +Use `testcontainers-rs` or equivalent to spin a Postgres container +per test run. Do not depend on a developer-local Postgres install; +CI must be hermetic. + +OPEN QUESTION: should the conformance suite in +`crates/core/trusty-store/src/testing.rs` be expanded with the +tri-state, concurrency, and OID tests so all backends run them? +Recommendation: yes, but in a follow-up trusty-store PR. Initially +add them inline in trusty-postgres; promote to the conformance suite +once stable and once trusty-sqlite is updated to support tri-state +TipState (default WARM). + +--- + +## 11b. Cross-cutting policy decisions (added post-research pass) + +Decisions on four policies that applied across multiple sections +and were surfaced in the S14-addendum research pass. These are +the memo's current position; any of them can be re-opened but all +four are in scope as implementation-ready defaults. + +### Payload version accept policy + +trusty-postgres accepts reads of v1, v2, and v3 payloads (dispatch +already lives in trusty-core via `row_hash_for_version` and +`extract_payload_version`, confirmed at +`crates/core/trusty-core/src/chain.rs:52,211`). For new appends +the adapter writes v3 only. This guarantees: + +- The `trusty-migrate sqlite -> postgres` tool copies v1/v2 rows + byte-verbatim; no recomputation, no upgrade of existing bytes. +- All new rows benefit from domain separation. +- `compute_record_signed` is v2-only today (handoff acknowledged + gap). trusty-postgres signed-append uses v2 bytes until + `compute_record_signed_multi` or a v3-signed path lands in + trusty-core -- tracked as a pre-req of #16 (patch-policy multi-sig). + +### Schema versioning discipline + +Schema migrations are planned, not forbidden. A `schema_meta` table +ships in v1 carrying `(key TEXT PRIMARY KEY, value TEXT)` with a +single row `('schema_version', '1')`. Any future schema change +ships with: + +- A new version number written to `schema_meta`. +- An idempotent forward migration tested end-to-end. +- A design memo at `docs/trusty-postgres-schema-vN.md` documenting + the change. + +Same rigor as payload v1/v2/v3 work. Do not pretend the schema is +locked; pretending is how drift happens. + +### External anchor publish + +App-level, not adapter-level. trusty-postgres exposes two read +APIs that downstream crates use to build publish pipelines: + +- `stream_anchors_since(chain_id, sequence) -> impl Iterator` +- `export_bundle_since(chain_id, sequence) -> ProofBundle` + +No S3, Git, CT-log, or DS-transport dependency in the adapter. +Retention/transport pipeline (#5) is a separate crate (likely +`trusty-archive` or the DS integration crate) that consumes these +iterators. Keeps trusty-postgres's TCB and dependency graph thin. + +### Observability + +`tracing` spans only. Every public method on the adapter emits a +span with `chain_id`, `entity_id`, and operation name. No +prometheus/opentelemetry crate dependency -- consumers plug in +any `tracing_subscriber` exporter they want (`tracing-opentelemetry`, +`tracing-prometheus-exporter`, etc.) at their layer. Zero-cost +when no subscriber is attached. + +Span namespacing: `trusty_postgres::append`, `trusty_postgres::verify`, +`trusty_postgres::with_txn`, `trusty_postgres::checkpoint`. Include +advisory-lock wait time as a span event so lock contention is +visible to the subscriber. + +--- + +## 11c. Multi-tenant federated deployment (real target) + +The original memo framed trusty-postgres as a generic mid-scale-cloud +Store adapter. The actual product goal is tighter: a **federated shared +knowledge base** spanning up to ~50 users, each with many agents writing +concurrently, glued between user nodes by Distant Star, with the +existing kb-mesh content (pins, synthesis artifacts, architectural +decisions) materialized as trusty chains. This section captures the +deployment shape implied by that goal; it supersedes no prior decision +but adds concrete structure where the original memo was generic. + +### Deployment shape + +- **Per-user Postgres schema.** Each user gets a Postgres schema + (namespace) named for their OID arc leaf, e.g. `usr_eric`, + `usr_partner`, `usr_agent03`. Inside that schema the three-table + layout (`provenance_envelope`, `hash_index`, `chain_tips`) is + replicated. Schema-per-user gives hard isolation (an agent cannot + accidentally write to another user's rows), per-schema index + locality, and straightforward per-user backup/archive. +- **Cross-user schema in `public`.** One schema-wide table set: + `public.producer_identity` (OID -> Ed25519 pubkey + SipHash key + + metadata, the known_producers registry), `public.schema_meta` + (global schema version), `public.received_bundles` (inbound + DS-delivered bundles awaiting or completed verification). +- **Cross-user read view.** `public.all_provenance_envelope AS SELECT + ... FROM usr_eric.provenance_envelope UNION ALL ... FROM + usr_partner.provenance_envelope ...` regenerated whenever a user + is provisioned or deprovisioned. Read-side queries spanning all + users (e.g. "every pin tagged trusty across the mesh") go through + this view; write-side never touches it. +- **One chain per (user, agent, topic) tuple.** `entity_id` encodes + `//`. `make_chain_id` on + top of that produces `chain_//_000001`. + Topic is caller-chosen (session id, KB pin topic, sensor stream + name, etc.). + +### Ownership and authorization + +- Every agent has its own Ed25519 keypair and an OID leaf under its + user's arc. `producer_identity` rows bind (OID -> pubkey). +- Append is rejected at the adapter if the signing key does not + match the `producer_identity` row for the target chain's OID. + This turns the two-party "one writer per chain" rule into a + general N-party invariant enforced at write time. +- Agents provisioned/deprovisioned by a user-level operator flow + (out of scope for this crate; in scope for a future + `trusty-admin` tool). + +### Concurrency at scale + +- Advisory lock per `entity_id` inside each user's schema. + `hashtext(entity_id)` already makes different entities contention- + free; per-entity serialization matters only when two of one user's + agents race the same chain. +- Connection pool (pgbouncer or equivalent) required in any + deployment exceeding ~100 concurrent agents. Memo-level + recommendation: transaction-pooling mode with a pool of + `max_connections / N_users` per user's app tier. +- Postgres's own MVCC handles the 50-user + multi-agent load + comfortably. The bottleneck is per-entity advisory locks, not + the database. Measured scaling will come from real deployment; + design-time assumption is that the chain-break rate stays O(errors) + not O(normal appends). + +### Federation via Distant Star + +- Each user runs their own Postgres + DS node. Writes land locally + under the user's schema. +- DS gossips ProofBundles peer-to-peer. Advertise-don't-copy: a DS + advertisement carries `(chain_id, tip_hash, size_hint)`; bundles + are pulled on demand. +- Inbound bundles land in `public.received_bundles` with: + `bundle_id`, `source_producer_oid`, `chain_id`, `tip_hash`, + `received_at`, `verify_status`, `verify_detail`, `bundle_blob` + (bytea). Verification runs against `producer_identity` (did the + claimed OID sign this?); failed bundles stay with `verify_status + = 'REJECTED'` and a reason -- not silently dropped, not replayed. +- A user's view of "what partner has" is a query against + `received_bundles WHERE source_producer_oid IN (partner's OIDs)`. + Historical verification remains local: even if the partner goes + offline or leaves the federation, bundles already ingested are + still cryptographically verifiable. + +### kb-mesh migration + +The existing kb-mesh primitive (ephemeral pins + promoted patterns) +maps cleanly: + +- Each pin becomes a chain row with `event_type = 'PIN'` and payload + data carrying `topic`, `turns`, `source_session`, `synthesis_level`, + pin body. +- `action = 'create' | 'update' | 'overwrite' | 'close' | 'promoted'` + from INDEX.jsonl becomes `extraction_method` or a structured field + inside `data`. +- Per-repo kb-mesh pins roll into a per-user chain named something + like `usr_/agent_/kb-mesh-pins`. INDEX.jsonl becomes + redundant (`SELECT ... FROM ...provenance_envelope WHERE event_type + = 'PIN'` replaces it). +- Promoted patterns (ephemeral pin -> durable `.kb/patterns/` file) + become CHECKPOINT envelopes over the pin chain + a supplementary + write to a pattern chain. + +A dedicated `trusty-kb-mesh` (or `trusty-kb`) crate owns the mapping; +trusty-postgres is the storage. Gitea issue to follow. + +### Changes to earlier sections this implies + +- §1 add non-goal: trusty-postgres is not a tenant manager; user + provisioning and schema creation live in a future `trusty-admin` + crate or operator script. +- §4 (tri-state null): `tip_state` is per-user; cold-start state + is per-agent per-chain, not global. +- §8 (OID + chain_id): chain_id convention tightened -- `//` + inside the template. `entity_id` encodes the same. Unchanged at + the byte level; just a convention on the strings. +- §9 (concurrency): advisory lock is per-entity within a schema, + not global. Hash collision risk on `hashtext(entity_id)` across + schemas is irrelevant because different schemas are different + critical sections anyway. +- §10 (migration): `trusty-migrate sqlite -> postgres` must now + specify a target schema (or default to `public` for single-user + deployments). + +### Open questions specific to the multi-tenant shape + +- How do we test 50-user + many-agent concurrency before deployment? + Probably a load-gen harness that spins N schemas + K async appenders + per schema. Out of scope for trusty-postgres v1; in scope for a + `trusty-load` test binary later. +- How do schema migrations roll out across N user schemas without + blocking active writers? Partial answer: ALTER runs per-schema in + sequence, advisory-locked against that schema's writers. Full + answer needs a real migration tool. +- Bundle ingestion rate limits per source producer? Probably yes, + to prevent a rogue peer from flooding; app-level concern, not + adapter. + +--- + +## 12. Open questions and explicit deferrals + +Items needing Eric's input before implementation. Items marked +RESOLVED were decided in the post-research pass; strike-through +anything that later changes. + +1. **RESOLVED: TipState additive trait change (S16).** Approved + as v1-compatible additive change with default WARM. Precedent: + v2 signing (`project_trusty_v2_signing_design`) added fields to + `PayloadFields` and a new top-level `compute_record_signed` + without breaking v1 callers; the same additive-default pattern + applies to trait fields. `TipState` is store metadata, not + chain-payload bytes, so v1 byte-shape is unaffected by + construction. + +2. **RESOLVED: Compaction primitive scope** (section 5). Decision: + ship `CHECKPOINT` (verification-layer only) for v1; archival and + row-aggregation compaction are v2. No trusty-core change. Retired + the word "compaction" from this crate's vocabulary. + +3. **RESOLVED: Revocation Condition placement (S16).** trusty-core. + Uniform across backends prevents two adapters from diverging on + what counts as revoked, which would silently fork chain + verification semantics. The cost (a small hash-set lookup in + the core verify path) is negligible vs. the consistency win. + +4. **RESOLVED: Sync vs async driver (S16).** Sync `postgres` crate + for v1 confirmed. Trait stays unchanged; `with_txn` is already + validated. Defer Option (c) trait-v2 split until ScyllaDB or + similar makes async unavoidable. + +5. **RESOLVED: OID validation (S16).** Enforce dotted-numeric + format at column level via `CHECK` constraint + (`chain_id ~ '^[0-9]+(\\.[0-9]+)*$'` or equivalent on the + producer_oid column). OID format is the trust spine of + federation; arbitrary strings invite ad-hoc IDs that defeat + the cryptographic identity model. Fail fast at the boundary. + +6. **RESOLVED: TimescaleDB column type (S16).** Use `TIMESTAMPTZ` + internally; trait surface stays RFC3339 strings. Indexing on + `TEXT` timestamps is bad; conversion is transparent at the + adapter boundary; conformance test enforces the RFC3339 + round-trip. No byte-shape impact (timestamps live in payload + data, not envelope columns subject to v1 lock). + +7. **RESOLVED: Bidirectional migration (S16).** v1 ships sqlite -> + postgres only. postgres -> sqlite is deferred to v2 (or never). + Federated multi-tenant is the production target; nobody + migrates back from federation to a single-file edge backend. + +8. **RESOLVED: Conformance suite expansion (S16).** Promote new + tests into `trusty-store::testing` AFTER the first stable + trusty-postgres release. Keeps churn local during development; + promotion gated on proven stability. + +9. **RESOLVED: Repo location (S16).** Separate Gitea repo + `mm-trusty/trusty-postgres` (already pushed via subtree split, + commit 427c7a7), depending on `trusty-core` and `trusty-store` + via path or git dep. Matches the workspace shape already in + flight; keeps trusty-core/sqlite TCB minimal; users pick + path/git/crates.io per deployment. + +10. **RESOLVED: TCB manifest (S16).** trusty-postgres README + page-one section. Matches `learn-LCG-ARCH-13` recommendation; + one place, explicit, no separate doc to drift. + +11. **RESOLVED: chain_id derivation** (section 8). Decision: keep + `make_chain_id` string template; move cryptographic binding + into genesis-row signature + `producer_identity` table. #14 + becomes "require genesis sig + identity lookup," additive only, + no trusty-core change to `make_chain_id`. + +12. **RESOLVED: payload version accept policy** (section 11b). + Decision: accept v1/v2/v3 reads; new appends are v3. + +13. **RESOLVED: schema versioning discipline** (section 11b). + Decision: plan migrations; ship `schema_meta` table in v1; + same rigor as payload-version work. + +14. **RESOLVED: external anchor publish** (section 11b). Decision: + app-level. Adapter exposes `stream_anchors_since` and + `export_bundle_since` read APIs; publishing lives in a + downstream crate. + +15. **RESOLVED: observability** (section 11b). Decision: `tracing` + spans only, no prometheus/otel dependency. Consumers plug in + any subscriber layer. + +16. **RESOLVED: `parse_generation` location (S16).** Promote to + `trusty-core` as part of the Tier 1 refactor + (`s14_insight_general_library_improvements`). Pure string + parser, no byte-shape involvement. Every future backend + would re-copy-paste otherwise. + +17. **RESOLVED: `compute_record_signed` v3 gap (S16).** + `compute_record_signed` now version-dispatches by + `fields.payload_version`: v1/v2 callers stay byte-identical + (v1 upgrades to v2 per S5 behavior), v3 callers get the + domain-tagged signed path via `row_hash_for_version` / + `anchor_hash_for_version`. Two new tests: + `v3_signed_record_verifies` and + `v3_tampered_signature_fails_verify`. 50 trusty-core tests + pass (was 48). HARD BLOCKER lifted; §13 P1/P4/P7 unblocked. + `compute_record_signed_multi` (tied to #16) remains a + separate follow-on for N-signer multi-sig. + +18. **First-class edges (Graphiti P6, deferred from §13).** Cross- + user / cross-agent provenance today rides on the chain_id + string template (`//`). Graphiti models + edges as their own chained relation with endpoint refs + + source-episode pointer, which would make federation traces + queryable. Conflicts mildly with the §8 chain-id-as-template + legibility decision and adds a parallel structure. Needs its + own scoping memo before any code; not adopted in §13. + +--- + +## 13. Patterns lifted from Cozo + Graphiti + +Cross-reference of design patterns from CozoDB (embedded Datalog +graph DB; pure Rust; HNSW + per-relation time-travel) and +Graphiti (Zep's temporal context graph; episode-based +provenance) against the trusty-postgres KB. Each pattern is +mapped to its existing anchor in the design memo or APERC tag +substrate. **All eight patterns are additive to v1 byte-shape +-- no change to `compute_record`, `build_payload`, or the +three locked tables' chained columns.** + +### 13.1 Substrate + +**P1 -- Stored-relation registry (Cozo).** A `chained_relations` +meta table where each app-level relation opts in to chaining +via a row + trigger calling `trusty_append`. Existing anchor: +`s15_insight_kb_surface_taxonomy_for_trusty_kb_adapter` +mandates "four clean mapping functions" (policy / decisions / +tags / pins) and "one entry point per concern." A registry +formalizes that mandate -- the four kb-mesh surfaces become +registry rows instead of bespoke crate-level glue. Slots into +§3 schema and §11c kb-mesh chain mapping. + +**P5 -- Episode as canonical append unit (Graphiti).** +`provenance_envelope` already is the episode; `event_type` +already names the kind (PIN / DECISION / POLICY_ROW per +s15_insight). Vocabulary alignment only -- glossary edit in +§1 and docstring edits referring to rows as "episodes." No +schema delta. + +**P6 -- First-class edges (Graphiti). DEFERRED.** See §12 +OQ #18. Heavy add; conflicts with chain-id-as-string-template +in §8; needs its own scoping memo. + +### 13.2 Temporality + +**P3 -- Per-relation time-travel (Cozo bitemporal).** Today +trusty has system-time only (chain_timestamp); supersession is +a payload-level concept (`supersedes: [prior_row_hash]` per +s15_insight). Add a companion table: + +```sql +CREATE TABLE bitemporal_index ( + chain_id TEXT NOT NULL, + row_hash TEXT NOT NULL, + valid_range tstzrange NOT NULL, + PRIMARY KEY (chain_id, row_hash), + FOREIGN KEY (chain_id, row_hash) + REFERENCES provenance_envelope (chain_id, row_hash) +); +CREATE INDEX bitemporal_index_range + ON bitemporal_index USING GIST (chain_id, valid_range); +``` + +**Critical invariant: `provenance_envelope` columns and the +`build_payload` byte-shape are NEVER modified by P3.** The +bitemporal index is a sidecar projection keyed by +(chain_id, row_hash). Conformance suite vs. trusty-sqlite stays +green. + +**P8 -- Bi-temporal supersession projection (Graphiti).** Ships +with P3. A Postgres view `current_facts` materializes +"facts whose `valid_range` contains `now()`," excluding +superseded rows. The supersession event itself remains a +chained envelope per s15_insight (never mutate prior rows); +P8 just makes the projection queryable. + +### 13.3 Retrieval (unblocks the trusty-monitor-daemon pin) + +**P4 -- HNSW vector indices on chained relations (Cozo).** +Adopt pgvector. Per chained relation that opts in (via the +registry from P1), an `embedding vector(N)` column on a +sidecar table keyed by `(chain_id, row_hash)` plus an HNSW +index. Existing anchor: pin +`../.kb-mesh/pins/trusty/trusty-monitor-daemon-chat-capture.md` +asks for "captures chat as unstructured data, auto-tags" -- +auto-tag-over-chain is exactly pgvector + HNSW. Currently no +path; P4 is the path. + +**P7 -- Hybrid retrieval contract (Graphiti).** One query API +combining vector similarity + tsvector keyword + recursive +graph traversal, with chain-verify result as a returned column +on every row. Existing anchor: `s15_insight` specifies +pre-write federation search as `kb_search(keywords) -> +Vec` across the user's schema + `received_bundles`. +P7 extends the signature to +`kb_search(keywords?, embedding?, graph_filter?) -> +Vec`. Single API surface for both the kb-mesh +adapter (s15_insight) and the trusty-monitor-daemon pin. + +### 13.4 Query API + +**P2 -- Datalog-shaped chain queries via recursive CTEs.** +Existing anchor: +`s14_insight_general_library_improvements_from_postgres_impl` +item (1) -- promote `history`, `breaks_since`, +`liveness_check`, `integrity` to `trusty-store` as a +`TrustyExt` trait. P2 reframes that refactor: the trusty-store +side ships a named-relation query library; each backend +provides the templates (recursive CTEs on Postgres, +WITH RECURSIVE on SQLite, Datalog rules on a future Cozo +adapter). Same surface, backend-specific implementation. + +### 13.5 Adoption summary + +| Pattern | KB anchor | Verdict | +|---------|-----------------------------------------------------------------|--------------------------------------| +| P1 | `s15_insight_kb_surface_taxonomy` | Adopt -- completes four-surface mandate | +| P2 | `s14_insight_general_library_improvements_from_postgres_impl` | Adopt -- reframes promote-to-store refactor | +| P3 | `s15_insight_kb_surface_taxonomy` (supersession), §11c | Adopt -- companion table only | +| P4 | pin `trusty-monitor-daemon-chat-capture`, §11c federation | Adopt -- direct unblocker | +| P5 | `provenance_envelope` already exists | Adopt -- vocabulary only | +| P6 | §8 chain-id-as-template, §11c federation | Defer -- §12 OQ #18 | +| P7 | `s15_insight_kb_surface_taxonomy` (kb_search spec), pin | Adopt -- direct extension | +| P8 | `s15_insight_kb_surface_taxonomy` (supersession decision) | Adopt -- bundles with P3 | + +### 13.6 Upstream blockers and locked-rule statement + +- **v3 signed-path gap (§12 OQ #17, escalated to HARD BLOCKER + in `s15_decision_multitenant_shared_kb_shape`)** is upstream + of P1 / P4 / P7. Federated hybrid retrieval over + `received_bundles` is meaningless without verifiable v3 + signed rows. Resolve OQ #17 before implementing §13. +- **Locked-rule check.** P1 adds a meta table. P2 ships pure + query library code. P3 adds a sidecar table keyed by + (chain_id, row_hash); never modifies `provenance_envelope`. + P4 adds a sidecar table per opted-in relation; never modifies + `provenance_envelope`. P5 is documentation. P7 is API + composition over P1 / P3 / P4. P8 is a view. **None touches + `compute_record`, `build_payload`, or the three core locked + columns.** Conformance suite vs. trusty-sqlite remains the + gate. + +--- + +## Appendix A. Lessons applied -- index + +| Lesson | Section | Where in design | +|---------------------------------|--------------------|-------------------------------------------------------------------------------| +| `insight-LCG-ARCH-01` meta | every section | "build it in v1 because retrofit is 5-10x more expensive" | +| `learn-LCG-ARCH-02` compaction | 5 | snapshot tables in v1; SNAPSHOT envelope; chain_compaction_state | +| `learn-LCG-ARCH-03` tri-state | 4 | `tip_state` column; `TipState` enum on the trait; cold-start representation | +| `learn-LCG-ARCH-06` revocation | 6 | `producer_revocation` table; REVOCATION envelope; runtime invariant test | +| `learn-LCG-ARCH-13` TCB | 1, 12 | TCB statement in README page one; postgres + libpq + driver named explicitly | + +| Trusty insight | Section | Where | +|--------------------------------------------------------|---------|------------------------------------------------------------------| +| `s14_insight_oid_anchored_chain_identity` | 8 | `producer_identity` table; OID-as-entity_id pattern | +| `s14_insight_store_trait_multi_backend_landscape` | 2 | scope of higher-level Trusty methods to re-implement | +| `s14_learn_with_txn_async_friction` | 7 | Option (a) sync `postgres` crate; trait unchanged | +| `s5_invariant_v1_byte_shape_preserved` | 3, 11 | TEXT not JSONB for raw_payload; conformance test vs sqlite | +| `project_trusty_v2_signing_design` | 6, 4 | additive-only precedent for trait additions | + +--- + +## Appendix B. Files referenced + +- `crates/core/trusty-store/src/lib.rs:22-48` -- Store trait definition. +- `crates/core/trusty-store/src/lib.rs:54-115` -- LastRow, TipRow, StoredRow, RowInsert, TipUpsert, KvCrossCheckReport. +- `crates/adapters/trusty-sqlite/src/store.rs:1-955` -- reference adapter implementation including all higher-level Trusty methods. +- `crates/adapters/trusty-sqlite/src/schema.rs:1-49` -- reference schema (three tables, locked). +- `APERC/tags/s14_insight_oid_anchored_chain_identity.toml` -- OID-anchored model. +- `APERC/tags/s14_insight_store_trait_multi_backend_landscape.toml` -- four-backend landscape; method scope. +- `APERC/tags/s14_learn_with_txn_async_friction.toml` -- with_txn options. +- `../legal-chaingraph/APERC/KB/architecture_antipatterns.md` -- the 18-lesson source. +- `../legal-chaingraph/APERC/tags/learn-LCG-ARCH-02.toml` -- compaction. +- `../legal-chaingraph/APERC/tags/learn-LCG-ARCH-03.toml` -- tri-state null. +- `../legal-chaingraph/APERC/tags/learn-LCG-ARCH-06.toml` -- revocation. +- `../legal-chaingraph/APERC/tags/insight-LCG-ARCH-01.toml` -- meta-principle. +- `CLAUDE.md` -- project rules; locked surfaces; library-not-platform discipline. +- `docs/payload-v2.md` -- v2 payload signing design. +- `docs/threat-model.md` -- existing trusty threat model (cross-reference for TCB). + +End of memo. diff --git a/docs/twin-coupling.md b/docs/twin-coupling.md new file mode 100644 index 0000000..cff22d4 --- /dev/null +++ b/docs/twin-coupling.md @@ -0,0 +1,92 @@ +# Twin Coupling -- Data Convention + +## Problem + +A digital-twin computation ingests readings from one or more sensor +chains and produces a derived result. The result is appended to its own +trusty chain (separate `entity_id`). How does an external verifier +confirm which sensor rows fed the computation? + +## Convention + +The twin row carries a `source_refs` array inside its `data` field: + +```json +{ + "computation": "rolling_avg_temp_c", + "result": 21.37, + "source_refs": [ + {"chain_id": "chain_sensor-001_...", "sequence": 0, "row_hash": "ae1cfe445cda1f6d"}, + {"chain_id": "chain_sensor-001_...", "sequence": 1, "row_hash": "dee9856d170787cf"}, + {"chain_id": "chain_sensor-001_...", "sequence": 2, "row_hash": "26cd125f051243cf"} + ] +} +``` + +Each entry is: + +| Field | Source | Purpose | +|------------|---------------------------------|--------------------------------------------| +| `chain_id` | `AppendResult.chain_id` | Identifies which sensor chain | +| `sequence` | `AppendResult.sequence` | Row position within that chain | +| `row_hash` | `AppendResult.row_hash` | Tamper-evident fingerprint of the row | + +Because `data` is already part of the payload that `row_hash` commits +to, the `source_refs` array is itself hash-protected. Altering any +reference after the fact invalidates the twin row's `row_hash` and +fails `verify_chain`. + +## Verification flow (two ProofBundles) + +An external verifier needs no database access -- only two +`ProofBundle` blobs exported via `Trusty::export_bundle`: + +1. `sensor_bundle.verify(&sip_key)` -- confirms the sensor chain is + internally consistent. +2. `twin_bundle.verify(&sip_key)` -- confirms the twin chain is + internally consistent. +3. Walk twin rows; for each `source_refs` entry, look up the matching + row in `sensor_bundle.rows` by `chain_id` + `sequence` and assert + `row_hash` matches. + +If all three checks pass, the verifier knows: +- The sensor readings are unaltered. +- The twin computation row is unaltered. +- The twin row commits to exactly those sensor row_hashes at write time. + +See `crates/trusty-sqlite/examples/twin_coupling.rs` for a runnable +end-to-end demo. + +## Design decisions + +**Why `data` field, not a new `break_reason` variant.** +`break_reason` marks loss-of-continuity (tamper, gap, domain mismatch). +A twin derivation is a planned event, not an anomaly. Mixing the two +pollutes `breaks_since()` as an alerting surface and requires callers +to filter by break kind. + +**Why not a separate `twin_reference` table.** +A join table outside the hash surface cannot be verified from a +ProofBundle alone. The verifier would need database access, breaking +the self-contained export contract. `source_refs` in `data` is +verifiable anywhere the bundle travels. + +**What this does NOT prove.** +The convention proves the twin row committed to those row_hashes at +write time. It does not prove the computation is correct (the twin +operator could have computed a wrong average and still referenced the +right sensor rows). Computation correctness is out of scope for trusty; +it belongs to a separate audit or reproducibility layer. + +## Multi-sensor / multi-chain refs + +`source_refs` can mix `chain_id` values from different sensor entities. +The verifier simply exports a ProofBundle per sensor entity and checks +each ref against the appropriate bundle. + +```json +"source_refs": [ + {"chain_id": "chain_sensor-001_...", "sequence": 5, "row_hash": "..."}, + {"chain_id": "chain_sensor-002_...", "sequence": 3, "row_hash": "..."} +] +``` diff --git a/docs/vocabulary.md b/docs/vocabulary.md new file mode 100644 index 0000000..6ecf7cb --- /dev/null +++ b/docs/vocabulary.md @@ -0,0 +1,140 @@ +# Vocabulary crosswalk + +Our words, the standard words, and which is which. Written so an +assessor can follow their own vocabulary into this system, and so we can +keep a term where ours is genuinely sharper without that being an +obstacle. + +This is the ordinary compliance practice of a **mapping**: NIST +publishes the same thing between 800-171 and 800-53. The value is not in +picking a winner. Picking one would be wrong anyway — XCCDF is +configuration scanning, C2PA is media provenance, in-toto is software +supply chain, and none was built for a P2P data mesh. + +**On sourcing.** Rows marked ✓ were checked against the specification +during this session. Rows marked ~ are from working knowledge and want +confirming before they go in front of an assessor. A term cited wrongly +is worse than a term not cited, because a citation reads as authority. + +--- + +## 1. Where we already use the standard word + +Nothing to do here. Listed so a reader can stop looking. + +| ours | standard | source | +|---|---|---| +| `principal` | principal | Kerberos, AWS IAM, Java security, NIST RBAC ~ | +| `delegation` | delegation | NIST RBAC, OAuth 2.0 ~ | +| `scope` | scope | OAuth 2.0 ~ | +| `grant` | grant | OAuth 2.0, XACML ~ | +| `Tier::sql_verb()` → SELECT/INSERT/UPDATE/DELETE | the SQL verbs | ISO SQL ✓ | +| `canonical_json()` | JSON Canonicalization Scheme | RFC 8785 ✓ | +| Ed25519, SHA-256, SipHash-2-4 | same | RFC 8032, FIPS 180-4 ✓ | + +--- + +## 2. Where we invented a word and a standard exists + +This is the list that matters. + +### 2.1 `Stance` — the verdict + +**Adopt the standard.** XCCDF is NIST's own, it is what a SCAP-literate +assessor reads, and our three values map exactly onto three of its nine. + +| ours | XCCDF `rule-result` ✓ | +|---|---| +| `Clean` | `pass` | +| `Violated` | `fail` | +| `Unverifiable` | `unknown` | + +XCCDF's full set is `pass`, `fail`, `error`, `unknown`, `notapplicable`, +`notchecked`, `notselected`, `informational`, `fixed`. + +**The one reservation.** `Unverifiable` says *why* nothing is known — +the evidence could not be checked — where `unknown` covers that and +several other cases. Ours is more precise; theirs is recognisable. For a +verdict an outsider reads, recognisable wins. + +**Prior art worth knowing:** OCSP answers the same shape with `good` / +`revoked` / `unknown` ~. Three values, and the third is "we cannot say" +— the same design decision, reached independently, which is a good sign +the three-value shape is right. + +### 2.2 `Annotation` — a claim about a datum + +| ours | C2PA ✓ | in-toto ✓ | +|---|---|---| +| `Annotation` | **assertion** — a statement the signer makes about the asset | **predicate** | +| the signed row carrying it | **claim** — the signed container grouping assertions | **Statement** (`_type`, `subject`, `predicateType`, `predicate`) | +| `ProofBundle` | **manifest** — assertions + claim + signature as one verifiable unit | — | +| `content_tag` | the asset | **subject** | + +**Keep ours, and say what it maps to.** C2PA's *assertion* is the closest +single word, and the C2PA shape is genuinely similar: individual +statements, grouped and signed together. But their *claim* and *manifest* +have media-provenance meanings that would mislead here, and adopting one +word of a three-word system imports the confusion without the benefit. + +The mapping is the deliverable, not a rename. + +### 2.3 The hash names + +| ours | standard | note | +|---|---|---| +| `row_hash` | Merkle leaf hash ~ | ours is a keyed MAC, theirs usually is not | +| `anchor_hash` | Signed Tree Head, RFC 6962 ~ | needs checking before it goes in a filing | + +**Do not rename.** These are frozen byte shapes; the field names feed +hashed payloads, and renaming a key that feeds a hash silently breaks +every artifact already signed under the old spelling. See the lesson +`freeze-tokens-for-hashed-keys`. + +### 2.4 Smaller ones + +| ours | standard | verdict | +|---|---|---| +| `producer registry` | PKI **issuer** ~; C2PA **claim generator** ✓ | keep — ours is a registry of keys, not an issuer of certificates | +| `Tier` | NIST RBAC **operation** ~ | keep — ours is a *damage* taxonomy that happens to map 1:1 onto the SQL verbs, which "operation" does not convey | +| `content_tag` | in-toto **subject** ✓; content-addressing **CID** ~ | keep, map to `subject` | + +--- + +## 3. Genuinely ours + +No standard equivalent found. These are the ideas worth defending as +novel rather than translating. + +- **chain-break-as-event.** A monitored condition failing is recorded as + a first-class row rather than raised as an error, so the violation is + evidence rather than something a log swallowed. +- **One chain per `(data, node)`.** Nobody writes to anybody else's + chain, so a node cannot forge a record it did not make, and the picture + is recovered by merging rather than by agreeing. +- **Tier → RLS with no translation layer.** The four tiers *are* the + four SQL verbs, so a grant compiles to a row-level-security predicate + directly. The translation layer is where an application's idea of a + permission and the database's drift apart. +- **`rows_valid` / `fully_signed` / `continuous` / `intact()`.** Four + separate questions instead of one boolean, because "did the signatures + check" and "was anything unsigned" and "was a break recorded" have + different answers and different consequences. + +--- + +## 4. What to do with this + +| | | +|---|---| +| **Rename now** | `Stance` → XCCDF's `pass` / `fail` / `unknown`. Days old, one consumer, cheap. | +| **Map, don't rename** | `Annotation`, `ProofBundle`, `content_tag`, `Tier`, `producer registry`. Load-bearing across crates and, for some, already in a committed migration. | +| **Never rename** | `row_hash`, `anchor_hash`, and any other key feeding a hashed payload. | +| **Defend as novel** | section 3. | + +**Sources checked this session:** [NIST XCCDF 1.2 +schema](https://csrc.nist.gov/schema/xccdf/1.2/xccdf_1.2.xsd) · +[OpenSCAP manual](https://static.open-scap.org/openscap-1.2/oscap_user_manual.html) · +[C2PA specification](https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html) · +[in-toto Statement +layer](https://github.com/in-toto/attestation/blob/main/spec/v1/statement.md)