feat(facts): add ADR 0002 - fact provenance as mandatory write path
Stacks on !595 (ADR 0001, fact-provenance and truth-maintenance layer). Please review/merge !595 first -- this MR's diff includes !595's changes because its target branch (feat/fact-provenance) doesn't exist on this project (only on my fork), so GitLab won't let me target it directly. Once !595 merges to master, I'll rebase this branch and the diff will shrink to just this MR's own changes.
Extends ADR 0001 to make provenance-fact assertion mandatory for untrusted/parsed-content ingestion paths (Tier A), rather than an opt-in tool surface -- and implements it.
Motivation
The memory_ingest_file incident: 14 mis-parsed rows from docs/BOOTSTRAP_MEMORIES.md's fenced JSON blocks went undetected because no write path asserted provenance for them. ADR 0001's fact-provenance/truth-maintenance layer is correct and tested, but scoped as an opt-in tool surface (memory_observe/memory_retract/memory_why) alongside the episodic memories table -- every other write path (memory_remember, memory_ingest_file, memory_import_memories, daemon observers, bulk_ingest, etc.) inserts directly into memories with zero interaction with the fact layer. Cleanup of the incident required a human to notice, investigate, recall, and manually archive each bad row -- exactly the manual burden ADR 0001 exists to eliminate, but couldn't, because the write path was never in scope.
Design (docs/adr/0002-fact-provenance-as-mandatory-write-path.md)
- Scope: tiered rollout (Tier A mandatory now / Tier B tracked follow-up / Tier C explicitly out of scope). GitLab entities are Tier A, not a lesser bookkeeping tier -- a merged MR or closed issue is exactly ADR 0001's own
has_status(mr_123, merged)worked example. - Enforcement point: new
insert_memory_with_provenancemethod (not a signature change toinsert_memory, which has ~226 call sites) that Tier A callers use instead. - Confidence sourcing: per-extractor, not a single global heuristic.
- Parser fix:
MarkdownParserJSON-block pre-pass, in BOTH runtimes -- the direct fix for the motivating incident. - Refutation automation: new
memory_why_memoryconvenience tool + the existingretract_facts_for_source_episodecascade now meaningful for every Tier A memory. - Boundary with ADR 0001's contradiction-escalation policy (no overlap).
Implementation
SqliteStorage::insert_memory_with_provenance+assert_ingestion_provenance(memory-core): asserts a self-referential(memory:<id>, ingested_via, <extractor>)base fact after insert; idempotent on dedup hits.ProvenanceDescriptor::trusted/untrusted+ reservedprovenance_predicatenamespace (ingested_via/parsed_from/derived_from_memory).MarkdownParserJSON-block pre-pass in both Rust and Python: fenced```jsonblocks with a validcategorybecome fully-typed memories instead of falling through to generic fact-chunking. Confidence-gated (1.0 clean parse, 0.4 unhandled JSON-shaped block) as a safety net. Also fixes a latent "no sections at all" fallback bug the pre-pass exposed in both parsers (it used to re-chunk the raw file/section content instead of the post-pre-pass content, double-storing extracted blocks).- Tier A call sites wired to
insert_memory_with_provenance:memory_ingest_file,memory_import_memories(per-record confidence: 1.0 with what/why/learned, 0.6 bare minimum), the daemon's OpenCode-DB and Claude-Code observers (confidence 1.0, structured/trusted), and thebulk_ingestGitLab summarizer (DuoSummarizer::with_fact_provenance, opt-in likeBulkIngestQuery::with_eval-- asserts aparsed_fromfact anchoring a storedkey_decisionsextraction at confidence 0.7/untrusted, and ahas_statusfact at confidence 1.0/gitlab when the entity's state is known, subjects likegitlab_issue:g/p#42). - New
memory_why_memory(memory_id)MCP tool: one call to see a memory's own provenance proof plus any facts derived from its content, instead of the incident's manual recall+inspect+archive sequence. - README +
mcp_tools.json+ tool-count assertions updated (100 -> 101 static schemas, 104 -> 105 dispatchable).
Deferred: the Serena observer named in Tier A's list ships on the separate feat/serena-memory-integration MR (!605), not yet on this branch -- wiring it to insert_memory_with_provenance is a fast-follow once that merges. The one-time backfill migration for pre-existing rows and the proactive low_confidence_ingest boot-context concern are also explicitly deferred (advisory-only follow-ups).
Testing
- 5 new
fact_provenance_invariantintegration tests (memory-core), 4 newDuoSummarizertests, 4 newmemory_why_memorytests, 2 new provenance tests (lib_tests/portability), 6 new Rust + 6 new Python parser JSON-block tests, plus 1 corrected pre-existing Python test that was pinning the old (buggy) re-read-from-disk behavior. - Full suites green: 2225 memory-core + 1072 memory-server Rust tests (only the known pre-existing
subconscious_parity::budget_timeout_returns_emptysandbox-timing flake); 5640 Python tests (only the 10 known pre-existing sandbox/environment flakes).cargo clippy --all-targets+cargo fmt --check+ruff check+ruff format --check: all clean.