feat(facts): add ADR 0002 - fact provenance as mandatory write path

Stacks on !595 (ADR 0001, fact-provenance and truth-maintenance layer). Please review/merge !595 first -- this MR's diff includes !595's changes because its target branch (feat/fact-provenance) doesn't exist on this project (only on my fork), so GitLab won't let me target it directly. Once !595 merges to master, I'll rebase this branch and the diff will shrink to just this MR's own changes.

Extends ADR 0001 to make provenance-fact assertion mandatory for untrusted/parsed-content ingestion paths (Tier A), rather than an opt-in tool surface -- and implements it.

Motivation

The memory_ingest_file incident: 14 mis-parsed rows from docs/BOOTSTRAP_MEMORIES.md's fenced JSON blocks went undetected because no write path asserted provenance for them. ADR 0001's fact-provenance/truth-maintenance layer is correct and tested, but scoped as an opt-in tool surface (memory_observe/memory_retract/memory_why) alongside the episodic memories table -- every other write path (memory_remember, memory_ingest_file, memory_import_memories, daemon observers, bulk_ingest, etc.) inserts directly into memories with zero interaction with the fact layer. Cleanup of the incident required a human to notice, investigate, recall, and manually archive each bad row -- exactly the manual burden ADR 0001 exists to eliminate, but couldn't, because the write path was never in scope.

Design (docs/adr/0002-fact-provenance-as-mandatory-write-path.md)

  • Scope: tiered rollout (Tier A mandatory now / Tier B tracked follow-up / Tier C explicitly out of scope). GitLab entities are Tier A, not a lesser bookkeeping tier -- a merged MR or closed issue is exactly ADR 0001's own has_status(mr_123, merged) worked example.
  • Enforcement point: new insert_memory_with_provenance method (not a signature change to insert_memory, which has ~226 call sites) that Tier A callers use instead.
  • Confidence sourcing: per-extractor, not a single global heuristic.
  • Parser fix: MarkdownParser JSON-block pre-pass, in BOTH runtimes -- the direct fix for the motivating incident.
  • Refutation automation: new memory_why_memory convenience tool + the existing retract_facts_for_source_episode cascade now meaningful for every Tier A memory.
  • Boundary with ADR 0001's contradiction-escalation policy (no overlap).

Implementation

  • SqliteStorage::insert_memory_with_provenance + assert_ingestion_provenance (memory-core): asserts a self-referential (memory:<id>, ingested_via, <extractor>) base fact after insert; idempotent on dedup hits.
  • ProvenanceDescriptor::trusted/untrusted + reserved provenance_predicate namespace (ingested_via/parsed_from/derived_from_memory).
  • MarkdownParser JSON-block pre-pass in both Rust and Python: fenced ```json blocks with a valid category become fully-typed memories instead of falling through to generic fact-chunking. Confidence-gated (1.0 clean parse, 0.4 unhandled JSON-shaped block) as a safety net. Also fixes a latent "no sections at all" fallback bug the pre-pass exposed in both parsers (it used to re-chunk the raw file/section content instead of the post-pre-pass content, double-storing extracted blocks).
  • Tier A call sites wired to insert_memory_with_provenance: memory_ingest_file, memory_import_memories (per-record confidence: 1.0 with what/why/learned, 0.6 bare minimum), the daemon's OpenCode-DB and Claude-Code observers (confidence 1.0, structured/trusted), and the bulk_ingest GitLab summarizer (DuoSummarizer::with_fact_provenance, opt-in like BulkIngestQuery::with_eval -- asserts a parsed_from fact anchoring a stored key_decisions extraction at confidence 0.7/untrusted, and a has_status fact at confidence 1.0/gitlab when the entity's state is known, subjects like gitlab_issue:g/p#42).
  • New memory_why_memory(memory_id) MCP tool: one call to see a memory's own provenance proof plus any facts derived from its content, instead of the incident's manual recall+inspect+archive sequence.
  • README + mcp_tools.json + tool-count assertions updated (100 -> 101 static schemas, 104 -> 105 dispatchable).

Deferred: the Serena observer named in Tier A's list ships on the separate feat/serena-memory-integration MR (!605), not yet on this branch -- wiring it to insert_memory_with_provenance is a fast-follow once that merges. The one-time backfill migration for pre-existing rows and the proactive low_confidence_ingest boot-context concern are also explicitly deferred (advisory-only follow-ups).

Testing

  • 5 new fact_provenance_invariant integration tests (memory-core), 4 new DuoSummarizer tests, 4 new memory_why_memory tests, 2 new provenance tests (lib_tests/portability), 6 new Rust + 6 new Python parser JSON-block tests, plus 1 corrected pre-existing Python test that was pinning the old (buggy) re-read-from-disk behavior.
  • Full suites green: 2225 memory-core + 1072 memory-server Rust tests (only the known pre-existing subconscious_parity::budget_timeout_returns_empty sandbox-timing flake); 5640 Python tests (only the 10 known pre-existing sandbox/environment flakes). cargo clippy --all-targets + cargo fmt --check + ruff check + ruff format --check: all clean.

Merge request reports

Loading
Loading