Ran Phase A's classifier against the frozen EMB-7KJ4VR4G archive. Since the
fleet shares one devbox image this is a reasonable prior for tor-ms22 and
MBP-M1-2020:
mined (source_file set) 15949 98.9% re-mine, never replay
diary entries 116 0.7% replay + §7.6 suffix skip
agent-authored drawers 60 0.4% replay, idempotent
KG open facts 34 -- server guard dedupes
KG closed facts 0 -- nothing to do
Two consequences that shrink this project: the replay-only surface is 176
records, not thousands, so the writer is a small job and the §7.6 diary guard
treated as the blocker governs 116 records; and there are ZERO closed KG facts,
so the unguarded-closed-fact gap is real in the code but empty in the data.
filed_at spread in the same archive -- 12 in May, 52 in June, 15174 in July, 887
in August -- is the concrete argument for the history-preserving regime.
Two corrections to the first draft:
1. CLOSETS were omitted entirely. mempalace_closets is a second Chroma
collection (1560 rows, ~10% of the palace) and there is NO MCP tool that
writes one -- mcp_server.py exposes only _purge_source_closets, and
closet_llm.py says outright that regex closets are always created by the
miner. So closets cannot be replayed even in principle; they return only by
re-mining. Same category as hallways/known_entities/palace-graph, and it
resolves itself for the ~99% that gets re-mined anyway.
2. Census gotcha: chroma.sqlite3 holds BOTH collections, so an embeddings-wide
query over-counts by ~10%. Must join segments->collections and keep
mempalace_drawers. Doing so reconciles exactly: 14,778 = the 14,777 seeded to
the primary + the one known post-snapshot chunk.
Also drops the earlier "438 of 14,829" figure, which conflated central's
no_source count (which includes its own later agent writes) with the archive's
actual replay surface.
RFC 001 §4.4 designs a join as "idempotent replay of local history", but no
replay tool exists and the two joins done so far were whole-palace file copies
that cannot merge. tor-ms22 and MBP-M1-2020 each hold a palace that needs to
reach the primary, so this scopes the tool.
The finding that drives the design: MCP replay CANNOT preserve filed_at.
add_drawer stamps it server-side (mcp_server.py:2580) with no override, and
diary_write builds its own now()-based id. A pure MCP replay would therefore
collapse months of history into the join instant -- on a palace whose value IS
its chronology, and where list_drawers filters on filed_at. RFC 001 does not
mention this. kg_add is the exception: it takes valid_from/valid_to, so fact
windows survive.
Hence two regimes, and a recommendation to build the history-preserving one
first: direct disk write on synlig (preserves ids + filed_at, bypasses the
server guards, needs the service stopped) vs MCP replay (guards work, timestamps
flatten). migrate.py is already a working model for the direct path -- it reads
drawers straight from the palace sqlite and re-adds them preserving ids,
documents and metadata.
Also concretised: every dedup key verified against mempalace 3.6.0 source rather
than assumed --
- agent-authored drawers: sha256(wing|room|content)[:24], fully deterministic
- mined drawers: sha256(source_file|chunk_index)[:24], PATH-dependent, so
replay duplicates instead of deduping -> re-mine on synlig, do not replay
- diaries: full ids never repeat (wall-clock component); match the
sha256(entry)[:12] suffix only -- this is §7.6
- KG closed facts: no server guard at all (it is scoped to valid_to IS NULL)
- hallways/known_entities/palace-graph are built at MINE time, so replayed
drawers arrive with no co-occurrence edges and traverse under-reports
Scope is phased so the useful half lands first: Phase A is a read-only census
that sizes the job and cannot break anything, and is the piece to run the moment
tor-ms22 is reachable. The replay-only surface is small -- only 438 of 14,829
drawers on the primary have no source_file.
Method note recorded in the doc: an attempt to gather these mechanisms via a
delegated subagent returned confident, fabricated code for a package path that
does not exist on this machine. Everything here is cited to file:line and was
re-read directly.
Joakim Persson