docs: the convos miner does check mtime — finish a correction that stopped half-way
ARCHITECTURE.md and README.md still carried the claim from954c3f2(initial commit) that the convos miner "keys on source_file path alone (convos miner doesn't check mtime)", and told the operator to delete the staging dir to force a re-mine.29e660ecorrected exactly that claim in AGENTS.md and SKILL.md — and missed these two files, so the repo has been documenting both behaviours at once ever since. Two files said mtime is checked, two said it is not. Ground truth, read off the deployed mempalace 3.7.1 rather than inferred: convo_miner.py:657 calls file_already_mined(..., check_mtime=True) inside mine_lock(source_file), and palace.py:1430 re-mines when no drawers exist for the source_file, when the stored normalize_version predates the current schema, or when the mtime differs. On a mismatch the file's existing drawers are purged (_source_file_delete_ids -> collection.delete) before refiling, so a changed transcript is replaced rather than doubled. The docstring names the case outright: transcripts are not assumed immutable, since a session keeps appending to its own file while active and /compact or /clear can rewrite one in place. The stale advice was not merely out of date, it was expensive. "Delete the staging dir to force a re-mine" is the one gesture that re-keys dedup: the staged path IS the key, so a stage that is wiped or recreated elsewhere makes the palace refile a whole wing as duplicates instead of replacing it. The docs now say so, name `touch` as the non-destructive way to force a single session, and record why staged copies must carry the source's mtime — with the corollary that an old mtime in a stage or a remote inbox says nothing about when the file was shipped, so a ship is judged by the feeder's log instead. Sample output blocks quoting "(dedup by source_file)" are deliberately left verbatim: that is what bin/mempalace-session:426 and bin/mempalace-pi-session:857 actually print. Tightening the wrappers' own wording is a separate change, because the samples have to move with it.
This commit is contained in:
@@ -460,7 +460,7 @@ mempalace-session --help
|
||||
- Tool outputs → `tool_result` blocks in a follow-up human message, folded back into the assistant turn by the mempalace normalizer.
|
||||
- `step-start` / `step-finish` parts are dropped as noise. `reasoning` parts are kept with a `[reasoning]` prefix.
|
||||
|
||||
**Dedup:** staging at `<palace-root>/opencode-stage/<wing>/` (override: `$MEMPALACE_SESSION_STAGE`) with deterministic per-session filenames (`<slug>_<id>.jsonl`). The convos miner keys on `source_file`, so re-runs skip unchanged sessions. To force re-mining a session, delete its JSONL from the staging dir.
|
||||
**Dedup:** staging at `<palace-root>/opencode-stage/<wing>/` (override: `$MEMPALACE_SESSION_STAGE`) with deterministic per-session filenames (`<slug>_<id>.jsonl`). The convos miner keys on `source_file` **and** `mtime`, so re-runs skip unchanged sessions, while a session that has grown or been rewritten in place is purged and refiled automatically — there is nothing to force. What must stay stable is the staged *path* itself: wiping the staging dir re-keys dedup rather than refreshing it, so those sources land as duplicates instead of replacing the drawers they already have.
|
||||
|
||||
**`--dry-run` is dedup-aware.** Each session is tagged `[NEW]` (would be filed) or `[SKIP]` (already in the palace), and the summary breaks down the count:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user