From cdb6fc0950587daed448ba8c634ebf50aed0f498 Mon Sep 17 00:00:00 2001 From: Joakim Persson Date: Thu, 27 Aug 2026 14:18:55 +0200 Subject: [PATCH] changelog: name what the floating toolkit ref will pull into the next tag MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MEMPALACE_TOOLKIT_REF=main floats and docker-publish.yml resolves it to a SHA at build time, so toolkit main moving 5b8d78f -> f0bffd1 (10 commits) ships in the next tagged image whether or not this repo has a commit. v1.8.9 adopted the rule after that shape bit twice (553d865, 5b8d78f): name the behaviour change BEFORE tagging. This is that rule obeyed rather than re-learned — the work was pushed to toolkit main earlier today and this entry was missing, which is exactly the gap that caused a cross-host misattribution in v1.8.7. Contents: the feeder-side secret scrubber (three tiers, T3 report-only after a measured 403 -> 29 false-positive calibration on 52 MB of real transcripts, fail closed); the symlink near-miss that fail-closed would have turned into a fleet-wide silent memory outage at bake time, caught before tagging; the mailbox work (hlc owed-set join, queued-until-next-turn note, explicit notify protocol modes, tmux path documented unverified); and the documentation set (RFC 003, fleet-memory.md, secret-hygiene.md). Also states what remains unscrubbed: the opencode bridge write path and the unbuilt server-side layer. No tag pushed — per the release protocol, no tag means no build. --- CHANGELOG.md | 112 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 89c5517..8b99f48 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,110 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`). ## Unreleased +**No tag yet, so nothing is built.** This section exists because +`MEMPALACE_TOOLKIT_REF=main` floats: `docker-publish.yml` resolves it to a +concrete SHA at build time, so whatever is on toolkit `main` when the next tag is +pushed ships in that image whether or not this repo has a commit. That is the +rule v1.8.9 adopted after `553d865`/`5b8d78f` shipped undocumented twice — *name +the behaviour change before tagging, not after* — and this entry is that rule +being obeyed rather than re-learned. + +**`mempalace-toolkit` main moves `5b8d78f` → `f0bffd1`** (10 commits, ~1800 +insertions / ~500 deletions). No pi-devbox commit implements any of it. + +### Transcripts get scrubbed before they are staged (`3d47937`, `836e35b`, `f0bffd1`) + +`bin/mempalace_redact.py`, called from `mempalace-pi-session` at the moment the +staged transcript is written — one hook covering both transports, because local +mode mines that file and remote mode rsyncs the same bytes. + +- **Why it exists, measured rather than argued.** One leaked bearer token had + reached 3 drawers, 13 feeder inbox files across all three devices, and 10 local + files spanning 10 days, from an agent printing an env var while debugging. A + second sweep then found `GITEA_ACCESS_TOKEN` in 2 more drawers and + `GITEA_EGL_ACCESS_TOKEN` in 3. This is routine agent behaviour, so the fix + belongs in the pipeline, not in discipline. +- **Detection is name-anchored, never entropy-anchored.** A palace's own primary + keys — drawer ids, chunk ids, event ids, replica ids, HLCs, commit SHAs — *are* + its high-entropy strings, so an entropy detector eats the memory it protects, + silently and unrecoverably. Three tiers instead: T1 literal values from this + process's env whose name says secret (zero false positives by construction); + T2 vendor shapes (`ghp_`, `glpat-`, `xox*-`, `sk-`, `AKIA`, JWT, PEM, URL + credentials, `Authorization:`); T3 key-name-says-secret. +- **T3 is report-only, because the false-positive rate was measured.** On 52 MB + of real fleet transcripts T3 fired 403 times, mostly `${VAR}` interpolation in + compose files, TypeScript identifiers, a *type annotation* + (`credentials: Credentials`), an IPA attribute holding a date + (`krbPasswordExpiration`), AAAK diary shorthand, and terminal output following + an ssh `Password:` prompt. With interpolation/code-context/key-suffix guards the + enforced count fell **403 → 29** on the same corpus. `MEMPALACE_REDACT_STRICT=1` + makes T3 enforce. +- **Operational shape.** Fail closed — no redactor, no staging (`exit 3`), + overridable with `MEMPALACE_FEED_ALLOW_UNSCRUBBED=1`. Every run prints a count + *including* `0 redaction(s)`, because silence is indistinguishable from a + scrubber that never ran. Findings carry rule, label, length and `sha256[:8]` — + never the value. + +**Near-miss this image would have shipped, caught before tagging (`f0bffd1`).** +The image installs `/usr/local/bin/mempalace-pi-session` as a **symlink** into +`/opt/mempalace-toolkit/bin`, and `${BASH_SOURCE[0]}` reports the invoked path, +not the target — so the sibling-module lookup resolved to `/usr/local/bin`, the +redactor was absent, and fail-closed did as instructed: `[FATAL] ... refusing to +stage`. Measured side by side, the symlinked invocation FATALed while the direct +one scrubbed 40 findings. **At the next bake that would have stopped every +feeder tick on every device — a silent fleet-wide memory outage, worse than the +leak the scrubber prevents.** Fixed by chasing the symlink chain in portable +shell (`readlink -f` avoided: GNU/newer-BSD only, and this script also runs +directly on macOS hosts) with colon-separated fallback candidates. Verified via +the symlink, the direct path, and a second-hop symlink. General lesson: fail-closed +converts "module not found" into an outage, which makes the module lookup +load-bearing infrastructure that must be tested through the invocation path the +fleet actually uses — not the convenient one from a checkout. + +### The mailbox becomes explainable and mesh-safe (`bfe9c5c`, `a92c75d`, `e917662`, `ecc2a9c`) + +- **Owed-set derivation joins on `hlc`, not `seq`** (`bfe9c5c`). `seq` is a + replica-local arrival counter — the same event is `#7` in one database and `#12` + in another — so a second replica would let already-answered asks resurrect. + `hlc` is immutable and replicated, fixed-width, so string comparison *is* causal + comparison. A safe no-op on today's single replica (verified: the positive-control + pair orders identically under both keys), correct once a mesh exists. +- **Delivered text now says it is queued** (`a92c75d`). Delivery uses `steer` + with no `triggerTurn`, and the poll fires on `agent_settled`, so nothing wakes + the model — a delivered ask sits until a human starts the next turn. Measured + case: a directed report sat unread for 2.5 hours. The note explains the agent is + not ignoring the ask, it is not running. +- **`MEMPALACE_MAILBOX_NOTIFY` gains explicit `=kitty` / `=osc777` modes** + (`e917662`). Terminal autodetection inside a container is not unreliable, it is + *blind*: `docker exec` forwards neither `KITTY_WINDOW_ID` nor `TERM_PROGRAM`, and + `TMUX` is unset because tmux runs on the host. Verified on a live process: + `TERM=xterm-256color` and nothing else. +- **The terminal path through tmux is documented as UNVERIFIED** (`ecc2a9c`). + Test sequences written to the pty produced no notification on a remote client; + tmux likely drops unknown OSC types without `allow-passthrough`, and multi-client + routing (one ask pinging every attached client) is an open question. + +### Documentation (`e1cc759`, `982b001`, `d4d8bb6`, `d2764bf`) + +- **RFC 003, the coordination-log spec the code had been citing all along** — it + did not exist anywhere. 11 sections, retrospective against mempalace 3.8.0 + `logstream.py`, incl. owed-set derivation, ten dogfooded landmines and seven open + decisions. Non-obvious findings: `event_append` has **no** idempotency guard on + the write path (verify-before-retry; the replication path *is* guarded), + coordination tools are exempt from both palace locks by design, + `GET /logstream/events` **never existed** in 3.8.0 (not proxy-blocked), and + `mempalace sync` never touches the logstream — the log is permanent and unbounded. +- **`docs/fleet-memory.md`**, operator-facing: five storage types, a decision tree, + latency expectations (~2–5 min live session; next session while offline), + broadcast exclusion by design, fan-out, and the search-before-answer / + diary-at-session-end / verify-don't-retry habits. +- **`docs/secret-hygiene.md`**, incl. the tier definitions, the measured FP data, + stated false negatives, and the three server-side call sites (specified, not + built — tier 2 only there, since the hub cannot see a client's env). +- Phase 1 exposure record moved to the private fleet repo with a moved-note stub; + retention direction for the unbounded log (logrotate-style: never rotate + still-owed events, rotation invalidates held cursors, archive-verify-delete). + ### The other memory system finally gets explained — `docs/observational-memory.md` `pi-observational-memory` has been baked for several releases and described in @@ -86,6 +190,14 @@ browser and read back as an image. Recorded because it generalises: `mermaid.parse()` proves syntax and says nothing about layout, so a diagram is unverified until someone has looked at it. +### Not covered by any of this + +The opencode bridge is a separate write path the feeder hook never sees, and the +server-side layer is unbuilt — so a secret typed straight into `add_drawer`, or +staged by a non-pi client, still lands unscrubbed. + +--- + --- ## v1.8.9 — 2026-08-26