changelog: release v1.8.10 — deploy the scrubber, and name the check that must not be skipped

Audited every component against upstream rather than assuming. Only two moved
since v1.8.9:

- mempalace-toolkit 5b8d78f -> b2b50af (ours): feeder scrubber, the symlink fix
  that stops it refusing to stage, hlc owed-set join, queued-delivery note,
  explicit MEMPALACE_MAILBOX_NOTIFY protocol modes, RFC 003 + fleet-memory docs.
- pi-studio v0.9.48 -> v0.9.52 (upstream, studio variant only): 22 commits, all
  additive — PDF previews, hideable header, contextual side questions. No removals
  or renames. Its pi floor is >=0.84.3 against our exact 0.84.3 pin: satisfied,
  zero headroom, named as a watch item because the next floor bump breaks the
  studio job only, after core has already published.

Verified unchanged, with dates proving they predate the v1.8.9 bake: pi 0.84.3 (=
npm latest), mempalace 3.8.0 (= PyPI latest), pi-atelier v0.8.2, playwright 1.62.1
(no drift this cycle despite floating on latest), pi-fork bf702b4, pi-obsmem
ce9fc98, pi-toolkit 0e1369e, pi-extensions 2022887. No breaking changes anywhere,
so everything can ship in one tag.

The reason for tagging now is not features. The scrubber has been live on exactly
ONE device since this morning; every other device has kept staging unscrubbed
transcripts into a shared palace, and cleanup after the fact is manual redaction
(done twice today, with freed-page residue left behind by choice).

The entry leads with the acceptance check instead of burying it, because this
release's worst failure is silent: the feeder is fail-closed, so a packaging or
path mistake stops the fleet's memory feed and nothing complains — refusing to
stage looks exactly like a quiet session. f0bffd1 exists because that very bug was
real (BASH_SOURCE reports the symlink path, not the target). First client on the
new image must confirm a [scrub] summary line appears, the drawer count moves, and
exit 3 did not fire. Silence is the failure signal, not success.

.env.example

@@ -18,6 +18,19 @@ SSH_KEY_PATH=~/.ssh
 # the staged files and the palace dedup keys pointing at them cannot be
 # separated.
 #
+# That palace root is resolved with mempalace's own precedence
+# ($MEMPALACE_PALACE_PATH -> $MEMPAL_PALACE_PATH -> ~/.mempalace/config.json ->
+# ~/.mempalace/palace), and the feeders derive their stage FROM it
+# (<palace-root>/pi-stage). Neither the image nor the entrypoint exports it, by
+# design: pinning the palace without carrying the stage along re-creates the
+# very split that a shared root removed. Override it only to move the palace off
+# the default -- e.g. onto a different mount -- and only to a path with the SAME
+# persistence as the palace itself. A stage that outlives its palace (or dies
+# first) makes a scoped `mempalace sync` prune conversation drawers, because
+# their dedup key is the staged path. Setting it to the default buys nothing.
+# Unlike WORKSPACE_PATH/SSH_KEY_PATH above, this is a path INSIDE the container.
+# MEMPALACE_PALACE_PATH=/home/developer/.mempalace/palace
+#
 # To instead share ONE MemPalace across containers/harnesses (pi + opencode
 # + native), set the URL below. When set, the extension connects over HTTP
 # and NO local mempalace-mcp is spawned; the devbox-palace volume is then

.gitea/workflows/docker-publish.yml

@@ -142,6 +142,7 @@ jobs:
       image: catthehacker/ubuntu:act-latest
     outputs:
       pi_version: ${{ steps.resolve.outputs.pi_version }}
+      mempalace_version: ${{ steps.resolve.outputs.mempalace_version }}
       fork_ref: ${{ steps.resolve.outputs.fork_ref }}
       obsmem_ref: ${{ steps.resolve.outputs.obsmem_ref }}
       toolkit_ref: ${{ steps.resolve.outputs.toolkit_ref }}
@@ -242,6 +243,54 @@ jobs:
           fi
           echo "pi_version=${PI_VERSION}" >> "$GITHUB_OUTPUT"
 
+          # ── mempalace core: same audit as pi, from Dockerfile.base ────
+          # Until now this pin had NO CI-side audit at all — a literal string
+          # in Dockerfile.base with zero references in this workflow, while
+          # PI_VERSION got a concreteness gate, a published-on-registry check
+          # and a drift warning. It is the same class of risk: the palace's MCP
+          # tool schema is the agent-facing contract, and a client/server skew
+          # against the shared central palace is a fleet-wide, not local,
+          # problem. Read from Dockerfile.base (not duplicated here) so a local
+          # `docker build` and CI install the same version by construction.
+          MEMPALACE_VERSION=$(sed -n 's/^ARG MEMPALACE_VERSION=\([^[:space:]]*\).*/\1/p' Dockerfile.base | head -n1)
+          if ! printf '%s' "${MEMPALACE_VERSION:-}" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+$'; then
+            echo "::error::ARG MEMPALACE_VERSION in Dockerfile.base is not a concrete version (got '${MEMPALACE_VERSION:-<empty>}'). CI refuses to build from a floating palace version — see the pin policy comment above that ARG."
+            exit 1
+          fi
+          # One fetch, two gates. `curl -sf` exits non-zero and prints nothing
+          # on 404 (PyPI's answer for an unpublished version), so an empty body
+          # lands in the "not published" branch with its own message.
+          MEMPALACE_PYPI=$(curl -sf "https://pypi.org/pypi/mempalace/${MEMPALACE_VERSION}/json" || true)
+          MEMPALACE_PUBLISHED=$(printf '%s' "$MEMPALACE_PYPI" | jq -r '.info.version // empty' 2>/dev/null || true)
+          if [ "${MEMPALACE_PUBLISHED:-}" != "${MEMPALACE_VERSION}" ]; then
+            echo "::error::Pinned mempalace version ${MEMPALACE_VERSION} is not published on PyPI (registry returned '${MEMPALACE_PUBLISHED:-<empty>}'). Fix ARG MEMPALACE_VERSION in Dockerfile.base."
+            exit 1
+          fi
+          # A yanked release still installs when pinned exactly (PEP 592), so
+          # `uv tool install mempalace==X` would succeed silently and ship a
+          # version upstream has withdrawn to the whole fleet. The escape hatch
+          # is the same one-line bump that got us here.
+          MEMPALACE_YANKED=$(printf '%s' "$MEMPALACE_PYPI" | jq -r '.info.yanked // false' 2>/dev/null || true)
+          if [ "${MEMPALACE_YANKED:-false}" = "true" ]; then
+            # Reason hoisted into its own variable rather than inlined as a
+            # $(...) inside the message: a jq program nested in a substitution
+            # inside a double-quoted string needs escaping that silently breaks
+            # the FILTER (jq compile error) while the surrounding `exit 1` still
+            # fires, so the gate looks correct and reports garbage. Caught by
+            # the mutation test, not by review.
+            MEMPALACE_YANK_REASON=$(printf '%s' "$MEMPALACE_PYPI" | jq -r '.info.yanked_reason // "no reason given"' 2>/dev/null || true)
+            echo "::error::Pinned mempalace version ${MEMPALACE_VERSION} is YANKED on PyPI (${MEMPALACE_YANK_REASON:-no reason given}). An exact pin installs a yanked release without complaint — bump ARG MEMPALACE_VERSION in Dockerfile.base."
+            exit 1
+          fi
+          # Informational only, exactly like pi's npm drift warning: a newer
+          # palace must never be adopted implicitly. `|| true` so a transient
+          # PyPI failure cannot fail a release whose pin is already verified.
+          MEMPALACE_PYPI_LATEST=$(curl -sf "https://pypi.org/pypi/mempalace/json" | jq -r '.info.version // empty' 2>/dev/null || true)
+          if [ -n "${MEMPALACE_PYPI_LATEST:-}" ] && [ "${MEMPALACE_PYPI_LATEST}" != "${MEMPALACE_VERSION}" ]; then
+            echo "::warning::mempalace ${MEMPALACE_PYPI_LATEST} is published; this build ships the audited pin ${MEMPALACE_VERSION}. To adopt it: read the upstream CHANGELOG for MCP tool-schema changes (the agent-facing contract) and for sync/delete semantics, check the skew it introduces against the central palace host's server version, then bump ARG MEMPALACE_VERSION in Dockerfile.base and note the audit in CHANGELOG.md."
+          fi
+          echo "mempalace_version=${MEMPALACE_VERSION}" >> "$GITHUB_OUTPUT"
+
           # pi-fork / pi-observational-memory (GitHub) → commit SHAs.
           FORK_REF=$(curl -sf -H "Accept: application/vnd.github.sha" \
             "https://api.github.com/repos/elpapi42/pi-fork/commits/master" || true)
@@ -337,6 +386,7 @@ jobs:
           echo "studio_tag=${STUDIO_TAG}" >> "$GITHUB_OUTPUT"
 
           echo "Resolved PI_VERSION=${PI_VERSION} (pinned in Dockerfile.variant; npm latest is ${PI_NPM_LATEST:-unknown})"
+          echo "Resolved MEMPALACE_VERSION=${MEMPALACE_VERSION} (pinned in Dockerfile.base; PyPI latest is ${MEMPALACE_PYPI_LATEST:-unknown})"
           echo "Resolved PI_ATELIER_REF=${ATELIER_REF} (pi-atelier ${ATELIER_TAG}, pinned)"
           echo "Resolved PI_FORK_REF=${FORK_REF}, PI_OBSMEM_REF=${OBSMEM_REF}"
           echo "Resolved PI_TOOLKIT_REF=${TOOLKIT_REF}, PI_EXTENSIONS_REF=${EXTENSIONS_REF}"
@@ -471,6 +521,7 @@ jobs:
       - name: Smoke test (amd64)
         env:
           EXPECTED_PI_VERSION: ${{ needs.resolve-versions.outputs.pi_version }}
+          EXPECTED_MEMPALACE_VERSION: ${{ needs.resolve-versions.outputs.mempalace_version }}
         run: bash scripts/smoke-test.sh pi-devbox:smoke
 
   # ── Phase 3b: amd64 smoke for the studio variant ────────────────────
@@ -533,6 +584,7 @@ jobs:
       - name: Smoke test studio (amd64)
         env:
           EXPECTED_PI_VERSION: ${{ needs.resolve-versions.outputs.pi_version }}
+          EXPECTED_MEMPALACE_VERSION: ${{ needs.resolve-versions.outputs.mempalace_version }}
         run: bash scripts/smoke-test.sh pi-devbox:smoke-studio
 
   # ── Phase 4: multi-arch publish ─────────────────────────────────────

.gitea/workflows/lint.yml

@@ -52,6 +52,55 @@ jobs:
           apt-get update
           apt-get install -y --no-install-recommends shellcheck python3-yaml
 
+      - name: "Shellcheck + syntax-check repository scripts (severity: error)"
+        # Gap being closed: everything else in this job shellchecks workflow
+        # `run:` steps ONLY, via actionlint. The repo's own shell scripts —
+        # entrypoint.sh, scripts/*.sh, and the extensionless tools under
+        # rootfs/usr/local/bin/ — have never been shellchecked. That exact gap
+        # (a sibling repo with no shell-script lint at all) is how a defect
+        # shipped invisibly for two months: `echo "$json" | python3 <<'EOF'
+        # ... json.load(sys.stdin)` cannot work — with no script argument
+        # python reads its SCRIPT from stdin, so the heredoc IS stdin and the
+        # json.load call hits EOF. shellcheck flags exactly this at severity
+        # ERROR (SC2259, "This redirection overrides piped input"); nothing
+        # ever ran it. Measured before adding this gate: `-S error` is 0
+        # findings across every shell file in THIS repo today, so it is free
+        # to add. `-S warning` is NOT free here (19x SC2088 tilde-in-quotes in
+        # scripts/recreate-sanity-check.sh, plus assorted SC2016 — both
+        # intentional), so warning-level would train people to ignore the job;
+        # hence error-only, matching the SHELLCHECK_OPTS philosophy below.
+        #
+        # Discovery is *.sh UNION a shebang scan, because rootfs/usr/local/
+        # bin/{pi-devbox-version,devbox-skill-reconcile,dot-watch,studio-expose}
+        # are shell scripts with no extension. -print0/mapfile -d '' so a path
+        # with a space cannot silently split, and the file count is asserted
+        # non-zero — a green tick over an empty file set is not a check.
+        run: |
+          # Union of two signals, because either alone misses a real case:
+          # a shebang scan misses a sourced fragment with no shebang, and a
+          # *.sh glob misses the extensionless tools in rootfs/usr/local/bin/.
+          # Silent skipping is precisely the failure mode this gate exists to
+          # prevent, so err toward over-collecting.
+          mapfile -d '' -t all_files < <(find . -not -path './.git/*' -type f -print0)
+          sh_files=()
+          for f in "${all_files[@]}"; do
+            case "$f" in *.sh) sh_files+=("$f"); continue;; esac
+            if head -n1 "$f" 2>/dev/null | grep -qE '^#!.*\b(bash|sh)\b'; then
+              sh_files+=("$f")
+            fi
+          done
+          echo "Checking ${#sh_files[@]} shell file(s)"
+          if [ "${#sh_files[@]}" -eq 0 ]; then
+            echo "::error::no shell files found — the shebang scan or the checkout is wrong"
+            exit 1
+          fi
+          shellcheck -S error -f gcc "${sh_files[@]}"
+          rc=0
+          for f in "${sh_files[@]}"; do
+            bash -n "$f" || { echo "::error file=$f::bash -n failed"; rc=1; }
+          done
+          exit "$rc"
+
       - name: Gitea shell guard (catches the actionlint blind spot)
         # actionlint models GitHub Actions, where the default run shell is
         # bash, so it does NOT flag bash syntax in a step that merely OMITS

AGENTS.md

@@ -64,17 +64,59 @@ re-brand of opencode-devbox's `pi-only` variant.
    (`curl -sf 'https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/latest' | jq -r .version`).
    Check release notes at https://github.com/earendil-works/pi/releases for
    the upstream changelog to include in `CHANGELOG.md`.
-2. Update `CHANGELOG.md` Unreleased → vX.Y.Z section.
-3. Verify `docker compose up` works locally with the current `latest` image
+2. **Refresh the vendored mempalace skill snapshot if the skillset moved:**
+   `scripts/vendor-mempalace-skill.sh --check` (reads a real skillset clone,
+   writes nothing). Three exit codes, not two — a stale-but-truthful record is
+   **not** a release blocker, so don't treat any non-zero exit as "must
+   refresh" without reading which one it was:
+     - **0** — the record is truthful. This includes stale-but-truthful
+       (upstream has moved past the recorded ref, or the local clone has
+       uncommitted changes) — a `NOTICE` is printed, but nothing is lying.
+       **Skipping the refresh in this case is the legitimate, sanctioned
+       outcome** — every enrolled host reads its own live skillset clone, so
+       the baked copy is only a no-mount fallback. What is not legitimate is
+       skipping it *silently*: the drift is visible here, in
+       `pi-devbox-version`, and in the manifest, so decide rather than forget.
+     - **1** — a confirmed problem: the vendored bytes provably do NOT match
+       the file at the recorded ref (a lying record), or the recorded ref
+       doesn't even resolve to that path in this clone. Refresh.
+     - **2** — cannot determine (the recorded ref itself isn't resolvable in
+       this clone — commonly a shallow checkout missing history). Fetch full
+       history and re-check before deciding; don't refresh blind.
+   Refresh with `scripts/vendor-mempalace-skill.sh`, which rewrites the file
+   **and** the ARG together so they cannot drift apart, and refuses (exit 1)
+   rather than silently rewinding provenance if the skillset clone's HEAD is
+   behind the already-recorded ref (detached HEAD, older checkout) — pass
+   `--force` only if that rewind is genuinely intended.
+   Two consequences to accept deliberately on an actual refresh: the snapshot
+   is hashed into `base_tag`, so it costs a base rebuild (~67 min); and if the
+   section the phrase canary names has changed, re-pin it in
+   `scripts/smoke-test.sh`.
+3. Update `CHANGELOG.md` Unreleased → vX.Y.Z section.
+4. Verify `docker compose up` works locally with the current `latest` image
    if you're upgrading users from a previous version. Then run the
    **post-recreate sanity check** inside the running container to confirm
    persisted volumes survived and the pi runtime wiring re-deployed (not just
    that the container booted):
-   `docker compose exec devbox bash scripts/recreate-sanity-check.sh --expected-version X.Y.Z`
-   (or just `pi-devbox-sanity --expected-version X.Y.Z` if `cli_utils/bin` is
-   on PATH). This is the runtime peer of the build-time `smoke-test.sh` gate.
-4. Push tag: `git tag vX.Y.Z && git push origin vX.Y.Z`.
-5. Watch CI: smoke job builds amd64 only and asserts size + extensions +
+   `docker compose exec devbox bash scripts/recreate-sanity-check.sh --expected-image-version X.Y.Z`
+   (or just `pi-devbox-sanity --expected-image-version X.Y.Z` if
+   `cli_utils/bin` is on PATH). This is the runtime peer of the build-time
+   `smoke-test.sh` gate.
+   **`X.Y.Z` here is the pi-devbox release tag** you are shipping (e.g.
+   `1.8.9`), which is what the rest of this checklist means by `vX.Y.Z`.
+   `--expected-image-version` is the flag that asserts it. There is also an
+   `--expected-version`, and it means something else — the **pi coding agent**
+   version (e.g. `0.84.3`, the `ARG PI_VERSION` pin). Handing the release tag
+   to that one used to report *"pi version mismatch: expected 1.8.8, got
+   0.84.3"*, i.e. a red on the final gate of the release accusing the wrong
+   component; it now tells you to use `--expected-image-version` instead, and
+   the reverse mix-up is caught too. Both flags are optional — with neither,
+   the live pi version is asserted against the version recorded in the image's
+   own build manifest (which catches a stale `pi` in the `~/.pi/npm-global`
+   volume shadowing the baked one) and the image tag is reported
+   informationally.
+5. Push tag: `git tag vX.Y.Z && git push origin vX.Y.Z`.
+6. Watch CI: smoke job builds amd64 only and asserts size + extensions +
    pi version + new-base-tooling presence. Variant build is multi-arch
    (amd64 + arm64) only after smoke passes. A tag push fires **only**
    `docker-publish.yml` — `lint.yml` is scoped to `branches: ['**']`, which
@@ -85,9 +127,9 @@ re-brand of opencode-devbox's `pi-only` variant.
    discovery on `head_sha` **and** the workflow `path` — see *Gitea API access*
    below — because that guard costs nothing and a future workflow added on `v*`
    would silently reintroduce the ambiguity.
-6. Verify the Hub tags appear (latest + vX.Y.Z, the `-studio` pair, plus
+7. Verify the Hub tags appear (latest + vX.Y.Z, the `-studio` pair, plus
    base-latest if the base was rebuilt this run).
-7. **Revoke any short-lived Gitea PAT** used during the release at
+8. **Revoke any short-lived Gitea PAT** used during the release at
    `gitea.jordbo.se/user/settings/applications`. N/A if you used the
    `GITEA_ACCESS_TOKEN` env var instead (see *Gitea API access* below) —
    its lifecycle is managed host-side, nothing to revoke.

DOCKER_HUB.md

@@ -65,12 +65,19 @@ The entrypoint deploys/registers all of these on first container start. Re-runni
 ### Document and image tooling
 
 - **pandoc** — universal Markdown↔HTML/Org/RST/etc. conversion. Useful well beyond pi: agent-driven doc exports, format conversion, etc.
+- **Typst** — markup-based typesetting, used as pandoc's `--pdf-engine`
 - **graphviz** (`dot`) — diagram rendering pipelines
 - **imagemagick** (`magick`) — image conversion / resizing
 
+### Browser automation
+
+- **agent-browser** — CLI for driving a real browser (open pages, click/fill/`eval`, snapshot the DOM, screenshots) so agents can verify front-end work instead of guessing
+- **Playwright** + a headless **Chromium** are pre-installed and pinned together; `AGENT_BROWSER_EXECUTABLE_PATH` is preset to the baked browser, so `agent-browser open <url>` works out of the box with no setup
+- **socat** — TCP bridge used to expose the pi-studio server outside the container's loopback
+
 ### Modern CLI tooling
 
-- **Editor**: neovim (LazyVim defaults), tmux (configured for 0-indexed sessions)
+- **Editor**: neovim (system-wide `termguicolors` default; bring your own config/plugins), tmux (configured for 0-indexed sessions)
 - **Search/nav**: ripgrep, fd, fzf, zoxide
 - **Display**: bat, eza, htop, tree
 - **Data**: jq, yq

Dockerfile.base

@@ -396,7 +396,48 @@ ARG INSTALL_MEMPALACE=true
 # (refuse the write) rather than fail open. Neither affects the container's
 # normal MCP-server-plus-CLI-feeder pattern, which already serialised on the
 # same lock under 3.6.0.
-ARG MEMPALACE_VERSION=3.7.1
+#
+# 3.8.0 (2026-08-23, PyPI, released hours after this project's own v1.8.5 tag
+# the same day) is additive/reliability only — reviewed for MCP tool-schema
+# changes before bumping, as always: there are NONE. Two PRs matter:
+#   - PR #2320/#2322: `sync --apply` no longer deletes a drawer solely because
+#     its source_file was unreachable AT THAT MOMENT — it now asks for
+#     corroboration first. This fixes losing a whole mined project to one
+#     `sync --apply` while its volume happened to be unmounted.
+#     IMPORTANT — do not over-read this fix: it addresses TRANSIENT
+#     unreachability, not the standing landmine (documented in the operator's
+#     global AGENTS.md) against running `mempalace_sync` / `mempalace_delete_by_source`
+#     beyond dry-run on the SHARED central palace. On that palace most
+#     source_file paths are PERMANENTLY absent from whichever host runs the
+#     sync — a different machine's paths simply do not exist here, ever, not
+#     merely "right now". That is a different failure shape than #2320/#2322
+#     fixes. The landmine still stands; this bump does not relax it.
+#   - PR #2307: long-running Chroma servers no longer invalidate their own
+#     HNSW cache on their own writes (server-side perf fix). This does NOT
+#     make `mempalace_reconnect` unnecessary — that tool exists for EXTERNAL
+#     writes bypassing the in-process client (e.g. direct sqlite backfills,
+#     CLI commands against a running server), a different scenario #2307
+#     does not touch.
+#
+# CI-side audit (added after v1.8.6, closing that release's "Still open" item):
+# resolve-versions now treats this pin exactly as it treats PI_VERSION — it
+# reads the ARG from THIS file, refuses a non-concrete value, verifies the
+# version is published on PyPI, refuses a YANKED release (an exact pin installs
+# one silently under PEP 592), and WARNS — never silently adopts — when PyPI has
+# a newer release. smoke-test.sh then asserts the installed core equals that
+# audited pin, which catches a stale cached base layer that no manifest-internal
+# check can see. So a bump here is now gated end to end; what remains manual is
+# the JUDGEMENT above (MCP schema review, server/client sequencing), which is
+# the part that should stay manual.
+#
+# Deployment sequencing note for whoever ships this bump: synlig (the shared
+# central palace host) currently serves mempalace 3.7.1 SERVER-SIDE via
+# docker-compose.mempalace.yml, which reuses this same devbox image. Bumping
+# this ARG changes only the CLIENT version baked into pi-devbox images: it
+# introduces client/server skew until synlig's compose stack is separately
+# rebuilt/redeployed with the new pin. Not something to code around here —
+# just sequence the redeploy.
+ARG MEMPALACE_VERSION=3.8.0
 ENV UV_TOOL_DIR=/opt/uv-tools
 ENV UV_TOOL_BIN_DIR=/usr/local/bin
 RUN if [ "${INSTALL_MEMPALACE}" = "true" ]; then \

Dockerfile.variant

@@ -56,7 +56,22 @@ ARG USER_NAME=developer
 # current when it was first populated (shipped the same bytes for pi-devbox
 # v0.74.0..v0.75.5; discovered + fixed in v0.75.5b, 2026-05-23). The `latest`
 # branch below is kept only for a deliberate local `docker build` override.
-ARG PI_VERSION=0.84.2
+#
+# AUDITED AT 0.84.3 (2026-08-25, was 0.84.2): upstream's notes carry a
+# "Breaking Changes" heading — `GoogleThinkingLevel` renamed to
+# `GoogleApiThinkingLevel`. INERT FOR THIS IMAGE: all four vendored companions
+# (/opt/pi-fork, /opt/pi-observational-memory, /opt/pi-atelier, /opt/pi-studio)
+# were grepped for that symbol and reference it ZERO times, so nothing here
+# couples to the renamed type. Recorded because the heading will look alarming
+# to the next reader doing step 1 above — the audit is done, don't redo it.
+# Adopted for two fixes that land squarely on this repo's own vendored-skill
+# wiring (see devbox-skill-reconcile, v1.8.5): nested Markdown skills inside
+# `.agents/skills/<group>/` directories were not discovered, and root Markdown
+# files such as README.md / AGENTS.md inside a skill dir were reported as
+# broken skills unless they declared valid skill frontmatter.
+# pi-atelier needs no companion bump: v0.8.2 clears the >=0.7.1 floor that
+# pi >= 0.84 requires (see PI_ATELIER_REF below).
+ARG PI_VERSION=0.84.3
 ARG PI_TOOLKIT_REF=main
 ARG PI_EXTENSIONS_REF=main
 # Repo URLs default to the canonical gitea origin but are overridable so a
@@ -262,6 +277,36 @@ ARG SOURCE_REVISION=
 # MEMPALACE_TOOLKIT_REF is consumed in Dockerfile.base; re-declared here
 # only so its intended ref lands in the label set alongside the others.
 ARG MEMPALACE_TOOLKIT_REF=main
+# ── Vendored skill provenance ─────────────────────────────────────────
+# The vendored mempalace SKILL.md is the ONLY baked artefact with no /opt
+# clone behind it: its upstream (the skillset repo) is PRIVATE, so the
+# image cannot clone it and CI cannot resolve its HEAD (see VENDORED.md).
+# Consequence through v1.8.7: the snapshot was ANONYMOUS — nothing in the
+# image or the repo recorded which skillset commit it was taken from, so
+# the only staleness check available was a hand-maintained phrase canary in
+# scripts/smoke-test.sh, which by construction can only detect "older than
+# what I remembered to pin", never "older than skillset main".
+#
+# Recording the ref costs nothing and makes the question answerable. It is
+# deliberately a plain ARG DEFAULT rather than a CI-resolved output:
+#   * the value is a fact about the committed snapshot, so it belongs in
+#     the tree next to it — not in a workflow that a local `docker build`
+#     never runs (same reasoning as MEMPALACE_VERSION living in
+#     Dockerfile.base rather than being duplicated in docker-publish.yml);
+#   * CI therefore needs NO new build-arg at any of its four
+#     Dockerfile.variant call sites (smoke, smoke-studio, build-variant,
+#     build-variant-studio) — a plumbing change that is easy to
+#     under-apply to only two of them;
+#   * and it needs no credential for a private repo.
+# Bump it with scripts/vendor-mempalace-skill.sh, which refreshes the file
+# and rewrites this line together, so the pair cannot drift apart by hand.
+# This ARG lives in Dockerfile.variant ON PURPOSE: Dockerfile.base and
+# rootfs/ are both hashed into base_tag, so recording provenance here costs
+# no ~67-minute base rebuild. (scripts/check-base-hash.sh scans only
+# Dockerfile.base, so no folding into the base hash is required — nor would
+# it be correct, since this ARG changes nothing about the base's contents.)
+ARG SKILLSET_SNAPSHOT_REF=6eb20af181f0147cb8c1377f6e36a6a47a68e8e5
+
 # Dockerfile.base sets description="pi-devbox — base image (variant-independent)"
 # and every variant INHERITS it, so both published images used to advertise
 # themselves on Docker Hub as the base image. A LABEL cannot branch on
@@ -286,7 +331,8 @@ LABEL org.opencontainers.image.version="${RELEASE_TAG}" \
       se.jordbo.pi-devbox.pi-atelier-version="${PI_ATELIER_VERSION}" \
       se.jordbo.pi-devbox.mempalace-toolkit-ref="${MEMPALACE_TOOLKIT_REF}" \
       se.jordbo.pi-devbox.pi-studio-ref="${PI_STUDIO_REF}" \
-      se.jordbo.pi-devbox.pi-studio-version="${PI_STUDIO_VERSION}"
+      se.jordbo.pi-devbox.pi-studio-version="${PI_STUDIO_VERSION}" \
+      se.jordbo.pi-devbox.skillset-snapshot-ref="${SKILLSET_SNAPSHOT_REF}"
 
 # The manifest is written from GROUND TRUTH — the actual checked-out HEAD
 # of each /opt clone and the live `pi --version` — not merely the intended
@@ -297,14 +343,69 @@ RUN set -e; \
     mkdir -p /etc/pi-devbox; \
     rev() { git -C "$1" rev-parse HEAD 2>/dev/null || echo "unknown"; }; \
     PI_V="$(pi --version 2>/dev/null | head -n1 | tr -d '\r\n')"; \
+    # mempalace CORE (the PyPI package behind the MCP tools) is installed in
+    # Dockerfile.base via `uv tool install`, so no /opt clone reveals it and
+    # until v1.8.6 the manifest could not answer "which palace shipped here?" —
+    # a palace bug could not be correlated to an image, which is precisely the
+    # correlation this file exists to provide. Read from the INSTALLED BINARY,
+    # not from ARG MEMPALACE_VERSION, per the ground-truth rule above: that is
+    # what catches an install which resolved to something other than the pin.
+    # `mempalace --version` prints "MemPalace 3.7.1" — NAME-PREFIXED, unlike
+    # pi's bare "0.84.2" — hence the $NF pick rather than a straight read. The
+    # leading-digit test then rejects usage/error text (a renamed flag prints a
+    # usage block) and degrades to JSON null, so this can never fail the build.
+    MP_V="$(mempalace --version 2>/dev/null | head -n1 | tr -d '\r' | awk '{print $NF}')"; \
+    case "$MP_V" in [0-9]*) MP_CORE="\"${MP_V}\"" ;; *) MP_CORE='null' ;; esac; \
     STUDIO_REV='null'; \
     if [ -d /opt/pi-studio/.git ]; then STUDIO_REV="\"$(rev /opt/pi-studio)\""; fi; \
+    # The vendored skill snapshot's fingerprint is MEASURED here, not passed
+    # in as a build-arg, per the ground-truth rule above: SKILLSET_SNAPSHOT_REF
+    # is a CLAIM about which skillset commit the file came from, while this
+    # hash is what the image actually ships. Recorded together they let any
+    # reader with the skillset checked out — which on this fleet is every
+    # host, since all four compose stacks mount it — verify the claim at
+    # RUNTIME, without CI ever needing access to the private repo. Degrades
+    # to JSON null rather than failing the build if the directory is absent;
+    # the smoke assertion is what turns that into a loud failure.
+    #
+    # Hashes the whole DIRECTORY, not just SKILL.md: a single-file hash
+    # answers "did this one file change", not "is the live copy the same
+    # skill" — a live checkout that added or edited a SIBLING file (a
+    # reference/ doc, a helper script) would still report "identical to
+    # baked snapshot" against a file-only hash. pi-extensions already ships
+    # two files for exactly this reason (SKILL.md + evaluate-extension-usage.py),
+    # so this is not a hypothetical. Deterministic over `find | sort`, never
+    # readdir order: relative paths + per-file sha256, folded into one hash.
+    # pi-devbox-version mirrors this exact pipeline over the live directory so
+    # the two sides are comparable — if you change this, change that too.
+    tree_sha256() { \
+      ( cd "$1" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum ) 2>/dev/null | sha256sum | cut -d' ' -f1; \
+    }; \
+    SKILL_SNAP='null'; \
+    _snap_dir=/usr/local/share/pi-devbox/skills/mempalace; \
+    if [ -d "$_snap_dir" ] && [ -n "$(find "$_snap_dir" -type f -print -quit)" ]; then \
+      SKILL_SNAP="\"$(tree_sha256 "$_snap_dir")\""; \
+    fi; \
     { \
       echo '{'; \
       echo "  \"release_tag\": \"${RELEASE_TAG}\","; \
       echo "  \"build_date\": \"${BUILD_DATE}\","; \
       echo "  \"source_revision\": \"${SOURCE_REVISION}\","; \
       echo "  \"pi_version\": \"${PI_V}\","; \
+      # Sibling of pi_version, NOT a member of components{}: that map holds git
+      # SHAs and `pi-devbox-version` renders it with .value[0:12], which would
+      # silently truncate a longer version string.
+      echo "  \"mempalace_version\": ${MP_CORE},"; \
+      # Siblings, NOT members of components{}, for two independent reasons:
+      # that map means "HEAD of a clone present in this image" and the
+      # skillset is not cloned here (calling it a component would be a
+      # lie a future reader would act on), and `pi-devbox-version` renders
+      # every components{} value with .value[0:12] — which would truncate
+      # a 64-hex sha256 into something that looks like a short commit.
+      # Named `_tree_sha256`, not `_sha256`: it measures every file under the
+      # vendored skill directory, not one file — see tree_sha256() above.
+      echo "  \"skillset_snapshot_ref\": \"${SKILLSET_SNAPSHOT_REF}\","; \
+      echo "  \"skillset_snapshot_tree_sha256\": ${SKILL_SNAP},"; \
       echo "  \"components\": {"; \
       echo "    \"pi-toolkit\": \"$(rev /opt/pi-toolkit)\","; \
       echo "    \"pi-extensions\": \"$(rev /opt/pi-extensions)\","; \

README.md

@@ -20,7 +20,9 @@ on the host.
 - `pi-extensions` — TypeScript extensions for pi (preview, MCP bridges,
   mempalace integration, etc.)
 - `pi-fork` — the `fork` tool for spawning sub-agents
-- `pi-observational-memory` — the `recall` tool for session compaction
+- `pi-observational-memory` — durable session memory: the ledger that makes
+  compaction cheap, plus the `recall` tool. See
+  [`docs/observational-memory.md`](docs/observational-memory.md)
 - `pi-atelier` — TUI sidebar: ordered panels, split-pane, themes. Pinned to an
   audited tag; see [Version pins](#version-pins-pi-pi-atelier-mempalace)
 
@@ -70,9 +72,27 @@ so `TERM=xterm-kitty` is understood. Override either in your own
 ### Document and image tooling
 
 - `pandoc` — universal Markdown↔HTML/Org/RST/etc. converter
+- `typst` — markup-based typesetting, wired up as pandoc's `--pdf-engine` (see
+  [Generating a PDF with pandoc + typst](#generating-a-pdf-with-pandoc--typst))
 - `graphviz` — `dot` rendering for diagram pipelines
 - `imagemagick` — image conversion / resizing (invoked as `magick`)
 
+### Browser automation
+
+- `agent-browser` — CLI for driving a real headless browser: open pages,
+  click/fill/`eval`, snapshot the DOM, take screenshots. Useful whenever a task
+  involves a web UI or verifying how a page actually renders (live DOM, WebGL,
+  layout, popup positioning) instead of guessing from source.
+- `playwright` + a pre-installed headless **Chromium** back it.
+  `AGENT_BROWSER_EXECUTABLE_PATH` is preset to the baked browser via a stable
+  `/usr/local/bin/agent-chrome` symlink (insulated from Playwright's
+  per-version/arch install directory), so `agent-browser open <url>` works
+  out of the box with no setup. Run `agent-browser skills get core --full`
+  for the command set and workflow patterns.
+- `socat` — TCP bridge used by `studio-expose` to reach pi-studio's
+  loopback-bound server from outside the container (see
+  [Using pi-studio](#using-pi-studio--studio-variant))
+
 ### Language toolchains
 
 - `python3` + `python3-venv` + `python3-pip` (system Python)
@@ -537,6 +557,75 @@ session/docs mining; the 29 MCP tools (search, kg-query, drawer-add,
 diary-write, etc.) are wired into pi automatically by the pi-extensions
 mempalace bridge.
 
+### Cross-machine agent coordination
+
+When `MEMPALACE_REMOTE_URL` points at a *shared* palace, the container gets more
+than shared search: it joins an append-only coordination log (RFC 003) that other
+machines' agents can address it on — used here for design review, patch handoff
+and retraction between hosts.
+
+Two container-side settings make it work:
+
+| Variable | Why it matters |
+|---|---|
+| `MEMPALACE_REMOTE_URL` | selects the shared palace; unset means a purely local palace, and the log then contains only this machine's own events |
+| `MEMPALACE_PI_DEVICE` | the bridge stamps `pi@<device>` as the writer, which is the **only** way the log can tell two machines apart when both are thin clients of one palace |
+
+So a container with no `MEMPALACE_PI_DEVICE` can read the log but is not
+reachable *on* it: messages addressed to a bare `pi` match nobody. Set both, or
+neither.
+
+What the agent is expected to *do* with this lives in the mempalace skill
+(`~/.agents/skills/mempalace/SKILL.md`) — the mailbox query at wake-up, and the
+convention that a directed event with `status="open"` is a request owed a reply
+while a `*` broadcast owes nothing. The mechanism side (what the bridge stamps,
+and why live SSE push depends on the palace deployment's reverse proxy rather
+than on this image) is documented in the toolkit's `extensions/pi/README.md`.
+
+**Since v1.8.9 the bridge reads the log for you.** Earlier images were write-only
+— they stamped provenance on the way out and never read back, so a directed ask
+reached an agent only if that agent happened to run `mempalace_event_list`
+itself. The mailbox is gated on the same two variables as the stamper, is on by
+default, and derives what is *owed* rather than trusting `status` (an acked event
+keeps matching a `status="open"` query forever, because the log is append-only):
+
+| Variable | Default | Effect |
+|---|---|---|
+| `MEMPALACE_MAILBOX` | unset (on) | `0` disables mailbox reads entirely |
+| `MEMPALACE_MAILBOX_POLL_MS` | `300000` | minimum gap between mid-session polls |
+| `MEMPALACE_MAILBOX_RESURFACE_MS` | `3600000` | re-announce a still-owed ask after this long |
+
+Delivery **queues, it never interrupts**: the poll runs when pi goes idle and the
+message is steered into the *next* turn, so nothing wakes the model on inbound
+fleet traffic. The practical consequence, measured on two devices: the message
+appears in your session window and the agent acts on it when the next turn
+starts — you are the trigger. (That describes the bridge **as baked in v1.8.9**,
+`mempalace-toolkit` `5b8d78f`; the mailbox's own mechanism and landmines live in
+the toolkit's `docs/rfc-003-coordination-log.md` §7.11–§7.12, which moves ahead of
+whatever this image has baked.)
+
+## Observational memory (in-session memory)
+
+The image also bakes [pi-observational-memory](https://github.com/elpapi42/pi-observational-memory),
+which is memory of a *different kind* from the palace and is easy to confuse with
+it. It keeps a small branch-local ledger of observations and reflections while a
+session runs, so when pi compacts the conversation the summary is a
+**deterministic fold of that ledger rather than a model call**, and every item
+keeps a 12-character id that `recall(<id>)` resolves back to the exact source.
+
+In one line: **observational memory keeps a session coherent; the palace keeps
+the fleet coherent.**
+
+It is on by default, needs no habit from you, and sends its background work to a
+cheaper model than your session (Haiku while the session runs Opus, in the seeded
+`~/.pi/agent/settings.json`). Inspect it from inside pi with `/om:status` and
+`/om:view`; turn all proactive work off for one run with
+`PI_OBSERVATIONAL_MEMORY_PASSIVE=1 pi`.
+
+What it is for, how the lifecycle works, what it costs, every setting and its
+default, and how it differs from MemPalace:
+[`docs/observational-memory.md`](docs/observational-memory.md).
+
 ## Agent skills
 
 pi discovers skills under `~/.agents/skills/`. Two delivery paths feed that
@@ -787,8 +876,9 @@ docker inspect --format '{{json .Config.Labels}}' joakimp/pi-devbox:latest | jq
 `org.opencontainers.image.{version,revision,created}` plus
 `se.jordbo.pi-devbox.*-ref` record the intended pi version and companion
 refs. The on-disk `/etc/pi-devbox/build-manifest.json` records **ground
-truth** — the actual checked-out commit of each `/opt` clone and the live
-`pi --version` — so a tag is reconstructable after CI logs rotate:
+truth** — the actual checked-out commit of each `/opt` clone, the live
+`pi --version`, and (from v1.8.6) the live `mempalace --version` of the
+installed palace core — so a tag is reconstructable after CI logs rotate:
 
 ```bash
 docker run --rm --entrypoint= joakimp/pi-devbox:latest cat /etc/pi-devbox/build-manifest.json
@@ -972,10 +1062,21 @@ After `docker compose up -d --force-recreate`, run the **runtime** peer of
 persisted volumes survived, and pi runtime wiring is intact:
 
 ```bash
-./scripts/recreate-sanity-check.sh                       # auto-detects variant
-./scripts/recreate-sanity-check.sh --expected-version 0.79.4   # assert pi version
+./scripts/recreate-sanity-check.sh                              # auto-detects variant
+./scripts/recreate-sanity-check.sh --expected-image-version 1.8.9  # assert the pi-devbox release tag
+./scripts/recreate-sanity-check.sh --expected-version 0.84.3        # assert the pi coding agent version
 ```
 
+Those are **two different versions**, and the flags are not interchangeable:
+`--expected-image-version` takes the pi-devbox release tag (`v` optional),
+`--expected-version` takes `pi --version`. Hand one the other's value and it
+says so by name instead of reporting a mismatch against the wrong component.
+With neither flag, both values are read from the image's own build manifest
+(`/etc/pi-devbox/build-manifest.json`): the live pi version is asserted against
+the one recorded at build time — which catches a stale `pi` in the
+`~/.pi/npm-global` volume shadowing the baked one — and the release tag is
+reported informationally.
+
 If `cli_utils` is on your PATH, the `pi-devbox-sanity` wrapper runs the same
 check by short name and locates the repo automatically (override with
 `PI_DEVBOX_REPO=/path/to/pi-devbox`). Like `smoke-test.sh`, this script is

THIRD_PARTY.md

@@ -18,8 +18,23 @@ for OS packages, the per-package copyright files inside the image at
 | pi-fork | github.com/elpapi42/pi-fork | MIT |
 | pi-observational-memory | github.com/elpapi42/pi-observational-memory | MIT |
 | pi-studio *(`-studio` variant only)* | github.com/omaclaren/pi-studio | MIT |
+| pi-atelier | github.com/michaelmjhhhh/pi-atelier | MIT |
 | pi-toolkit, pi-extensions, mempalace-toolkit | authored by the maintainer (Joakim Persson) | MIT |
 
+## MemPalace (AI memory)
+
+| Component | Upstream | License |
+| --- | --- | --- |
+| mempalace (core, MCP server) | github.com/MemPalace/mempalace (PyPI: `mempalace`) | MIT — the GitHub repo declares MIT; the PyPI package's own metadata omits a license classifier, so if you need clearance from the package artifact alone, verify against the repo's `LICENSE` file rather than the sdist/wheel metadata |
+
+## Browser automation
+
+| Component | Upstream | License |
+| --- | --- | --- |
+| agent-browser | github.com/vercel-labs/agent-browser (npm: `agent-browser`) | Apache-2.0 |
+| Playwright | github.com/microsoft/playwright (npm: `playwright`) | Apache-2.0 |
+| Chromium | chromium.googlesource.com/chromium/src | BSD-3-Clause for Chromium's own code, plus a large set of bundled third-party components each under their own license (see Chromium's own `LICENSE`/`about:credits`). The binary in this image is **not compiled here** — it is the build Playwright downloads for its pinned version ("Chrome for Testing"), installed via `playwright install --with-deps chromium` at `/usr/local/share/ms-playwright/`. Treat Playwright's own distribution terms for that build as authoritative over any summary here. |
+
 ## Tooling baked into the base image
 
 | Component | Upstream | License (best effort) |

docs/observational-memory.md

@@ -0,0 +1,371 @@
+# Observational memory — why this image has it, and what it does for you
+
+**Audience:** anyone using this container for long pi sessions who has wondered
+what `recall`, `/om:status` and "compacted memory" are, or whether they should
+leave any of it switched on.
+
+**Companion documents:** the extension ships its own reference docs at
+`/opt/pi-observational-memory/docs/` —
+[`concepts.md`](https://github.com/elpapi42/pi-observational-memory/blob/main/docs/concepts.md)
+(the model),
+[`how-it-works.md`](https://github.com/elpapi42/pi-observational-memory/blob/main/docs/how-it-works.md)
+(hooks and internals) and
+[`configuration.md`](https://github.com/elpapi42/pi-observational-memory/blob/main/docs/configuration.md)
+(every setting). Pi's own compaction mechanics are in
+`/usr/lib/node_modules/@earendil-works/pi-coding-agent/docs/compaction.md`.
+Those are normative; this document is the **deployment** view — what is pinned
+here, how it is wired, what it costs, and how it differs from MemPalace. For the
+palace, see
+[`mempalace-toolkit/docs/fleet-memory.md`](https://gitea.jordbo.se/joakimp/mempalace-toolkit/src/branch/main/docs/fleet-memory.md).
+
+> Verified on pi-devbox **v1.8.9** (`release_tag v1.8.9`, source `aac4a1c`),
+> which bakes pi-observational-memory **v3.0.4** at commit `ce9fc98` — the value
+> in `/etc/pi-devbox/build-manifest.json` → `components.pi-observational-memory`.
+> Every number below was read from that tree, from pi 0.84.3's own docs, or from
+> the live container.
+
+---
+
+## 1. The problem it solves
+
+A long pi session outgrows the model's context window. Pi's answer is
+**compaction**: fold the older part of the conversation into a summary and keep
+recent messages verbatim. That is unavoidable, and it is where sessions go
+wrong — the summary is produced *at the moment of pressure*, by a model, about a
+transcript that is about to leave the context.
+
+Observational memory changes *when* the remembering happens. Instead of
+summarising in a panic at the end, it keeps a small **ledger** up to date while
+the session runs, and compaction then just folds that ledger.
+
+```mermaid
+flowchart LR
+    A0["plain compaction"] --> A1["context fills"]
+    A1 --> A2["a model summarises<br/>under pressure"]
+    A2 --> A3["prose summary,<br/>no way back"]
+    B0["with observational<br/>memory"] --> B1["context fills"]
+    B1 --> B2["ledger written<br/>as you work"]
+    B2 --> B3["compaction folds<br/>the ledger"]
+    B3 --> B4["ids you can<br/>recall"]
+```
+
+Top row is pi on its own: one model call at the worst possible moment, detail
+chosen in a hurry, and the original wording gone from view. Bottom row is this
+image's default: the thinking happened earlier on a cheap model, the fold is
+deterministic, and every line in the result carries an id that resolves back to
+the exact source.
+
+## 2. The mental model: three layers and a ledger
+
+| Layer | What it is | Example |
+|---|---|---|
+| **Observation** | a timestamped, source-backed event from the conversation | "user rejected option B because it needs a base rebuild" |
+| **Reflection** | a durable conclusion *backed by* observations | "the user optimises for avoiding 67-minute rebuilds" |
+| **Drop** | a tombstone retiring an observation from active memory | the superseded detail of a bug that is now fixed |
+
+These are appended to the session as silent ledger entries
+(`om.observations.recorded`, `om.reflections.recorded`,
+`om.observations.dropped`) and **folded** — replayed in order — to produce the
+memory state. The ledger is the source of truth; what you see in a compacted
+session is a rendering of it.
+
+Two properties follow, and both matter later:
+
+- **The ledger itself costs no context.** Those entries are pi `custom` entries,
+  which *"do not participate in LLM context"* (pi `docs/session-format.md`). They
+  sit in the session file and reach the model only via the fold at compaction.
+- **Memory is branch-local.** A pi session is a tree (resume, fork), and the fold
+  follows the current branch only, so a forked branch does not inherit another
+  branch's view.
+
+## 3. The lifecycle
+
+Three background workers and one compaction hook, driven by *raw token
+progress* rather than wall-clock time. Defaults in brackets.
+
+```mermaid
+flowchart TD
+    T(["turn_end"]) --> O{"10k raw tokens<br/>since observing?"}
+    O -- yes --> OBS["<b>observer</b> runs"]
+    O -- "no" --> R{"20k tokens<br/>since reflecting?"}
+    R -- yes --> REF["<b>reflector</b> runs"]
+    REF -- "if pool over 10k" --> DR["<b>dropper</b> prunes"]
+    S(["agent_settled"]) --> C{"81k tokens<br/>since compacting?"}
+    C -- yes --> CP["ctx.compact()"]
+    CP --> H(["session_before_compact"])
+    H --> F["fold the ledger<br/>no model call"]
+    F --> VIS["compacted memory"]
+```
+
+- **observer** — `observeAfterTokens` [10000]: writes observations for the
+  conversation it has not covered yet.
+- **reflector** — `reflectAfterTokens` [20000]: promotes patterns across
+  observations into durable reflections.
+- **dropper** — no clock of its own. It is post-reflection maintenance, gated on
+  a *successful same-turn* reflection **and** an active pool above
+  `observationsPoolTargetTokens` [10000]. Not a third worker on a third
+  threshold.
+- **compaction** — `compactAfterTokens` [81000], checked when pi goes idle, so it
+  never interrupts a turn. Pi will also compact on its own when the context is
+  nearly full (`contextTokens > contextWindow - reserveTokens`, `reserveTokens`
+  [16384]).
+
+## 4. What compaction actually does to your context
+
+This is the question the rest of the document used to leave hanging: if the old
+conversation is folded away, is the session back to knowing nothing?
+
+**No.** Compaction replaces *part* of the context, not all of it, and it deletes
+nothing at all from disk.
+
+```mermaid
+flowchart LR
+    SYS["system prompt<br/>+ AGENTS.md"] --> CTX["what the model sees<br/>on the next turn"]
+    SUM["folded memory:<br/>reflections + observations"] --> CTX
+    TAIL["recent turns,<br/>verbatim"] --> CTX
+    DISK[("session .jsonl: all of it")] -. "recall(id)" .-> CTX
+```
+
+Where each piece comes from:
+
+- **System prompt and `AGENTS.md` — never compacted, because they were never
+  conversation.** Pi rebuilds them from disk on every request
+  (`loadContextFileFromDir`), so they cannot be lost by compaction.
+- **The verbatim tail — sized by a token budget, not a message count.** Pi walks
+  backwards from the newest entry accumulating token estimates until
+  `keepRecentTokens` [20000] is reached; that entry becomes `firstKeptEntryId`,
+  and *everything from there on is kept unchanged*. Cut points land on turn
+  boundaries, never mid-tool-call. So the most recent ~20k tokens of real work —
+  your last instructions, the diffs, the test output — survive word for word.
+- **The folded memory — replaces only what came before that cut.** Rendered from
+  the ledger's records: reflections and observations, each with its 12-hex id.
+- **The session file — untouched.** Compaction *appends* a `compaction` entry
+  (`{"type":"compaction", summary, firstKeptEntryId, tokensBefore, …}`) and
+  rebuilds context from it on later turns. Nothing is rewritten in place; the
+  only documented way to remove session content is deleting the whole `.jsonl`.
+
+That last point is what makes the answer to "is the detail gone?" *no* rather
+than *mostly*: `recall` does not read the context window at all. It calls
+`sessionManager.getBranch()` — the full branch from the root — and resolves an
+observation id back to the original entries. Detail that left the model's view
+an hour ago is still one `recall` away.
+
+**Repeated compaction does not summarise the summary.** The rendered text is
+always built from live observation/reflection *records*, never from the previous
+compaction's prose, so there is no generation-loss spiral. (Mechanically the
+projection is incremental — it re-derives back to the last full-fold boundary and
+carries the rest forward, escalating to a genuine re-fold from the branch root
+when the observation pool reaches `observationsPoolMaxTokens` [20000].)
+
+So the honest summary of the state after compaction: **the model keeps its
+instructions, keeps recent work verbatim, trades older turns for a dense
+id-carrying digest of them, and can pull any of it back on demand.** Not a fresh
+start — a smaller, cheaper, still-navigable one.
+
+### One caveat about "no model call"
+
+If the ledger is empty — compaction fires before the observer has ever run — the
+hook returns nothing and *declines ownership*, and pi's own model-based
+summariser runs instead:
+
+```ts
+const summary = renderSummary(projection.reflections, projection.observations);
+if (summary.length === 0) {
+  // Decline ownership so Pi's native summarizer preserves the pre-cut context.
+  return;
+}
+```
+
+In steady state (any session old enough to have produced one observation) om's
+hook wins and compaction is model-free. "Never calls a model" is true in practice
+and false in principle; the fallback is deliberate, so an empty ledger degrades
+to normal pi rather than to no summary at all.
+
+## 5. What you actually get
+
+- **Compaction stops being a stall.** In steady state the latency path is
+  deterministic work over ledger entries, not a summarisation call.
+- **Nothing important vanishes silently.** Compaction is lossy by design, but
+  every item keeps a 12-character id, and `recall(<id>)` returns the exact
+  evidence — original wording, reasoning, file path, error text.
+- **The bookkeeping runs on a cheaper model than your session.** In this image
+  that is deliberate and visible (§7): background workers on Haiku, session on
+  Opus.
+- **It is automatic.** No habit to maintain, unlike the palace protocol — which is
+  exactly why the two complement each other (§11).
+- **Forks stay clean.** Branch-local memory means a `fork` sub-agent's noise does
+  not leak into the parent's folded memory.
+
+## 6. `recall` is not a search tool
+
+`recall` takes **one specific 12-hex id** that already appears in compacted
+memory or in `/om:view`. It cannot be given a topic. It can return an observation
+(marked `active` or `dropped`), or a reflection together with the observations
+supporting it.
+
+```mermaid
+sequenceDiagram
+    participant M as compacted memory
+    participant A as agent
+    participant L as ledger
+    M->>A: "[high] user rejected option B (a1b2c3d4e5f6)"
+    A->>L: recall("a1b2c3d4e5f6")
+    L-->>A: exact observation + source ids
+    Note over A: acts on the original wording
+```
+
+The rule of thumb the agent skill uses: recall **before a load-bearing action**
+that rests on a compressed memory — shipping a change, asserting a fact,
+answering "why do you believe that". One recall is cheap; redoing finished work
+is not.
+
+## 7. How it is wired in this image
+
+```mermaid
+flowchart TB
+    IMG["baked in the image:<br/>v3.0.4 @ ce9fc98"] --> REG["settings.json<br/>packages[]"]
+    REG --> SESS["your pi session"]
+    SESS -- "your turns" --> SM["session model:<br/>Opus"]
+    SESS -- "observer, reflector,<br/>dropper" --> WM["memory model:<br/>Haiku"]
+    SESS -- "ledger entries" --> JL["session .jsonl"]
+    JL --> VOL[("devbox-pi-config<br/>volume")]
+```
+
+Four consequences of that wiring:
+
+1. **There is no separate database.** Memory *is* entries inside the ordinary pi
+   session file (`~/.pi/agent/sessions/<project>/<timestamp>_<uuid>.jsonl`).
+   Nothing extra to back up, nothing to migrate.
+2. **It survives container recreate**, because `~/.pi` is the `devbox-pi-config`
+   named volume (`docker-compose.yml`) — the same one holding your pi config and
+   session history.
+3. **`packages[]` is the only source of truth for which copy is loaded.** A clone
+   at `/workspace/pi-observational-memory` may exist (and today matches `/opt`
+   byte-for-byte at `ce9fc98`) — its presence proves nothing. To run a patched
+   build you point `packages[]` at it explicitly and start a new session.
+4. **The worker model is a deliberate choice, and it is yours to change.** The
+   seeded config sends background work to Haiku while your session runs Opus:
+
+```json
+"observational-memory": {
+  "model": { "provider": "amazon-bedrock", "id": "eu.anthropic.claude-haiku-4-5-20251001-v1:0" },
+  "debugLog": false
+}
+```
+
+## 8. What it costs
+
+| Resource | Cost |
+|---|---|
+| Model calls | up to **three** background calls per consolidation pass (observer, reflector, dropper), each capped at `agentMaxTurns` [16], on the configured memory model — not your session model |
+| Latency in your turns | none by construction: workers run from `turn_end`, compaction runs when pi is idle, and the fold itself does no model work |
+| Disk | negligible — JSON lines inside a session file that would exist anyway (measured here: `~/.pi/agent/sessions` = 30 MB total, tens of `om.*` entries per session) |
+| Context window | **zero until compaction.** `custom` entries do not enter LLM context; only the folded summary does |
+| Attention | none once configured; there is no protocol for you or the agent to remember |
+
+If that is still more than you want on a given run, §9's `passive` switch turns
+off all proactive work while keeping `recall` and `/om:*` usable.
+
+## 9. Configuration
+
+Global: `~/.pi/agent/settings.json` (persisted in the volume). Per project:
+`<project>/.pi/settings.json`, which overrides global. Precedence is
+project → global → environment, and the environment can only override `passive`.
+
+| Key | Default | What it changes |
+|---|---|---|
+| `observeAfterTokens` | `10000` | observer cadence — lower means smaller chunks and more calls |
+| `reflectAfterTokens` | `20000` | reflector cadence (and thereby dropper opportunities) |
+| `observerChunkMaxTokens` | 20% of the memory model's context window, else `60000` | cap on one observer run's input |
+| `compactAfterTokens` | `81000` | when proactive auto-compaction fires |
+| `observationsPoolMaxTokens` | `20000` | pool size at which compaction does a full re-fold from the branch root |
+| `observationsPoolTargetTokens` | half of max (`10000`) | what the dropper aims back down to |
+| `agentMaxTurns` | `16` | shared turn cap for the three workers |
+| `model` | unset → session model | send background work to a cheaper/faster model |
+| `showWorkerNotifications` | `true` | routine "observer ran" notices |
+| `passive` | `false` | **kill switch** for all proactive background work; `recall` and `/om:*` still work |
+| `debugLog` | `false` | per-session NDJSON trace at `~/.pi/agent/observational-memory/debug/<session-id>.ndjson` |
+
+Pi's own compaction knobs live under a separate `compaction` key —
+`keepRecentTokens` [20000] sets the verbatim tail from §4, `reserveTokens`
+[16384] the headroom that triggers pi's own compaction.
+
+One-off passive run, no config edit:
+
+```bash
+PI_OBSERVATIONAL_MEMORY_PASSIVE=1 pi
+```
+
+Invalid values are ignored rather than fatal, so a typo degrades to the default
+instead of breaking your session — which also means a typo is silent. Check with
+`/om:status`.
+
+## 10. Confirming it is actually working
+
+Do not infer health from the absence of a warning; look:
+
+```bash
+# 1. inside pi — the authoritative view
+/om:status          # visible-vs-full drift, thresholds, worker state
+/om:view            # what the agent currently sees
+/om:view full       # full ledger truth at the branch tip
+
+# 2. from a shell — are ledger entries being written, and has it compacted?
+grep -o '"customType":"om\.[a-z.]*"' \
+  "$(ls -t ~/.pi/agent/sessions/*/*.jsonl | head -1)" | sort | uniq -c
+grep -c '"type":"compaction"' "$(ls -t ~/.pi/agent/sessions/*/*.jsonl | head -1)"
+
+# 3. which copy is loaded, and at what commit
+python3 -c "import json;print(json.load(open('$HOME/.pi/agent/settings.json'))['packages'])"
+git -c safe.directory=/opt/pi-observational-memory -C /opt/pi-observational-memory rev-parse HEAD
+```
+
+Ledger entries are `"type":"custom"` with `"customType":"om.…"`. Do not grep for
+`custom_message` — that is a *different* pi API for entries that **do** enter LLM
+context, used here by the MemPalace mailbox (`customType: "mempalace-mailbox"`),
+not by om.
+
+## 11. It is not the same thing as MemPalace
+
+Both are called "memory" and they solve different problems. Nothing is wrong with
+running both — this image does, and they cover each other's failure modes.
+
+```mermaid
+flowchart LR
+    O0["observational<br/>memory"] --> O1["horizon:<br/>this session"]
+    O1 --> O2["scope: one branch,<br/>one machine"]
+    O2 --> O3["automatic"]
+    O3 --> O4["retrieval:<br/>recall(id)"]
+    P0["MemPalace"] --> P1["horizon: months,<br/>machines"]
+    P1 --> P2["scope:<br/>the fleet"]
+    P2 --> P3["protocol-driven"]
+    P3 --> P4["retrieval:<br/>search, KG, mailbox"]
+```
+
+| Question | Answer |
+|---|---|
+| "What did we decide 200 turns ago in *this* session?" | observational memory (and `recall` for the exact wording) |
+| "What did we decide last month, or on another machine?" | MemPalace (`mempalace_search`, diaries) |
+| "What is true *right now* about version X?" | MemPalace knowledge graph |
+| "Does another machine need something from me?" | MemPalace coordination log — see [Cross-machine agent coordination](../README.md#cross-machine-agent-coordination) |
+| "Why is compaction not losing my session?" | observational memory |
+
+The crisp version: **observational memory keeps a session coherent; the palace
+keeps the fleet coherent.** A container recreate wipes neither — but only because
+`~/.pi` and the palace both live outside the container filesystem.
+
+## 12. Gotchas
+
+- **Branch-local means branch-local.** Resuming or forking changes which ledger
+  is folded. Memory that "disappeared" is usually on another branch.
+- **`recall` needs an id, not a topic.** If you only have a topic, that is a
+  palace search, not a recall.
+- **A `/workspace` clone is not evidence of what is loaded** — see §7.3.
+- **`showWorkerNotifications: true` is not proof of work**; it reports runs, and
+  an observer that deliberately emits nothing writes no ledger entry and simply
+  retries after another `observeAfterTokens`.
+- **A turn bigger than `keepRecentTokens` splits.** The cut then lands mid-turn at
+  an assistant message and pi merges two summaries — rare, but it is why a very
+  large single turn can lose more verbatim detail than you would expect.
+- **`git log` in the baked tree needs `safe.directory`** (`/opt` is root-owned):
+  `git -c safe.directory=/opt/pi-observational-memory -C /opt/pi-observational-memory log`.

entrypoint-user.sh

@@ -7,7 +7,11 @@ set -euo pipefail
 # so this reaches the same stream as the interactive shell the user lands
 # in). Reads the ground-truth manifest baked in Dockerfile.variant; a no-op
 # with a short stderr notice on images built before it existed.
-command -v pi-devbox-version >/dev/null 2>&1 && pi-devbox-version || true
+# `--no-skills`: this runs FIRST, before the baked skill links are created
+# below and long before the skillset deploy + devbox-skill-reconcile run at the
+# end of this script, so the skill-source section would report a pre-reconcile
+# state that is about to change. Wrong-but-plausible is worse than absent.
+command -v pi-devbox-version >/dev/null 2>&1 && pi-devbox-version --no-skills || true
 
 # ── SSH ControlMaster socket dir ────────────────────────────────
 # Companion to /etc/ssh/ssh_config.d/00-devbox-controlmaster.conf in the

rootfs/usr/local/bin/pi-devbox-version

@@ -14,6 +14,8 @@
 #   pi-devbox-version           human-readable summary (default)
 #   pi-devbox-version --json    raw manifest JSON (for scripting)
 #   pi-devbox-version --quiet   one-line "release_tag (source_revision)" form
+#   pi-devbox-version --no-skills   skip the skill-source section (used at
+#                               container start, where it would be premature)
 #
 # EXIT STATUS
 #   0 on success. 1 if the manifest is missing (e.g. an image built before
@@ -24,15 +26,35 @@ set -euo pipefail
 
 MANIFEST=/etc/pi-devbox/build-manifest.json
 MODE="human"
+SHOW_SKILLS="yes"
 
-case "${1:-}" in
-  --json)  MODE="json" ;;
-  --quiet|-q) MODE="quiet" ;;
-  --help|-h)
-    sed -n '2,20p' "$0" | sed 's/^# \?//'
-    exit 0
-    ;;
-esac
+# A `case "${1:-}"` here only ever looked at the FIRST argument, so
+# `--no-skills --json` matched --no-skills, silently dropped --json, and
+# printed human text to a caller expecting JSON (a real failure: a jq
+# consumer piping that output gets a parse error, not a wrong-but-parseable
+# answer). Loop over every argument instead, and reject anything unknown
+# rather than silently ignoring it the same way.
+for _arg in "$@"; do
+  case "$_arg" in
+    --json)      MODE="json" ;;
+    --quiet|-q)  MODE="quiet" ;;
+    --no-skills) SHOW_SKILLS="no" ;;
+    --help|-h)
+      # Print the leading `#`-comment block verbatim, stopping at the first
+      # non-comment line, rather than a hardcoded line range: `sed -n
+      # '2,22p'` was silently truncating --help because this file has grown
+      # usage lines since that range was written, and a fixed range will
+      # drift again the next time a comment is added above it.
+      awk 'NR==1{next} /^#/{sub(/^# ?/,""); print; next} {exit}' "$0"
+      exit 0
+      ;;
+    *)
+      echo "pi-devbox-version: unknown option: $_arg" >&2
+      echo "  try --help" >&2
+      exit 2
+      ;;
+  esac
+done
 
 if [ ! -f "$MANIFEST" ]; then
   echo "pi-devbox-version: no build manifest at $MANIFEST" >&2
@@ -55,6 +77,10 @@ release_tag=$(jq -r '.release_tag' "$MANIFEST")
 build_date=$(jq -r '.build_date' "$MANIFEST")
 source_rev=$(jq -r '.source_revision' "$MANIFEST")
 pi_version_baked=$(jq -r '.pi_version' "$MANIFEST")
+# `// empty` matters: images built before v1.8.6 have no such field, and
+# `jq -r` renders a JSON null as the 4-char string "null" — which would
+# print as a bogus version rather than being treated as absent.
+mp_version_baked=$(jq -r '.mempalace_version // empty' "$MANIFEST")
 
 if [ "$MODE" = "quiet" ]; then
   printf '%s (%s)\n' "$release_tag" "${source_rev:0:7}"
@@ -71,6 +97,16 @@ if command -v pi >/dev/null 2>&1; then
   pi_version_live=$(pi --version 2>/dev/null | head -n1 | tr -d '\r\n')
 fi
 
+# Same check for the palace, which matters more than it looks: mempalace is
+# the one component that is BOTH client (here) and server (synlig runs this
+# same image), so a skew between the two is a real failure mode rather than
+# cosmetic. `mempalace --version` prints "MemPalace 3.8.0" — name-prefixed,
+# unlike pi's bare "0.84.3" — hence $NF rather than reading the whole line.
+mp_version_live=""
+if command -v mempalace >/dev/null 2>&1; then
+  mp_version_live=$(mempalace --version 2>/dev/null | head -n1 | awk '{print $NF}' | tr -d '\r\n')
+fi
+
 printf 'pi-devbox %s\n' "$release_tag"
 printf '  built:  %s  (source %s)\n' "$build_date" "${source_rev:0:12}"
 if [ -n "$pi_version_live" ] && [ "$pi_version_live" != "$pi_version_baked" ]; then
@@ -79,5 +115,120 @@ else
   printf '  pi:     %s\n' "${pi_version_live:-$pi_version_baked}"
 fi
 
+# Printed only when known, so this degrades quietly on pre-v1.8.6 images
+# instead of showing an empty or "null" palace line.
+if [ -n "$mp_version_live" ] || [ -n "$mp_version_baked" ]; then
+  if [ -n "$mp_version_live" ] && [ -n "$mp_version_baked" ] && [ "$mp_version_live" != "$mp_version_baked" ]; then
+    printf '  palace: %s  \033[33m(baked as %s — drift detected)\033[0m\n' "$mp_version_live" "$mp_version_baked"
+  else
+    printf '  palace: %s\n' "${mp_version_live:-$mp_version_baked}"
+  fi
+fi
+
 printf '  components:\n'
 jq -r '.components | to_entries[] | select(.value != null) | "    \(.key): \(.value[0:12])"' "$MANIFEST"
+
+# ── Which copy of each vendored skill is actually being read? ─────────
+# The image bakes fallback skills under /usr/local/share/pi-devbox/skills/,
+# but for skills the skillset repo OWNS (skillset-owned.txt) a mounted live
+# clone takes over at container start via devbox-skill-reconcile. Nothing
+# reported which copy won, so a stale baked snapshot and a current live clone
+# looked identical from inside — and on this fleet the baked mempalace copy is
+# read by NOBODY (all four compose stacks mount a workspace containing the
+# skillset), which is exactly the sort of fact that should be visible rather
+# than reasoned about. Same "drift detected" shape as the pi/palace lines
+# above: what is live, annotated with what was baked, when they disagree.
+#
+# Skipped with --no-skills at container start (entrypoint-user.sh calls this
+# FIRST, before the baked links exist and long before the skillset deploy and
+# reconcile run last), because a section that is accurate only after boot
+# finishes is worse than no section at all.
+BAKED_SKILLS=/usr/local/share/pi-devbox/skills
+SKILLS_DIR="${HOME:-/home/developer}/.agents/skills"
+
+if [ "$SHOW_SKILLS" = "yes" ] && [ -d "$BAKED_SKILLS" ] && [ -d "$SKILLS_DIR" ]; then
+  # Recorded provenance of the vendored mempalace snapshot (absent on images
+  # built before this existed — `// empty` so a JSON null never prints as the
+  # 4-char string "null", the same trap noted for mempalace_version above).
+  # `_tree_sha256`, not `_sha256`: it is a hash over every file in the
+  # vendored skill DIRECTORY (see tree_sha256() below), not one file, because
+  # a single-file hash reports "identical" against a live checkout that added
+  # or edited a sibling file — pi-extensions already ships two files, so this
+  # is not hypothetical.
+  snap_ref=$(jq -r '.skillset_snapshot_ref // empty' "$MANIFEST")
+  snap_sha=$(jq -r '.skillset_snapshot_tree_sha256 // empty' "$MANIFEST")
+  # Same pipeline Dockerfile.variant uses to measure the baked directory at
+  # build time: relative paths in `find | sort` order, each hashed, the whole
+  # listing folded into one sha256. Keep the two definitions identical — they
+  # run in different processes (image build vs. this container) and are
+  # meaningless to compare unless they agree byte-for-byte on the algorithm.
+  tree_sha256() {
+    ( cd "$1" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum ) 2>/dev/null | sha256sum | cut -d' ' -f1
+  }
+
+  # Iterate the baked tree rather than a hardcoded name list, so vendoring a
+  # fourth skill needs no edit here. The header prints only if the tree is
+  # non-empty, so this can never emit a dangling "skills:" label.
+  _printed_header="no"
+  for _dir in "$BAKED_SKILLS"/*/; do
+    [ -d "$_dir" ] || continue
+    if [ "$_printed_header" = "no" ]; then
+      printf '  skills:\n'
+      _printed_header="yes"
+    fi
+    _name=$(basename "$_dir")
+    _link="$SKILLS_DIR/$_name"
+
+    if [ ! -e "$_link" ]; then
+      printf '    %-22s not linked\n' "$_name"
+      continue
+    fi
+
+    _target=$(readlink -f "$_link" 2>/dev/null || echo "$_link")
+    case "$_target" in
+      "$BAKED_SKILLS"/*|"$BAKED_SKILLS")
+        printf '    %-22s baked\n' "$_name"
+        continue
+        ;;
+    esac
+
+    # Outside the baked tree: a mounted skillset clone, or a user override.
+    # The link target is <repo>/skills/<name>, so the repo root is two up.
+    # Everything here is guarded: this script runs on the container-start path
+    # and must never fail, and `set -e` is in force.
+    _root=$(cd "$_target/../.." 2>/dev/null && pwd) || _root=""
+    _head=""
+    if [ -n "$_root" ]; then
+      _head=$(git -C "$_root" rev-parse HEAD 2>/dev/null || echo "")
+    fi
+    _where="live ${_root:-$_target}"
+    [ -n "$_head" ] && _where="$_where @ ${_head:0:7}"
+
+    # For the one skill whose baked fingerprint we recorded, say plainly
+    # whether the live copy differs from what shipped. This is the check CI
+    # cannot perform (the skillset is private) and the container can, free.
+    # Hash the whole live DIRECTORY with the same tree_sha256() used to
+    # measure the baked one in Dockerfile.variant — a SKILL.md-only compare
+    # would silently ignore a changed or added sibling file.
+    _live_sha=""
+    if [ -n "$snap_sha" ] && [ "$_name" = "mempalace" ] && [ -d "$_target" ]; then
+      _live_sha=$(tree_sha256 "$_target")
+    fi
+    if [ -z "$_live_sha" ]; then
+      printf '    %-22s %s\n' "$_name" "$_where"
+    elif [ "$_live_sha" = "$snap_sha" ]; then
+      printf '    %-22s %s  (identical to baked snapshot)\n' "$_name" "$_where"
+    elif [ -n "$_head" ] && [ "$_head" = "$snap_ref" ]; then
+      # Same commit, different bytes — i.e. uncommitted edits in the live
+      # checkout. Distinguished from plain drift because otherwise the line
+      # reads as a self-contradiction ("@ c04cd15 ... baked snapshot c04cd15
+      # — live copy differs") and a reader would suspect the tool, not the
+      # working tree.
+      printf '    %-22s %s  \033[33m(baked snapshot %s + uncommitted edits)\033[0m\n' \
+        "$_name" "$_where" "${snap_ref:0:7}"
+    else
+      printf '    %-22s %s  \033[33m(baked snapshot %s — live copy differs)\033[0m\n' \
+        "$_name" "$_where" "${snap_ref:0:7}"
+    fi
+  done
+fi

rootfs/usr/local/lib/pi-devbox/setup-lan-access.sh

@@ -197,6 +197,45 @@ EOF
 )
 fi
 
+# ── Multiplexing default, deliberately LAST ───────────────────────────
+# Why this block exists: ControlPath above is forced, but ControlMaster is not
+# set anywhere for targets that come from the user's own ~/.ssh/config. A target
+# whose entry omits ControlMaster therefore opens a NEW TCP connection per ssh
+# call, and an agent doing a dozen calls in a few minutes can trip fail2ban or a
+# CGNAT flow-table cap on the far end — observed 2026-08-25: ~12 connections in
+# 15 min and port 22 stopped answering while HTTPS to the same estate stayed fine.
+#
+# WHY IT IS AT THE BOTTOM, and ControlPath is at the top. ssh_config is
+# first-value-wins, so position encodes intent:
+#   * BEFORE the Include = an OVERRIDE. Correct for ControlPath, whose value in
+#     the user's config points at read-only ~/.ssh and simply cannot work here.
+#   * AFTER  the Include = a DEFAULT. Correct for ControlMaster, because an
+#     explicit per-host 'ControlMaster no' (or 'auto', or any value) in the
+#     user's own config must keep winning. We are supplying an opinion only
+#     where the user expressed none.
+# That asymmetry is the whole design: force what is broken, default what is
+# merely absent. It also means this needs no audit of anyone's ~/.ssh/config —
+# which matters because that file is per-machine, differs across the fleet, and
+# future machines' versions do not exist yet to be audited.
+#
+# Caveat worth knowing (and documented in the pi-devbox-environment skill): a
+# stale master socket — file present, daemon gone, e.g. after the host suspends
+# or changes network — makes every later ssh to that host hang. Recovery is
+# 'ssh -F ~/.ssh-local/config -O exit <host>'. ControlPersist is deliberately
+# short (10m idle, and each new session resets the idle timer) so an abandoned
+# socket ages out on its own rather than lingering for hours.
+MULTIPLEX_DEFAULT_BLOCK=$(cat <<'EOF'
+
+# Multiplexing DEFAULT — intentionally after the Include above, so any explicit
+# per-host ControlMaster in your own ~/.ssh/config still wins (first-value-wins).
+# Applies only to targets that never mentioned ControlMaster at all.
+# Stale socket after a suspend/network change? ssh -O exit <host>.
+Host *
+    ControlMaster auto
+    ControlPersist 10m
+EOF
+)
+
 cat > "$CONFIG" <<EOF
 # AUTO-GENERATED by setup-lan-access.sh on every container start. Do not edit
 # by hand — edits are overwritten. Used via: ssh -F ~/.ssh-local/config <host>
@@ -216,6 +255,7 @@ ${JUMP_BLOCK}
 ${LAN_CONF_BLOCK}
 ${AUTOJUMP_BLOCK}
 ${INCLUDE_BLOCK}
+${MULTIPLEX_DEFAULT_BLOCK}
 EOF
 chmod 600 "$CONFIG" 2>/dev/null || true
 

rootfs/usr/local/share/pi-devbox/skills/VENDORED.md

@@ -39,6 +39,33 @@ its skill file needed baking.
   *different* skill, `opencode-mempalace-bridge`), so there is no public
   package source to copy from. This snapshot is refreshed manually per release.
 
+  **Refresh it with `scripts/vendor-mempalace-skill.sh <skillset-root>`, not
+  `cp`.** Because the image cannot clone the private upstream, the snapshot used
+  to be *anonymous* — nothing recorded which skillset commit the bytes came
+  from, so the only staleness check possible was a hand-maintained phrase canary
+  in `scripts/smoke-test.sh`, which by construction detects "older than the
+  phrase I remembered to pin", never "older than skillset main". Two facts now
+  travel with the file:
+
+  | Fact | Where | Kind |
+  |---|---|---|
+  | `ARG SKILLSET_SNAPSHOT_REF` in `Dockerfile.variant` | manifest `skillset_snapshot_ref` + OCI label `se.jordbo.pi-devbox.skillset-snapshot-ref` | a **claim** about which commit these bytes are |
+  | `sha256sum` of this file, measured in the manifest layer | manifest `skillset_snapshot_sha256` | the bytes that **actually shipped** |
+
+  The script writes both together, refuses when the upstream file has
+  uncommitted modifications (no commit describes those bytes), and
+  `--check` verifies the claim against a real clone. Deliberately an `ARG`
+  default rather than a CI-resolved value: no credential for a private repo, no
+  change at any of the four `Dockerfile.variant` build call sites, and a local
+  `docker build` records the same thing CI does.
+
+  Verifying "is this snapshot current?" is **not** a CI job and was deliberately
+  not made one — see the Unreleased CHANGELOG entry for why (private repo;
+  another repo's branch must not be able to fail this build; and the artefact it
+  would guard is read by no host on this fleet). The check belongs where the
+  skillset actually is: `vendor-mempalace-skill.sh --check` for a maintainer,
+  and `pi-devbox-version`'s `skills:` section for an agent inside a container.
+
 ## Runtime precedence (v1.8.5+)
 
 The baked links are created **early** in `entrypoint-user.sh` (before pi-deploy,
@@ -59,6 +86,21 @@ repoints the links for skills the **skillset owns**, listed one per line in
 3. **baked snapshot** — everything else, and every skill when no skillset is
    mounted
 
+**Which one won is now reportable from inside the container:**
+`pi-devbox-version` prints a `skills:` section naming, per vendored skill,
+`baked` or `live <repo> @ <sha>` — and for `mempalace` whether that live copy is
+identical to the baked fingerprint, at the same commit but with uncommitted
+edits, or genuinely divergent. Before that, a stale baked snapshot and a current
+live clone were indistinguishable from inside, which is how the freshness of
+this file went unexamined for three releases. The section is suppressed with
+`--no-skills` on the container-start banner, because `entrypoint-user.sh` prints
+the version *before* the links exist and long before the reconcile below runs.
+
+On this fleet, precedence 2 wins for `mempalace` on **every** host — all four
+compose stacks mount a workspace containing the skillset — so the baked copy is
+exercised only by CI and by a hypothetical no-mount container. Worth
+remembering before spending effort on its freshness.
+
 Ownership is per-skill on purpose: `pi-extensions`' authoritative source is the
 package repo (copied over the snapshot at build), and `skillset` carries a
 downstream copy that can lag, so handing it to the clone would *regress* the
@@ -72,15 +114,31 @@ mounted, plus a fabricated-skillset run of the reconciler).
 
     cp <pi-extensions-pkg>/skill/SKILL.md                    pi-extensions/SKILL.md
     cp <pi-extensions-pkg>/skill/evaluate-extension-usage.py pi-extensions/
-    cp <skillset>/skills/mempalace/SKILL.md                  mempalace/SKILL.md
 
-Copy each snapshot **from its owner in the table above** — `pi-extensions` from
-the package repo's `skill/` (since `a7f3044` co-located it there; `skillset`
-also carries a copy, but it is a downstream duplicate and can lag), and
-`mempalace` from `skillset`. Copying `pi-extensions` from `skillset` would
-regress the snapshot to whatever that repo last mirrored.
+Copy `pi-extensions` **from its owner in the table above** — the package
+repo's `skill/` (since `a7f3044` co-located it there; `skillset` also carries a
+copy, but it is a downstream duplicate and can lag). Copying `pi-extensions`
+from `skillset` would regress the snapshot to whatever that repo last mirrored.
 
-Snapshot provenance at last refresh: skillset `670f7f1`, pi-extensions pkg `e73cb9f`.
+`mempalace` is **not** refreshed by `cp` — see the *Freshness model* section
+above: `scripts/vendor-mempalace-skill.sh <skillset-root>` is the only thing
+that should ever touch that snapshot, because a bare copy can update the bytes
+without updating the ref that claims to describe them, which produces a
+manifest that confidently lies.
+
+Neither vendored skill has a hand-maintained "last refreshed at" line here on
+purpose — one previously existed (skillset `670f7f1`, pi-extensions pkg
+`e73cb9f`) and went stale within hours, because nothing forced it to move
+when the ARGs did. `670f7f1` is now a cautionary example rather than a fact
+worth recording: it is the commit that told agents to hand-stamp `added_by`,
+which a later skillset commit (and the pi-devbox edge stamper) withdrew — so a
+reader trusting that line would have been pointed at superseded guidance.
+Both facts it tried to capture now live somewhere that cannot drift by hand:
+
+| Fact | Where |
+|---|---|
+| which skillset commit `mempalace`'s bytes came from | `ARG SKILLSET_SNAPSHOT_REF` (Dockerfile.variant) + `skillset_snapshot_ref` in `build-manifest.json`, written *only* by `vendor-mempalace-skill.sh` |
+| which pi-extensions package commit was vendored | `ARG PI_EXTENSIONS_REF` (Dockerfile.variant, CI-resolved to a 40-hex commit) → OCI label `se.jordbo.pi-devbox.pi-extensions-ref` and `build-manifest.json`'s `components.pi-extensions`, both read from the actual `/opt/pi-extensions` checkout, not from intent |
 
 When you refresh the `mempalace` snapshot, also update the phrase asserted by
 the "mempalace skill snapshot is current" smoke test — it deliberately pins the

rootfs/usr/local/share/pi-devbox/skills/mempalace/SKILL.md

@@ -41,6 +41,21 @@ Run these immediately when a session begins, before responding to the user:
    mempalace_kg_query(entity="<project_or_person>")
    ```
 
+4. **Check your mailbox.** Just run it — an empty result is a fine answer and
+   costs one call. Do not try to decide first whether coordination "applies to
+   you"; that test is what used to be wrong here (see *Cross-Machine
+   Coordination* below):
+   ```
+   mempalace_event_list(to_agent="<harness>@<device>", status="open")
+   ```
+   This is a candidate list, not a to-do list — `status` never changes after an
+   event is written, so finished asks keep matching. Subtract the ones you have
+   already answered using the rule in *What you actually owe*, below.
+   Another machine may have asked you something, or corrected something you are
+   about to rely on. This costs one call and is the only way you will find out:
+   nothing pushes an event into your session unless your bridge delivers it for
+   you, and if it does you will already have seen it before reading this.
+
 Do NOT announce this to the user. Just do it silently to orient yourself.
 
 ### Temporal grounding — compute time deltas, don't guess
@@ -79,6 +94,79 @@ mempalace_search(query="<keywords>", wing="<project>")
 
 **Never guess about facts that might be in the palace.** Wrong is worse than slow. Say "let me check" and query.
 
+#### Search Before You *Probe*
+
+The rule above covers **questions**. This one covers **actions** — and it is the one
+that actually gets skipped, because mid-task the impulse is to go and *look* rather
+than to remember. The palace is a **fleet** record: another machine's agent has
+usually already paid the cost of discovering how this environment is wired, and its
+notes include the corrections that came afterwards, which a fresh probe cannot show
+you.
+
+**Before you SSH somewhere to find out how it is set up, enumerate infrastructure,
+or derive a deployment — search.** Concrete triggers, all meaning *search first*:
+
+- about to run `ssh <host> …`, `docker ps`, `systemctl list-units`, `ip addr` to
+  discover how something is deployed or connected
+- about to establish topology: which hosts/runners/services exist, where they live,
+  which of them can reach which
+- about to conclude "this isn't documented anywhere" or "there's no way to know"
+- about to assert an environment fact you learned **earlier in this same session**
+
+**That last trigger is the sharp edge.** A compacted session summary is lossy by
+design, and a belief you formed 40 turns ago may already be *retracted* in the
+palace by another machine. Trusting your own context over the shared record is how a
+withdrawn claim gets re-published as fact.
+
+Search broadly before narrowing — fleet knowledge often sits in another machine's
+wing, or inside a mined conversation, not where you would file it yourself:
+
+```
+mempalace_search(query="<topic> <host> <mechanism>")     # no wing filter first
+mempalace_search(query="…", wing="<likely-wing>")        # then narrow
+```
+
+Two or three searches cost seconds. Re-deriving infrastructure costs minutes **and
+can be wrong**: a probe shows one host's present state, while the palace records
+intent, history, and what was already disproved.
+
+> **Worked example (real, 2026-08-25).** An agent evaluating whether to add an ARM
+> CI runner probed hosts directly instead of searching. It concluded "the runner
+> lives on synlig" — there are **four** — and that "synlig is on the home LAN" —
+> it is an OpenStack VM with a public floating IP that cannot reach the home LAN at
+> all. Both facts were already in the palace, the second one as an **explicit
+> retraction of the very same mistake** made weeks earlier. The palace also held
+> the runner labels and the deliberate `capacity: 1` setting, which the probe never
+> revealed. Cost: a wrong recommendation written into the palace twice, then
+> corrected twice.
+
+
+**A search that comes back empty is not an answer — least of all about recent work.**
+Semantic search is weakest exactly where the fleet record is freshest: a drawer filed
+minutes ago is unranked against a keyword-shaped query, and the drawer you most need
+is *by construction* the newest one, because the other machine files its release,
+handoff and correction drawers at the **end** of its session. So a single miss proves
+nothing. **If the work is 0-2 days old and the first search looks stale or empty,
+enumerate before concluding:**
+
+```
+mempalace_list_drawers(wing="<wing>", since="<today>")   # or room=, or no filter
+mempalace_diary_read(agent_name="<you>", wing="<wing>")  # the other machine's handoff
+```
+
+Enumeration is exact where embeddings are probabilistic. Treat "I searched and found
+nothing" as a hypothesis you have not yet tested, and never as licence to go probing.
+
+> **Worked example (real, 2026-08-25, same fleet as above).** An agent asked to
+> orient on an in-flight release *did* search first — `"v1.8.6 release run 579
+> Docker Hub verification"` — and got back only v1.6.4 / v0.78.0 era hits, because
+> the release drawer it needed was **58 seconds old**. It accepted the miss and went
+> off to probe Docker Hub and the Gitea API. The user had to prompt "maybe there is a
+> note in mempalace"; `list_drawers(wing="pi-devbox", since=<today>)` then returned
+> the drawer immediately, along with the diary entry naming the exact open item. The
+> rule above was present and correct in this very file at the time — the failure was
+> not knowing to *retry differently* after a bad first hit.
+
 #### Mine New Projects
 
 When working on a new codebase for the first time:
@@ -267,6 +355,176 @@ mempalace_kg_invalidate(subject="...", predicate="...", object="...", ended="<to
 mempalace_kg_add(subject="...", predicate="...", object="...", valid_from="<today>")
 ```
 
+## Cross-Machine Coordination — the logstream
+
+The palace stores what you *know*. The logstream (`mempalace_event_*`,
+`mempalace_artifact_*`) carries what you want to *say to another agent* —
+delegation, review, patch handoff, retraction. It is the only channel on which
+another machine can reach you.
+
+**Does this apply to you at all? Do not use `mempalace_mesh_peers` to decide.**
+It answers a different question than it appears to. A shared palace can be
+*hub-and-spoke* — many machines as thin clients of one central replica — and
+then `mesh_peers` reports `peers: []` because there are no peer *replicas*,
+even while four machines are actively writing to the same log. Measured on this
+fleet: `peers: []`, one replica authoring every event from every machine. An
+earlier version of this section told you to read `mesh_peers` and skip the
+mailbox when it came back empty, which disabled the mailbox on precisely the
+fleet it was written for.
+
+The honest discriminators, cheapest first: **just run the mailbox query** (empty
+is a fine answer); check whether `MEMPALACE_REMOTE_URL` is set, which is what
+actually selects a shared palace; or look for any event whose `from_agent` is
+not you. On a solitary palace the event tools still work — you are writing to
+yourself and your mailbox stays empty. That is not a fault to debug.
+
+**It is a durable log, not a bus — nobody is "listening".** Events are appended
+and persist; there is no subscription, no delivery window, and nothing is lost
+by being offline when one is written. A message waits indefinitely for you, and
+your reply waits just as patiently for a sender who has since gone away. Machines
+in a fleet are rarely awake at the same time, which is exactly why this is a log
+and not a chat.
+
+**Agent name is the only identity the log has.** Depending on deployment, every
+client may share one `origin_replica` — on the fleet this skill was written for,
+all machines are thin MCP clients of a single central replica, so `origin_replica`
+is identical for every event and cannot tell two machines apart. `from_agent` /
+`to_agent` carry the whole distinction, which is why the `<harness>@<device>`
+stamping in *Provenance is stamped for you* is load-bearing here and not mere
+tidiness.
+
+### Reading your mailbox
+
+```
+mempalace_event_list(to_agent="<harness>@<device>", status="open")
+```
+
+- `to_agent=<you>` **also matches `*` broadcasts**, so one call covers both. No
+  second query needed.
+- `status="open"` narrows the mailbox to what a sender *said was an ask at the
+  time of writing* — that is all it can do. It is a good first filter (on a real
+  stream it cut 5 events to 2), but it is **not** a list of what you owe, and it
+  never shrinks as you work. Treating it as owed-ness is the mistake this
+  section previously made: an earlier draft cited "5 unfiltered, exactly 1
+  filtered — the one that needed a reply" as proof the filter tracked
+  obligation. It did not. That single result was an event which had *already
+  been acked* half an hour earlier; the filter looked decisive only because the
+  stream happened to contain one directed `open` event. **Unfiltered mailboxes
+  train you to ignore them — and so does a filter that keeps showing you
+  finished work.**
+- To resume where you left off, use `since_event_id`, **never**
+  `since_created_at`. A timestamp cursor permanently skips an event that synced
+  in late — it is a time window ("what happened today"), not a cursor.
+- Read `metadata` before acting: senders put the load-bearing specifics there
+  (which host verified what, which run failed, what a change retracts).
+
+### The ack contract — the sender declares whether a reply is owed
+
+An obligation you never agreed to is noise, so the sender states it:
+
+| Sender writes | Means | Recipient owes |
+|---|---|---|
+| `to_agent="<specific agent>"` + `status="open"` | an ask | an ack or a reply (the event itself keeps matching forever — see below) |
+| `to_agent="*"` (any status) | broadcast FYI | nothing |
+| any other status (`ready`, `applied`, `blocked`, …) | a statement of fact | nothing |
+
+Ack with `mempalace_event_ack(event_id=…, from_agent="<you>", status=…)`. It
+**appends a new event** and never mutates the original; the correlation id is
+copied for you, and `metadata.ack_of` is set to the event you answered.
+
+#### What you actually owe — derive it, do not read it off `status`
+
+The log is append-only and `status` is written **once**, so it is an honest
+statement about an item *at the moment it was written* and nothing more. It is
+not mutable state, and asking it to carry mutable state is what breaks:
+acking appends a new event and changes nothing about the old one, so **a
+directed `open` event matches your mailbox query forever, answered or not.**
+Nothing is ever "dismissed" — which also means a deferred ask cannot be
+accidentally lost, only that you must compute what is outstanding:
+
+```
+candidates = mempalace_event_list(to_agent="<you>", status="open")
+mine       = mempalace_event_list(from_agent="<you>")
+```
+
+A candidate is **answered** when one of your own events
+
+1. has a **higher `seq`** than the candidate, and
+2. joins to it — `metadata.ack_of == candidate.id` (exact, written for you by
+   `event_ack`) or the same `correlation_id` (the fallback), and
+3. carries a **terminal** status: `applied`, `superseded`, `failed`, `blocked`.
+
+Everything else is still owed. Two calls, constant cost.
+
+**Compare `seq`, never `created_at`** — the same reason you resume with
+`since_event_id`. Without the ordering test, one terminal reply would suppress
+every later ask on the same `correlation_id` for good; verified on a live thread
+where a `ready` reply at `seq` 16 sits *before* the request at `seq` 17 that it
+obviously cannot have answered.
+
+**On a real mesh, compare `hlc` instead.** `seq` is *replica-local*: it equals
+`origin_seq` today only because a single replica authors events for every
+machine. Enrol a second replica and a late-syncing peer event gets a late local
+`seq`, so two replicas can order the same pair differently and derive different
+owed-sets from the same log. Every event already carries `hlc`
+(`<millis>-<counter>-<replica_id>`), which is total and causally consistent.
+So: compare `seq` while `mempalace_mesh_peers` reports no peers, `hlc` once it
+reports any, and `created_at` never. (This is a legitimate use of `mesh_peers` —
+choosing an ordering key — not the discredited gate on *whether* to read your
+mailbox at all.)
+
+**The failure directions are not symmetric, which is why this is safe to get
+slightly wrong.** Local-`seq` skew can make an already-answered item *resurface*
+as owed: noise, self-correcting, and visible. A timestamp comparison can
+*suppress an unanswered ask forever*: silent and permanent. So if you ever see an
+item you know you answered come back, do **not** "fix" it by reaching for
+`created_at` — you would be trading the safe failure for the dangerous one.
+
+This also supplies the "taken, not finished" state that looked missing:
+`claimed` and `ready` are deliberately **not** terminal, so work you have picked
+up keeps resurfacing until you close it out. No extra convention, no new field.
+
+Two consequences worth internalising:
+
+- **"Seen, not doing it" is a legitimate ack** — `status="blocked"` or
+  `"superseded"` plus the reason. Silence is not, and it is not merely rude:
+  with no terminal event of yours to join to, the ask stays in the owed set
+  indefinitely and there is nothing anyone can do about it from the other end.
+- **Nothing expires, and it should not.** An `open` with no terminal reply is
+  still live by definition, and the finished threads are valuable history. If
+  content is genuinely perishable ("do not push to main for the next hour"), say
+  so in `metadata.expires_at` — metadata is stored verbatim — and honour it as a
+  hint when reading. An old `open` that the derivation still counts as owed is a
+  signal, not garbage: it means somebody asked and nobody answered.
+
+### Writing to another machine
+
+- **Address the stamped name you actually saw** in a `from_agent` field, e.g.
+  `pi@tor-ms22`. A bare `pi` reaches nobody's mailbox once stamping is live, and
+  older events in the log still carry bare names — do not copy them.
+- **Use `status="open"` only when you truly need an answer.** It places an
+  obligation on another machine.
+- **Never broadcast an ask.** `to_agent="*"` + `status="open"` obliges everyone
+  and therefore no one.
+- **Always set a `correlation_id` on a directed `open`,** and reply with the
+  same one. It is not just for reconstructing a conversation later: it is the
+  join the owed-set derivation depends on. An uncorrelated ask can only ever be
+  closed by an `event_ack` (which sets `ack_of` for you) — a plain reply cannot
+  be matched to it at all.
+- **Corrections are new events, never edits.** Say explicitly what you retract
+  and name the id — drawer or event — that carried the withdrawn claim.
+- **Put a retraction where the reader will look.** An event reaches a live agent;
+  a *drawer* is what a future semantic search finds. If you filed advice as a
+  drawer and later withdraw it, file the withdrawal as a drawer too — otherwise
+  the next agent finds your original confident advice and no trace of the
+  correction. (This is a real incident, not a hypothetical.)
+- **Hand over exact content as an artifact**, not prose: `mempalace_artifact_put`
+  or `mempalace_patch_submit` store bytes with a sha256, and the event references
+  the id. Never paste a diff into a body and hope it survives.
+- **Waiting on a specific reply?** `mempalace_event_wait` blocks with backoff —
+  do not poll `event_list` in a loop. A timeout there is a normal result, not an
+  error.
+
 ## Palace Structure
 
 ### Wings
@@ -290,13 +548,14 @@ Zechner's pi-coding-agent). Implications:
 - **Session feeders run on different schedules.** Pi sessions are fed Tue 03:00, opencode sessions Mon 03:00 (launchd `Weekday`: `0`/`7`=Sunday, `1`=Monday, `2`=Tuesday — misreading this by one day is easy). Recent sessions from either harness can lag the palace by up to a week, so absence-of-evidence in `wing_conversations` is not evidence-of-absence for recent work.
 - **Reading another harness's diary is useful.** When orienting after a gap, `mempalace_diary_read agent_name=pi` (or whichever sibling agent has been active) often gives a fresher picture than waiting for the conversations feeder to catch up.
 
-When the palace is **central** (shared across machines), five more things apply:
+When the palace is **central** (shared across machines), these further things apply:
 
 - **Check which machine a conversation came from.** Transcripts are fed per device, so `source_path` reads `…/mempalace-feed/<device>/pi_<uuid>.jsonl` while the displayed `source_file` is only the basename. One search can legitimately return hits from several machines at once — look at the device segment before attributing a decision to *this* project.
-- **Attribute what you file yourself.** Drawers now carry `device` and `agent_kind` metadata (plus `device_source`/`agent_kind_source` recording *how* each was determined, so an inference is never mistaken for a fact). Mined content gets these for free — the inbox path gives the device, the filename shape gives the harness — and a timer on the palace host re-stamps hourly, because live re-mining replaces metadata rows and silently drops earlier stamps. But for anything **you** file by hand, the only signal is what you pass: set `added_by="<harness>@<device>"` (e.g. `pi@emb-7kj4vr4g`, from `$MEMPALACE_PI_DEVICE`) on `add_drawer`/`checkpoint`/`mine`. Skip it and your drawer joins the ~16k historic `/workspace` project mines that are permanently unattributable, because `/workspace` exists identically on every devbox. Note the palace preserves `source_file` in full (see `source_path`) but *displays* only the basename — so a device prefix there survives storage even though it looks stripped.
+- **Provenance is stamped for you — leave it alone.** Drawers carry `device` and `agent_kind` metadata (plus `device_source`/`agent_kind_source` recording *how* each was determined, so an inference is never mistaken for a fact). You do **not** set these, and you no longer set `added_by` either: the pi bridge defaults the writer field to `<harness>@<device>` on `add_drawer`/`checkpoint`/`mine`/`event_append`/`artifact_put`, and prefixes diary entries with `HOST:<device>|`, from host-supplied `$MEMPALACE_PI_DEVICE`. RFC 001 §7.3.2 ranks "agent stamps it via a skill instruction" as the *worst possible* place for exactly the reason you would expect — it is per-call boilerplate that gets forgotten, and it did: the agent who wrote the previous version of this bullet then filed its own provenance drawer as `added_by=checkpoint`. **Confirm the bridge in your image actually stamps before trusting it:** the extension is baked at image build time, so a container on an image older than the stamping commit (pi-devbox < v1.8.7) stamps nothing while still satisfying both gates — the env vars are set and the code is simply absent. Check with `grep -c MEMPALACE_PI_DEVICE "$(readlink -f ~/.pi/agent/extensions/mempalace.ts)"`; zero means keep passing `added_by="<harness>@<device>"` and a manual `HOST:<device>|` diary prefix until the container is recreated on a newer image. Two things remain yours: pass `source_drawer_id` on `kg_add` (triples have no provenance field, so that pointer is the only path back to a device), and pass an explicit `added_by` **only** when deliberately filing on behalf of another device. Never invent values for `device`/`agent_kind`/`origin_device` — a fabricated value is worse than a blank, because it silently corrupts a future merge.
+- **Metadata is invisible to search — so check the text, not the fields.** `search` results are built from a fixed key list and `diary_read` returns content, so neither ever shows `device`/`added_by`. Only `mempalace_get_drawer` reveals them. This is why diary entries carry an in-text `HOST:<device>` marker: it is the only attribution a reader actually sees. **A diary entry with no `HOST:` marker predates the convention and may be from any machine — do not assume it is this one's history.**
 - **Mined drawers carry the MINE date, not the session date.** When history is imported, or re-mined on the palace host, `filed_at`/`created_at` is the *import* time — so sorting by them does not give chronological order. Real session time is recoverable from the UUIDv7 in `pi_<uuid>.jsonl`: the first 12 hex digits are milliseconds since the epoch (and UUIDv7 sorts lexicographically in time order, so a plain filename sort is already chronological). Agent-authored drawers and diaries have no such backdoor — for those `filed_at` is the only chronology, which is why it must never be restamped.
 - **Beware the timezone mismatch when you combine those.** Palace `filed_at`/`created_at` are naive timestamps in the palace host's local time, while a UUIDv7 decodes to UTC. Comparing them directly introduces a silent offset (2 h for a CEST host). Normalise before drawing conclusions about ordering.
-- **`agent_name` is not device-scoped.** `mempalace_diary_read(agent_name="pi")` returns *every* machine's `pi` diary, interleaved. Read the entry before assuming it is your own history.
+- **`agent_name` is not device-scoped.** `mempalace_diary_read(agent_name="pi")` returns *every* machine's `pi` diary, interleaved. Read the entry before assuming it is your own history — and note that a container cannot tell you which machine it is on (`hostname` is a docker hash, `$DEVBOX_HOST_ALIAS` is generic). `$MEMPALACE_PI_DEVICE` is the cheap answer; `ssh -F ~/.ssh-local/config host hostname` is the independent one.
 - **One writer, no queue.** A concurrent mine returns a structured `already-running` error rather than waiting its turn, and one large mine can make the palace unresponsive to every client for minutes. After another client's mine, call `mempalace_reconnect` to see the new drawers. A client-side timeout is not evidence of failure — verify before retrying, or you file a duplicate.
 
 ### Rooms
@@ -330,10 +589,15 @@ Entity-relationship triples with temporal validity. Query with `mempalace_kg_que
 ## Anti-Patterns
 
 - **Don't guess when you can search.** If a question touches past work, search first.
+- **Don't probe what the fleet already knows.** Before SSH-ing into a host, enumerating infrastructure, or deriving how something is deployed, search the palace. A probe reveals one host's present state; the palace holds intent, history and prior corrections — including the ones that contradict what you are about to conclude.
+- **Don't trust this session's context over the palace.** A compacted summary is lossy, and another machine may have corrected the fact since. Verify load-bearing environment claims against the shared record before acting on them.
+- **Don't take one empty search as proof the palace is silent.** Fresh drawers rank worst, and the drawer that matters is usually the newest one. For anything 0-2 days old, enumerate with `mempalace_list_drawers(since=…)` and read the other machine's diary before you go and probe.
 - **Don't infer elapsed time from session or container boundaries.** A restart isn't a new day. Compare the actual timestamp (`timestamp` / `created_at`) against the current date/time before saying "yesterday", "last week", etc.
 - **Don't skip the diary.** A session without a diary entry is a session forgotten.
 - **Don't summarize drawer content.** File verbatim — the embedding model needs the original words.
 - **Don't mine .git directories or node_modules.** The CLI miner respects .gitignore by default.
 - **Don't create duplicate drawers.** Use `mempalace_check_duplicate` before adding manually.
 - **Don't treat the palace as a task list.** It's for knowledge and context, not todos.
-- **Don't hand-craft provenance.** Leave `added_by` alone (and never put a machine name in a diary's `agent_name` — it becomes the wing name and hides your entries from `diary_read`). Recording *which device* wrote a record is client/server infrastructure, not your job: a hostname or container ID is not a stable identity, and an invented value is worse than none because it silently corrupts any future palace merge. If you find notes in the palace describing an `origin_device` scheme, that is a design for the client to implement — not an instruction for you to start stamping.
+- **Don't broadcast an ask, and don't leave one unanswered.** On a shared palace, `to_agent="*"` + `status="open"` obliges every machine and therefore none of them. And don't expect acking to tidy your mailbox: `status` is immutable, so the event keeps matching either way — what a terminal reply buys you is that the *derived* owed set (see *What you actually owe*) stops counting it. Leave asks unanswered and that set only grows, until everyone learns to stop looking. "Seen, not doing it" is a complete answer — silence is not.
+- **Don't assume you would have heard.** Nothing pushes another machine's message into your session. If you did not run the mailbox query at wake-up, a correction addressed to you by name can sit unread while you confidently rebuild the thing it warned you about.
+- **Don't invent provenance metadata, and don't hand-stamp it either.** An earlier version of this list told you to set `added_by="<harness>@<device>"` by hand; that instruction has been withdrawn, because RFC 001 §7.3.2 places provenance at the client/server boundary and the pi bridge now does it uniformly (see *Provenance is stamped for you* above) — but the withdrawal only holds where the bridge is live, so run the one-line check in that bullet first; on an older image hand-stamping is still the only signal a hand-filed drawer gets. DO NOT invent values for the palace's own metadata fields (`device`, `agent_kind`, `origin_device`): those are stamped by infrastructure that also records *how* each was determined, and a fabricated value is worse than none because it silently corrupts a future merge. DO pass `source_drawer_id` on `kg_add`. And never put a machine name in a diary's `agent_name` — it becomes the wing name and hides your entries from `diary_read`.

rootfs/usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md

@@ -185,6 +185,16 @@ entrypoint's `setup-lan-access.sh` writes a **writable SSH sidecar** at
 - A `Host *` block redirecting `ControlPath` into the writable `~/.ssh-local/cm`
   (because `~/.ssh` is typically bind-mounted **read-only**, so a master socket
   can't be created under it), plus `Include ~/.ssh/config`.
+- A **trailing** `Host *` block supplying `ControlMaster auto` + `ControlPersist
+  10m` as a *default*. Position is the design: `ControlPath` sits **before** the
+  `Include` (an override — the value in your own config points at read-only
+  `~/.ssh` and cannot work here), while `ControlMaster` sits **after** it (a
+  default — an explicit per-host `ControlMaster no`/`auto` in your own config
+  still wins, because ssh_config is first-value-wins). **Force what is broken,
+  default what is merely absent.** Without this, a target whose entry never
+  mentioned `ControlMaster` opens a fresh TCP connection per `ssh` call, and an
+  agent making a dozen calls in a few minutes can trip fail2ban or a CGNAT
+  flow-table cap on the far end.
 - Aliases **`host` / `mac`** → `host.docker.internal` (user comes from
   `HOST_SSH_USER`) — i.e. SSH back into the Docker host.
 - On VM-backed hosts only: an **SSH-jump-via-host** block so the container can
@@ -199,6 +209,25 @@ ssh -F "$HOME/.ssh-local/config" mac 'hostname; whoami'      # reach the host
 ssh -F "$HOME/.ssh-local/config" <lan-peer> '…'             # reach a LAN peer (if configured)
 ```
 
+**Always go through the sidecar, never `-F ~/.ssh/config`.** This is the single
+easiest way to break SSH from inside the container, and the failure actively
+misleads: the read-only path makes the master socket uncreatable, so
+multiplexing appears *impossible* rather than misconfigured. What follows is a
+burst of fresh connections and, on a rate-limiting peer, a block that looks like
+an outage. The tell that it is rate-limiting and not an outage: HTTPS to the same
+estate keeps working while port 22 stops answering. (Recorded 2026-08-25 — an
+agent hit exactly this, concluded "ControlMaster is impossible here", disabled
+multiplexing, and filed that as a lesson. The sidecar had solved it since v1.4.)
+
+If every `ssh` to one host suddenly hangs, suspect a **stale master** — socket
+file present, daemon gone, typically after the host suspended or changed
+network. Check and clear it:
+
+```sh
+ssh -F "$HOME/.ssh-local/config" -O check <host>   # "Master running (pid=…)" or no master
+ssh -F "$HOME/.ssh-local/config" -O exit  <host>   # tear down a stale one
+```
+
 Two related mechanisms (don't reinvent them):
 
 - **ControlMaster multiplexing** is preconfigured (`/tmp/sshcm/`) to survive

scripts/recreate-sanity-check.sh

@@ -2,8 +2,8 @@
 # Runtime post-recreate verification for pi-devbox.
 #
 # Verifies that after `docker compose up -d --force-recreate`:
-#   - The new image is actually live (pi version matches, when an expected
-#     version is supplied — see the version note below)
+#   - The new image is actually live (both the pi version and — when asked —
+#     the pi-devbox image release tag; see the two version notes below)
 #   - Persisted named volumes survived (~/.pi config, shell history, zoxide,
 #     nvim data, uv cache, ssh-local)
 #   - pi runtime wiring is intact: keybindings symlink, AGENTS.md symlink,
@@ -25,13 +25,33 @@
 # the pi-devbox repo (which a maintainer already has for CI builds). A plain
 # `docker pull` consumer is not the audience and will not have this file.
 #
-# Version note: pi's version is resolved from `latest` at CI build time and is
-# NOT pinned to a concrete value in Dockerfile.variant (ARG PI_VERSION=latest).
-# So unlike opencode-devbox, this script cannot self-derive an expected version
-# from the Dockerfile. Pass --expected-version to assert a match; without it the
-# live pi version is reported as an informational WARN, not a failure.
+# TWO DIFFERENT VERSIONS, TWO DIFFERENT FLAGS. This distinction has already
+# cost a release day, so it is spelled out here and in AGENTS.md step 4:
 #
-# Usage: ./scripts/recreate-sanity-check.sh [--expected-version X.Y.Z] [--variant studio|plain]
+#   --expected-version        the PI CODING AGENT version, e.g. 0.84.3
+#                             (`pi --version`; pinned as ARG PI_VERSION in
+#                             Dockerfile.variant, which CI reads as the source
+#                             of truth)
+#   --expected-image-version  the PI-DEVBOX IMAGE release tag, e.g. 1.8.9 or
+#                             v1.8.9 (the `release_tag` baked into
+#                             /etc/pi-devbox/build-manifest.json)
+#
+# Passing a release tag to --expected-version used to report
+# "pi version mismatch: expected 1.8.8, got 0.84.3" — an accusation aimed at
+# the wrong component, on the last gate of a release. Both flags now detect
+# being handed the other one's value and say so instead.
+#
+# Neither flag is required. Both values are derivable from the image's own
+# build manifest, so by default the script asserts the LIVE pi version against
+# the version recorded at build time — which is not a tautology: a stale
+# `pi` in the ~/.pi/npm-global volume can shadow the baked one, exactly the
+# way a stale npm:pi-atelier can (see the packages[] check below). Pass the
+# flags when you want an assertion against a value you name yourself, which
+# is what a release checklist wants.
+#
+# Usage: ./scripts/recreate-sanity-check.sh [--expected-version X.Y.Z]
+#                                           [--expected-image-version X.Y.Z]
+#                                           [--variant studio|plain]
 #
 # Exit codes:
 #   0  all checks passed
@@ -41,22 +61,61 @@
 set -euo pipefail
 
 EXPECTED_VERSION=""
+EXPECTED_IMAGE_VERSION=""
 VARIANT=""
 REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+MANIFEST=/etc/pi-devbox/build-manifest.json
 
-# Parse arguments
+usage() {
+  cat >&2 <<'EOF'
+usage: recreate-sanity-check.sh [--expected-version X.Y.Z]
+                                [--expected-image-version X.Y.Z]
+                                [--variant studio|plain]
+
+  --expected-version        pi coding agent version, e.g. 0.84.3 (`pi --version`)
+  --expected-image-version  pi-devbox image release tag, e.g. 1.8.9 or v1.8.9
+  --variant                 studio|plain (auto-detected when omitted)
+
+These are two different versions. Both are read from the image's own build
+manifest when the corresponding flag is omitted.
+EOF
+}
+
+# Parse arguments. Every flag takes a value, so reject a missing one rather
+# than swallowing the next flag as if it were the value.
+need_value() {
+  case "${2:-}" in
+    ""|-*)
+      echo "$1 requires a value" >&2
+      usage
+      exit 2
+      ;;
+  esac
+}
 while [[ $# -gt 0 ]]; do
   case "$1" in
     --expected-version)
+      need_value "$@"
       EXPECTED_VERSION="$2"
       shift 2
       ;;
+    --expected-image-version)
+      need_value "$@"
+      EXPECTED_IMAGE_VERSION="$2"
+      shift 2
+      ;;
     --variant)
+      need_value "$@"
       VARIANT="$2"
       shift 2
       ;;
+    --help|-h)
+      usage
+      exit 0
+      ;;
     *)
-      echo "usage: $0 [--expected-version X.Y.Z] [--variant studio|plain]" >&2
+      echo "unknown option: $1" >&2
+      usage
       exit 2
       ;;
   esac
@@ -67,6 +126,19 @@ pass() { echo "  ✓ $1"; }
 fail() { echo "  ✗ $1" >&2; FAILED=$((FAILED + 1)); }
 warn() { echo "  ⚠ $1" >&2; }
 
+# Read one top-level field from the build manifest, or print nothing. The
+# manifest is the image's own ground truth (written at `docker build` time by
+# Dockerfile.variant), so it needs no checkout and no network. Absent on an
+# image built before it existed, hence every caller treats "" as unknown.
+manifest_field() {
+  [ -f "$MANIFEST" ] || return 0
+  command -v jq >/dev/null 2>&1 || return 0
+  jq -r --arg k "$1" '.[$k] // empty' "$MANIFEST" 2>/dev/null || true
+}
+# Release tags are written with a leading v in the manifest and quoted without
+# one in checklists; compare on the bare number so both spellings work.
+strip_v() { printf '%s' "${1#v}"; }
+
 # Auto-detect variant if not provided. The studio variant vendors pi-studio to
 # /opt/pi-studio; the plain variant does not.
 if [ -z "$VARIANT" ]; then
@@ -86,21 +158,59 @@ else
 fi
 echo
 
-echo "-- pi version --"
+MANIFEST_PI_VERSION=$(manifest_field pi_version)
+MANIFEST_RELEASE_TAG=$(manifest_field release_tag)
+
+echo "-- pi (coding agent) version --"
 if ACTUAL_VERSION=$(pi --version 2>&1 | head -1); then
   if [ -n "$EXPECTED_VERSION" ]; then
-    if [ "$ACTUAL_VERSION" = "$EXPECTED_VERSION" ]; then
-      pass "pi version $ACTUAL_VERSION"
+    if [ "$(strip_v "$EXPECTED_VERSION")" = "$(strip_v "$ACTUAL_VERSION")" ]; then
+      pass "pi version $ACTUAL_VERSION (matches --expected-version)"
+    elif [ -n "$MANIFEST_RELEASE_TAG" ] &&
+         [ "$(strip_v "$EXPECTED_VERSION")" = "$(strip_v "$MANIFEST_RELEASE_TAG")" ]; then
+      # Exact, not heuristic: the value handed over IS this image's release
+      # tag, so it cannot be a pi version anyone meant.
+      fail "--expected-version $EXPECTED_VERSION is the pi-devbox IMAGE version, not the pi version — use --expected-image-version $EXPECTED_VERSION (live pi is $ACTUAL_VERSION)"
     else
-      fail "pi version mismatch: expected $EXPECTED_VERSION, got $ACTUAL_VERSION"
+      fail "pi version mismatch: expected $EXPECTED_VERSION, got $ACTUAL_VERSION (this flag asserts the pi coding agent version; for the image release tag use --expected-image-version)"
+    fi
+  elif [ -n "$MANIFEST_PI_VERSION" ]; then
+    # Not a tautology: the manifest records what pi reported at BUILD time,
+    # while `pi --version` resolves through PATH, which a stale npm-global
+    # volume install can shadow.
+    if [ "$MANIFEST_PI_VERSION" = "$ACTUAL_VERSION" ]; then
+      pass "pi version $ACTUAL_VERSION (matches this image's build manifest)"
+    else
+      fail "live pi $ACTUAL_VERSION != $MANIFEST_PI_VERSION recorded in $MANIFEST — a stale pi in the ~/.pi/npm-global volume is shadowing the baked one"
     fi
   else
-    warn "pi version $ACTUAL_VERSION (no --expected-version given; pi is built from 'latest', cannot self-derive — informational only)"
+    warn "pi version $ACTUAL_VERSION (no --expected-version and no build manifest to compare against — informational only)"
   fi
 else
   fail "pi --version failed"
 fi
 
+echo
+echo "-- pi-devbox image version --"
+if [ -z "$MANIFEST_RELEASE_TAG" ]; then
+  if [ -n "$EXPECTED_IMAGE_VERSION" ]; then
+    fail "cannot verify --expected-image-version $EXPECTED_IMAGE_VERSION: no readable release_tag in $MANIFEST (image built before the manifest existed, or jq missing)"
+  else
+    warn "image release tag unknown (no readable $MANIFEST) — pi-devbox-version would say the same"
+  fi
+elif [ -n "$EXPECTED_IMAGE_VERSION" ]; then
+  if [ "$(strip_v "$EXPECTED_IMAGE_VERSION")" = "$(strip_v "$MANIFEST_RELEASE_TAG")" ]; then
+    pass "image version $MANIFEST_RELEASE_TAG (matches --expected-image-version)"
+  elif [ -n "$MANIFEST_PI_VERSION" ] &&
+       [ "$(strip_v "$EXPECTED_IMAGE_VERSION")" = "$MANIFEST_PI_VERSION" ]; then
+    fail "--expected-image-version $EXPECTED_IMAGE_VERSION is the pi version, not the image release tag — use --expected-version $EXPECTED_IMAGE_VERSION (this image is $MANIFEST_RELEASE_TAG)"
+  else
+    fail "image version mismatch: expected $EXPECTED_IMAGE_VERSION, got $MANIFEST_RELEASE_TAG — the recreate did not pick up the intended image"
+  fi
+else
+  warn "image version $MANIFEST_RELEASE_TAG (no --expected-image-version given — informational only)"
+fi
+
 echo
 echo "-- Persisted named volumes (must survive --force-recreate) --"
 

scripts/smoke-test.sh

@@ -5,8 +5,9 @@
 #
 # Verifies:
 #   - pi binary present and (if EXPECTED_PI_VERSION set) matches CI's resolved version
+#   - mempalace core matches the audited pin (if EXPECTED_MEMPALACE_VERSION set)
 #   - new v1.0.0 base additions (pandoc, graphviz, imagemagick, yq, tealdeer)
-#   - typst PDF engine for pandoc (Unreleased) — `pandoc --pdf-engine=typst`
+#   - typst PDF engine for pandoc (v1.4.0) — `pandoc --pdf-engine=typst`
 #   - non-modal editors nano + micro (alongside nvim)
 #   - terminfo for modern emulators: xterm-kitty, xterm-ghostty, wezterm,
 #     alacritty, foot (kitty-terminfo + ncurses-term + compiled ghostty alias)
@@ -151,6 +152,33 @@ run "pi stage follows MEMPALACE_PALACE_PATH" '
         mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
   echo "$out" | grep -q "stage=/tmp/alt/.mempalace/pi-stage/"
 '
+# The feeder's --agent default is WHO a drawer is attributed to. mempalace core
+# records neither the machine nor the harness on a write, and one shared bearer
+# token means the server cannot tell clients apart, so toolkit c64ffa1 changed
+# this default from $USER to pi@$MEMPALACE_PI_DEVICE — the one string that makes
+# a write attributable to both. Nothing ever PRINTED the resolved value (the
+# banner shows mode= and stage= only), so an image built from a pre-c64ffa1
+# toolkit ref would ship unattributed writes with every check still green.
+#
+# `--help` assigns AGENT (script top) before it parses args, then exits 0 with
+# no side effects — so `bash -x` observes the REAL resolution, env interpolation
+# and fallback included, rather than grepping the source for a literal line that
+# any reformat would break. Two-sided on purpose: device set => pi@<device>;
+# device UNSET => must not be pi@anything. The second half is what fails against
+# the old unconditional $USER default, which ignored the device entirely.
+#
+# Probes the PATH entry (a symlink into the /opt clone) rather than that clone
+# path directly: this is the invocation the systemd/launchd timers and
+# entrypoint-user.sh actually use, so it is the default that reaches the palace.
+run "feeder resolves --agent to pi@<device> (drawer attribution)" '
+  f=$(command -v mempalace-pi-session) || { echo "feeder not on PATH" >&2; exit 1; }
+  with=$(MEMPALACE_PI_DEVICE=smoke-device bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
+  without=$(env -u MEMPALACE_PI_DEVICE bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
+  echo "resolved with-device=[$with] without-device=[$without]" >&2
+  [ "$with" = "pi@smoke-device" ] || exit 1
+  case "$without" in pi@*) exit 1 ;; esac
+  echo ok
+'
 # Regression guard for the pi transcript exporter. If pi ever changes its
 # session JSONL shape, the exporter stops recognising sessions and the palace
 # silently gets nothing (or, worse, raw JSON chunked as prose). Feed it a
@@ -263,6 +291,26 @@ run "pi-fork clone + node_modules" \
     "test -f /opt/pi-fork/package.json && test -d /opt/pi-fork/node_modules"
 run "pi-observational-memory clone + node_modules" \
     "test -f /opt/pi-observational-memory/package.json && test -d /opt/pi-observational-memory/node_modules"
+# ...and that the clone carries the AUTH FIX, not merely that it exists. om's
+# pre-flight hasUsableAuth() check silently disabled `recall` for ~8 weeks once
+# pi moved to request-time SigV4 signing and stopped exposing a static Bedrock
+# key; upstream fixed it in ce9fc98, adopted in v1.8.4. PI_OBSMEM_REF tracks
+# master, so an upstream revert or force-push would ship a dead `recall` with
+# the clone assertion above still green — the exact gap flagged as open in the
+# v1.8.5 changelog.
+#
+# Pin the markers to src/runtime.ts, the fix SITE, rather than grepping the
+# repo: two of these three strings also appear under tests/, so a repo-wide
+# grep stays green with runtime.ts itself reverted. That is a false green of the
+# same family as the old skill-snapshot canary.
+run "pi-observational-memory carries the ce9fc98 auth fix (recall stays alive)" '
+  f=/opt/pi-observational-memory/src/runtime.ts
+  test -f "$f" || { echo "fix site missing: $f" >&2; exit 1; }
+  for m in availability_recheck providerCredentialConfigured hasConfiguredAuth; do
+    grep -q "$m" "$f" || { echo "marker absent from runtime.ts: $m" >&2; exit 1; }
+  done
+  echo ok
+'
 # pi-atelier: deliberately NO node_modules assertion, unlike its siblings —
 # it declares zero runtime dependencies (only peerDeps, satisfied by the baked
 # pi) and has no build step, so Dockerfile.variant skips `npm install` for it.
@@ -301,26 +349,157 @@ echo ""
 echo "── Build provenance ──"
 run "/etc/pi-devbox/build-manifest.json present" \
     "test -f /etc/pi-devbox/build-manifest.json"
-run_expect "manifest records pi-extensions component" \
-    "cat /etc/pi-devbox/build-manifest.json" '"pi-extensions"'
-run_expect "manifest records pi-atelier" \
-    "cat /etc/pi-devbox/build-manifest.json" '"pi-atelier"'
-run_expect "manifest records pi_version" \
-    "cat /etc/pi-devbox/build-manifest.json" '"pi_version"'
+# These next checks replace three that grepped the manifest for the FIELD NAME
+# and never looked at the value:
+#
+#     run_expect "manifest records pi_version" "cat …manifest.json" '"pi_version"'
+#
+# which passes on {"pi_version": ""} and on {"pi_version": null}. The tell was
+# visible in its own passing output — `✅ manifest records pi_version (got
+# "pi_version")` echoes the key back as the thing it claims to have found.
+# Two failure modes were therefore invisible: a key that survives with an empty
+# or garbage value, and a key that vanishes from the manifest while every
+# remaining value still looks fine.
+#
+# Those two need SEPARATE assertions, and the reason is a trap worth keeping in
+# writing: an "every component value is a valid SHA" loop passes VACUOUSLY on
+# components:{} — jq's all() over an empty list is true — so the value check
+# alone would go green on a manifest that lost every component. Mutation-tested
+# 2026-08-25 across nine fabricated manifests (empty map, deleted key, "",
+# null, "unknown", 12-hex truncation, 40 non-hex chars, legit null pi-studio).
+run "manifest declares every required component key" '
+  req="pi-toolkit pi-extensions pi-fork pi-observational-memory pi-atelier mempalace-toolkit pi-studio"
+  for k in $req; do
+    jq -e --arg k "$k" "(.components|has(\$k))" /etc/pi-devbox/build-manifest.json >/dev/null \
+      || { echo "manifest lost component key: $k" >&2; exit 1; }
+  done
+'
+# Subsumes the old `! grep -q \"unknown\"` check ("unknown" is not 40-hex), and
+# also catches "", null and truncated SHAs, which that grep let through. null is
+# legitimate for pi-studio alone: the non-studio variant has no such clone.
+run "manifest component values are resolved 40-hex commits" '
+  jq -e "
+    .components
+    | to_entries
+    | all(if .key == \"pi-studio\" and .value == null then true
+           else (.value|type) == \"string\" and (.value|test(\"^[0-9a-f]{40}\$\")) end)
+  " /etc/pi-devbox/build-manifest.json >/dev/null
+'
+# pi_version against ground truth, same shape as the mempalace check below.
+# Chains with the "pi version matches build arg" assertion earlier in this file:
+# together they tie build arg -> installed binary -> recorded manifest, so a
+# manifest written from a stale variable cannot pass by agreeing with itself.
+run "manifest pi_version matches the installed pi" '
+  m=$(jq -r ".pi_version // empty" /etc/pi-devbox/build-manifest.json)
+  b=$(pi --version 2>/dev/null | head -n1 | tr -d "\r")
+  echo "manifest=[$m] installed=[$b]" >&2
+  [ -n "$m" ] && [ "$m" = "$b" ]
+'
+# Top-level provenance fields: assert the SHAPE of each value, and only when the
+# field is populated. source_revision and build_date legitimately default to
+# empty (Dockerfile.variant ARGs) on a plain local `docker build`, so demanding
+# them would fail honest local smoke runs; a populated-but-malformed value is
+# the actual defect. release_tag defaults to "dev", so empty means a broken write.
+run "manifest top-level fields are well-formed, not merely present" '
+  j=/etc/pi-devbox/build-manifest.json
+  t=$(jq -r ".release_tag // empty" $j)
+  r=$(jq -r ".source_revision // empty" $j)
+  d=$(jq -r ".build_date // empty" $j)
+  echo "release_tag=[$t] source_revision=[$r] build_date=[$d]" >&2
+  [ -n "$t" ] || { echo "release_tag empty (ARG default is dev)" >&2; exit 1; }
+  if [ -n "$r" ]; then
+    printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" || { echo "source_revision not a 40-hex commit" >&2; exit 1; }
+  fi
+  if [ -n "$d" ]; then
+    printf "%s" "$d" | grep -qE "^[0-9]{4}-[0-9]{2}-[0-9]{2}T" || { echo "build_date not ISO-8601" >&2; exit 1; }
+  fi
+'
+# mempalace CORE was absent from the manifest through v1.8.5: the toolkit SHA
+# was recorded but the palace version behind the MCP tools was not, so a palace
+# bug could not be correlated to an image version. Assert the field exists AND
+# equals the installed binary — recording it from ARG MEMPALACE_VERSION instead
+# would look identical here yet drift silently the first time an install
+# resolved to something other than the pin, which is the whole reason this file
+# is built from ground truth. `// empty` matters: jq -r prints the 4-char
+# string "null" for a JSON null, which would satisfy a naive -n test.
+run "manifest mempalace_version matches the installed core" '
+  m=$(jq -r ".mempalace_version // empty" /etc/pi-devbox/build-manifest.json)
+  b=$(mempalace --version 2>/dev/null | head -n1 | tr -d "\r"); b=${b##* }
+  echo "manifest=[$m] installed=[$b]" >&2
+  [ -n "$m" ] && [ "$m" = "$b" ]
+'
+# ... and, when CI supplies it, that the installed core is the version CI
+# actually AUDITED (published + not yanked on PyPI, in resolve-versions). This
+# does NOT duplicate the check above, which compares two properties of one
+# image and so cannot notice that BOTH are the wrong version. The live failure
+# mode it covers: the variant builds `FROM` a base tag chosen by base-decide's
+# content hash, so a bug in that hashing (the reason scripts/check-base-hash.sh
+# exists) could reuse a cached base built from an OLDER MEMPALACE_VERSION pin —
+# internally consistent, silently stale, invisible to every other assertion.
+if [ -n "${EXPECTED_MEMPALACE_VERSION:-}" ]; then
+  run "installed mempalace matches CI's audited pin (${EXPECTED_MEMPALACE_VERSION})" "
+    b=\$(mempalace --version 2>/dev/null | head -n1 | tr -d '\r'); b=\${b##* }
+    echo \"installed=[\$b] audited_pin=[${EXPECTED_MEMPALACE_VERSION}]\" >&2
+    [ \"\$b\" = \"${EXPECTED_MEMPALACE_VERSION}\" ]
+  "
+fi
 # Every component must be a resolved commit (or null for pi-studio in the
-# non-studio variant) — 'unknown' means a clone silently failed to resolve.
-run "manifest has no unresolved ('unknown') components" \
-    "! grep -q '\"unknown\"' /etc/pi-devbox/build-manifest.json"
-# pi-devbox-version wraps the manifest into a human-first command (this
-# PR); verify the binary is present, executable, and both output modes work.
+# non-studio variant) — now enforced by the 40-hex value check above, which
+# strictly subsumes the old whole-file grep for '"unknown"'. Only rev() ever
+# emits "unknown" and rev() feeds components only, so nothing is lost.
+# pi-devbox-version wraps the manifest into a human-first command; verify the
+# binary is present, executable, and that all three output modes work.
 run "pi-devbox-version binary present + executable" \
     "test -x /usr/local/bin/pi-devbox-version"
 run_expect "pi-devbox-version human output shows release tag" \
     "pi-devbox-version" "pi-devbox "
-run_expect "pi-devbox-version --json round-trips the manifest" \
-    "pi-devbox-version --json" '"release_tag"'
+# --json is a verbatim `cat` of the manifest, so "round-trips" is assertable
+# literally. The old form grepped the output for the string "release_tag" — the
+# key name again — which would pass on a truncated or re-serialised dump.
+run "pi-devbox-version --json round-trips the manifest byte-for-byte" '
+  a=$(cat /etc/pi-devbox/build-manifest.json)
+  b=$(pi-devbox-version --json)
+  [ "$a" = "$b" ] || { echo "--json output differs from the manifest on disk" >&2; exit 1; }
+'
 run_expect "pi-devbox-version --quiet is a compact one-liner" \
     "pi-devbox-version --quiet | wc -l" "1"
+# ── Vendored skill snapshot provenance ─────────────────────────────────
+# The vendored mempalace skill is the one baked artefact with no /opt clone
+# behind it (private upstream — see VENDORED.md), so until now the manifest
+# could not say which skillset commit it came from. Two fields now travel with
+# it: the CLAIMED ref (ARG default in Dockerfile.variant) and the MEASURED
+# sha256 of the shipped bytes. Assert both are well-formed, and — separately —
+# that the measurement still describes the file in the image.
+#
+# Kept as two assertions for the same reason the component checks are: one
+# proves the fields are not empty/garbage, the other proves they are not merely
+# self-consistent. A single combined check could pass on a manifest whose hash
+# was computed from a file that was later overwritten (the pi-extensions skill
+# copy at Dockerfile.variant:165 does exactly that kind of overwrite, one stage
+# earlier), which is the failure this second one exists to catch.
+run "manifest records the vendored skill snapshot provenance" '
+  j=/etc/pi-devbox/build-manifest.json
+  r=$(jq -r ".skillset_snapshot_ref // empty" $j)
+  s=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
+  echo "ref=[$r] tree_sha256=[$s]" >&2
+  printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" \
+    || { echo "skillset_snapshot_ref is not a 40-hex commit" >&2; exit 1; }
+  printf "%s" "$s" | grep -qxE "[0-9a-f]{64}" \
+    || { echo "skillset_snapshot_tree_sha256 is not a 64-hex digest" >&2; exit 1; }
+'
+# Recomputes over the whole DIRECTORY with the same tree_sha256() pipeline
+# Dockerfile.variant used to measure it, not a plain `sha256sum SKILL.md` —
+# a file-only compare here would pass even if the manifest recorded a
+# fingerprint over a directory that has since grown a second file (this is
+# not hypothetical: pi-extensions already ships two files for its skill).
+run "manifest skill fingerprint matches the baked snapshot" '
+  j=/etc/pi-devbox/build-manifest.json
+  d=/usr/local/share/pi-devbox/skills/mempalace
+  m=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
+  a=$( (cd "$d" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum) | sha256sum | cut -d" " -f1)
+  echo "manifest=[$m] actual=[$a]" >&2
+  [ -n "$m" ] && [ "$m" = "$a" ]
+'
 # OCI labels live in the image config, not the container fs — inspect them
 # from the host docker rather than via `docker run`.
 LBL=$(docker inspect --format '{{ index .Config.Labels "se.jordbo.pi-devbox.pi-extensions-ref" }}' "$IMAGE" 2>/dev/null || true)
@@ -385,7 +564,37 @@ exec_test "mempalace skill linked (fallback)"      'test -L $HOME/.agents/skills
 # multiple harnesses", a phrase present in BOTH the stale and the fresh copy.
 # A snapshot canary must pin the NEWEST section, so update this string whenever
 # the snapshot is refreshed — that is the point of it.
-exec_test "mempalace skill snapshot is current"    'grep -q "Attribute what you file yourself" $HOME/.agents/skills/mempalace/SKILL.md && echo ok'
+#
+# v1.8.7: this fired for real, and on the release that changed the snapshot. The
+# pinned phrase was "Attribute what you file yourself", the heading of the
+# instruction telling agents to hand-stamp added_by — which that same release
+# WITHDREW (RFC 001 §7.3.2 ranks agent-side stamping worst-possible; the bridge
+# now does it). So the canary correctly reported "snapshot changed, expectation
+# did not", and blocked publication of an otherwise-green build (81 passed, 1
+# failed, twice). Two lessons kept in the assertion itself:
+#   * it is now BIDIRECTIONAL — the new phrase must be present AND the withdrawn
+#     one absent, so a re-vendored stale snapshot fails just as loudly as a
+#     forgotten bump. A one-way canary only catches half the drift.
+#   * a phrase canary can only ever detect "older than what I remembered to pin",
+#     never "older than skillset main".
+#
+# That structural limit is now addressed, but NOT by the "CI job diffing this
+# file against the skillset repo" this comment used to point at (that pointer
+# also dangled: it referenced an Unreleased changelog note that had become the
+# v1.8.7 heading). A CI diff cannot be done without granting CI a credential
+# for the PRIVATE skillset repo, and it would guard a file that on this fleet
+# NO host reads — all four compose stacks mount a workspace containing the
+# skillset, so devbox-skill-reconcile repoints this link at the live clone and
+# the baked copy is a CI/no-mount fallback only. Instead the snapshot now
+# carries its provenance (skillset_snapshot_ref + a measured
+# skillset_snapshot_sha256 in build-manifest.json, written by
+# scripts/vendor-mempalace-skill.sh), which moves the check to where the
+# skillset actually IS: `scripts/vendor-mempalace-skill.sh --check` for a
+# maintainer, and `pi-devbox-version` for an agent inside any container.
+# This assertion is kept because it is orthogonal and free: it pins content,
+# not provenance, so it still catches a re-vendored snapshot whose ref was
+# bumped correctly but whose bytes came from the wrong place.
+exec_test "mempalace skill snapshot is current"    'f=$HOME/.agents/skills/mempalace/SKILL.md; grep -q "Provenance is stamped for you" "$f" && ! grep -q "Attribute what you file yourself" "$f" && echo ok'
 # Link TARGETS, not just link existence: with no skillset mounted (as here) the
 # baked tree must be what resolves, for all three vendored skills.
 exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
@@ -395,6 +604,30 @@ exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
          *) echo "$s resolves to $(readlink -f $HOME/.agents/skills/$s)" >&2; exit 1 ;;
        esac
      done; echo ok'
+# ... and that the tool REPORTS that resolution, which is the half that was
+# missing: a stale baked snapshot and a current live clone were
+# indistinguishable from inside the container. CI mounts no skillset, so every
+# vendored skill must report "baked" here — which also makes this a real test of
+# the fallback path rather than of the environment it happens to run in.
+exec_test "pi-devbox-version reports skill sources (all baked, no skillset here)" \
+    'out=$(pi-devbox-version)
+     echo "$out" | grep -q "skills:" || { echo "no skills section" >&2; exit 1; }
+     for s in mempalace pi-extensions pi-devbox-environment; do
+       echo "$out" | grep -qE "^    $s +baked$" \
+         || { echo "$s not reported as baked" >&2; exit 1; }
+     done; echo ok'
+# The boot banner must NOT carry the section: entrypoint-user.sh prints the
+# version FIRST, before the baked links exist and long before the skillset
+# deploy + reconcile run last, so anything it said about skill sources would be
+# a pre-reconcile state that is about to change.
+# A bare negative (`! grep -q "skills:"`) passes if the tool crashes or
+# prints nothing at all — it cannot tell "correctly omitted the section"
+# apart from "the binary is broken". Anchor it positively: the command must
+# still succeed and still print its normal release-tag line.
+exec_test "pi-devbox-version --no-skills omits the skills section" \
+    'out=$(pi-devbox-version --no-skills) && echo "$out" | grep -q "^pi-devbox " && ! echo "$out" | grep -q "skills:"'
+exec_test "entrypoint prints the version banner with --no-skills" \
+    'grep -q "pi-devbox-version --no-skills" /usr/local/bin/entrypoint-user.sh'
 # The handover path itself. CI never mounts a skillset, so without this the
 # v1.8.5 fix would ship untested: fabricate a skillset + a skills dir holding
 # baked-style links, run the reconciler, and assert all three outcomes —

scripts/vendor-mempalace-skill.sh

@@ -0,0 +1,293 @@
+#!/usr/bin/env bash
+# vendor-mempalace-skill.sh — refresh the vendored mempalace skill snapshot
+# AND its recorded provenance, together, so the two cannot drift apart.
+#
+# WHY THIS EXISTS
+# ---------------
+# rootfs/usr/local/share/pi-devbox/skills/mempalace/SKILL.md is a snapshot of a
+# file owned by the PRIVATE skillset repo (see VENDORED.md). Because the image
+# cannot clone that repo, refreshing the snapshot was a manual `cp` — and the
+# result was anonymous: nothing recorded WHICH skillset commit the bytes came
+# from. The only staleness check available was a hand-maintained phrase canary
+# in scripts/smoke-test.sh, which by construction detects "older than the phrase
+# I remembered to pin", never "older than skillset main".
+#
+# Two facts now travel with the snapshot: the skillset commit it was taken from
+# (ARG SKILLSET_SNAPSHOT_REF in Dockerfile.variant) and the sha256 of the bytes
+# themselves (measured at build time into build-manifest.json). This script is
+# the only thing that should ever write the first one, because a `cp` without a
+# matching ARG bump produces a manifest that CONFIDENTLY LIES — worse than the
+# anonymous snapshot it replaced.
+#
+# HARDENED after peer review (pi@emb-7kj4vr4g, logstream correlation
+# skills-provenance-review, 2026-08-26) proved the original --check could print
+# OK and exit 0 without actually verifying anything: `git show <ref>:<path>`
+# emits NOTHING when the ref/path doesn't resolve, and `sha256sum` still hashes
+# that empty stdin, so "ref not found" silently collided with "the file really
+# is 0 bytes". Depending on which side of the comparison hit the collision this
+# fell through as either a false MISMATCH (blaming provenance for what was
+# really an incomplete clone) or, worse, a false OK. See EXIT STATUS below —
+# "cannot determine" is now its own outcome, distinct from "confirmed wrong",
+# which is the same distinction the phrase canary this script replaced lacked.
+#
+# USAGE
+#   scripts/vendor-mempalace-skill.sh [skillset-root] [--force]
+#       refresh: rewrite the snapshot and the ARG together.
+#   scripts/vendor-mempalace-skill.sh --check [skillset-root]
+#       verify only, writes nothing. The root path and any flag may appear in
+#       either order — a positional-only parser previously made `<root>
+#       --check` silently run a refresh instead of the verification asked for.
+#
+#   skillset-root defaults to /workspace/skillset, then $HOME/skillset.
+#
+#   --force  (refresh mode only) proceed even when the recorded ref cannot be
+#            proven to be an ancestor of the skillset's current HEAD — i.e.
+#            skip the guard against silently REWINDING provenance, which a
+#            detached HEAD, an older checkout, or a shallow clone lacking the
+#            recorded commit can all trigger. Meant to be used deliberately,
+#            not habitually: each use is a human deciding a rewind is fine.
+#
+# --check answers "is the committed snapshot really skillset@<recorded ref>?"
+# — the question CI cannot answer without a credential for the private repo,
+# and which anyone with the skillset checked out can answer for free.
+#
+# EXIT STATUS (same three codes in both modes)
+#   0  the operation succeeded, or (--check) the record is verified truthful.
+#      This INCLUDES a truthful record that is merely stale — upstream has
+#      moved on since the recorded ref, or the local working tree has since
+#      diverged. A NOTICE is printed to stderr, but the snapshot is not being
+#      accused of lying, so this is not a release-blocking failure. Skipping a
+#      refresh is a legitimate release-day choice (see AGENTS.md); this exit
+#      code is what makes that choice checkable rather than merely asserted.
+#   1  refused: a CONFIRMED problem. Dirty upstream file; a refresh that would
+#      rewind past the recorded ref; or (--check) the vendored bytes provably
+#      do NOT match the file at the recorded ref — a lying record.
+#   2  cannot determine: the recorded ref, or the path at that ref, is not
+#      resolvable in this clone. Commonly a shallow clone missing history, or
+#      a ref that was rewritten or never pushed. Deliberately NOT the same as
+#      1 — "I can't tell" must never be reported as "it's wrong".
+set -euo pipefail
+
+cd "$(dirname "$0")/.."
+
+DOCKERFILE="Dockerfile.variant"
+VENDORED="rootfs/usr/local/share/pi-devbox/skills/mempalace/SKILL.md"
+ARG_NAME="SKILLSET_SNAPSHOT_REF"
+REL_PATH="skills/mempalace/SKILL.md"
+
+die() { printf '%s: %s\n' "$(basename "$0")" "$1" >&2; exit 1; }
+
+# Parse flags and the optional root path in either order, and reject anything
+# unrecognised rather than silently absorbing it.
+MODE="refresh"
+FORCE=0
+ROOT=""
+for arg in "$@"; do
+  case "$arg" in
+    --check) MODE="check" ;;
+    --force) FORCE=1 ;;
+    # This is the one script whose argument ORDER was itself a landmine, so the
+    # path that documents the trap must not be the path that errors.
+    -h|--help)
+      awk 'NR>1 && /^#/ { sub(/^# ?/, ""); print; next } NR>1 { exit }' "$0"
+      exit 0
+      ;;
+    --*) die "unknown option: $arg (try --help)" ;;
+    *)
+      [ -z "$ROOT" ] || die "unexpected extra argument: $arg (root already set to $ROOT)"
+      ROOT="$arg"
+      ;;
+  esac
+done
+if [ "$MODE" = "check" ] && [ "$FORCE" = 1 ]; then
+  die "--force has no effect with --check (nothing is written); remove it"
+fi
+
+if [ -z "$ROOT" ]; then
+  for candidate in /workspace/skillset "$HOME/skillset"; do
+    if [ -d "$candidate/.git" ]; then
+      ROOT="$candidate"
+      break
+    fi
+  done
+fi
+
+[ -n "$ROOT" ] || die "no skillset clone found (pass one: $(basename "$0") /path/to/skillset)"
+[ -d "$ROOT/.git" ] || die "not a git clone: $ROOT"
+[ -f "$ROOT/$REL_PATH" ] || die "no $REL_PATH in $ROOT"
+[ -f "$VENDORED" ] || die "vendored snapshot missing: $VENDORED"
+# LOAD-BEARING, DO NOT DELETE AS "REDUNDANT WITH THE EXISTENCE PROBES": -f
+# accepts an empty file, and sha256 of an empty file equals sha256 of a failed
+# pipeline's empty stdin. Guarding it HERE, before mode dispatch, makes that
+# collision unreachable by construction rather than by a probe further down --
+# which also means no test below exercises the collision any more. Remove this
+# line and the false "OK" for a nonexistent ref returns with nothing failing.
+[ -s "$VENDORED" ] || die "vendored snapshot is empty: $VENDORED"
+
+head_sha=$(git -C "$ROOT" rev-parse HEAD 2>/dev/null) || die "cannot read HEAD of $ROOT"
+recorded=$(grep -oE "^ARG ${ARG_NAME}=[0-9a-f]{40}$" "$DOCKERFILE" | cut -d= -f2 || true)
+[ -n "$recorded" ] || die "no 'ARG ${ARG_NAME}=<40-hex>' line in $DOCKERFILE"
+
+sha_of() { sha256sum "$1" | cut -d' ' -f1; }
+vendored_sha=$(sha_of "$VENDORED")
+upstream_sha=$(sha_of "$ROOT/$REL_PATH")
+
+# Does $REL_PATH exist at HEAD at all? Proven with `cat-file -e` BEFORE
+# hashing anything. Piping a failed `git show` straight into sha256sum, as
+# this script used to, hashes an EMPTY stream and produces sha256(""): a real,
+# collidable value — not a representation of absence. That collapsed "doesn't
+# exist" and "exists and happens to be empty" into the same signal, which is
+# exactly the defect class the peer review found in --check's at_ref, below.
+blob_sha=""
+if git -C "$ROOT" cat-file -e "HEAD:$REL_PATH" 2>/dev/null; then
+  blob_sha=$(git -C "$ROOT" show "HEAD:$REL_PATH" | sha256sum | cut -d' ' -f1)
+fi
+
+upstream_dirty=""
+if [ -z "$blob_sha" ]; then
+  upstream_dirty="not present at HEAD (untracked, or absent at this commit)"
+elif [ "$blob_sha" != "$upstream_sha" ]; then
+  if ! git -C "$ROOT" diff --quiet -- "$REL_PATH" 2>/dev/null; then
+    upstream_dirty="modified but not committed"
+  elif ! git -C "$ROOT" diff --cached --quiet -- "$REL_PATH" 2>/dev/null; then
+    upstream_dirty="staged but not committed"
+  else
+    upstream_dirty="different at HEAD than in the working tree"
+  fi
+fi
+
+if [ "$MODE" = "check" ]; then
+  # Resolve the recorded ref the same careful way: existence is proven with
+  # `cat-file -e` before anything is hashed, and "the ref itself is missing"
+  # is reported distinctly from "the ref resolves but the path isn't there
+  # at it" — both used to be silently swallowed into a plausible sha256("").
+  ref_exists=0
+  path_at_ref_exists=0
+  at_ref=""
+  if git -C "$ROOT" cat-file -e "${recorded}^{commit}" 2>/dev/null; then
+    ref_exists=1
+    if git -C "$ROOT" cat-file -e "${recorded}:${REL_PATH}" 2>/dev/null; then
+      path_at_ref_exists=1
+      at_ref=$(git -C "$ROOT" show "${recorded}:${REL_PATH}" | sha256sum | cut -d' ' -f1)
+    fi
+  fi
+
+  printf 'recorded ref:      %s\n' "$recorded"
+  printf 'vendored sha256:   %s\n' "$vendored_sha"
+  if [ "$path_at_ref_exists" = 1 ]; then
+    printf 'sha256 at ref:     %s\n' "$at_ref"
+  elif [ "$ref_exists" = 1 ]; then
+    printf 'sha256 at ref:     <%s not present at %s>\n' "$REL_PATH" "${recorded:0:7}"
+  else
+    printf 'sha256 at ref:     <%s not present in this clone>\n' "${recorded:0:7}"
+  fi
+  printf 'skillset HEAD:     %s (%s)\n' "$head_sha" "$upstream_sha"
+  if [ -n "$upstream_dirty" ]; then
+    printf 'live working tree: %s\n' "$upstream_dirty"
+  fi
+
+  rc=0
+  if [ "$ref_exists" != 1 ]; then
+    printf 'CANNOT-DETERMINE: %s is not present in %s — fetch, or check against a complete clone\n' "$recorded" "$ROOT" >&2
+    rc=2
+  elif [ "$path_at_ref_exists" != 1 ]; then
+    printf 'MISMATCH: %s does not exist at %s in this clone — the recorded ref cannot be describing these bytes\n' "$REL_PATH" "$recorded" >&2
+    rc=1
+  elif [ "$at_ref" != "$vendored_sha" ]; then
+    printf 'MISMATCH: the vendored snapshot is NOT the file at the recorded ref\n' >&2
+    rc=1
+  else
+    printf 'OK: the vendored snapshot is exactly skillset@%s:%s\n' "${recorded:0:7}" "$REL_PATH"
+  fi
+
+  # Staleness is orthogonal to truthfulness: a record can correctly describe
+  # an old commit even after upstream has moved on, and a dirty local working
+  # tree in $ROOT doesn't rewrite git history either — it says nothing about
+  # whether the RECORDED, committed ref describes the RECORDED, committed
+  # bytes. Only worth reporting once we already know rc=0 (truthful) — a
+  # MISMATCH or CANNOT-DETERMINE is the dominant fact and a staleness note
+  # would only muddy it.
+  if [ "$rc" = 0 ] && [ "$vendored_sha" != "$upstream_sha" ]; then
+    # Name the ACTUAL cause. "working tree differs" is wrong when the tree is
+    # clean and the ref simply moved on — a message that names the wrong cause
+    # is the same defect class as a canary pinned to a deleted phrase.
+    if [ "$recorded" != "$head_sha" ] && [ "$blob_sha" = "$upstream_sha" ]; then
+      # Do not ASSERT which side is newer — test it. Asserting that HEAD is the
+      # newer side points the operator at a refresh (which costs a ~67-minute
+      # base rebuild) when the real remedy may be `git pull` in this clone. The
+      # refresh path below already uses this primitive; reuse it here.
+      if git -C "$ROOT" merge-base --is-ancestor "$recorded" "$head_sha" 2>/dev/null; then
+        printf 'NOTICE: %s has moved on to %s; the snapshot describes the older %s (stale, not untruthful — refresh to catch up)\n' \
+          "$ROOT" "${head_sha:0:7}" "${recorded:0:7}" >&2
+      elif git -C "$ROOT" merge-base --is-ancestor "$head_sha" "$recorded" 2>/dev/null; then
+        printf 'NOTICE: %s is BEHIND at %s; the snapshot describes the newer %s — pull this clone, do NOT refresh the snapshot\n' \
+          "$ROOT" "${head_sha:0:7}" "${recorded:0:7}" >&2
+      else
+        printf 'NOTICE: %s (HEAD %s) and the recorded %s have DIVERGED — neither is an ancestor of the other; reconcile the clone before refreshing\n' \
+          "$ROOT" "${head_sha:0:7}" "${recorded:0:7}" >&2
+      fi
+    else
+      printf 'NOTICE: the working tree of %s differs from the snapshot (HEAD %s)\n' \
+        "$ROOT" "${head_sha:0:7}" >&2
+    fi
+  fi
+
+  exit "$rc"
+fi
+
+[ -z "$upstream_dirty" ] || die "$ROOT/$REL_PATH is $upstream_dirty — commit it first, or the recorded ref would not describe these bytes"
+
+if [ "$vendored_sha" = "$upstream_sha" ] && [ "$recorded" = "$head_sha" ]; then
+  printf 'already current: snapshot == skillset@%s\n' "${head_sha:0:7}"
+  exit 0
+fi
+
+# Refuse to silently REWIND provenance. `git checkout <tag>`, a detached HEAD,
+# or an older checkout can all leave $ROOT's HEAD behind the already-recorded
+# ref; without this guard a refresh there would happily rewrite both the ARG
+# and the bytes backwards and report it as an ordinary update.
+if [ "$recorded" != "$head_sha" ]; then
+  if git -C "$ROOT" cat-file -e "${recorded}^{commit}" 2>/dev/null; then
+    if ! git -C "$ROOT" merge-base --is-ancestor "$recorded" "$head_sha" 2>/dev/null; then
+      if [ "$FORCE" != 1 ]; then
+        die "refusing: $ROOT's HEAD ($head_sha) is not a descendant of the recorded ref ($recorded) — this looks like a rewind. Pass --force if this is intentional."
+      fi
+      printf 'WARNING: --force set; %s is not an ancestor of HEAD %s — proceeding anyway\n' "${recorded:0:7}" "${head_sha:0:7}" >&2
+    fi
+  else
+    if [ "$FORCE" != 1 ]; then
+      printf 'CANNOT-DETERMINE: %s is not present in %s (shallow clone?) — fetch full history to verify this refresh moves forward, or pass --force to proceed without that guarantee\n' "$recorded" "$ROOT" >&2
+      exit 2
+    fi
+    printf 'WARNING: --force set; %s could not be resolved in %s — proceeding without verifying forward motion\n' "${recorded:0:7}" "$ROOT" >&2
+  fi
+fi
+
+# Written FROM THE REF, not copied from the working tree, so the pair cannot
+# be a lie by construction. Via a temp file so a failed write cannot leave a
+# half-vendored snapshot behind.
+snap_tmp=$(mktemp)
+if ! git -C "$ROOT" show "HEAD:$REL_PATH" > "$snap_tmp" 2>/dev/null; then
+  rm -f -- "$snap_tmp"
+  die "cannot read HEAD:$REL_PATH from $ROOT"
+fi
+chmod 0644 -- "$snap_tmp"
+mv -- "$snap_tmp" "$VENDORED"
+[ "$(sha_of "$VENDORED")" = "$blob_sha" ] \
+  || die "internal: written snapshot does not match HEAD:$REL_PATH"
+
+# In-place, and only the exact pinned line: a broad sed on this Dockerfile
+# could rewrite one of the other *_REF ARGs.
+tmp=$(mktemp)
+sed "s|^ARG ${ARG_NAME}=.*\$|ARG ${ARG_NAME}=${head_sha}|" "$DOCKERFILE" > "$tmp"
+chmod 0644 -- "$tmp"
+mv -- "$tmp" "$DOCKERFILE"
+
+new_recorded=$(grep -oE "^ARG ${ARG_NAME}=[0-9a-f]{40}$" "$DOCKERFILE" | cut -d= -f2 || true)
+[ "$new_recorded" = "$head_sha" ] || die "failed to rewrite ${ARG_NAME} in $DOCKERFILE"
+
+printf 'snapshot:  %s -> %s\n' "${vendored_sha:0:12}" "$(sha_of "$VENDORED" | cut -c1-12)"
+printf 'ref:       %s -> %s\n' "${recorded:0:7}" "${head_sha:0:7}"
+printf '\nNOTE: %s is hashed into base_tag, so this costs a base rebuild\n' "$VENDORED"
+printf 'on the next tag (~67 min). Also re-pin the phrase canary in\n'
+printf 'scripts/smoke-test.sh if the section it names changed.\n'