Compare commits
78 Commits
v1.6.4
...
3a44e81cad
| Author | SHA1 | Date | |
|---|---|---|---|
| 3a44e81cad | |||
| 35964abd01 | |||
| 6353d59e63 | |||
| 8f0960e134 | |||
| 1c905480e3 | |||
| ff6fd1492a | |||
| edc7659add | |||
| cac5e00a31 | |||
| ecfd2fc2e5 | |||
| 15a3728ae9 | |||
| 361babd4fd | |||
| 70e675afee | |||
| 601fc98a49 | |||
| 7e0e66997d | |||
| 6bd8b79d3a | |||
| fabf1274aa | |||
| 5972a2c535 | |||
| aa0fbc5ec0 | |||
| 702dd71f4c | |||
| f561acc89a | |||
| 0d984b1414 | |||
| adcf56f829 | |||
| c8622ece9d | |||
| 05843ecfae | |||
| a2846a5f7e | |||
| 58c22afb04 | |||
| 30094782df | |||
| 9b5783f9dd | |||
| d9a7fe101b | |||
| 36e65fe657 | |||
| f0ebea2d98 | |||
| b615571913 | |||
| 495b7e3859 | |||
| 45850bc973 | |||
| 6891dc32b8 | |||
| 8a673ec143 | |||
| cdb6fc0950 | |||
| 14371e2da6 | |||
| aac4a1c323 | |||
| 34cf1e3810 | |||
| e8ddeaf89f | |||
| 49a6534093 | |||
| e070e0bcbf | |||
| dbb78798fb | |||
| f645e6654f | |||
| 657b1ad856 | |||
| ebd0de0be2 | |||
| 4f1aa0d0dd | |||
| 9e744d701f | |||
| 2b8c3a4db4 | |||
| cb7b8ad2ae | |||
| 93f986e90e | |||
| 26f223568d | |||
| 01abda3456 | |||
| b5810654f6 | |||
| 4f6f470518 | |||
| fbc1f86612 | |||
| 2ebf00d6d4 | |||
| c3b6d36778 | |||
| 3a509077c2 | |||
| a55f6369b3 | |||
| ffd54750b9 | |||
| d8b745c164 | |||
| ae13c2264e | |||
| a2f0a4a441 | |||
| 53b41cd76b | |||
| 29b62093f0 | |||
| cbd7cf5c67 | |||
| 7c00dd6001 | |||
| 7649d53f3b | |||
| ade58131d6 | |||
| ffd44ad9cf | |||
| 43cd6e22f2 | |||
| 62a2a79b1c | |||
| f20b2a7926 | |||
| 66a19aa394 | |||
| 572430237f | |||
| 1fd524e7fb |
+120
-3
@@ -12,16 +12,110 @@ SSH_KEY_PATH=~/.ssh
|
||||
# ── MemPalace memory (local by default) ───────────────────────────
|
||||
# By default the mempalace.ts extension spawns a LOCAL mempalace-mcp stdio
|
||||
# server (palace at ~/.mempalace). Uncomment the devbox-palace volume in
|
||||
# docker-compose.yml to persist it across container recreation.
|
||||
# docker-compose.yml to persist it across container recreation — that one
|
||||
# volume now covers the mined conversation transcripts too, since the pi and
|
||||
# opencode feeders stage inside the palace root (<palace-root>/pi-stage), so
|
||||
# the staged files and the palace dedup keys pointing at them cannot be
|
||||
# separated.
|
||||
#
|
||||
# That palace root is resolved with mempalace's own precedence
|
||||
# ($MEMPALACE_PALACE_PATH -> $MEMPAL_PALACE_PATH -> ~/.mempalace/config.json ->
|
||||
# ~/.mempalace/palace), and the feeders derive their stage FROM it
|
||||
# (<palace-root>/pi-stage). Neither the image nor the entrypoint exports it, by
|
||||
# design: pinning the palace without carrying the stage along re-creates the
|
||||
# very split that a shared root removed. Override it only to move the palace off
|
||||
# the default -- e.g. onto a different mount -- and only to a path with the SAME
|
||||
# persistence as the palace itself. A stage that outlives its palace (or dies
|
||||
# first) makes a scoped `mempalace sync` prune conversation drawers, because
|
||||
# their dedup key is the staged path. Setting it to the default buys nothing.
|
||||
# Unlike WORKSPACE_PATH/SSH_KEY_PATH above, this is a path INSIDE the container.
|
||||
# MEMPALACE_PALACE_PATH=/home/developer/.mempalace/palace
|
||||
#
|
||||
# To instead share ONE MemPalace across containers/harnesses (pi + opencode
|
||||
# + native), set the URL below. When set, the extension connects over HTTP
|
||||
# and NO local mempalace-mcp is spawned; the devbox-palace volume is then
|
||||
# irrelevant. MEMPALACE_REMOTE_TOKEN, if set, is sent as a bearer token.
|
||||
# Serve it with: mempalace-mcp --transport http --host 0.0.0.0 --port 8765
|
||||
# MEMPALACE_REMOTE_URL=http://mempalace.lan:8765/mcp
|
||||
#
|
||||
# Serve it with: mempalace serve --host 172.17.0.1 --port 8765
|
||||
#
|
||||
# NOT `mempalace-mcp --transport http --host 0.0.0.0`: `serve` is the turnkey
|
||||
# wrapper that mints/keeps a bearer token (0600, passed via env so it stays out
|
||||
# of `ps`) and can terminate TLS. Two binds to avoid:
|
||||
# 0.0.0.0 - exposes the palace to the whole LAN.
|
||||
# 127.0.0.1 - behind a tunnel this 403s every proxied request (the Host pin
|
||||
# is only enforced on loopback binds) AND silently starts with
|
||||
# no token at all, since auto-minting is gated on the bind being
|
||||
# non-loopback. Bind the docker0 gateway: reachable from the host
|
||||
# and its containers (so a newt/proxy container works), not from
|
||||
# the LAN. Set MEMPALACE_MCP_HTTP_TOKEN explicitly server-side.
|
||||
# MEMPALACE_REMOTE_URL=https://mempalace.example.com/mcp
|
||||
# MEMPALACE_REMOTE_TOKEN=
|
||||
|
||||
# ── MemPalace: automatic capture of pi sessions ───────────────────────
|
||||
# The mempalace.ts extension feeds this container's pi transcripts into the
|
||||
# palace by itself: on session_shutdown, and on a debounced agent_settled so a
|
||||
# crash loses at most one window rather than the whole session. The entrypoint
|
||||
# also runs a catch-up at container start, which is the only thing that can
|
||||
# recover transcripts after a hard kill (no handler runs on SIGKILL).
|
||||
# Nothing below is required for the local-palace case; the defaults work.
|
||||
#
|
||||
# MEMPALACE_FEED=0 # disable automatic capture entirely
|
||||
# MEMPALACE_FEED_DEBOUNCE_MS=600000 # min gap between mid-session feeds (10 min)
|
||||
# MEMPALACE_FEED_WING=wing_conversations
|
||||
#
|
||||
# REMOTE PALACE ONLY (MEMPALACE_REMOTE_URL set above): the palace is on another
|
||||
# host, and `mempalace_mine` resolves its source path in the SERVER process, so
|
||||
# the server cannot see this container's transcripts. The feeder therefore
|
||||
# rsyncs its staged exports into a per-device inbox on the palace host and asks
|
||||
# the server to mine its own local copy. Without MEMPALACE_PI_SSH_TARGET the
|
||||
# feeder is skipped (a remote palace with no inbox has nothing to mine).
|
||||
# MEMPALACE_PI_SSH_TARGET where to rsync to, as user@host:path
|
||||
# MEMPALACE_PI_REMOTE_PATH what that inbox is called ON THE SERVER — i.e. the
|
||||
# path the SERVER PROCESS can open. If the palace
|
||||
# server runs in Docker, that is the container path
|
||||
# (see docker-compose.mempalace.yml). If it runs
|
||||
# NATIVELY (systemd unit / uv tool / plain
|
||||
# `mempalace serve`), it sees host paths, so this
|
||||
# must equal the path half of
|
||||
# MEMPALACE_PI_SSH_TARGET. Getting this wrong is
|
||||
# quiet: rsync still succeeds and only the mine
|
||||
# fails with "source directory not found", so
|
||||
# transcripts ship and are filed nowhere. The feeder
|
||||
# warns in preflight when the two paths disagree.
|
||||
# MEMPALACE_PI_DEVICE inbox subdirectory for this machine (default: hostname)
|
||||
# MEMPALACE_PI_SSH_TARGET=user@palace-host:/srv/mempalace-feed
|
||||
# MEMPALACE_PI_REMOTE_PATH=/data/feed
|
||||
# MEMPALACE_PI_DEVICE=
|
||||
|
||||
# ── Mailbox notification: MUST BE NAMED, auto-detect CANNOT work here ──
|
||||
# The mempalace extension polls the logstream for fleet asks addressed to this
|
||||
# device and queues them into the next turn. That part needs no config. The
|
||||
# NOTIFICATION that tells the human it happened does, and unset means SILENT
|
||||
# outside the pi TUI.
|
||||
#
|
||||
# Why there is no working default: terminal identity lives in env vars set by
|
||||
# the emulator (KITTY_WINDOW_ID, TERM_PROGRAM) and `docker exec` does NOT
|
||||
# forward them — inside the container pi sees only TERM=xterm-256color no matter
|
||||
# what is rendering it. So "desktop" auto-detection always falls through to
|
||||
# OSC 777, which Kitty does not implement, and the notification silently does
|
||||
# nothing: the worst outcome for a feature whose only job is to break a silence.
|
||||
# Naming the protocol is what makes it fire.
|
||||
#
|
||||
# kitty OSC 99 desktop notification (correct for Kitty, incl. over SSH)
|
||||
# osc777 OSC 777 (tmux/iTerm2/foot and others)
|
||||
# desktop OSC 99 if KITTY_WINDOW_ID is visible, else OSC 777 — inside a
|
||||
# container that means effectively always OSC 777, so prefer naming
|
||||
# 0 / off suppress entirely (in-TUI notify still shows)
|
||||
# MEMPALACE_MAILBOX_NOTIFY=kitty
|
||||
#
|
||||
# Cadence, if the delivery ever feels late: the poll is coupled to session
|
||||
# activity (it runs when the agent settles), NOT to a wall clock.
|
||||
# MEMPALACE_MAILBOX_POLL_MS is therefore a FLOOR BETWEEN POLLS (default 300000),
|
||||
# not a promise of one every 5 minutes — an idle session polls zero times, and
|
||||
# session start does the first look.
|
||||
# MEMPALACE_MAILBOX_POLL_MS=300000
|
||||
# MEMPALACE_MAILBOX_RESURFACE_MS=3600000
|
||||
|
||||
# ── LAN access from the container (host-OS-agnostic) ─────────────────
|
||||
# On VM-backed hosts (macOS OrbStack / Docker Desktop) the container can't
|
||||
# reach the host's directly-attached LAN peers by default. The entrypoint
|
||||
@@ -43,7 +137,22 @@ SSH_KEY_PATH=~/.ssh
|
||||
# the host, so bare `dssh user@<ip>` works on whatever LAN you're roaming on.
|
||||
# DEVBOX_LAN_AUTOJUMP_PRIVATE=0
|
||||
|
||||
# ── pi-atelier (TUI sidebar) ─────────────────────────────────────────
|
||||
# The image vendors pi-atelier at a pinned, audited tag and registers it on
|
||||
# container start. Set to 0 to opt out: the entrypoint then removes it from
|
||||
# pi's `packages[]` instead of registering it. This lives here rather than
|
||||
# being a `pi uninstall` because a broken TUI extension's failure mode is
|
||||
# "pi will not start", which you cannot fix from inside pi.
|
||||
# DEVBOX_ATELIER=1
|
||||
|
||||
# ── Git Configuration ────────────────────────────────────────────────
|
||||
# Set BOTH. If unset, every repo inside the container fails with
|
||||
# "Author identity unknown" on first commit, and an agent asked to commit
|
||||
# will guess an identity from git log — often the wrong one. The e-mail is
|
||||
# per-machine (work machines use the corporate address, personal machines the
|
||||
# private one), so it belongs in this per-machine .env, never in a skill or a
|
||||
# repo-local override. Consumed by entrypoint-user.sh -> ~/.gitconfig, which is
|
||||
# NOT persistent across container recreate — this file is the source of truth.
|
||||
GIT_USER_NAME=
|
||||
GIT_USER_EMAIL=
|
||||
|
||||
@@ -66,6 +175,14 @@ GIT_USER_EMAIL=
|
||||
# Detection is automatic if the skillset lives at WORKSPACE_PATH/skillset.
|
||||
# SKILLSET_CONTAINER_PATH=
|
||||
|
||||
# ── cli_utils (standalone commands from a mounted checkout) ──────────
|
||||
# If a cli_utils repo is mounted, the entrypoint symlinks its bin/ commands
|
||||
# into ~/.local/bin on every start, so they survive container recreate and
|
||||
# resolve in non-interactive shells too (docker exec, agent tool shells).
|
||||
# Detection is automatic at WORKSPACE_PATH/cli_utils (or one level below).
|
||||
# CLI_UTILS_CONTAINER_PATH=
|
||||
# CLI_UTILS_LINK=0 # disable the linking entirely
|
||||
|
||||
# ── Locale ───────────────────────────────────────────────────────────
|
||||
# LANG=sv_SE.UTF-8
|
||||
# LANGUAGE=sv_SE:sv
|
||||
|
||||
@@ -18,6 +18,14 @@ name: Publish Docker Image
|
||||
# 5. build-variant multi-arch push of latest + vX.Y.Z tags.
|
||||
# 6. promote-base-latest re-tag base-<hash> → base-latest with `crane copy`.
|
||||
# 7. update-description patch Docker Hub description.
|
||||
#
|
||||
# Note the trigger: `push: tags: v*` (plus workflow_dispatch). Nothing here runs
|
||||
# on a push to main, so a smoke assertion added outside a release is UNVALIDATED
|
||||
# until the next tag — which is exactly how v1.8.0 shipped a broken assertion
|
||||
# written three days earlier (it asserted a literal /home/developer stage path,
|
||||
# while `run` executes `docker run --entrypoint=""` as root with HOME=/root).
|
||||
# The `smoke_only` dispatch input exists to close that gap: it runs steps 1-4
|
||||
# against HEAD and stops before anything is published.
|
||||
|
||||
on:
|
||||
push:
|
||||
@@ -25,14 +33,39 @@ on:
|
||||
- 'v*'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
# `type:` is REQUIRED for Gitea to render these fields in the "Run
|
||||
# workflow" dialog. Without it (Gitea 1.26.2) the dispatch form shows a
|
||||
# branch selector and NO inputs at all, so a manual run silently uses
|
||||
# every default — which for `release_tag: ''` means RELEASE_TAG resolves
|
||||
# empty, the variant tag list becomes `<image>:`, and the run dies on an
|
||||
# invalid reference AFTER paying the full base + smoke cost (~70 min).
|
||||
# That made the documented `smoke_only` escape hatch below unreachable
|
||||
# from the UI for its whole existence; found 2026-09-06 trying to use it.
|
||||
#
|
||||
# Deliberately `string` and not `boolean`, even though these two read as
|
||||
# flags: every consumption is a STRING comparison against 'true'
|
||||
# (`inputs.smoke_only != 'true'` at the build-variant gates,
|
||||
# `inputs.promote_latest == 'true'` at the promote gates) plus string
|
||||
# interpolation into env.PROMOTE_LATEST. A boolean-typed input yields a
|
||||
# real boolean, so `!= 'true'` would compare across types and could
|
||||
# invert a publish gate rather than fail loudly. Changing the type here
|
||||
# would mean re-auditing all six call sites; keeping it string is a
|
||||
# rendering fix with provably zero semantic change.
|
||||
release_tag:
|
||||
description: 'Release tag to publish (e.g. v1.0.0). Used only for workflow_dispatch runs.'
|
||||
required: false
|
||||
default: ''
|
||||
type: string
|
||||
promote_latest:
|
||||
description: 'Update latest aliases (default true for tag-push, false for manual test runs)'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: string
|
||||
smoke_only:
|
||||
description: 'Build base + run both smoke jobs against HEAD, then stop. Publishes nothing. Use to validate smoke assertions without cutting a tag. Set to the literal string true.'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: string
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
@@ -124,20 +157,63 @@ jobs:
|
||||
# buildcache silently reuses the layer from whatever pi version was
|
||||
# current when the cache was first populated. Same class of bug as
|
||||
# pi-devbox v0.74.0..v0.75.5 (fixed in v0.75.5b 2026-05-23).
|
||||
# ── release gate ──────────────────────────────────────────────
|
||||
# Refuse to spend a base build on a tree whose own shell scripts do not lint.
|
||||
#
|
||||
# v1.8.14's first attempt is why this exists. smoke and smoke-studio both failed
|
||||
# at scripts/smoke-test.sh:770 AFTER build-base had already spent ~46 minutes,
|
||||
# on a defect shellcheck had flagged as SC2289 (severity error) a day earlier:
|
||||
# the lint workflow went red on the very push that introduced it (run 186) and
|
||||
# stayed red for runs 187 and 188, unread.
|
||||
#
|
||||
# lint.yml deliberately does not run on tag pushes, and its reasoning is sound
|
||||
# (the tagged tree was already linted on main; a tag-ref lint run sorts above
|
||||
# the publish run and makes a release look finished before anything ships). The
|
||||
# missing invariant was never "lint the tag" -- it was "do not RELEASE a tree
|
||||
# whose lint failed", and only a job inside THIS workflow can enforce that.
|
||||
#
|
||||
# ~40 s, ahead of everything expensive, and it runs scripts/lint-shell.sh --
|
||||
# the same file lint.yml calls, not a second copy that drifts.
|
||||
lint-gate:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: catthehacker/ubuntu:act-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Install shellcheck
|
||||
run: |
|
||||
apt-get update
|
||||
apt-get install -y --no-install-recommends shellcheck
|
||||
|
||||
- name: "Shellcheck + syntax-check repository scripts (severity: error)"
|
||||
run: bash scripts/lint-shell.sh
|
||||
|
||||
resolve-versions:
|
||||
# Gated: a defective tree must not reach a 46-minute base build.
|
||||
needs: [lint-gate]
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: catthehacker/ubuntu:act-latest
|
||||
outputs:
|
||||
pi_version: ${{ steps.resolve.outputs.pi_version }}
|
||||
mempalace_version: ${{ steps.resolve.outputs.mempalace_version }}
|
||||
fork_ref: ${{ steps.resolve.outputs.fork_ref }}
|
||||
obsmem_ref: ${{ steps.resolve.outputs.obsmem_ref }}
|
||||
toolkit_ref: ${{ steps.resolve.outputs.toolkit_ref }}
|
||||
extensions_ref: ${{ steps.resolve.outputs.extensions_ref }}
|
||||
studio_ref: ${{ steps.resolve.outputs.studio_ref }}
|
||||
studio_tag: ${{ steps.resolve.outputs.studio_tag }}
|
||||
atelier_ref: ${{ steps.resolve.outputs.atelier_ref }}
|
||||
atelier_tag: ${{ steps.resolve.outputs.atelier_tag }}
|
||||
mempalace_toolkit_ref: ${{ steps.resolve.outputs.mempalace_toolkit_ref }}
|
||||
steps:
|
||||
# Needed since v1.7.0: the pi version and the pi-atelier tag are now
|
||||
# PINNED IN Dockerfile.variant and read from it here, so this job has to
|
||||
# see the repo. Keeping the pins in the Dockerfile (rather than duplicated
|
||||
# in this workflow) means a local `docker build` and CI ship the same
|
||||
# versions by construction, and a bump is one reviewable line.
|
||||
- uses: actions/checkout@v4
|
||||
- name: Resolve pi version + companion refs
|
||||
id: resolve
|
||||
shell: bash
|
||||
@@ -157,15 +233,119 @@ jobs:
|
||||
fi
|
||||
}
|
||||
|
||||
# pi version from npm (catthehacker/ubuntu:act-latest's npm is not
|
||||
# reliably on PATH in act_runner job containers, so query directly).
|
||||
PI_VERSION=$(curl -sf "https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/latest" | jq -r '.version' 2>/dev/null || true)
|
||||
if ! printf '%s' "${PI_VERSION:-}" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+'; then
|
||||
echo "::error::Could not resolve pi version from npm (got '${PI_VERSION:-<empty>}')."
|
||||
# Read a commit SHA from Gitea, surviving a bad build token.
|
||||
#
|
||||
# These repos are public (see the note at the call sites), so auth is
|
||||
# a convenience, not a requirement — but Gitea REJECTS an invalid
|
||||
# token (401) rather than ignoring it, so a revoked or malformed
|
||||
# GITEA_BUILD_TOKEN could fail an entire release on reads that work
|
||||
# fine anonymously. An ABSENT secret was always safe (Gitea ignores an
|
||||
# empty `token ` value and serves the request, 200); a STALE one was
|
||||
# not. So: try authed, and on 401/403 retry anonymously.
|
||||
#
|
||||
# A non-200 after that emits nothing and returns 0 deliberately, so
|
||||
# require_sha raises the loud explicit abort rather than this helper
|
||||
# inventing a fallback ref.
|
||||
#
|
||||
# Messages go to STDERR, not as ::warning:: annotations: this
|
||||
# function's stdout IS the SHA, so anything written there would be
|
||||
# captured into the ref by the command substitution.
|
||||
gitea_sha() { # $1=repo
|
||||
local repo="$1" url resp code
|
||||
url="https://gitea.jordbo.se/api/v1/repos/joakimp/${repo}/commits?limit=1&sha=main"
|
||||
resp=$(curl -s -w '\n%{http_code}' -H "$AUTH_HEADER" "$url" || printf '\n000')
|
||||
code=${resp##*$'\n'}
|
||||
if [ "$code" = "401" ] || [ "$code" = "403" ]; then
|
||||
printf 'WARNING: Gitea rejected the build token for %s (HTTP %s); retrying anonymously. The read should succeed (public repo), but GITEA_BUILD_TOKEN is stale or malformed and should be rotated.\n' "$repo" "$code" >&2
|
||||
resp=$(curl -s -w '\n%{http_code}' "$url" || printf '\n000')
|
||||
code=${resp##*$'\n'}
|
||||
fi
|
||||
if [ "$code" != "200" ]; then
|
||||
printf 'WARNING: Gitea commit lookup for %s returned HTTP %s\n' "$repo" "$code" >&2
|
||||
return 0
|
||||
fi
|
||||
printf '%s' "${resp%$'\n'*}" | jq -r '.[0].sha // empty' 2>/dev/null || true
|
||||
}
|
||||
|
||||
# ── pi version: from the PIN, not from npm `latest` ───────────
|
||||
# Until v1.7.0 this followed npm `latest`, which meant every release
|
||||
# silently adopted whatever pi had shipped that morning — unaudited —
|
||||
# in the same build that then got tagged and published. A pi minor
|
||||
# can move the TUI/renderer internals that pi-atelier wraps (0.84 vs
|
||||
# atelier 0.6.0: startup hang, sustained CPU) or the session `.jsonl`
|
||||
# format that pi-session-repair parses. The pin makes adoption an
|
||||
# explicit, reviewable act; the drift warning below makes it a
|
||||
# prompt rather than a surprise.
|
||||
PI_VERSION=$(sed -n 's/^ARG PI_VERSION=\([^[:space:]]*\).*/\1/p' Dockerfile.variant | head -n1)
|
||||
if ! printf '%s' "${PI_VERSION:-}" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "::error::ARG PI_VERSION in Dockerfile.variant is not a concrete version (got '${PI_VERSION:-<empty>}'). CI refuses to build from a floating pi version — see the pin policy comment above that ARG."
|
||||
exit 1
|
||||
fi
|
||||
# The pin must actually exist on npm: catches a typo, an unpublished
|
||||
# version, or one yanked after we audited it — at resolve time, with
|
||||
# a clear message, instead of as an `npm install` failure mid-build.
|
||||
PI_PUBLISHED=$(curl -sf "https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/${PI_VERSION}" | jq -r '.version // empty' 2>/dev/null || true)
|
||||
if [ "${PI_PUBLISHED:-}" != "${PI_VERSION}" ]; then
|
||||
echo "::error::Pinned pi version ${PI_VERSION} is not published on npm (registry returned '${PI_PUBLISHED:-<empty>}'). Fix ARG PI_VERSION in Dockerfile.variant."
|
||||
exit 1
|
||||
fi
|
||||
# Informational only — a newer pi must never be adopted implicitly.
|
||||
# `|| true`: a transient registry failure must not fail a release
|
||||
# whose version is already pinned and verified above.
|
||||
PI_NPM_LATEST=$(curl -sf "https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/latest" | jq -r '.version // empty' 2>/dev/null || true)
|
||||
if [ -n "${PI_NPM_LATEST:-}" ] && [ "${PI_NPM_LATEST}" != "${PI_VERSION}" ]; then
|
||||
echo "::warning::pi ${PI_NPM_LATEST} is published; this build ships the audited pin ${PI_VERSION}. To adopt it: read the upstream CHANGELOG for every version in between (TUI/theme API, session .jsonl format, extension loader, Node engine), re-check pi-atelier's floor, then bump ARG PI_VERSION in Dockerfile.variant and note the audit in CHANGELOG.md."
|
||||
fi
|
||||
echo "pi_version=${PI_VERSION}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# ── mempalace core: same audit as pi, from Dockerfile.base ────
|
||||
# Until now this pin had NO CI-side audit at all — a literal string
|
||||
# in Dockerfile.base with zero references in this workflow, while
|
||||
# PI_VERSION got a concreteness gate, a published-on-registry check
|
||||
# and a drift warning. It is the same class of risk: the palace's MCP
|
||||
# tool schema is the agent-facing contract, and a client/server skew
|
||||
# against the shared central palace is a fleet-wide, not local,
|
||||
# problem. Read from Dockerfile.base (not duplicated here) so a local
|
||||
# `docker build` and CI install the same version by construction.
|
||||
MEMPALACE_VERSION=$(sed -n 's/^ARG MEMPALACE_VERSION=\([^[:space:]]*\).*/\1/p' Dockerfile.base | head -n1)
|
||||
if ! printf '%s' "${MEMPALACE_VERSION:-}" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "::error::ARG MEMPALACE_VERSION in Dockerfile.base is not a concrete version (got '${MEMPALACE_VERSION:-<empty>}'). CI refuses to build from a floating palace version — see the pin policy comment above that ARG."
|
||||
exit 1
|
||||
fi
|
||||
# One fetch, two gates. `curl -sf` exits non-zero and prints nothing
|
||||
# on 404 (PyPI's answer for an unpublished version), so an empty body
|
||||
# lands in the "not published" branch with its own message.
|
||||
MEMPALACE_PYPI=$(curl -sf "https://pypi.org/pypi/mempalace/${MEMPALACE_VERSION}/json" || true)
|
||||
MEMPALACE_PUBLISHED=$(printf '%s' "$MEMPALACE_PYPI" | jq -r '.info.version // empty' 2>/dev/null || true)
|
||||
if [ "${MEMPALACE_PUBLISHED:-}" != "${MEMPALACE_VERSION}" ]; then
|
||||
echo "::error::Pinned mempalace version ${MEMPALACE_VERSION} is not published on PyPI (registry returned '${MEMPALACE_PUBLISHED:-<empty>}'). Fix ARG MEMPALACE_VERSION in Dockerfile.base."
|
||||
exit 1
|
||||
fi
|
||||
# A yanked release still installs when pinned exactly (PEP 592), so
|
||||
# `uv tool install mempalace==X` would succeed silently and ship a
|
||||
# version upstream has withdrawn to the whole fleet. The escape hatch
|
||||
# is the same one-line bump that got us here.
|
||||
MEMPALACE_YANKED=$(printf '%s' "$MEMPALACE_PYPI" | jq -r '.info.yanked // false' 2>/dev/null || true)
|
||||
if [ "${MEMPALACE_YANKED:-false}" = "true" ]; then
|
||||
# Reason hoisted into its own variable rather than inlined as a
|
||||
# $(...) inside the message: a jq program nested in a substitution
|
||||
# inside a double-quoted string needs escaping that silently breaks
|
||||
# the FILTER (jq compile error) while the surrounding `exit 1` still
|
||||
# fires, so the gate looks correct and reports garbage. Caught by
|
||||
# the mutation test, not by review.
|
||||
MEMPALACE_YANK_REASON=$(printf '%s' "$MEMPALACE_PYPI" | jq -r '.info.yanked_reason // "no reason given"' 2>/dev/null || true)
|
||||
echo "::error::Pinned mempalace version ${MEMPALACE_VERSION} is YANKED on PyPI (${MEMPALACE_YANK_REASON:-no reason given}). An exact pin installs a yanked release without complaint — bump ARG MEMPALACE_VERSION in Dockerfile.base."
|
||||
exit 1
|
||||
fi
|
||||
# Informational only, exactly like pi's npm drift warning: a newer
|
||||
# palace must never be adopted implicitly. `|| true` so a transient
|
||||
# PyPI failure cannot fail a release whose pin is already verified.
|
||||
MEMPALACE_PYPI_LATEST=$(curl -sf "https://pypi.org/pypi/mempalace/json" | jq -r '.info.version // empty' 2>/dev/null || true)
|
||||
if [ -n "${MEMPALACE_PYPI_LATEST:-}" ] && [ "${MEMPALACE_PYPI_LATEST}" != "${MEMPALACE_VERSION}" ]; then
|
||||
echo "::warning::mempalace ${MEMPALACE_PYPI_LATEST} is published; this build ships the audited pin ${MEMPALACE_VERSION}. To adopt it: read the upstream CHANGELOG for MCP tool-schema changes (the agent-facing contract) and for sync/delete semantics, check the skew it introduces against the central palace host's server version, then bump ARG MEMPALACE_VERSION in Dockerfile.base and note the audit in CHANGELOG.md."
|
||||
fi
|
||||
echo "mempalace_version=${MEMPALACE_VERSION}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# pi-fork / pi-observational-memory (GitHub) → commit SHAs.
|
||||
FORK_REF=$(curl -sf -H "Accept: application/vnd.github.sha" \
|
||||
"https://api.github.com/repos/elpapi42/pi-fork/commits/master" || true)
|
||||
@@ -176,15 +356,47 @@ jobs:
|
||||
echo "fork_ref=${FORK_REF}" >> "$GITHUB_OUTPUT"
|
||||
echo "obsmem_ref=${OBSMEM_REF}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# pi-toolkit / pi-extensions (Gitea) → commit SHAs. Gitea API
|
||||
# requires auth even for public-repo commit listing.
|
||||
TOOLKIT_REF=$(curl -sf -H "$AUTH_HEADER" \
|
||||
"https://gitea.jordbo.se/api/v1/repos/joakimp/pi-toolkit/commits?limit=1&sha=main" \
|
||||
| jq -r '.[0].sha // empty' 2>/dev/null || true)
|
||||
# pi-atelier → the PINNED TAG's commit SHA. Unlike fork/obsmem
|
||||
# (which track a branch head) atelier wraps pi's private TUI
|
||||
# renderer, so its version is pinned in Dockerfile.variant and read
|
||||
# from there; we only resolve tag → SHA, for reproducibility and to
|
||||
# defeat the cache-hit footgun. Never floats to a branch.
|
||||
ATELIER_TAG=$(sed -n 's/^ARG PI_ATELIER_REF=\([^[:space:]]*\).*/\1/p' Dockerfile.variant | head -n1)
|
||||
if ! printf '%s' "${ATELIER_TAG:-}" | grep -qE '^v?[0-9]+\.[0-9]+\.[0-9]+$'; then
|
||||
echo "::error::ARG PI_ATELIER_REF in Dockerfile.variant is not a semver tag (got '${ATELIER_TAG:-<empty>}'). pi-atelier must stay pinned to a tag — see the floor note above that ARG."
|
||||
exit 1
|
||||
fi
|
||||
ATELIER_LS=$(git ls-remote --tags "https://github.com/michaelmjhhhh/pi-atelier.git" || true)
|
||||
# Peeled ^{} line first (annotated tags), then the direct ref.
|
||||
ATELIER_REF=$(printf '%s\n' "$ATELIER_LS" | awk -v t="refs/tags/${ATELIER_TAG}^{}" '$2==t{print $1}')
|
||||
if [ -z "$ATELIER_REF" ]; then
|
||||
ATELIER_REF=$(printf '%s\n' "$ATELIER_LS" | awk -v t="refs/tags/${ATELIER_TAG}" '$2==t{print $1}')
|
||||
fi
|
||||
require_sha PI_ATELIER_REF "$ATELIER_REF"
|
||||
echo "atelier_ref=${ATELIER_REF}" >> "$GITHUB_OUTPUT"
|
||||
echo "atelier_tag=${ATELIER_TAG}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# pi-toolkit / pi-extensions (Gitea) → commit SHAs. All three Gitea
|
||||
# repos read in this step are PUBLIC: an unauthenticated GET of these
|
||||
# commit endpoints returns 200 with the IDENTICAL sha (verified
|
||||
# 2026-08-15 for pi-toolkit, pi-extensions and mempalace-toolkit).
|
||||
# The comment that used to sit here claimed the Gitea API "requires
|
||||
# auth even for public-repo commit listing" — it does not. Only
|
||||
# /api/v1/repos/*/actions/* refuses anonymous reads (401), which is
|
||||
# what that claim was almost certainly generalised from.
|
||||
#
|
||||
# The header is still passed on purpose: it keeps working if a repo is
|
||||
# ever flipped private, and an ABSENT secret degrades cleanly, because
|
||||
# Gitea ignores an empty `token ` value and serves the request
|
||||
# anonymously (200). The real hazard is the opposite one — a REVOKED or
|
||||
# malformed token returns 401 where anonymous would have returned 200,
|
||||
# so a stale GITEA_BUILD_TOKEN turns a healthy public read into a
|
||||
# require_sha failure that reads like an API or network fault. If this
|
||||
# step ever fails on a repo you can browse anonymously, suspect the
|
||||
# token before you suspect Gitea.
|
||||
TOOLKIT_REF=$(gitea_sha pi-toolkit)
|
||||
require_sha PI_TOOLKIT_REF "$TOOLKIT_REF"
|
||||
EXTENSIONS_REF=$(curl -sf -H "$AUTH_HEADER" \
|
||||
"https://gitea.jordbo.se/api/v1/repos/joakimp/pi-extensions/commits?limit=1&sha=main" \
|
||||
| jq -r '.[0].sha // empty' 2>/dev/null || true)
|
||||
EXTENSIONS_REF=$(gitea_sha pi-extensions)
|
||||
require_sha PI_EXTENSIONS_REF "$EXTENSIONS_REF"
|
||||
echo "toolkit_ref=${TOOLKIT_REF}" >> "$GITHUB_OUTPUT"
|
||||
echo "extensions_ref=${EXTENSIONS_REF}" >> "$GITHUB_OUTPUT"
|
||||
@@ -194,9 +406,7 @@ jobs:
|
||||
# into the base-decide hash (see that job) to force a base rebuild
|
||||
# when the toolkit moves — otherwise a toolkit-only fix silently
|
||||
# fails to land unless Dockerfile.base itself changes.
|
||||
MEMPALACE_TOOLKIT_REF=$(curl -sf -H "$AUTH_HEADER" \
|
||||
"https://gitea.jordbo.se/api/v1/repos/joakimp/mempalace-toolkit/commits?limit=1&sha=main" \
|
||||
| jq -r '.[0].sha // empty' 2>/dev/null || true)
|
||||
MEMPALACE_TOOLKIT_REF=$(gitea_sha mempalace-toolkit)
|
||||
require_sha MEMPALACE_TOOLKIT_REF "$MEMPALACE_TOOLKIT_REF"
|
||||
echo "mempalace_toolkit_ref=${MEMPALACE_TOOLKIT_REF}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
@@ -230,7 +440,9 @@ jobs:
|
||||
echo "studio_ref=${STUDIO_REF}" >> "$GITHUB_OUTPUT"
|
||||
echo "studio_tag=${STUDIO_TAG}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "Resolved PI_VERSION=${PI_VERSION}"
|
||||
echo "Resolved PI_VERSION=${PI_VERSION} (pinned in Dockerfile.variant; npm latest is ${PI_NPM_LATEST:-unknown})"
|
||||
echo "Resolved MEMPALACE_VERSION=${MEMPALACE_VERSION} (pinned in Dockerfile.base; PyPI latest is ${MEMPALACE_PYPI_LATEST:-unknown})"
|
||||
echo "Resolved PI_ATELIER_REF=${ATELIER_REF} (pi-atelier ${ATELIER_TAG}, pinned)"
|
||||
echo "Resolved PI_FORK_REF=${FORK_REF}, PI_OBSMEM_REF=${OBSMEM_REF}"
|
||||
echo "Resolved PI_TOOLKIT_REF=${TOOLKIT_REF}, PI_EXTENSIONS_REF=${EXTENSIONS_REF}"
|
||||
echo "Resolved PI_STUDIO_REF=${STUDIO_REF} (pi-studio ${STUDIO_TAG})"
|
||||
@@ -357,12 +569,20 @@ jobs:
|
||||
PI_TOOLKIT_REF=${{ needs.resolve-versions.outputs.toolkit_ref }}
|
||||
PI_EXTENSIONS_REF=${{ needs.resolve-versions.outputs.extensions_ref }}
|
||||
MEMPALACE_TOOLKIT_REF=${{ needs.resolve-versions.outputs.mempalace_toolkit_ref }}
|
||||
PI_ATELIER_REF=${{ needs.resolve-versions.outputs.atelier_ref }}
|
||||
PI_ATELIER_VERSION=${{ needs.resolve-versions.outputs.atelier_tag }}
|
||||
RELEASE_TAG=smoke
|
||||
SOURCE_REVISION=${{ github.sha }}
|
||||
- name: Smoke test (amd64)
|
||||
env:
|
||||
EXPECTED_PI_VERSION: ${{ needs.resolve-versions.outputs.pi_version }}
|
||||
run: bash scripts/smoke-test.sh pi-devbox:smoke
|
||||
EXPECTED_MEMPALACE_VERSION: ${{ needs.resolve-versions.outputs.mempalace_version }}
|
||||
run: |
|
||||
# Single source of truth for the node major is Dockerfile.base's ARG.
|
||||
# Asserting the BUILT image matches it also catches a stale cached layer.
|
||||
EXPECTED_NODE_MAJOR=$(sed -n 's/^ARG NODE_VERSION=\([0-9][0-9]*\).*/\1/p' Dockerfile.base)
|
||||
export EXPECTED_NODE_MAJOR
|
||||
bash scripts/smoke-test.sh pi-devbox:smoke
|
||||
|
||||
# ── Phase 3b: amd64 smoke for the studio variant ────────────────────
|
||||
# Additive + independent of the core `smoke` job: gates ONLY
|
||||
@@ -417,16 +637,32 @@ jobs:
|
||||
PI_STUDIO_REF=${{ needs.resolve-versions.outputs.studio_ref }}
|
||||
PI_STUDIO_VERSION=${{ needs.resolve-versions.outputs.studio_tag }}
|
||||
MEMPALACE_TOOLKIT_REF=${{ needs.resolve-versions.outputs.mempalace_toolkit_ref }}
|
||||
PI_ATELIER_REF=${{ needs.resolve-versions.outputs.atelier_ref }}
|
||||
PI_ATELIER_VERSION=${{ needs.resolve-versions.outputs.atelier_tag }}
|
||||
RELEASE_TAG=smoke-studio
|
||||
SOURCE_REVISION=${{ github.sha }}
|
||||
- name: Smoke test studio (amd64)
|
||||
env:
|
||||
EXPECTED_PI_VERSION: ${{ needs.resolve-versions.outputs.pi_version }}
|
||||
run: bash scripts/smoke-test.sh pi-devbox:smoke-studio
|
||||
EXPECTED_MEMPALACE_VERSION: ${{ needs.resolve-versions.outputs.mempalace_version }}
|
||||
run: |
|
||||
# Single source of truth for the node major is Dockerfile.base's ARG.
|
||||
# Asserting the BUILT image matches it also catches a stale cached layer.
|
||||
EXPECTED_NODE_MAJOR=$(sed -n 's/^ARG NODE_VERSION=\([0-9][0-9]*\).*/\1/p' Dockerfile.base)
|
||||
export EXPECTED_NODE_MAJOR
|
||||
bash scripts/smoke-test.sh pi-devbox:smoke-studio
|
||||
|
||||
# ── Phase 4: multi-arch publish ─────────────────────────────────────
|
||||
build-variant:
|
||||
needs: [base-decide, smoke, resolve-versions]
|
||||
# A `smoke_only` dispatch stops the pipeline here: base is probed/built and
|
||||
# both smoke jobs run, but nothing is published. Deliberately NOT wrapped in
|
||||
# always() — specifying `if:` keeps the implicit "all needs succeeded" gate,
|
||||
# so a failing smoke still blocks the release. On a tag push `inputs` is
|
||||
# unset, and `null != 'true'` is true, so releases are unaffected.
|
||||
# promote-base-latest and update-description need build-variant to have
|
||||
# succeeded, so they skip on their own — no extra guard required.
|
||||
if: inputs.smoke_only != 'true'
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: catthehacker/ubuntu:act-latest
|
||||
@@ -471,6 +707,8 @@ jobs:
|
||||
TOOLKIT_REF: ${{ needs.resolve-versions.outputs.toolkit_ref }}
|
||||
EXTENSIONS_REF: ${{ needs.resolve-versions.outputs.extensions_ref }}
|
||||
MEMPALACE_TOOLKIT_REF: ${{ needs.resolve-versions.outputs.mempalace_toolkit_ref }}
|
||||
ATELIER_REF: ${{ needs.resolve-versions.outputs.atelier_ref }}
|
||||
ATELIER_TAG: ${{ needs.resolve-versions.outputs.atelier_tag }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TAG_FLAGS=()
|
||||
@@ -490,6 +728,10 @@ jobs:
|
||||
--build-arg "PI_TOOLKIT_REF=${TOOLKIT_REF}" \
|
||||
--build-arg "PI_EXTENSIONS_REF=${EXTENSIONS_REF}" \
|
||||
--build-arg "MEMPALACE_TOOLKIT_REF=${MEMPALACE_TOOLKIT_REF}" \
|
||||
--build-arg "PI_ATELIER_REF=${ATELIER_REF}" \
|
||||
--build-arg "PI_ATELIER_VERSION=${ATELIER_TAG}" \
|
||||
--build-arg "IMAGE_TITLE=pi-devbox" \
|
||||
--build-arg "IMAGE_DESCRIPTION=pi-devbox ${RELEASE_TAG} — core variant: pi coding agent CLI ${PI_VERSION}, pi-toolkit, extensions (fork + observational-memory + atelier ${ATELIER_TAG} TUI sidebar), MemPalace. No browser UI — see the -studio tags for that." \
|
||||
--build-arg "RELEASE_TAG=${RELEASE_TAG}" \
|
||||
--build-arg "BUILD_DATE=${BUILD_DATE}" \
|
||||
--build-arg "SOURCE_REVISION=${GITHUB_SHA:-}" \
|
||||
@@ -513,6 +755,7 @@ jobs:
|
||||
# or fail independently of the core release.
|
||||
build-variant-studio:
|
||||
needs: [base-decide, smoke-studio, resolve-versions]
|
||||
if: inputs.smoke_only != 'true'
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: catthehacker/ubuntu:act-latest
|
||||
@@ -559,6 +802,8 @@ jobs:
|
||||
STUDIO_REF: ${{ needs.resolve-versions.outputs.studio_ref }}
|
||||
STUDIO_TAG: ${{ needs.resolve-versions.outputs.studio_tag }}
|
||||
MEMPALACE_TOOLKIT_REF: ${{ needs.resolve-versions.outputs.mempalace_toolkit_ref }}
|
||||
ATELIER_REF: ${{ needs.resolve-versions.outputs.atelier_ref }}
|
||||
ATELIER_TAG: ${{ needs.resolve-versions.outputs.atelier_tag }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TAG_FLAGS=()
|
||||
@@ -579,6 +824,10 @@ jobs:
|
||||
--build-arg "PI_EXTENSIONS_REF=${EXTENSIONS_REF}" \
|
||||
--build-arg "MEMPALACE_TOOLKIT_REF=${MEMPALACE_TOOLKIT_REF}" \
|
||||
--build-arg "INSTALL_STUDIO=true" \
|
||||
--build-arg "IMAGE_TITLE=pi-devbox (studio)" \
|
||||
--build-arg "PI_ATELIER_REF=${ATELIER_REF}" \
|
||||
--build-arg "PI_ATELIER_VERSION=${ATELIER_TAG}" \
|
||||
--build-arg "IMAGE_DESCRIPTION=pi-devbox ${RELEASE_TAG} — studio variant: everything in the core variant (pi ${PI_VERSION}, pi-toolkit, fork + observational-memory + atelier ${ATELIER_TAG}, MemPalace) plus the pi-studio browser UI ${STUDIO_TAG}." \
|
||||
--build-arg "PI_STUDIO_REF=${STUDIO_REF}" \
|
||||
--build-arg "PI_STUDIO_VERSION=${STUDIO_TAG}" \
|
||||
--build-arg "RELEASE_TAG=${RELEASE_TAG}" \
|
||||
|
||||
+128
-4
@@ -6,11 +6,28 @@ name: Lint
|
||||
# actionlint runs shellcheck against each `run:` step using its *effective*
|
||||
# shell, so `set -o pipefail` under dash is flagged as SC3040 before any
|
||||
# expensive build runs. This is cheap (~10s) and independent of the build
|
||||
# pipeline, so it fires on every push/PR — not just on release tags, which
|
||||
# is where the build workflow (docker-publish.yml) is otherwise only
|
||||
# pipeline, so it fires on every branch push/PR — not just on release tags,
|
||||
# which is where the build workflow (docker-publish.yml) is otherwise only
|
||||
# triggered.
|
||||
#
|
||||
# `branches: ['**']` (rather than a bare `push:`) deliberately EXCLUDES tag
|
||||
# pushes. A bare `push:` also fires on `refs/tags/v*`, which was duplicate work —
|
||||
# the tagged tree was already linted when the same commit was pushed to main
|
||||
# (v1.6.4: lint id=529 on refs/heads/main, then id=531 again on
|
||||
# refs/tags/v1.6.4, same sha e86e5df). The wasted compute is small (measured:
|
||||
# lint here runs 0.3-0.9 min, against a 77.6 min release build for v1.6.4 — so
|
||||
# runner contention is NOT a real argument in this repo, unlike opencode-devbox
|
||||
# where actionlint installs shellcheck and takes 6-15 min). The substantive
|
||||
# reason is discovery ambiguity: the runs listing is newest-first, so the
|
||||
# tag-ref lint run sorts ABOVE the publish run, and "first run matching
|
||||
# refs/tags/<tag>" picks lint — which goes green in under a minute while the
|
||||
# image is still building, making a release look finished before anything is
|
||||
# published. See AGENTS.md "Gitea API access" for the head_sha-filtered
|
||||
# discovery pattern.
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- '**'
|
||||
pull_request:
|
||||
workflow_dispatch:
|
||||
|
||||
@@ -35,6 +52,35 @@ jobs:
|
||||
apt-get update
|
||||
apt-get install -y --no-install-recommends shellcheck python3-yaml
|
||||
|
||||
- name: "Shellcheck + syntax-check repository scripts (severity: error)"
|
||||
# Gap being closed: everything else in this job shellchecks workflow
|
||||
# `run:` steps ONLY, via actionlint. The repo's own shell scripts —
|
||||
# entrypoint.sh, scripts/*.sh, and the extensionless tools under
|
||||
# rootfs/usr/local/bin/ — have never been shellchecked. That exact gap
|
||||
# (a sibling repo with no shell-script lint at all) is how a defect
|
||||
# shipped invisibly for two months: `echo "$json" | python3 <<'EOF'
|
||||
# ... json.load(sys.stdin)` cannot work — with no script argument
|
||||
# python reads its SCRIPT from stdin, so the heredoc IS stdin and the
|
||||
# json.load call hits EOF. shellcheck flags exactly this at severity
|
||||
# ERROR (SC2259, "This redirection overrides piped input"); nothing
|
||||
# ever ran it. Measured before adding this gate: `-S error` is 0
|
||||
# findings across every shell file in THIS repo today, so it is free
|
||||
# to add. `-S warning` is NOT free here (19x SC2088 tilde-in-quotes in
|
||||
# scripts/recreate-sanity-check.sh, plus assorted SC2016 — both
|
||||
# intentional), so warning-level would train people to ignore the job;
|
||||
# hence error-only, matching the SHELLCHECK_OPTS philosophy below.
|
||||
#
|
||||
# Discovery is *.sh UNION a shebang scan, because rootfs/usr/local/
|
||||
# bin/{pi-devbox-version,devbox-skill-reconcile,dot-watch,studio-expose}
|
||||
# are shell scripts with no extension. -print0/mapfile -d '' so a path
|
||||
# with a space cannot silently split, and the file count is asserted
|
||||
# non-zero — a green tick over an empty file set is not a check.
|
||||
#
|
||||
# The implementation moved to scripts/lint-shell.sh on 2026-09-08 so the
|
||||
# release gate in docker-publish.yml runs the SAME code rather than a
|
||||
# second copy that drifts. Edit the script, not a copy of it.
|
||||
run: bash scripts/lint-shell.sh
|
||||
|
||||
- name: Gitea shell guard (catches the actionlint blind spot)
|
||||
# actionlint models GitHub Actions, where the default run shell is
|
||||
# bash, so it does NOT flag bash syntax in a step that merely OMITS
|
||||
@@ -46,7 +92,7 @@ jobs:
|
||||
|
||||
- name: Install actionlint (pinned)
|
||||
env:
|
||||
ACTIONLINT_VERSION: 1.7.7
|
||||
ACTIONLINT_VERSION: 1.7.12
|
||||
run: |
|
||||
curl -fsSL \
|
||||
"https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" \
|
||||
@@ -81,7 +127,7 @@ jobs:
|
||||
|
||||
- name: Install hadolint (pinned)
|
||||
env:
|
||||
HADOLINT_VERSION: 2.14.0
|
||||
HADOLINT_VERSION: 2.15.1
|
||||
run: |
|
||||
curl -fsSL \
|
||||
"https://github.com/hadolint/hadolint/releases/download/v${HADOLINT_VERSION}/hadolint-Linux-x86_64" \
|
||||
@@ -91,3 +137,81 @@ jobs:
|
||||
|
||||
- name: Run hadolint
|
||||
run: hadolint Dockerfile.base Dockerfile.variant
|
||||
|
||||
skill-floor:
|
||||
# Gate the VENDORED pi-extensions skill snapshot in rootfs/ against the
|
||||
# package repo it is a snapshot of. Its own job rather than a step in
|
||||
# `actionlint`, so "the floor is stale" is a distinct red name in the runs
|
||||
# list instead of being buried in a lint job that is about something else.
|
||||
#
|
||||
# The gap it closes, measured 2026-09-10: the floor sat at 34284 B, untouched
|
||||
# since fa04d20 (2026-07-30), while the package copy was 38973 B.
|
||||
# Dockerfile.variant copies the fresh package copy over the SERVED path but
|
||||
# never writes back to the floor, so nothing in the repo ever noticed. That
|
||||
# matters because the floor is a FALLBACK: the copy is guarded by
|
||||
# `if [ -f /opt/pi-extensions/skill/SKILL.md ]`, so a build whose clone
|
||||
# yields no skill/ ships the vendored snapshot and still goes green, with no
|
||||
# manifest flag or label saying which copy was served.
|
||||
#
|
||||
# Gating on another repo is normally a smell; it is proportionate here
|
||||
# because the check compares the skill DIRECTORY hash, so it can only fire
|
||||
# when that directory actually changed — which is exactly when the floor has
|
||||
# gone stale. pi-extensions commits that leave skill/ alone cannot turn this
|
||||
# red. No secret is needed either: the repo is anonymously clonable (verified
|
||||
# 2026-09-10 with `git ls-remote` and no credentials), so this cannot start
|
||||
# failing when a token expires.
|
||||
#
|
||||
# Exit codes are 0 in sync / 1 drift / 2 cannot-run, matching
|
||||
# scripts/lint-shell.sh: a gate that cannot run must not pass, so an
|
||||
# unreachable package repo is a red 2 rather than a green tick.
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: catthehacker/ubuntu:act-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Vendored pi-extensions skill floor matches the package
|
||||
run: bash scripts/check-skill-floor.sh
|
||||
|
||||
doc-drift:
|
||||
# Gate hand-maintained doc claims against the build files they describe.
|
||||
# Its own job for the same reason as skill-floor: "the docs lie" should be a
|
||||
# distinct red name, not a line buried in a job about workflow syntax.
|
||||
#
|
||||
# The gap it closes, measured 2026-09-10 while preparing v1.9.0 — five
|
||||
# claims had rotted, every one of them a fact written by hand in a file
|
||||
# nothing verified:
|
||||
# * README.md's "Version pins" table was wrong on ALL THREE rows (pi
|
||||
# 0.84.4 vs 0.85.1, pi-atelier v0.10.0 vs v0.10.1, mempalace 3.8.0 vs
|
||||
# 3.9.0) — and that table exists specifically to be the reviewable
|
||||
# record of what the repo freezes on purpose, so a wrong row destroys
|
||||
# the only thing it is for.
|
||||
# * README.md listed already-shipped typst PDF export under "Planned for
|
||||
# an upcoming minor release", marked "(shipped in Unreleased/base)".
|
||||
# * DOCKER_HUB.md claimed "Node.js v22" while v1.9.0 ships Node 24.
|
||||
#
|
||||
# DOCKER_HUB.md is why this is a gate and not a habit. It is PUBLISHED —
|
||||
# update-description POSTs it to Docker Hub as full_description on every tag
|
||||
# — and it had gone eight releases (v1.8.6 -> v1.9.0) untouched. Nothing
|
||||
# generates it and nothing checked it, so the only thing keeping it true was
|
||||
# someone remembering. It is also read from the TAG, so a fix pushed to main
|
||||
# after tagging never reaches the published page.
|
||||
#
|
||||
# Cheap and hermetic on purpose: every check compares a doc string against a
|
||||
# value that exists in this repo, so no network, no token, no built image,
|
||||
# and no sibling clone. Claims that genuinely need a running container (image
|
||||
# sizes, the "N mempalace_* tools" count) are deliberately left out — a gate
|
||||
# that cannot evaluate a claim honestly would have to guess, and a guessing
|
||||
# gate is worse than none. Assert those in scripts/smoke-test.sh instead.
|
||||
#
|
||||
# Exit codes 0 in sync / 1 drift / 2 cannot-run, matching lint-shell.sh and
|
||||
# check-skill-floor.sh. A renamed ARG makes the gate blind, so that is a red
|
||||
# 2, not a green tick.
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: catthehacker/ubuntu:act-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Doc claims match the build files
|
||||
run: bash scripts/check-doc-drift.sh
|
||||
|
||||
@@ -64,26 +64,141 @@ re-brand of opencode-devbox's `pi-only` variant.
|
||||
(`curl -sf 'https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/latest' | jq -r .version`).
|
||||
Check release notes at https://github.com/earendil-works/pi/releases for
|
||||
the upstream changelog to include in `CHANGELOG.md`.
|
||||
2. Update `CHANGELOG.md` Unreleased → vX.Y.Z section.
|
||||
3. Verify `docker compose up` works locally with the current `latest` image
|
||||
2. **Refresh the vendored mempalace skill snapshot if the skillset moved:**
|
||||
`scripts/vendor-mempalace-skill.sh --check` (reads a real skillset clone,
|
||||
writes nothing). Three exit codes, not two — a stale-but-truthful record is
|
||||
**not** a release blocker, so don't treat any non-zero exit as "must
|
||||
refresh" without reading which one it was:
|
||||
- **0** — the record is truthful. This includes stale-but-truthful
|
||||
(upstream has moved past the recorded ref, or the local clone has
|
||||
uncommitted changes) — a `NOTICE` is printed, but nothing is lying.
|
||||
**Skipping the refresh in this case is the legitimate, sanctioned
|
||||
outcome** — every enrolled host reads its own live skillset clone, so
|
||||
the baked copy is only a no-mount fallback. What is not legitimate is
|
||||
skipping it *silently*: the drift is visible here, in
|
||||
`pi-devbox-version`, and in the manifest, so decide rather than forget.
|
||||
- **1** — a confirmed problem: the vendored bytes provably do NOT match
|
||||
the file at the recorded ref (a lying record), or the recorded ref
|
||||
doesn't even resolve to that path in this clone. Refresh.
|
||||
- **2** — cannot determine (the recorded ref itself isn't resolvable in
|
||||
this clone — commonly a shallow checkout missing history). Fetch full
|
||||
history and re-check before deciding; don't refresh blind.
|
||||
Refresh with `scripts/vendor-mempalace-skill.sh`, which rewrites the file
|
||||
**and** the ARG together so they cannot drift apart, and refuses (exit 1)
|
||||
rather than silently rewinding provenance if the skillset clone's HEAD is
|
||||
behind the already-recorded ref (detached HEAD, older checkout) — pass
|
||||
`--force` only if that rewind is genuinely intended.
|
||||
Two consequences to accept deliberately on an actual refresh: the snapshot
|
||||
is hashed into `base_tag`, so it costs a base rebuild (~67 min); and if the
|
||||
section the phrase canary names has changed, re-pin it in
|
||||
`scripts/smoke-test.sh`.
|
||||
3. **Update the docs this release makes stale — BEFORE you tag.** Rename
|
||||
`CHANGELOG.md`'s `## Unreleased` to `## vX.Y.Z — YYYY-MM-DD` (em dash, as
|
||||
every prior release heading uses), then run the gate:
|
||||
|
||||
```bash
|
||||
bash scripts/check-doc-drift.sh # 0 in sync / 1 drift / 2 cannot run
|
||||
```
|
||||
|
||||
It compares README.md's version-pin table against the ARGs it names, and
|
||||
DOCKER_HUB.md's Node claim against `ARG NODE_VERSION`, plus Hub's
|
||||
25 000-char limit, unsubstituted `{{PLACEHOLDERS}}`, and stale `Unreleased`
|
||||
pointers in user-facing docs.
|
||||
|
||||
**Why before and not after:** `docker-publish.yml` runs `actions/checkout@v4`
|
||||
with no `ref:`, so every job reads `github.ref` — the **tag**. A doc fix
|
||||
pushed to `main` after tagging does not reach the release, and for
|
||||
`DOCKER_HUB.md` it does not reach the published Hub page either, because
|
||||
`update-description` POSTs that file as Docker Hub's `full_description` from
|
||||
the tag's tree. Getting it in afterwards means re-pointing the tag, which is
|
||||
its own hazard (v1.8.14 went `601fc98` → `361babd` and broke deploy
|
||||
verification until `git fetch --tags --force`).
|
||||
|
||||
The gate is deliberately narrow — it only checks claims verifiable from files
|
||||
in this repo. Still eyeball, because these are NOT gated:
|
||||
- counts and sizes (`~1.1 GB`, "N `mempalace_*` tools", "7 extensions") —
|
||||
they need a running image; assert them in `scripts/smoke-test.sh` instead
|
||||
- feature prose that quietly became false, e.g. a "Planned for an upcoming
|
||||
release" section describing something that already shipped
|
||||
- `Dockerfile.base`'s `# BASE_REBUILD_DATE:` marker. Ungated on purpose:
|
||||
`base_tag` hashes Dockerfile.base's content, comments included, so
|
||||
demanding it be current would force a ~60 min base rebuild on a release
|
||||
that touched no base files. **Fix it when the base is already rebuilding —
|
||||
then it is free.**
|
||||
|
||||
Measured cost of skipping this, 2026-09-10 (v1.9.0): five stale claims, one
|
||||
of them published. README's pin table was wrong on all three rows, and
|
||||
DOCKER_HUB.md — untouched for eight releases — still said Node v22 while the
|
||||
image shipped Node 24.
|
||||
4. Verify `docker compose up` works locally with the current `latest` image
|
||||
if you're upgrading users from a previous version. Then run the
|
||||
**post-recreate sanity check** inside the running container to confirm
|
||||
persisted volumes survived and the pi runtime wiring re-deployed (not just
|
||||
that the container booted):
|
||||
`docker compose exec devbox bash scripts/recreate-sanity-check.sh --expected-version X.Y.Z`
|
||||
(or just `pi-devbox-sanity --expected-version X.Y.Z` if `cli_utils/bin` is
|
||||
on PATH). This is the runtime peer of the build-time `smoke-test.sh` gate.
|
||||
4. Push tag: `git tag vX.Y.Z && git push origin vX.Y.Z`.
|
||||
5. Watch CI: smoke job builds amd64 only and asserts size + extensions +
|
||||
`docker compose exec devbox bash scripts/recreate-sanity-check.sh --expected-image-version X.Y.Z`
|
||||
(or just `pi-devbox-sanity --expected-image-version X.Y.Z` if
|
||||
`cli_utils/bin` is on PATH). This is the runtime peer of the build-time
|
||||
`smoke-test.sh` gate.
|
||||
**`X.Y.Z` here is the pi-devbox release tag** you are shipping (e.g.
|
||||
`1.8.9`), which is what the rest of this checklist means by `vX.Y.Z`.
|
||||
`--expected-image-version` is the flag that asserts it. There is also an
|
||||
`--expected-version`, and it means something else — the **pi coding agent**
|
||||
version (e.g. `0.84.3`, the `ARG PI_VERSION` pin). Handing the release tag
|
||||
to that one used to report *"pi version mismatch: expected 1.8.8, got
|
||||
0.84.3"*, i.e. a red on the final gate of the release accusing the wrong
|
||||
component; it now tells you to use `--expected-image-version` instead, and
|
||||
the reverse mix-up is caught too. Both flags are optional — with neither,
|
||||
the live pi version is asserted against the version recorded in the image's
|
||||
own build manifest (which catches a stale `pi` in the `~/.pi/npm-global`
|
||||
volume shadowing the baked one) and the image tag is reported
|
||||
informationally.
|
||||
5. Push tag: `git tag vX.Y.Z && git push origin vX.Y.Z`.
|
||||
6. Watch CI: smoke job builds amd64 only and asserts size + extensions +
|
||||
pi version + new-base-tooling presence. Variant build is multi-arch
|
||||
(amd64 + arm64) only after smoke passes.
|
||||
6. Verify the Hub tags appear (latest + vX.Y.Z, the `-studio` pair, plus
|
||||
(amd64 + arm64) only after smoke passes. A tag push fires **only**
|
||||
`docker-publish.yml` — `lint.yml` is scoped to `branches: ['**']`, which
|
||||
excludes tag refs on purpose (the tagged tree was already linted when the
|
||||
commit hit `main`, and a fast lint run sorting above the slow publish run
|
||||
made releases look finished before anything shipped). Verified on v1.8.4:
|
||||
`refs/tags/v1.8.4` produced run 571 (publish) and nothing else. Still filter
|
||||
discovery on `head_sha` **and** the workflow `path` — see *Gitea API access*
|
||||
below — because that guard costs nothing and a future workflow added on `v*`
|
||||
would silently reintroduce the ambiguity.
|
||||
7. Verify the Hub tags appear (latest + vX.Y.Z, the `-studio` pair, plus
|
||||
base-latest if the base was rebuilt this run).
|
||||
7. **Revoke any short-lived Gitea PAT** used during the release at
|
||||
8. **Revoke any short-lived Gitea PAT** used during the release at
|
||||
`gitea.jordbo.se/user/settings/applications`. N/A if you used the
|
||||
`GITEA_ACCESS_TOKEN` env var instead (see *Gitea API access* below) —
|
||||
its lifecycle is managed host-side, nothing to revoke.
|
||||
|
||||
## Verifying this repo's reality from inside a container
|
||||
|
||||
Most work on this repo happens **inside** a pi-devbox container, inspecting a
|
||||
host or a peer over SSH. That setup manufactures convincing false negatives, so
|
||||
when you are about to report that something is **absent, unreachable, or not
|
||||
running**, suspect your own command first. Recurring instances:
|
||||
|
||||
- **`docker` is not on the host's non-interactive SSH `PATH`.** `ssh mac 'docker
|
||||
ps'` says *command not found* on a host that plainly runs Docker; use
|
||||
`/usr/local/bin/docker` (or `command -v docker` first). Every step in the
|
||||
*Release-day checklist* that inspects a running container hits this.
|
||||
- **Don't `| head -N` a search whose answer you don't already know.** The host's
|
||||
`~/.ssh/config` is ~500 lines; a `head -20` "proved" a peer absent that was
|
||||
defined at line 454.
|
||||
- **The deployment compose file is not this repo's.** `docker-compose.yml` here
|
||||
is a template pinning `:latest`; a real host runs its own per-machine file
|
||||
(find it with `docker inspect <container> --format '{{ index .Config.Labels
|
||||
"com.docker.compose.project.config_files" }}'`). Recreating from the repo copy
|
||||
can silently move a host off `:latest-studio` onto `:latest`.
|
||||
- **A live SSH ControlMaster hides remote auth changes** — after editing a
|
||||
peer's `authorized_keys`, prove access with `-o ControlPath=none -o
|
||||
ControlMaster=no`, or the breakage surfaces in a later session instead.
|
||||
|
||||
Depth and further mechanisms: the repo-authored `pi-devbox-environment` skill
|
||||
(`rootfs/usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md`) §2
|
||||
and §3 — that file is the one an agent actually loads mid-session, whereas this
|
||||
`AGENTS.md` is only auto-read when the cwd *is* this repo.
|
||||
|
||||
## Gitea API access (env token)
|
||||
|
||||
`GITEA_ACCESS_TOKEN` + `GITEA_HOST` are passed into the container from the
|
||||
@@ -92,13 +207,61 @@ host `.env` via `docker-compose.yml` (`${GITEA_ACCESS_TOKEN:-}` /
|
||||
**not** baked into the image. When configured, they are also available for
|
||||
**any** direct Gitea API interaction from inside the container — inspecting
|
||||
CI runs, checking published tags, listing commits — e.g.
|
||||
`curl -H "Authorization: token $GITEA_ACCESS_TOKEN" "$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs?limit=5"`.
|
||||
`curl -H "Authorization: token $GITEA_ACCESS_TOKEN" "$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs?limit=20"`.
|
||||
Prefer this over a short-lived PAT file when the env token is present (the
|
||||
`ci-release-watcher` skill auto-detects it). Public-repo GET listings work
|
||||
unauthenticated too, so the token matters mainly for private repos or
|
||||
rate-limit headroom; its lifecycle is host-managed, so there is nothing to
|
||||
revoke after use. Never echo the token value (including into logs).
|
||||
|
||||
**Gotcha — a tag push fires EVERY workflow whose triggers match the tag ref.**
|
||||
`lint.yml` uses a bare `push:` trigger, so a release tag yields *both* a lint run
|
||||
and the publish run. The listing is newest-first and lint sorts **above** the
|
||||
publish run, so "take the first run whose `path` contains `refs/tags/<tag>`"
|
||||
picks the wrong one **reliably, not occasionally**. Real listing for v1.6.4:
|
||||
|
||||
```
|
||||
id=531 #104 lint.yml@refs/tags/v1.6.4 <- wrong; sorts first
|
||||
id=530 #103 docker-publish.yml@refs/tags/v1.6.4 <- the release build
|
||||
id=529 #102 lint.yml@refs/heads/main <- same commit, linted on push
|
||||
```
|
||||
|
||||
Lint goes green in minutes while the image is still building, so watching it
|
||||
makes a release look finished when nothing has been published yet.
|
||||
|
||||
**Gotcha — the jobs endpoint takes the internal `id`, NOT the `run_number` the
|
||||
UI shows as `#104`.** The two diverge widely, and `GET
|
||||
.../actions/runs/<run_number>/jobs` does **not** error — it silently returns a
|
||||
*different* run's jobs. Always read `id` from the run listing:
|
||||
|
||||
```bash
|
||||
# Which runs did this tag/commit trigger? Filter on head_sha; never trust
|
||||
# ordering or run numbering. limit=20, not 5 — with two runs per push the
|
||||
# publish run falls off a 5-item window fast.
|
||||
curl -sS -H "Authorization: token $GITEA_ACCESS_TOKEN" \
|
||||
"$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs?limit=20" \
|
||||
| jq --arg sha "$(git rev-list -n1 vX.Y.Z)" \
|
||||
'.workflow_runs[] | select(.head_sha==$sha) | {id, run_number, path, status, conclusion}'
|
||||
# pick the id whose .path starts with docker-publish.yml, then:
|
||||
curl -sS -H "Authorization: token $GITEA_ACCESS_TOKEN" \
|
||||
"$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs/<id>/jobs" \
|
||||
| jq '.jobs[] | {name, status, conclusion}'
|
||||
```
|
||||
|
||||
**Watcher config for this repo** (`ci-release-watcher` skill, hub-only shape —
|
||||
pi-devbox has no downstream host to deploy to):
|
||||
|
||||
- `EXPECT_WORKFLOW=docker-publish.yml` — the skill's `preflight_run()` aborts at
|
||||
startup if the run id belongs to lint instead.
|
||||
- `EXPECTED_FRESH_TAGS='vX.Y.Z latest vX.Y.Z-studio latest-studio'`
|
||||
- `EXPECTED_EXISTS_TAGS='base-latest'` — existence only: it is content-addressed
|
||||
and legitimately keeps its old timestamp when the base is a cache hit.
|
||||
- `CRITICAL_JOBS='build-variant build-variant-studio'` — job names are matched
|
||||
**exactly** (`critical.issubset(succeeded)`), so the studio variant must be
|
||||
listed explicitly; the skill's default omits it. Leave `promote-base-latest`
|
||||
out: it legitimately skips on a base cache hit, which would misclassify a good
|
||||
run. `update-description` is the cosmetic post-publish job.
|
||||
|
||||
## Cache-hit footgun (must-know)
|
||||
|
||||
`PI_VERSION` defaults to `latest` in `Dockerfile.variant` but **CI must
|
||||
|
||||
+3340
File diff suppressed because it is too large
Load Diff
+11
-3
@@ -46,7 +46,8 @@ Full setup guide — authentication for each provider (Anthropic, OpenAI, Gemini
|
||||
|
||||
### pi and companions
|
||||
|
||||
- **pi `{{PI_VERSION}}`** ([`@earendil-works/pi-coding-agent`](https://www.npmjs.com/package/@earendil-works/pi-coding-agent)) — installed at `/usr/bin/pi`
|
||||
- **pi `{{PI_VERSION}}`** ([`@earendil-works/pi-coding-agent`](https://www.npmjs.com/package/@earendil-works/pi-coding-agent)) — installed at `/usr/bin/pi`, pinned to an audited version (not npm `latest`)
|
||||
- **pi-atelier** — TUI sidebar (ordered panels, split-pane, themes), vendored at `/opt/pi-atelier` and pinned to an audited tag; the exact tag is in the image labels (`se.jordbo.pi-devbox.pi-atelier-version`) and `/etc/pi-devbox/build-manifest.json`
|
||||
- **[pi-toolkit](https://gitea.jordbo.se/joakimp/pi-toolkit)** — keybindings (mosh/tmux-friendly Shift+Enter, Ctrl+J, Alt+J newline bindings), AWS env loader, settings template
|
||||
- **[pi-extensions](https://gitea.jordbo.se/joakimp/pi-extensions)** — 7 user-facing extensions: `ext-toggle`, `mcp-loader`, `todo`, `ssh-controlmaster`, `notify`, `git-checkpoint`, `confirm-destructive`
|
||||
- **`fork`** ([pi-fork](https://github.com/elpapi42/pi-fork)) and **`recall`** ([pi-observational-memory](https://github.com/elpapi42/pi-observational-memory)) tools
|
||||
@@ -64,12 +65,19 @@ The entrypoint deploys/registers all of these on first container start. Re-runni
|
||||
### Document and image tooling
|
||||
|
||||
- **pandoc** — universal Markdown↔HTML/Org/RST/etc. conversion. Useful well beyond pi: agent-driven doc exports, format conversion, etc.
|
||||
- **Typst** — markup-based typesetting, used as pandoc's `--pdf-engine`
|
||||
- **graphviz** (`dot`) — diagram rendering pipelines
|
||||
- **imagemagick** (`magick`) — image conversion / resizing
|
||||
|
||||
### Browser automation
|
||||
|
||||
- **agent-browser** — CLI for driving a real browser (open pages, click/fill/`eval`, snapshot the DOM, screenshots) so agents can verify front-end work instead of guessing
|
||||
- **Playwright** + a headless **Chromium** are pre-installed and pinned together; `AGENT_BROWSER_EXECUTABLE_PATH` is preset to the baked browser, so `agent-browser open <url>` works out of the box with no setup
|
||||
- **socat** — TCP bridge used to expose the pi-studio server outside the container's loopback
|
||||
|
||||
### Modern CLI tooling
|
||||
|
||||
- **Editor**: neovim (LazyVim defaults), tmux (configured for 0-indexed sessions)
|
||||
- **Editor**: neovim (system-wide `termguicolors` default; bring your own config/plugins), tmux (configured for 0-indexed sessions)
|
||||
- **Search/nav**: ripgrep, fd, fzf, zoxide
|
||||
- **Display**: bat, eza, htop, tree
|
||||
- **Data**: jq, yq
|
||||
@@ -86,7 +94,7 @@ The entrypoint deploys/registers all of these on first container start. Re-runni
|
||||
uv run --with jupyterlab jupyter lab --no-browser --port 8888
|
||||
uv run --with marimo marimo edit
|
||||
```
|
||||
- **Node.js** v22 + npm (used by pi itself)
|
||||
- **Node.js** v24 LTS + npm (used by pi itself)
|
||||
- **Rust** — `rustup-init` is on PATH; install toolchains on demand
|
||||
- **Go** — opt-in via `--build-arg INSTALL_GO=true` if rebuilding from source
|
||||
|
||||
|
||||
+199
-6
@@ -83,6 +83,98 @@ ENV DEBIAN_FRONTEND=noninteractive
|
||||
# above); TERM=xterm-ghostty is compiled from an alias further
|
||||
# down (ncurses ships `ghostty`, not `xterm-ghostty`). iTerm2
|
||||
# defaults to xterm-256color (ncurses-base), so needs nothing.
|
||||
# iproute2 — `ss` (socket statistics) and `ip`. Measured 2026-08-30 on
|
||||
# v1.8.11: NEITHER was present, so the container could not
|
||||
# answer "what is listening in here" by any means, and
|
||||
# cli_utils' `portcheck` was a hard stub — it prints
|
||||
# "portcheck requires at least one of: ss, lsof, netstat" and
|
||||
# all three were absent. `ss` satisfies its preferred branch
|
||||
# (`ss -tlnp`), which is also the branch that reports the
|
||||
# owning PID, so nothing further is needed: net-tools is
|
||||
# deliberately NOT added (`netstat` is deprecated and only a
|
||||
# fallback branch) and neither is lsof (~500 KB for a third
|
||||
# path to the same answer). ~5.5 MB total: iproute2 itself is
|
||||
# 4.2 MB and pulls 6 libs under --no-install-recommends
|
||||
# (libbpf1, libmnl0, libtirpc-common, libtirpc3t64,
|
||||
# libxtables12, libcap2-bin — libpam-cap is a Recommends and
|
||||
# is correctly dropped). Verified end-to-end in a live
|
||||
# container: `ss` lands at /usr/bin/ss, `ip` at /usr/sbin/ip
|
||||
# (both already on the developer PATH), and `portcheck --all`
|
||||
# then correctly identifies the socat listener on 8765.
|
||||
# shellcheck — shell linter. Added 2026-09-09 to close a CAPABILITY gap, not
|
||||
# a style preference. `scripts/lint-shell.sh` is the release
|
||||
# GATE (the `lint-gate` job that `resolve-versions` depends
|
||||
# on), and it correctly refuses to pass when shellcheck is
|
||||
# missing — "a gate that cannot run must not pass". Measured on
|
||||
# v1.8.14: shellcheck was absent from this image by all three
|
||||
# routes (PATH, dpkg, filesystem), so `bash
|
||||
# scripts/lint-shell.sh` exited 2 in EVERY devbox container and
|
||||
# no developer could run the release gate locally at all. The
|
||||
# loop was therefore write-shell → push → wait for CI → discover,
|
||||
# which is the loop the gate was added to shorten: v1.8.14's
|
||||
# first attempt burned ~46 min on a tree whose lint had already
|
||||
# been red for 24 h. This is also what makes a client-side
|
||||
# pre-push hook possible (see hooks/pre-push); without the
|
||||
# binary that hook would refuse every push. ~39 MB installed
|
||||
# (Installed-Size 40112 KB, shellcheck 0.10.0-1) and measured
|
||||
# to pull ZERO additional packages under
|
||||
# --no-install-recommends: its deps (libc6, libffi8, libgmp10)
|
||||
# are already present. NOTE this file feeds the base-decide
|
||||
# hash (Dockerfile.base + rootfs/), so adding it forces one
|
||||
# full base rebuild.
|
||||
# bind9-dnsutils — `dig` and `nslookup`. Added 2026-09-10 to close a
|
||||
# DIAGNOSTIC gap measured during the gitea.egl.lan/FreeIPA
|
||||
# work: the container could resolve names but had NO way to
|
||||
# ask a SPECIFIC nameserver anything. `getent hosts` only
|
||||
# follows the resolver's default path, so the whole "gateway
|
||||
# 172.16.88.1 returns NXDOMAIN for the egl.lan zone while
|
||||
# 10.20.253.1 is authoritative for it" diagnosis had to be
|
||||
# hand-rolled in python3 — dig, host AND nslookup were all
|
||||
# absent. `dig @10.20.253.1 freeipa-4.egl.lan` is the
|
||||
# one-liner that replaces it, and split-horizon DNS is a
|
||||
# recurring class of bug on this fleet, not a one-off. NOTE
|
||||
# the package to name is bind9-dnsutils: plain `dnsutils` is
|
||||
# a transitional package in trixie. ~6.1 MB total (6210 KB
|
||||
# measured): bind9-dnsutils 721 KB + bind9-host 161 KB +
|
||||
# bind9-libs 3804 KB plus 7 small libs (libfstrm0,
|
||||
# libjson-c5, liblmdb0, libmaxminddb0, libprotobuf-c1,
|
||||
# liburcu8t64, libuv1t64) under --no-install-recommends.
|
||||
# ldap-utils — `ldapsearch`/`ldapmodify`. Added 2026-09-10. This fleet
|
||||
# authenticates against FreeIPA (EGL.LAN), and every LDAP
|
||||
# probe during the Gitea auth work had to be run by SSHing to
|
||||
# an already-enrolled host because the container had no LDAP
|
||||
# client at all. 1244 KB and pulls NOTHING extra under
|
||||
# --no-install-recommends — its deps (libldap, libsasl2) are
|
||||
# already present. CAVEAT: this gives SIMPLE binds only,
|
||||
# which is what Gitea itself uses and what most probes need.
|
||||
# GSSAPI binds (`ldapsearch -Y GSSAPI`) additionally require
|
||||
# krb5-user + libsasl2-modules-gssapi-mit, deliberately NOT
|
||||
# added here — that is a Kerberos-client decision with
|
||||
# /etc/krb5.conf implications, not just a tool.
|
||||
# xxd — hex dump. 198 KB, no extra deps. Convenience, and honestly
|
||||
# marginal: `od -c` from coreutils is always present and does
|
||||
# the same job. Earned its place because verifying that
|
||||
# git-crypt actually encrypted a staged blob (the \0GITCRYPT\0
|
||||
# magic) is a recurring check in myconfigs and xxd is the
|
||||
# muscle-memory command for it.
|
||||
# NOT added — netcat-openbsd (133 KB): measured redundant on
|
||||
# 2026-09-10, because socat is already baked above AND bash's
|
||||
# /dev/tcp does reachability checks with zero packages
|
||||
# (verified against gitea.egl.lan:3000). Recorded here so the
|
||||
# omission reads as a decision rather than an oversight.
|
||||
# python3-yaml — PyYAML. Added 2026-09-10 for precisely the same reason as
|
||||
# shellcheck above: a gate this repo ALREADY OWNS could not be
|
||||
# run locally by anyone. scripts/check-workflow-shell.sh — the
|
||||
# guard that catches the "bash-only syntax under Gitea's default
|
||||
# sh/dash shell" footgun that broke resolve-versions (ed49b8d)
|
||||
# and promote-base-latest (b7197e8) — hard-exits with "ERROR:
|
||||
# python3 yaml module missing" without it. lint.yml installs it
|
||||
# explicitly in CI (`shellcheck python3-yaml`), which is itself
|
||||
# the evidence that the image lacked it. Measured 2026-09-10
|
||||
# while wiring the skill-floor job: the guard could not be run
|
||||
# before pushing — the same write → push → wait-for-CI loop that
|
||||
# shellcheck was baked to shorten. 552 KB, and pulls ZERO extra
|
||||
# packages under --no-install-recommends.
|
||||
RUN apt-get update && \
|
||||
apt-get upgrade -y --no-install-recommends && \
|
||||
apt-get install -y --no-install-recommends \
|
||||
@@ -102,6 +194,7 @@ RUN apt-get update && \
|
||||
make \
|
||||
patch \
|
||||
diffutils \
|
||||
shellcheck \
|
||||
git-crypt \
|
||||
age \
|
||||
file \
|
||||
@@ -122,6 +215,11 @@ RUN apt-get update && \
|
||||
nano \
|
||||
kitty-terminfo \
|
||||
ncurses-term \
|
||||
iproute2 \
|
||||
bind9-dnsutils \
|
||||
ldap-utils \
|
||||
xxd \
|
||||
python3-yaml \
|
||||
&& ln -s /usr/bin/fdfind /usr/local/bin/fd \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
@@ -367,13 +465,90 @@ ARG INSTALL_MEMPALACE=true
|
||||
# diary_write schema. Pinning makes mempalace upgrades a reviewable diff
|
||||
# rather than a surprise.
|
||||
#
|
||||
# 3.5.0 (2026-06) ships the upstream fix for the top-level-anyOf diary_write
|
||||
# 3.5.0 (2026-06) shipped the upstream fix for the top-level-anyOf diary_write
|
||||
# schema (issue #1728 / PR #1717, merged 2026-06-14): the advertised schema
|
||||
# is now `"required": ["agent_name"]` with entry/content enforced at dispatch,
|
||||
# which Anthropic's tools API accepts — so the old mcp_server.py perl
|
||||
# workaround that used to live below is gone. Keep in lockstep with
|
||||
# opencode-devbox when bumping.
|
||||
ARG MEMPALACE_VERSION=3.5.0
|
||||
# workaround that used to live below is gone.
|
||||
#
|
||||
# 3.6.0 (2026-07-17, PyPI latest) is additive/reliability only — secure
|
||||
# `mempalace serve` remote mode, optional Milvus backend, atomic KG
|
||||
# supersede(), conversation chronology, mining exclusions, plus recovery and
|
||||
# locking fixes. Reviewed for MCP tool-schema changes before bumping (that
|
||||
# being the exact regression class this pin exists to catch): there are NONE,
|
||||
# and nothing touches diary_write. Two fixes matter for how this image uses
|
||||
# mempalace: read-only mode now covers checkpoint + delete_by_source in
|
||||
# _MUTATING_TOOLS (#1930), and agent attribution is preserved in
|
||||
# mempalace_checkpoint (#2023/#2034).
|
||||
#
|
||||
# Keep in lockstep with opencode-devbox when bumping.
|
||||
#
|
||||
# 3.7.1 (from 3.6.0) is safe for anyone with an EXISTING LOCAL palace: verified
|
||||
# against the 3.7.1 source, not the changelog. Legacy drawers lack the new
|
||||
# `chunk_total` marker and both decision sites trust them ("trust the match as
|
||||
# before"), NORMALIZE_VERSION is 2 in both, chromadb stays <2 (no index-format
|
||||
# migration), there is no auto-migration ("We do NOT auto-migrate"), and the one
|
||||
# new palace file (logstream.sqlite3) is created lazily on first logstream use.
|
||||
# Two behaviour changes to know: MEMPALACE_MCP_ALLOW_PEER_WRITER no longer works
|
||||
# on local/chroma palaces, and writer-lock setup failures now fail CLOSED
|
||||
# (refuse the write) rather than fail open. Neither affects the container's
|
||||
# normal MCP-server-plus-CLI-feeder pattern, which already serialised on the
|
||||
# same lock under 3.6.0.
|
||||
#
|
||||
# 3.8.0 (2026-08-23, PyPI, released hours after this project's own v1.8.5 tag
|
||||
# the same day) is additive/reliability only — reviewed for MCP tool-schema
|
||||
# changes before bumping, as always: there are NONE. Two PRs matter:
|
||||
# - PR #2320/#2322: `sync --apply` no longer deletes a drawer solely because
|
||||
# its source_file was unreachable AT THAT MOMENT — it now asks for
|
||||
# corroboration first. This fixes losing a whole mined project to one
|
||||
# `sync --apply` while its volume happened to be unmounted.
|
||||
# IMPORTANT — do not over-read this fix: it addresses TRANSIENT
|
||||
# unreachability, not the standing landmine (documented in the operator's
|
||||
# global AGENTS.md) against running `mempalace_sync` / `mempalace_delete_by_source`
|
||||
# beyond dry-run on the SHARED central palace. On that palace most
|
||||
# source_file paths are PERMANENTLY absent from whichever host runs the
|
||||
# sync — a different machine's paths simply do not exist here, ever, not
|
||||
# merely "right now". That is a different failure shape than #2320/#2322
|
||||
# fixes. The landmine still stands; this bump does not relax it.
|
||||
# - PR #2307: long-running Chroma servers no longer invalidate their own
|
||||
# HNSW cache on their own writes (server-side perf fix). This does NOT
|
||||
# make `mempalace_reconnect` unnecessary — that tool exists for EXTERNAL
|
||||
# writes bypassing the in-process client (e.g. direct sqlite backfills,
|
||||
# CLI commands against a running server), a different scenario #2307
|
||||
# does not touch.
|
||||
#
|
||||
# CI-side audit (added after v1.8.6, closing that release's "Still open" item):
|
||||
# resolve-versions now treats this pin exactly as it treats PI_VERSION — it
|
||||
# reads the ARG from THIS file, refuses a non-concrete value, verifies the
|
||||
# version is published on PyPI, refuses a YANKED release (an exact pin installs
|
||||
# one silently under PEP 592), and WARNS — never silently adopts — when PyPI has
|
||||
# a newer release. smoke-test.sh then asserts the installed core equals that
|
||||
# audited pin, which catches a stale cached base layer that no manifest-internal
|
||||
# check can see. So a bump here is now gated end to end; what remains manual is
|
||||
# the JUDGEMENT above (MCP schema review, server/client sequencing), which is
|
||||
# the part that should stay manual.
|
||||
#
|
||||
# Deployment sequencing note for whoever ships this bump: synlig (the shared
|
||||
# central palace host) serves mempalace 3.8.0 SERVER-SIDE via
|
||||
# docker-compose.mempalace.yml, which reuses this same devbox image. (Measured
|
||||
# 2026-09-06 over ssh: synlig's UV_TOOL_DIR mempalace entry last changed
|
||||
# 2026-08-25 15:33 — this comment previously said 3.7.1, which was stale.)
|
||||
# Bumping this ARG changes only the CLIENT version baked into pi-devbox
|
||||
# images: it introduces client/server skew until synlig's compose stack is
|
||||
# separately rebuilt/redeployed with the new pin. Not something to code around
|
||||
# here — just sequence the redeploy.
|
||||
#
|
||||
# v1.8.13: 3.8.0 -> 3.9.0. Audited: no Breaking/Removed changelog headings.
|
||||
# Adopted mainly for #2281 (`mempalace_mine` accepts a single conversation
|
||||
# file again) — though note that does NOT unblock this image's own feeder,
|
||||
# which was measured to mine DIRECTORIES, not files, so it was never hitting
|
||||
# that bug. Four behaviour changes ride along and are skew-relevant while
|
||||
# synlig stays on 3.8.0: hub-forward escaping, an HTTP lock split, similarity
|
||||
# score semantics, and parsed-output compatibility. 3.9.0-only features
|
||||
# (release awareness, `task create`/`task launch` MCP tools) are SERVER-side,
|
||||
# so they stay dark until synlig is redeployed — a client bump alone cannot
|
||||
# light them up.
|
||||
ARG MEMPALACE_VERSION=3.9.0
|
||||
ENV UV_TOOL_DIR=/opt/uv-tools
|
||||
ENV UV_TOOL_BIN_DIR=/usr/local/bin
|
||||
RUN if [ "${INSTALL_MEMPALACE}" = "true" ]; then \
|
||||
@@ -413,9 +588,15 @@ RUN if [ "${INSTALL_MEMPALACE}" = "true" ] && [ "${INSTALL_MEMPALACE_TOOLKIT}" =
|
||||
[ "$ok" = "1" ] && \
|
||||
ln -sf /opt/mempalace-toolkit/bin/mempalace-session /usr/local/bin/mempalace-session && \
|
||||
ln -sf /opt/mempalace-toolkit/bin/mempalace-docs /usr/local/bin/mempalace-docs && \
|
||||
chmod +x /opt/mempalace-toolkit/bin/mempalace-session /opt/mempalace-toolkit/bin/mempalace-docs && \
|
||||
ln -sf /opt/mempalace-toolkit/bin/mempalace-pi-session /usr/local/bin/mempalace-pi-session && \
|
||||
ln -sf /opt/mempalace-toolkit/bin/mempalace-census /usr/local/bin/mempalace-census && \
|
||||
chmod +x /opt/mempalace-toolkit/bin/mempalace-session /opt/mempalace-toolkit/bin/mempalace-docs \
|
||||
/opt/mempalace-toolkit/bin/mempalace-pi-session \
|
||||
/opt/mempalace-toolkit/bin/mempalace-census && \
|
||||
mempalace-session --help >/dev/null && \
|
||||
mempalace-docs --help >/dev/null && \
|
||||
mempalace-pi-session --help >/dev/null && \
|
||||
mempalace-census --help >/dev/null && \
|
||||
echo "mempalace-toolkit installed at $(cd /opt/mempalace-toolkit && git rev-parse --short HEAD)" ; \
|
||||
fi
|
||||
|
||||
@@ -452,7 +633,17 @@ ENV COLORTERM=truecolor
|
||||
ENV PATH="/home/developer/.local/bin:/home/developer/.cargo/bin:${PATH}"
|
||||
|
||||
# ── Node.js (required for pi + MCP servers + tldr) ──
|
||||
ARG NODE_VERSION=22
|
||||
# 24 (LTS "Krypton"), raised from 22 on 2026-09-10 because the image was BELOW a
|
||||
# DECLARED requirement, not merely behind the newest release: `agent-browser`
|
||||
# publishes engines.node ">=24.0.0", so every build on 22 installed it with an npm
|
||||
# EBADENGINE warning and then ran it outside its supported range — measured on
|
||||
# v1.8.14, which shipped node 22.23.2 with agent-browser 0.37.1. The other two npm
|
||||
# consumers are satisfied either way: pi declares ">=22.19.0" and playwright
|
||||
# ">=20". Verified before bumping, because a missing NodeSource suite would break
|
||||
# the build for every arch at once: deb.nodesource.com/setup_24.x returns HTTP 200
|
||||
# and the node_24.x suite advertises `Architectures: amd64 arm64 armhf x86_64`, so
|
||||
# both the arm64 fleet and the amd64 CI runners resolve.
|
||||
ARG NODE_VERSION=24
|
||||
RUN curl -fsSL --retry 5 --retry-delay 5 --retry-all-errors https://deb.nodesource.com/setup_${NODE_VERSION}.x | bash - && \
|
||||
apt-get install -y --no-install-recommends nodejs && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
@@ -657,12 +848,14 @@ COPY rootfs/usr/local/share/pi-devbox/ /usr/local/share/pi-devbox/
|
||||
COPY rootfs/usr/local/bin/studio-expose /usr/local/bin/studio-expose
|
||||
COPY rootfs/usr/local/bin/dot-watch /usr/local/bin/dot-watch
|
||||
COPY rootfs/usr/local/bin/pi-devbox-version /usr/local/bin/pi-devbox-version
|
||||
COPY rootfs/usr/local/bin/devbox-skill-reconcile /usr/local/bin/devbox-skill-reconcile
|
||||
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
|
||||
COPY entrypoint-user.sh /usr/local/bin/entrypoint-user.sh
|
||||
RUN chmod +x /usr/local/bin/entrypoint.sh /usr/local/bin/entrypoint-user.sh \
|
||||
/usr/local/bin/studio-expose \
|
||||
/usr/local/bin/dot-watch \
|
||||
/usr/local/bin/pi-devbox-version \
|
||||
/usr/local/bin/devbox-skill-reconcile \
|
||||
/usr/local/lib/pi-devbox/*.sh 2>/dev/null || true
|
||||
|
||||
# Start as root — entrypoint adjusts UID/GID then drops to developer
|
||||
|
||||
+297
-12
@@ -29,16 +29,91 @@ ARG USER_NAME=developer
|
||||
# runs each repo's install.sh on container start so symlinks land under
|
||||
# ~/.pi/agent/ on the named volume.
|
||||
#
|
||||
# PI_VERSION should be passed explicitly by CI as a concrete version
|
||||
# (resolved from `npm view @earendil-works/pi-coding-agent version`).
|
||||
# The default `latest` is for local dev convenience only — it has a
|
||||
# known cache-hit footgun in registry-cached CI builds: the resulting
|
||||
# build-arg string is byte-identical across builds, the layer-hash is
|
||||
# identical, and the registry buildcache silently reuses the layer
|
||||
# from whatever pi version was current when the cache was first
|
||||
# populated. CI MUST pass a resolved concrete version. See pi-devbox
|
||||
# v0.75.5b 2026-05-23 for the discovery + canonical fix.
|
||||
ARG PI_VERSION=latest
|
||||
# ── pi version pin: an AUDITED CHECKPOINT, not a freeze ──────────────
|
||||
# PI_VERSION is pinned to a version whose upstream CHANGELOG has been read
|
||||
# against this image's integration surface: the theme/TUI API that pi-atelier
|
||||
# couples to, the session `.jsonl` format that `pi-session-repair` parses, the
|
||||
# extension/package loader, and the Node engine floor. CI reads THIS LINE as
|
||||
# the single source of truth (see the `resolve-versions` job) and no longer
|
||||
# follows npm `latest` — following it meant every release silently adopted
|
||||
# whatever pi shipped that morning, unaudited, in the very build that then got
|
||||
# tagged and published.
|
||||
#
|
||||
# BUMPING IS ROUTINE AND EXPECTED — the pin exists to force a look, not to
|
||||
# hold a version forever:
|
||||
# 1. Read the upstream CHANGELOG for every version between old and new.
|
||||
# 2. Re-check the companions that couple to pi's private TUI/renderer
|
||||
# internals — pi-atelier above all (see PI_ATELIER_REF below for the
|
||||
# 0.6.0-under-pi-0.84 startup-hang precedent).
|
||||
# 3. Bump this line, record the audit in CHANGELOG.md, then tag.
|
||||
# CI fails the build if this pin is not a published npm version, and warns —
|
||||
# without adopting it — when npm `latest` has moved ahead. That warning is the
|
||||
# prompt to do step 1; it is not something to silence.
|
||||
#
|
||||
# A concrete version here ALSO defeats the registry-buildcache cache-hit
|
||||
# footgun that `latest` carried: a byte-identical build-arg string produced an
|
||||
# identical layer hash, so the cache reused the layer from whatever pi was
|
||||
# current when it was first populated (shipped the same bytes for pi-devbox
|
||||
# v0.74.0..v0.75.5; discovered + fixed in v0.75.5b, 2026-05-23). The `latest`
|
||||
# branch below is kept only for a deliberate local `docker build` override.
|
||||
#
|
||||
# AUDITED AT 0.84.4 (2026-08-31, was 0.84.3): NO "Breaking Changes" and no
|
||||
# "Removed" heading in the 0.84.4 section (grepped, 0 matches) — unlike 0.84.3,
|
||||
# whose heading is described in the paragraph below and stays audited. Adopted
|
||||
# for three fixes that land on machinery this fleet actually runs:
|
||||
# - #6879 large tool results crossing the auto-compaction threshold were sent
|
||||
# to the provider BEFORE compacting; pi now compacts between tool execution
|
||||
# and the next assistant response in the same run. This is the shape of
|
||||
# nearly every session here (multi-hundred-KB logstream/palace tool output).
|
||||
# - #8345 a resumed session corrupted its next appended entry when the JSONL
|
||||
# lacked a trailing newline. That file is the memory feeder's own input.
|
||||
# Measured on tor-ms22 before the bump: 49/49 transcripts end in a newline,
|
||||
# 0 lines fail json.loads — the bug had not bitten this corpus.
|
||||
# - #8537 extension messages sent with `triggerTurn: false` WHILE THE AGENT IS
|
||||
# RUNNING were inserted between a tool call and its result, so
|
||||
# order-validating providers rejected the replayed history. The mempalace
|
||||
# mailbox is outside that precondition — it delivers at `agent_settled`
|
||||
# (idle) with `{deliverAs:"steer"}` and deliberately no `triggerTurn` — and
|
||||
# 0.84.4 leaves the documented steer semantics unchanged, so RFC 003 §7.11
|
||||
# still holds. Recorded because the fix is what would make a future mid-run
|
||||
# delivery safe, which is the only reason we would ever change that call.
|
||||
# One doc consequence, fixed in this same release: pi's own docs/compaction.md
|
||||
# gained exactly one paragraph — the autoCompact threshold is now ALSO checked
|
||||
# mid-run, after a tool batch's results are appended. See
|
||||
# docs/observational-memory.md §3, which had said compaction is only checked
|
||||
# when pi goes idle.
|
||||
#
|
||||
# AUDITED AT 0.84.3 (2026-08-25, was 0.84.2): upstream's notes carry a
|
||||
# "Breaking Changes" heading — `GoogleThinkingLevel` renamed to
|
||||
# `GoogleApiThinkingLevel`. INERT FOR THIS IMAGE: all four vendored companions
|
||||
# (/opt/pi-fork, /opt/pi-observational-memory, /opt/pi-atelier, /opt/pi-studio)
|
||||
# were grepped for that symbol and reference it ZERO times, so nothing here
|
||||
# couples to the renamed type. Recorded because the heading will look alarming
|
||||
# to the next reader doing step 1 above — the audit is done, don't redo it.
|
||||
# Adopted for two fixes that land squarely on this repo's own vendored-skill
|
||||
# wiring (see devbox-skill-reconcile, v1.8.5): nested Markdown skills inside
|
||||
# `.agents/skills/<group>/` directories were not discovered, and root Markdown
|
||||
# files such as README.md / AGENTS.md inside a skill dir were reported as
|
||||
# broken skills unless they declared valid skill frontmatter.
|
||||
#
|
||||
# v1.8.13: 0.84.4 -> 0.85.1. SKIP 0.85.0 deliberately — it accidentally
|
||||
# published internal experimental code and extra subpaths, breaking SDK
|
||||
# imports (upstream #9132); 0.85.1 exists specifically to undo that, with the
|
||||
# supported SDK and stdio RPC API unchanged. Audited: no Breaking/Removed
|
||||
# changelog headings in either release, engine floor unchanged (>=22.19.0,
|
||||
# container runs 22.23.2), runtime deps 20 -> 19. User-visible changes are the
|
||||
# streaming indicator moving into the editor border and faster fullscreen
|
||||
# transcript search; no deprecation language anywhere.
|
||||
#
|
||||
# Verified EMPIRICALLY rather than from the changelog, because a pi bump has
|
||||
# hung the TUI before (pi-atelier < 0.7.1 + pi >= 0.84): 0.85.1 was
|
||||
# side-installed and driven under a pty against all four companion extensions,
|
||||
# with atelier v0.10.0 AND v0.10.1 — five combinations, each rendering alive
|
||||
# with a CPU delta of 0.00-0.01s over a 5s window, where the known hang
|
||||
# signature is ~5s of sustained CPU. Two-sided check: the atelier sidebar
|
||||
# painted ACTIVITY+WORKSPACE identically to the 0.84.4 control, so the test
|
||||
# could distinguish "loaded" from "silently absent".
|
||||
ARG PI_VERSION=0.85.1
|
||||
ARG PI_TOOLKIT_REF=main
|
||||
ARG PI_EXTENSIONS_REF=main
|
||||
# Repo URLs default to the canonical gitea origin but are overridable so a
|
||||
@@ -54,6 +129,50 @@ ARG PI_FORK_REPO=https://github.com/elpapi42/pi-fork.git
|
||||
ARG PI_FORK_REF=master
|
||||
ARG PI_OBSMEM_REPO=https://github.com/elpapi42/pi-observational-memory.git
|
||||
ARG PI_OBSMEM_REF=master
|
||||
# pi-atelier (TUI sidebar: ordered panels, split-pane, themes) is PINNED TO A
|
||||
# TAG, which CI resolves to that tag's commit SHA — same treatment as
|
||||
# pi-studio, for reproducibility plus cache-busting.
|
||||
#
|
||||
# This floor is hard-earned. pi-atelier 0.6.0/0.7.0 wrapped pi's PRIVATE TUI
|
||||
# renderer, and under pi 0.84 that wrapper recursed: pi hung at startup with
|
||||
# sustained CPU. Upstream fixed the recursion in 0.7.1 and restored the
|
||||
# non-overlapping split in 0.7.2 — "avoiding the recursive render path that
|
||||
# caused startup hangs and sustained CPU usage". Its own peerDependencies
|
||||
# still say `>=0.80.7`, which does NOT encode that floor, so nothing would
|
||||
# have warned us: NEVER pair pi-atelier < 0.7.1 with pi >= 0.84. Bump this
|
||||
# pin and PI_VERSION together, checking atelier's CHANGELOG for the pi
|
||||
# version it claims to track.
|
||||
#
|
||||
# AUDITED AT v0.10.0 (2026-08-31, was v0.8.2 — two minor releases): no
|
||||
# BREAKING notice in either release, and both are UI-only (Sidebar calm during
|
||||
# an active Turn, composer frame + Status Rail, fullscreen-copy-safe Sidebar,
|
||||
# Windows path normalisation, Workspace Pulse deferred until pi trusts the
|
||||
# project). The one coupling that matters runs the OPPOSITE way to the floor
|
||||
# above: v0.9.0 renders the Sidebar as a separate split-layout child and
|
||||
# therefore "raises the minimum supported Pi version to 0.84.0", which its
|
||||
# peerDependencies do encode this time (`>=0.84.0`, up from `>=0.80.7`).
|
||||
# Satisfied with room to spare by PI_VERSION 0.84.4 above — and note that both
|
||||
# executable floors (scripts/smoke-test.sh, scripts/recreate-sanity-check.sh)
|
||||
# compare with `sort -V`, so 0.10.0 >= 0.7.1 is evaluated correctly rather than
|
||||
# as the string comparison that would read 0.10.0 as older than 0.7.1.
|
||||
# Pairs deliberately with pi 0.84.4's own fullscreen selection-copy controls:
|
||||
# atelier keeps Sidebar content out of the transcript selection, pi adds
|
||||
# `fullscreenCopyOnSelect` + Ctrl+X for the selection itself.
|
||||
#
|
||||
# No `npm install` step, unlike pi-fork/pi-observational-memory/pi-studio:
|
||||
# pi-atelier declares ZERO runtime dependencies (only peerDeps, satisfied by
|
||||
# the baked pi) and has no build step — pi loads its TypeScript directly from
|
||||
# the /opt checkout. Adding an install here would be a no-op that only costs
|
||||
# build time.
|
||||
ARG PI_ATELIER_REPO=https://github.com/michaelmjhhhh/pi-atelier.git
|
||||
# v1.8.13: v0.10.0 -> v0.10.1. Refactor-only upstream (formatters, tests,
|
||||
# panel identity); peerDependencies declare pi >=0.84.0, so it spans both the
|
||||
# old and new pin. Included because it was already exercised: the pty matrix
|
||||
# for PI_VERSION above ran atelier v0.10.1 against pi 0.85.1 and painted the
|
||||
# sidebar identically to v0.10.0.
|
||||
ARG PI_ATELIER_REF=v0.10.1
|
||||
# Human-readable tag PI_ATELIER_REF was resolved from; recorded as a label.
|
||||
ARG PI_ATELIER_VERSION=v0.10.1
|
||||
|
||||
RUN set -e && \
|
||||
# git_fetch_ref: clone-equivalent helper that accepts EITHER a branch name
|
||||
@@ -87,12 +206,14 @@ RUN set -e && \
|
||||
git_fetch_ref "${PI_EXTENSIONS_REPO}" "${PI_EXTENSIONS_REF}" /opt/pi-extensions && \
|
||||
git_fetch_ref "${PI_FORK_REPO}" "${PI_FORK_REF}" /opt/pi-fork && \
|
||||
git_fetch_ref "${PI_OBSMEM_REPO}" "${PI_OBSMEM_REF}" /opt/pi-observational-memory && \
|
||||
git_fetch_ref "${PI_ATELIER_REPO}" "${PI_ATELIER_REF}" /opt/pi-atelier && \
|
||||
(cd /opt/pi-fork && npm install --omit=dev --no-audit --no-fund) && \
|
||||
(cd /opt/pi-observational-memory && npm install --omit=dev --no-audit --no-fund) && \
|
||||
echo "pi-toolkit at $(cd /opt/pi-toolkit && git rev-parse --short HEAD)" && \
|
||||
echo "pi-extensions at $(cd /opt/pi-extensions && git rev-parse --short HEAD)" && \
|
||||
echo "pi-fork at $(cd /opt/pi-fork && git rev-parse --short HEAD)" && \
|
||||
echo "pi-observational-memory at $(cd /opt/pi-observational-memory && git rev-parse --short HEAD)"
|
||||
echo "pi-observational-memory at $(cd /opt/pi-observational-memory && git rev-parse --short HEAD)" && \
|
||||
echo "pi-atelier at $(cd /opt/pi-atelier && git rev-parse --short HEAD) (${PI_ATELIER_VERSION})"
|
||||
|
||||
# ── Image-baked skill refresh: pi-extensions (Option 1 over Option 2) ──
|
||||
# rootfs ships a VENDORED snapshot of the pi-extensions skill at
|
||||
@@ -161,6 +282,30 @@ ARG PI_STUDIO_REF=main
|
||||
# PI_STUDIO_VERSION is the human-readable tag (e.g. v0.9.36) that PI_STUDIO_REF
|
||||
# was resolved from; recorded as a label below for at-a-glance identification.
|
||||
# Only meaningful for the studio variant (default `none` otherwise).
|
||||
#
|
||||
# v1.8.13 — READ THIS BEFORE REASONING ABOUT WHICH pi-studio SHIPS. Neither
|
||||
# default below survives a CI build. `resolve-versions` in
|
||||
# .gitea/workflows/docker-publish.yml passes BOTH as build-args (studio_ref and
|
||||
# studio_tag), and it deliberately selects the newest STABLE semver tag: its
|
||||
# filter is `^v?[0-9]+\.[0-9]+\.[0-9]+$`, which excludes pre-releases. So a
|
||||
# PUBLISHED v1.8.13 studio image contains pi-studio v0.9.59 (commit 9eed84f,
|
||||
# = refs/tags/v0.9.59^{}), NOT the v0.9.60-rc.0 that `main` currently points at
|
||||
# (658536f). The `main` default here only applies to a local `docker build`
|
||||
# that passes no studio args.
|
||||
#
|
||||
# That upstream-tag-over-main choice is intentional and documented at the
|
||||
# resolve step: pi-studio keeps tagging every version but stopped publishing
|
||||
# GitHub Releases at v0.5.55 and pushes freely to main, so pinning main risked
|
||||
# baking half-finished commits that land after a tag.
|
||||
#
|
||||
# Corrected here on 2026-09-06 after reading the run-639 resolve-versions
|
||||
# output: the v1.8.13 audit had recorded "RC adopted deliberately" and set this
|
||||
# ARG to v0.9.60-rc.0, which was measured at the wrong layer — a Dockerfile
|
||||
# default cannot answer "what will CI publish?" when CI overrides it. Left at
|
||||
# `none` rather than pinned to a tag, because a hardcoded pre-release here goes
|
||||
# stale the moment main moves and would re-tell the same lie to the next reader.
|
||||
# Consequence worth keeping: the RC's opt-in Studio network binding is NOT in
|
||||
# any published v1.8.13 image, so it needs no audit for this release.
|
||||
ARG PI_STUDIO_VERSION=none
|
||||
RUN if [ "${INSTALL_STUDIO}" = "true" ]; then \
|
||||
set -e; \
|
||||
@@ -219,18 +364,62 @@ ARG SOURCE_REVISION=
|
||||
# MEMPALACE_TOOLKIT_REF is consumed in Dockerfile.base; re-declared here
|
||||
# only so its intended ref lands in the label set alongside the others.
|
||||
ARG MEMPALACE_TOOLKIT_REF=main
|
||||
# ── Vendored skill provenance ─────────────────────────────────────────
|
||||
# The vendored mempalace SKILL.md is the ONLY baked artefact with no /opt
|
||||
# clone behind it: its upstream (the skillset repo) is PRIVATE, so the
|
||||
# image cannot clone it and CI cannot resolve its HEAD (see VENDORED.md).
|
||||
# Consequence through v1.8.7: the snapshot was ANONYMOUS — nothing in the
|
||||
# image or the repo recorded which skillset commit it was taken from, so
|
||||
# the only staleness check available was a hand-maintained phrase canary in
|
||||
# scripts/smoke-test.sh, which by construction can only detect "older than
|
||||
# what I remembered to pin", never "older than skillset main".
|
||||
#
|
||||
# Recording the ref costs nothing and makes the question answerable. It is
|
||||
# deliberately a plain ARG DEFAULT rather than a CI-resolved output:
|
||||
# * the value is a fact about the committed snapshot, so it belongs in
|
||||
# the tree next to it — not in a workflow that a local `docker build`
|
||||
# never runs (same reasoning as MEMPALACE_VERSION living in
|
||||
# Dockerfile.base rather than being duplicated in docker-publish.yml);
|
||||
# * CI therefore needs NO new build-arg at any of its four
|
||||
# Dockerfile.variant call sites (smoke, smoke-studio, build-variant,
|
||||
# build-variant-studio) — a plumbing change that is easy to
|
||||
# under-apply to only two of them;
|
||||
# * and it needs no credential for a private repo.
|
||||
# Bump it with scripts/vendor-mempalace-skill.sh, which refreshes the file
|
||||
# and rewrites this line together, so the pair cannot drift apart by hand.
|
||||
# This ARG lives in Dockerfile.variant ON PURPOSE: Dockerfile.base and
|
||||
# rootfs/ are both hashed into base_tag, so recording provenance here costs
|
||||
# no ~67-minute base rebuild. (scripts/check-base-hash.sh scans only
|
||||
# Dockerfile.base, so no folding into the base hash is required — nor would
|
||||
# it be correct, since this ARG changes nothing about the base's contents.)
|
||||
ARG SKILLSET_SNAPSHOT_REF=4d7c0ea9caeb3a1d6d9b04cf34f3fca5f9df4985
|
||||
|
||||
# Dockerfile.base sets description="pi-devbox — base image (variant-independent)"
|
||||
# and every variant INHERITS it, so both published images used to advertise
|
||||
# themselves on Docker Hub as the base image. A LABEL cannot branch on
|
||||
# INSTALL_STUDIO, so the description arrives as a build-arg: CI passes the
|
||||
# variant-specific string (see docker-publish.yml), and the default below keeps
|
||||
# a plain `docker build -f Dockerfile.variant` honest rather than misleading.
|
||||
ARG IMAGE_TITLE="pi-devbox"
|
||||
ARG IMAGE_DESCRIPTION="pi-devbox — development container for the pi coding agent"
|
||||
|
||||
LABEL org.opencontainers.image.version="${RELEASE_TAG}" \
|
||||
org.opencontainers.image.revision="${SOURCE_REVISION}" \
|
||||
org.opencontainers.image.created="${BUILD_DATE}" \
|
||||
org.opencontainers.image.title="${IMAGE_TITLE}" \
|
||||
org.opencontainers.image.description="${IMAGE_DESCRIPTION}" \
|
||||
description="${IMAGE_DESCRIPTION}" \
|
||||
se.jordbo.pi-devbox.pi-version="${PI_VERSION}" \
|
||||
se.jordbo.pi-devbox.pi-toolkit-ref="${PI_TOOLKIT_REF}" \
|
||||
se.jordbo.pi-devbox.pi-extensions-ref="${PI_EXTENSIONS_REF}" \
|
||||
se.jordbo.pi-devbox.pi-fork-ref="${PI_FORK_REF}" \
|
||||
se.jordbo.pi-devbox.pi-obsmem-ref="${PI_OBSMEM_REF}" \
|
||||
se.jordbo.pi-devbox.pi-atelier-ref="${PI_ATELIER_REF}" \
|
||||
se.jordbo.pi-devbox.pi-atelier-version="${PI_ATELIER_VERSION}" \
|
||||
se.jordbo.pi-devbox.mempalace-toolkit-ref="${MEMPALACE_TOOLKIT_REF}" \
|
||||
se.jordbo.pi-devbox.pi-studio-ref="${PI_STUDIO_REF}" \
|
||||
se.jordbo.pi-devbox.pi-studio-version="${PI_STUDIO_VERSION}"
|
||||
se.jordbo.pi-devbox.pi-studio-version="${PI_STUDIO_VERSION}" \
|
||||
se.jordbo.pi-devbox.skillset-snapshot-ref="${SKILLSET_SNAPSHOT_REF}"
|
||||
|
||||
# The manifest is written from GROUND TRUTH — the actual checked-out HEAD
|
||||
# of each /opt clone and the live `pi --version` — not merely the intended
|
||||
@@ -241,19 +430,115 @@ RUN set -e; \
|
||||
mkdir -p /etc/pi-devbox; \
|
||||
rev() { git -C "$1" rev-parse HEAD 2>/dev/null || echo "unknown"; }; \
|
||||
PI_V="$(pi --version 2>/dev/null | head -n1 | tr -d '\r\n')"; \
|
||||
# mempalace CORE (the PyPI package behind the MCP tools) is installed in
|
||||
# Dockerfile.base via `uv tool install`, so no /opt clone reveals it and
|
||||
# until v1.8.6 the manifest could not answer "which palace shipped here?" —
|
||||
# a palace bug could not be correlated to an image, which is precisely the
|
||||
# correlation this file exists to provide. Read from the INSTALLED BINARY,
|
||||
# not from ARG MEMPALACE_VERSION, per the ground-truth rule above: that is
|
||||
# what catches an install which resolved to something other than the pin.
|
||||
# `mempalace --version` prints "MemPalace 3.7.1" — NAME-PREFIXED, unlike
|
||||
# pi's bare "0.84.2" — hence the $NF pick rather than a straight read. The
|
||||
# leading-digit test then rejects usage/error text (a renamed flag prints a
|
||||
# usage block) and degrades to JSON null, so this can never fail the build.
|
||||
MP_V="$(mempalace --version 2>/dev/null | head -n1 | tr -d '\r' | awk '{print $NF}')"; \
|
||||
case "$MP_V" in [0-9]*) MP_CORE="\"${MP_V}\"" ;; *) MP_CORE='null' ;; esac; \
|
||||
STUDIO_REV='null'; \
|
||||
if [ -d /opt/pi-studio/.git ]; then STUDIO_REV="\"$(rev /opt/pi-studio)\""; fi; \
|
||||
# The vendored skill snapshot's fingerprint is MEASURED here, not passed
|
||||
# in as a build-arg, per the ground-truth rule above: SKILLSET_SNAPSHOT_REF
|
||||
# is a CLAIM about which skillset commit the file came from, while this
|
||||
# hash is what the image actually ships. Recorded together they let any
|
||||
# reader with the skillset checked out — which on this fleet is every
|
||||
# host, since all four compose stacks mount it — verify the claim at
|
||||
# RUNTIME, without CI ever needing access to the private repo. Degrades
|
||||
# to JSON null rather than failing the build if the directory is absent;
|
||||
# the smoke assertion is what turns that into a loud failure.
|
||||
#
|
||||
# Hashes the whole DIRECTORY, not just SKILL.md: a single-file hash
|
||||
# answers "did this one file change", not "is the live copy the same
|
||||
# skill" — a live checkout that added or edited a SIBLING file (a
|
||||
# reference/ doc, a helper script) would still report "identical to
|
||||
# baked snapshot" against a file-only hash. pi-extensions already ships
|
||||
# two files for exactly this reason (SKILL.md + evaluate-extension-usage.py),
|
||||
# so this is not a hypothetical. Deterministic over `find | sort`, never
|
||||
# readdir order: relative paths + per-file sha256, folded into one hash.
|
||||
# pi-devbox-version mirrors this exact pipeline over the live directory so
|
||||
# the two sides are comparable — if you change this, change that too.
|
||||
tree_sha256() { \
|
||||
( cd "$1" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum ) 2>/dev/null | sha256sum | cut -d' ' -f1; \
|
||||
}; \
|
||||
SKILL_SNAP='null'; \
|
||||
_snap_dir=/usr/local/share/pi-devbox/skills/mempalace; \
|
||||
if [ -d "$_snap_dir" ] && [ -n "$(find "$_snap_dir" -type f -print -quit)" ]; then \
|
||||
SKILL_SNAP="\"$(tree_sha256 "$_snap_dir")\""; \
|
||||
fi; \
|
||||
# ── WHICH pi-extensions skill copy actually shipped ──
|
||||
# Closes the silent-fallback hole. The refresh step above is guarded by
|
||||
# `[ -f /opt/pi-extensions/skill/SKILL.md ]`, so a build whose clone predates
|
||||
# the co-located skill (or a fork pointing at a mirror without it) keeps the
|
||||
# vendored floor and still succeeds — GREEN, with nothing anywhere recording
|
||||
# that a snapshot shipped instead of the package copy. Measured 2026-09-10:
|
||||
# the floor had been stale since 2026-07-30, so that fallback would have
|
||||
# shipped a six-week-old skill silently. The floor is fresh now and gated by
|
||||
# the skill-floor CI job, but "the fallback is currently harmless" is not the
|
||||
# same as "you can tell which copy you got", and only the second survives.
|
||||
#
|
||||
# MEASURED, never claimed, per the ground-truth rule above: the branch
|
||||
# condition is re-derived from the same test the refresh step used, and the
|
||||
# served bytes are then compared against the clone. A build-arg could not
|
||||
# express this at all, since the outcome depends on the clone's contents.
|
||||
# package served bytes == the clone's skill/ (the normal path)
|
||||
# vendored-floor the clone has no skill/ at this ref (fallback shipped)
|
||||
# divergent both exist but differ — e.g. the clone ships SKILL.md but
|
||||
# not evaluate-extension-usage.py, so the served directory is
|
||||
# a MIX of package and floor. Worth its own value: it is the
|
||||
# one state neither of the other two names honestly.
|
||||
# No OCI label mirrors this, deliberately: LABEL cannot take a value computed
|
||||
# in a RUN, and a label fed from an ARG would be exactly the claim-not-
|
||||
# measurement this block exists to avoid.
|
||||
_px_dir=/usr/local/share/pi-devbox/skills/pi-extensions; \
|
||||
PIEXT_SRC='null'; PIEXT_HASH='null'; \
|
||||
if [ -d "$_px_dir" ] && [ -n "$(find "$_px_dir" -type f -print -quit)" ]; then \
|
||||
PIEXT_HASH="\"$(tree_sha256 "$_px_dir")\""; \
|
||||
if [ -f /opt/pi-extensions/skill/SKILL.md ]; then \
|
||||
if [ "$(tree_sha256 "$_px_dir")" = "$(tree_sha256 /opt/pi-extensions/skill)" ]; then \
|
||||
PIEXT_SRC='"package"'; \
|
||||
else \
|
||||
PIEXT_SRC='"divergent"'; \
|
||||
fi; \
|
||||
else \
|
||||
PIEXT_SRC='"vendored-floor"'; \
|
||||
fi; \
|
||||
fi; \
|
||||
{ \
|
||||
echo '{'; \
|
||||
echo " \"release_tag\": \"${RELEASE_TAG}\","; \
|
||||
echo " \"build_date\": \"${BUILD_DATE}\","; \
|
||||
echo " \"source_revision\": \"${SOURCE_REVISION}\","; \
|
||||
echo " \"pi_version\": \"${PI_V}\","; \
|
||||
# Sibling of pi_version, NOT a member of components{}: that map holds git
|
||||
# SHAs and `pi-devbox-version` renders it with .value[0:12], which would
|
||||
# silently truncate a longer version string.
|
||||
echo " \"mempalace_version\": ${MP_CORE},"; \
|
||||
# Siblings, NOT members of components{}, for two independent reasons:
|
||||
# that map means "HEAD of a clone present in this image" and the
|
||||
# skillset is not cloned here (calling it a component would be a
|
||||
# lie a future reader would act on), and `pi-devbox-version` renders
|
||||
# every components{} value with .value[0:12] — which would truncate
|
||||
# a 64-hex sha256 into something that looks like a short commit.
|
||||
# Named `_tree_sha256`, not `_sha256`: it measures every file under the
|
||||
# vendored skill directory, not one file — see tree_sha256() above.
|
||||
echo " \"skillset_snapshot_ref\": \"${SKILLSET_SNAPSHOT_REF}\","; \
|
||||
echo " \"skillset_snapshot_tree_sha256\": ${SKILL_SNAP},"; \
|
||||
echo " \"pi_extensions_skill_source\": ${PIEXT_SRC},"; \
|
||||
echo " \"pi_extensions_skill_tree_sha256\": ${PIEXT_HASH},"; \
|
||||
echo " \"components\": {"; \
|
||||
echo " \"pi-toolkit\": \"$(rev /opt/pi-toolkit)\","; \
|
||||
echo " \"pi-extensions\": \"$(rev /opt/pi-extensions)\","; \
|
||||
echo " \"pi-fork\": \"$(rev /opt/pi-fork)\","; \
|
||||
echo " \"pi-observational-memory\": \"$(rev /opt/pi-observational-memory)\","; \
|
||||
echo " \"pi-atelier\": \"$(rev /opt/pi-atelier)\","; \
|
||||
echo " \"mempalace-toolkit\": \"$(rev /opt/mempalace-toolkit)\","; \
|
||||
echo " \"pi-studio\": ${STUDIO_REV}"; \
|
||||
echo " }"; \
|
||||
|
||||
@@ -20,7 +20,11 @@ on the host.
|
||||
- `pi-extensions` — TypeScript extensions for pi (preview, MCP bridges,
|
||||
mempalace integration, etc.)
|
||||
- `pi-fork` — the `fork` tool for spawning sub-agents
|
||||
- `pi-observational-memory` — the `recall` tool for session compaction
|
||||
- `pi-observational-memory` — durable session memory: the ledger that makes
|
||||
compaction cheap, plus the `recall` tool. See
|
||||
[`docs/observational-memory.md`](docs/observational-memory.md)
|
||||
- `pi-atelier` — TUI sidebar: ordered panels, split-pane, themes. Pinned to an
|
||||
audited tag; see [Version pins](#version-pins-pi-pi-atelier-mempalace)
|
||||
|
||||
### MemPalace (AI memory)
|
||||
|
||||
@@ -68,9 +72,27 @@ so `TERM=xterm-kitty` is understood. Override either in your own
|
||||
### Document and image tooling
|
||||
|
||||
- `pandoc` — universal Markdown↔HTML/Org/RST/etc. converter
|
||||
- `typst` — markup-based typesetting, wired up as pandoc's `--pdf-engine` (see
|
||||
[Generating a PDF with pandoc + typst](#generating-a-pdf-with-pandoc--typst))
|
||||
- `graphviz` — `dot` rendering for diagram pipelines
|
||||
- `imagemagick` — image conversion / resizing (invoked as `magick`)
|
||||
|
||||
### Browser automation
|
||||
|
||||
- `agent-browser` — CLI for driving a real headless browser: open pages,
|
||||
click/fill/`eval`, snapshot the DOM, take screenshots. Useful whenever a task
|
||||
involves a web UI or verifying how a page actually renders (live DOM, WebGL,
|
||||
layout, popup positioning) instead of guessing from source.
|
||||
- `playwright` + a pre-installed headless **Chromium** back it.
|
||||
`AGENT_BROWSER_EXECUTABLE_PATH` is preset to the baked browser via a stable
|
||||
`/usr/local/bin/agent-chrome` symlink (insulated from Playwright's
|
||||
per-version/arch install directory), so `agent-browser open <url>` works
|
||||
out of the box with no setup. Run `agent-browser skills get core --full`
|
||||
for the command set and workflow patterns.
|
||||
- `socat` — TCP bridge used by `studio-expose` to reach pi-studio's
|
||||
loopback-bound server from outside the container (see
|
||||
[Using pi-studio](#using-pi-studio--studio-variant))
|
||||
|
||||
### Language toolchains
|
||||
|
||||
- `python3` + `python3-venv` + `python3-pip` (system Python)
|
||||
@@ -153,12 +175,10 @@ Currently published:
|
||||
| `joakimp/pi-devbox:latest-studio` | `latest` + [pi-studio](https://github.com/omaclaren/pi-studio) (browser prompt editor, KaTeX/Mermaid preview, tmux-backed literate REPLs) | ~3.25 GB |
|
||||
| `joakimp/pi-devbox:vX.Y.Z-studio` | pinned-version studio equivalent | ~3.25 GB |
|
||||
|
||||
Planned for an upcoming minor release:
|
||||
|
||||
- *(shipped in Unreleased/base)* **PDF export from Studio/pandoc** now works:
|
||||
the base image ships **`typst`** as the PDF engine (`pandoc --pdf-engine=typst`),
|
||||
a single ~30 MB static binary — no separate `-tex` variant needed.
|
||||
`texlive-xetex` stays the higher-fidelity fallback (install on demand).
|
||||
Both variants ship **`typst`** as the pandoc PDF engine
|
||||
(`pandoc --pdf-engine=typst`), a single ~30 MB static binary, so PDF export from
|
||||
Studio/pandoc works out of the box — no separate `-tex` variant needed.
|
||||
`texlive-xetex` stays the higher-fidelity fallback (install on demand).
|
||||
|
||||
## Using pi-studio (`-studio` variant)
|
||||
|
||||
@@ -338,6 +358,59 @@ DOT syntax errors instead of crashing. Then in Studio: open the PNG (or a
|
||||
`.md` that embeds it) and hit **refresh-from-disk** after each edit.
|
||||
Note: SVG is **not** in Studio's local-image-link allowlist — use PNG.
|
||||
|
||||
## Using pi-atelier (TUI sidebar)
|
||||
|
||||
`pi-atelier` is bundled in **both** variants (vendored at `/opt/pi-atelier`,
|
||||
pinned — see [Version pins](#version-pins-pi-pi-atelier-mempalace)). It adds two
|
||||
things to pi's terminal UI:
|
||||
|
||||
- a **status rail** — activity, token/cost metrics, context usage, model, git
|
||||
state, extension statuses, and a menu;
|
||||
- a **sidebar** — ordered panels (agent, activity, alerts, TODOs, context,
|
||||
workspace, usage, tools) in a split pane beside the transcript.
|
||||
|
||||
Nothing needs installing; the entrypoint registers it on container start, and it
|
||||
binds on the next pi start (or `/reload`).
|
||||
|
||||
| Action | How |
|
||||
|---|---|
|
||||
| Open the atelier menu | `alt+a`, or `/atelier` |
|
||||
| Toggle the sidebar for this session | `/atelier sidebar on` / `off` |
|
||||
| Change settings persistently | atelier menu → **Settings**, then **Save** |
|
||||
| Turn the whole thing off | `DEVBOX_ATELIER=0` in `.env` |
|
||||
|
||||
If your terminal or keymap swallows `alt+a`, use `/atelier` and pick a different
|
||||
`shortcut` in the config file below.
|
||||
|
||||
### Config
|
||||
|
||||
Config lives at `~/.pi/agent/pi-atelier.json` on the `devbox-pi-config` volume,
|
||||
seeded from pi-toolkit with container-appropriate defaults: compact density,
|
||||
context warnings at 60/85 % (earlier than upstream's 70/90), sidebar tool names
|
||||
on, and desktop completion notifications **off** (a container has nowhere useful
|
||||
to pop a toast).
|
||||
|
||||
It is **copied, not symlinked** — atelier rewrites this exact path when you hit
|
||||
**Save**, using write-temp-then-`rename(2)`, and `rename` replaces a symlink with
|
||||
a regular file instead of following it. A symlink would silently detach on your
|
||||
first save. Consequently pi-toolkit's `install.sh` only seeds the file when it is
|
||||
absent: once you have saved your own preferences, image upgrades leave them
|
||||
alone, and `install.sh` prints a diff hint instead of clobbering.
|
||||
|
||||
The seeded file uses atelier's **current** schema — `segmentLayout` with explicit
|
||||
per-segment visibility, plus `showSidebarAgent` / `showSidebarTodos` /
|
||||
`showSidebarOnStartup`. Older configs written against the pre-0.7 vocabulary
|
||||
(`segments`, `ornament`, `showExtensionStatuses`) still load, but only through
|
||||
upstream's legacy-compatibility shims — so if you are carrying one on an old
|
||||
volume, expect it to keep working while missing every sidebar control added
|
||||
since. `sidebarPanelLayout` is deliberately left unset so the panel set follows
|
||||
upstream's product default as atelier adds panels; set it only if you want to
|
||||
pin the order yourself.
|
||||
|
||||
The sidebar auto-hides below 92 terminal columns and keeps the main pane at
|
||||
least 64 columns wide, so a narrow terminal degrades to the plain TUI rather
|
||||
than a squeezed one.
|
||||
|
||||
## docker-compose.yml — basic shape
|
||||
|
||||
```yaml
|
||||
@@ -463,6 +536,35 @@ to refresh.
|
||||
Anything not on a volume is on the writable layer and is lost on
|
||||
container recreate.
|
||||
|
||||
### Rebuilding ephemeral shell state at start
|
||||
|
||||
Two entrypoint steps put back the kind of state that the writable layer eats, so a
|
||||
recreate does not cost you a manual re-install:
|
||||
|
||||
- **`cli_utils` commands.** If a `cli_utils` checkout is mounted, every
|
||||
executable in its `bin/` is symlinked into `~/.local/bin` on start, so
|
||||
`git-status-all` and friends are on `PATH` without a path prefix. Detection:
|
||||
`CLI_UTILS_CONTAINER_PATH` → `/workspace/cli_utils` → `$HOME/cli_utils` →
|
||||
`/workspace/*/cli_utils`. Set `CLI_UTILS_LINK=0` to disable. Existing real files
|
||||
in `~/.local/bin` and symlinks pointing elsewhere are left alone, so a
|
||||
deliberate override still wins; links whose target disappeared are pruned.
|
||||
Do **not** run a host installer's `install.sh` inside the container to achieve
|
||||
this — it writes to the ephemeral home and dies on the next recreate.
|
||||
- **A per-device boot hook.** If `~/.config/devbox-shell/init.sh` exists it is run
|
||||
once at start (`bash`, never sourced, exit status ignored), with output in
|
||||
`~/.pi/agent/devbox-init.log`. `~/.config/devbox-shell/` is the host-owned
|
||||
bind-mount whose `bash_aliases` is already sourced into every interactive shell,
|
||||
so a hook there persists across recreates with no image change. Use it for
|
||||
fixups that must exist *before any shell* — symlinks, directories, one-off
|
||||
migrations.
|
||||
|
||||
The distinction that decides which mechanism you want: `~/.local/bin` is on `ENV
|
||||
PATH`, so symlinks there work in **non-interactive** shells too (`docker exec <c>
|
||||
<cmd>`, agent tool shells, scripts). A `PATH` edit in `bash_aliases` reaches only
|
||||
*interactive* shells, because `~/.bashrc` returns early when non-interactive —
|
||||
which is also why shell **functions** (fzf helpers and the like) can only come
|
||||
from the sourced file, never from a symlink.
|
||||
|
||||
## MemPalace integration
|
||||
|
||||
MemPalace is installed in the base image and pre-warmed with the
|
||||
@@ -482,6 +584,75 @@ session/docs mining; the 29 MCP tools (search, kg-query, drawer-add,
|
||||
diary-write, etc.) are wired into pi automatically by the pi-extensions
|
||||
mempalace bridge.
|
||||
|
||||
### Cross-machine agent coordination
|
||||
|
||||
When `MEMPALACE_REMOTE_URL` points at a *shared* palace, the container gets more
|
||||
than shared search: it joins an append-only coordination log (RFC 003) that other
|
||||
machines' agents can address it on — used here for design review, patch handoff
|
||||
and retraction between hosts.
|
||||
|
||||
Two container-side settings make it work:
|
||||
|
||||
| Variable | Why it matters |
|
||||
|---|---|
|
||||
| `MEMPALACE_REMOTE_URL` | selects the shared palace; unset means a purely local palace, and the log then contains only this machine's own events |
|
||||
| `MEMPALACE_PI_DEVICE` | the bridge stamps `pi@<device>` as the writer, which is the **only** way the log can tell two machines apart when both are thin clients of one palace |
|
||||
|
||||
So a container with no `MEMPALACE_PI_DEVICE` can read the log but is not
|
||||
reachable *on* it: messages addressed to a bare `pi` match nobody. Set both, or
|
||||
neither.
|
||||
|
||||
What the agent is expected to *do* with this lives in the mempalace skill
|
||||
(`~/.agents/skills/mempalace/SKILL.md`) — the mailbox query at wake-up, and the
|
||||
convention that a directed event with `status="open"` is a request owed a reply
|
||||
while a `*` broadcast owes nothing. The mechanism side (what the bridge stamps,
|
||||
and why live SSE push depends on the palace deployment's reverse proxy rather
|
||||
than on this image) is documented in the toolkit's `extensions/pi/README.md`.
|
||||
|
||||
**Since v1.8.9 the bridge reads the log for you.** Earlier images were write-only
|
||||
— they stamped provenance on the way out and never read back, so a directed ask
|
||||
reached an agent only if that agent happened to run `mempalace_event_list`
|
||||
itself. The mailbox is gated on the same two variables as the stamper, is on by
|
||||
default, and derives what is *owed* rather than trusting `status` (an acked event
|
||||
keeps matching a `status="open"` query forever, because the log is append-only):
|
||||
|
||||
| Variable | Default | Effect |
|
||||
|---|---|---|
|
||||
| `MEMPALACE_MAILBOX` | unset (on) | `0` disables mailbox reads entirely |
|
||||
| `MEMPALACE_MAILBOX_POLL_MS` | `300000` | minimum gap between mid-session polls |
|
||||
| `MEMPALACE_MAILBOX_RESURFACE_MS` | `3600000` | re-announce a still-owed ask after this long |
|
||||
|
||||
Delivery **queues, it never interrupts**: the poll runs when pi goes idle and the
|
||||
message is steered into the *next* turn, so nothing wakes the model on inbound
|
||||
fleet traffic. The practical consequence, measured on two devices: the message
|
||||
appears in your session window and the agent acts on it when the next turn
|
||||
starts — you are the trigger. (That describes the bridge **as baked in v1.8.9**,
|
||||
`mempalace-toolkit` `5b8d78f`; the mailbox's own mechanism and landmines live in
|
||||
the toolkit's `docs/rfc-003-coordination-log.md` §7.11–§7.12, which moves ahead of
|
||||
whatever this image has baked.)
|
||||
|
||||
## Observational memory (in-session memory)
|
||||
|
||||
The image also bakes [pi-observational-memory](https://github.com/elpapi42/pi-observational-memory),
|
||||
which is memory of a *different kind* from the palace and is easy to confuse with
|
||||
it. It keeps a small branch-local ledger of observations and reflections while a
|
||||
session runs, so when pi compacts the conversation the summary is a
|
||||
**deterministic fold of that ledger rather than a model call**, and every item
|
||||
keeps a 12-character id that `recall(<id>)` resolves back to the exact source.
|
||||
|
||||
In one line: **observational memory keeps a session coherent; the palace keeps
|
||||
the fleet coherent.**
|
||||
|
||||
It is on by default, needs no habit from you, and sends its background work to a
|
||||
cheaper model than your session (Haiku while the session runs Opus, in the seeded
|
||||
`~/.pi/agent/settings.json`). Inspect it from inside pi with `/om:status` and
|
||||
`/om:view`; turn all proactive work off for one run with
|
||||
`PI_OBSERVATIONAL_MEMORY_PASSIVE=1 pi`.
|
||||
|
||||
What it is for, how the lifecycle works, what it costs, every setting and its
|
||||
default, and how it differs from MemPalace:
|
||||
[`docs/observational-memory.md`](docs/observational-memory.md).
|
||||
|
||||
## Agent skills
|
||||
|
||||
pi discovers skills under `~/.agents/skills/`. Two delivery paths feed that
|
||||
@@ -492,7 +663,8 @@ directory, and they compose:
|
||||
`~/.agents/skills/` by `entrypoint-user.sh` on every start. They need no
|
||||
external mount, survive volume recreate (the source is an image path, not a
|
||||
home dir a named volume would shadow), and are created only when absent so a
|
||||
same-named skillset skill or user override is never clobbered. The bundled
|
||||
user override is never clobbered. Precedence against a mounted `skillset` repo
|
||||
is per-skill, not blanket — see *Skillset repo* below. The bundled
|
||||
**`pi-devbox-environment`** skill is delivered this way — it teaches agents
|
||||
the container's persistence model, host/LAN SSH reachability, split-DNS
|
||||
mechanisms, the interactive-vs-tool-shell alias gotcha (`dssh`/`dscp`),
|
||||
@@ -503,8 +675,11 @@ directory, and they compose:
|
||||
pi session to read `~/.agents/skills/pi-extensions/SKILL.md` at start (to fix
|
||||
fork/recall under-utilisation). That pointer would dangle in a container
|
||||
started *without* the private `skillset` repo, so the image also bakes
|
||||
fallback copies of **`pi-extensions`** and **`mempalace`**. They are
|
||||
symlinked only when absent, so a mounted skillset always overrides them. The
|
||||
fallback copies of **`pi-extensions`** and **`mempalace`**. Whether a mounted
|
||||
skillset overrides them depends on who *owns* the skill (see *Skillset repo*):
|
||||
`mempalace` is skillset-owned, so the live clone wins; `pi-extensions` is
|
||||
owned by its package repo, so the baked copy keeps winning — the skillset's
|
||||
copy of it is a downstream duplicate that can lag. The
|
||||
`pi-extensions` skill is *layered*: a committed snapshot in `rootfs/` is the
|
||||
floor, and `Dockerfile.variant` copies the canonical, package-owned copy from
|
||||
the pinned `pi-extensions` clone (`/opt/pi-extensions/skill/`) over it at
|
||||
@@ -518,7 +693,16 @@ directory, and they compose:
|
||||
- **Skillset repo (optional).** If a `skillset` repo is mounted (at
|
||||
`$HOME/skillset` or `/workspace/skillset`, or via `SKILLSET_CONTAINER_PATH`),
|
||||
`deploy-skills.sh` symlinks its skills in too. Image-baked skills are
|
||||
classified as foreign-links by its `--prune-stale` pass and left untouched.
|
||||
classified as foreign-links by its `--prune-stale` pass and left untouched —
|
||||
which through v1.8.4 meant the baked copy *always* won, so an edit pushed to a
|
||||
skillset-owned skill was invisible until the next image build. Since v1.8.5
|
||||
`devbox-skill-reconcile` runs right after the deploy and repoints the links for
|
||||
skills the skillset owns, listed in
|
||||
`/usr/local/share/pi-devbox/skills/skillset-owned.txt` (today: `mempalace`).
|
||||
Effective precedence, highest first: **user override** (a real directory, or a
|
||||
symlink pointing outside the baked tree) → **live skillset clone** (owned names
|
||||
only) → **baked snapshot** (everything else, and every skill when no skillset
|
||||
is mounted). Check with `readlink -f ~/.agents/skills/<skill>`.
|
||||
|
||||
To make agents *proactively* load a baked skill at session start (rather than
|
||||
only on description match), the image appends a short, gated pointer to the
|
||||
@@ -554,6 +738,35 @@ User-level overrides in `~/.ssh/config` win because Debian's
|
||||
`/etc/ssh/ssh_config` includes `/etc/ssh/ssh_config.d/*.conf` before
|
||||
the `Host *` block.
|
||||
|
||||
### macOS-only keywords in a shared `~/.ssh/config`
|
||||
|
||||
The same `~/.ssh/config` is read by macOS ssh *and* by the Linux OpenSSH inside
|
||||
the container (the sidecar `Include`s it). macOS-only keywords are **fatal**
|
||||
there, not ignored — a single `UseKeychain yes` in a `Host *` block takes down
|
||||
every ssh call in the container:
|
||||
|
||||
```
|
||||
/home/developer/.ssh/config: line 2: Bad configuration option: usekeychain
|
||||
/home/developer/.ssh/config: terminating, 1 bad configuration options
|
||||
```
|
||||
|
||||
That breaks `dssh`/`dscp`, `pi --ssh`, `scp`, and anything that shells out to
|
||||
ssh (including CI/deploy helpers), while the host keeps working perfectly — so
|
||||
it presents as a container regression rather than a host config error. Guard the
|
||||
keyword on the host, *before* it is used:
|
||||
|
||||
```diff
|
||||
Host *
|
||||
+ IgnoreUnknown UseKeychain
|
||||
UseKeychain yes
|
||||
AddKeysToAgent yes
|
||||
```
|
||||
|
||||
`IgnoreUnknown` is understood by both implementations: macOS still honours
|
||||
`UseKeychain`, Linux skips it. Also keep such a `Host *` block **below** any
|
||||
`Include` that must come first — OrbStack's own `Include ~/.orbstack/ssh/config`
|
||||
says so in a comment, and a `Host *` block above it silently violates that.
|
||||
|
||||
### Per-host `ControlPath` on a read-only `~/.ssh`
|
||||
|
||||
`~/.ssh` is usually bind-mounted read-only, so a user `~/.ssh/config` that
|
||||
@@ -573,7 +786,10 @@ this without editing the read-only config:
|
||||
jump via the host, add `ProxyJump host` overrides in the host-owned
|
||||
`~/.config/devbox-shell/ssh-lan.conf` (see
|
||||
[Naming LAN peers](#naming-lan-peers)) rather than the read-only
|
||||
`~/.ssh/config`.
|
||||
`~/.ssh/config`. If the peer also rejects the host's key — the usual case,
|
||||
since host keys are normally passphrase-protected and the container has no
|
||||
Keychain or agent — see
|
||||
[Giving the container its own key for a peer](#giving-the-container-its-own-key-for-a-peer).
|
||||
|
||||
## tmux and 0-indexed sessions
|
||||
|
||||
@@ -646,6 +862,7 @@ repoint each one at a mirror, another host, or a local `file://` path
|
||||
| `MEMPALACE_TOOLKIT_REPO` | `https://gitea.jordbo.se/joakimp/mempalace-toolkit.git` | base |
|
||||
| `PI_FORK_REPO` | `https://github.com/elpapi42/pi-fork.git` | variant |
|
||||
| `PI_OBSMEM_REPO` | `https://github.com/elpapi42/pi-observational-memory.git` | variant |
|
||||
| `PI_ATELIER_REPO` | `https://github.com/michaelmjhhhh/pi-atelier.git` | variant |
|
||||
| `PI_STUDIO_REPO` | `https://github.com/omaclaren/pi-studio.git` | variant |
|
||||
|
||||
Each has a matching `*_REF` arg (branch name or commit SHA). Example — build
|
||||
@@ -686,8 +903,9 @@ docker inspect --format '{{json .Config.Labels}}' joakimp/pi-devbox:latest | jq
|
||||
`org.opencontainers.image.{version,revision,created}` plus
|
||||
`se.jordbo.pi-devbox.*-ref` record the intended pi version and companion
|
||||
refs. The on-disk `/etc/pi-devbox/build-manifest.json` records **ground
|
||||
truth** — the actual checked-out commit of each `/opt` clone and the live
|
||||
`pi --version` — so a tag is reconstructable after CI logs rotate:
|
||||
truth** — the actual checked-out commit of each `/opt` clone, the live
|
||||
`pi --version`, and (from v1.8.6) the live `mempalace --version` of the
|
||||
installed palace core — so a tag is reconstructable after CI logs rotate:
|
||||
|
||||
```bash
|
||||
docker run --rm --entrypoint= joakimp/pi-devbox:latest cat /etc/pi-devbox/build-manifest.json
|
||||
@@ -699,16 +917,23 @@ through `jq` yourself:
|
||||
|
||||
```console
|
||||
$ pi-devbox-version
|
||||
pi-devbox v1.5.0
|
||||
built: 2026-07-13T17:53:16Z (source d68674d11e06)
|
||||
pi: 0.80.6
|
||||
pi-devbox v1.8.14
|
||||
built: 2026-09-08T21:54:07Z (source 361babd4fd61)
|
||||
pi: 0.85.1
|
||||
palace: 3.9.0
|
||||
components:
|
||||
pi-toolkit: 9a8f6faeaa08
|
||||
pi-extensions: 61c98e004e3d
|
||||
pi-fork: 4a09af4ef527
|
||||
pi-observational-memory: 27a5195eaf90
|
||||
mempalace-toolkit: 96699f2a1781
|
||||
pi-studio: 2ef38ef31cea
|
||||
pi-toolkit: adfb553f5c8a
|
||||
pi-extensions: 2610545c83bb
|
||||
pi-fork: e69725c39603
|
||||
pi-observational-memory: ce9fc982b3a2
|
||||
pi-atelier: 734258bbcb62
|
||||
mempalace-toolkit: e45f6b430181
|
||||
pi-studio: e04fc7aa3275
|
||||
skills:
|
||||
credential-incident-response baked
|
||||
mempalace live /workspace/skillset @ 4d7c0ea (identical to baked snapshot)
|
||||
pi-devbox-environment baked
|
||||
pi-extensions baked
|
||||
```
|
||||
|
||||
It also flags **live drift** — if `pi --version` no longer matches what was
|
||||
@@ -750,14 +975,109 @@ Host pve pve-2 alpserv-2 lagret
|
||||
ProxyJump host
|
||||
```
|
||||
|
||||
`HostName` / `User` / `IdentityFile` are inherited from the matching block in
|
||||
your real `~/.ssh/config` (first-value-wins, so only `ProxyJump` is taken from
|
||||
here). This file is `Include`d *before* `~/.ssh/config` and read fresh on every
|
||||
connection — newly added peers work immediately, no container or session
|
||||
restart needed — and the peer names stay out of the published image (they're a
|
||||
fact about your specific LAN, not the image). Alternatively, set
|
||||
`DEVBOX_LAN_AUTOJUMP_PRIVATE=1` to ProxyJump *any* RFC1918 address through the
|
||||
host without naming peers (see `.env.example`).
|
||||
Any option can be set here, not just `ProxyJump`: the file is `Include`d
|
||||
*before* `~/.ssh/config` and ssh takes the **first** value it sees for each
|
||||
option, so whatever you put here wins while everything you omit is inherited
|
||||
from the matching block in your real `~/.ssh/config`. Peer names stay out of the
|
||||
published image (they are a fact about your LAN, not the image). Alternatively,
|
||||
set `DEVBOX_LAN_AUTOJUMP_PRIVATE=1` to ProxyJump *any* RFC1918 address through
|
||||
the host without naming peers (see `.env.example`).
|
||||
|
||||
Once the file exists it is re-read on every connection, so *edits* take effect
|
||||
immediately — no container or session restart. **Creating it for the first time
|
||||
does need one restart**, because `setup-lan-access.sh` only emits the
|
||||
`Include ~/.config/devbox-shell/ssh-lan.conf` line when the file is already
|
||||
readable at container start (`if [ -r "$SSH_LAN_CONF" ]`). Until then ssh never
|
||||
looks at it — which reads exactly like "my override is being ignored".
|
||||
|
||||
#### Giving the container its own key for a peer
|
||||
|
||||
`ProxyJump` fixes *routing*; it does not fix *authentication*, and inheriting
|
||||
the host's `IdentityFile` usually fails inside the container:
|
||||
|
||||
- Host keys are commonly passphrase-protected, and that passphrase is unlocked
|
||||
by the macOS Keychain or a running `ssh-agent`. The container has neither, so
|
||||
the key can never be decrypted — `Permission denied (publickey)` even though
|
||||
the identical `ssh peer` works in a host terminal.
|
||||
- `~/.ssh` is mounted read-only, so you can neither drop a container-usable key
|
||||
in there nor edit `~/.ssh/config` from inside.
|
||||
|
||||
The answer is a **container-only keypair** in `~/.ssh-local/` — the named volume
|
||||
`devbox-ssh-local`, so it survives `docker compose up -d --force-recreate` —
|
||||
plus an `IdentityFile` override in the host-owned `ssh-lan.conf`. Note that
|
||||
nothing is baked into the *published image*: that volume is created on your
|
||||
machine at runtime, so no private key ever ships to Docker Hub, and a fresh pull
|
||||
elsewhere generates its own. (Every key below is a throwaway example.)
|
||||
|
||||
**1. In the container** — generate a passphraseless key (there is no agent to
|
||||
unlock a protected one):
|
||||
|
||||
```bash
|
||||
ssh-keygen -t ed25519 -N '' -C "devbox-$(hostname)" \
|
||||
-f ~/.ssh-local/mypeer_devbox_ed25519
|
||||
cat ~/.ssh-local/mypeer_devbox_ed25519.pub
|
||||
# ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEXAMPLE0000EXAMPLE0000EXAMPLE0000ex devbox-0d11ec7731c7
|
||||
```
|
||||
|
||||
**2. On the peer** — append that public key to `~/.ssh/authorized_keys` **of
|
||||
the account you will log in as** (the `User` from step 3), narrowly authorized
|
||||
rather than bare:
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.ssh && chmod 700 ~/.ssh
|
||||
cat >> ~/.ssh/authorized_keys <<'KEY'
|
||||
from="192.168.1.0/24,192.168.4.0/24,10.8.0.7",restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEXAMPLE0000EXAMPLE0000EXAMPLE0000ex devbox-mymachine
|
||||
KEY
|
||||
chmod 600 ~/.ssh/authorized_keys
|
||||
```
|
||||
|
||||
Both lines are safe on a peer that is already set up: `mkdir -p` is a no-op
|
||||
when the directory exists, the `chmod`s only tighten, and appending never
|
||||
touches keys already listed. Use `>>`, never `>` — one stray truncation
|
||||
revokes every other key on that account. The options prefix must sit on the
|
||||
**same physical line** as the key, comma-separated with no spaces: a paste
|
||||
that wrapped is the likeliest reason a key that looks right is refused.
|
||||
`ssh-copy-id` cannot add that prefix, so append by hand (or let it copy the
|
||||
bare key and edit the line afterwards). If authentication still fails with no
|
||||
clear reason, suspect permissions — sshd's `StrictModes` silently ignores
|
||||
`authorized_keys` when the home directory, `~/.ssh` or the file itself is
|
||||
group- or world-writable, and says why only in the peer's own log
|
||||
(`journalctl -u ssh`, `/var/log/auth.log`).
|
||||
|
||||
`restrict` disables pty, agent/X11 and port forwarding; append
|
||||
`port-forwarding` and `permitopen="127.0.0.1:<port>"` after it if you need one
|
||||
specific tunnel. `from=` must list the **host's** addresses, not the
|
||||
container's: container egress is NAT'd through the host, so the peer sees the
|
||||
host's LAN address (confirm with `echo $SSH_CLIENT` on first login). List every
|
||||
network the host roams — e.g. both home WLAN subnets plus its VPN address —
|
||||
because a `from=` mismatch is indistinguishable from a wrong key in the error
|
||||
message.
|
||||
|
||||
**3. On the host** — point the peer at that key in
|
||||
`~/.config/devbox-shell/ssh-lan.conf`:
|
||||
|
||||
```
|
||||
Host mypeer mypeer.home.arpa
|
||||
HostName 192.168.1.142
|
||||
User myuser
|
||||
IdentityFile ~/.ssh-local/mypeer_devbox_ed25519
|
||||
IdentitiesOnly yes
|
||||
# ProxyJump host # only if the container cannot reach the peer directly
|
||||
```
|
||||
|
||||
That path exists only inside containers, which is why it belongs here rather
|
||||
than in the shared `~/.ssh/config`.
|
||||
|
||||
**4. First time only** — restart the container so the `Include` is emitted (see
|
||||
above), then verify with the master socket bypassed, so a warm connection cannot
|
||||
fake a pass:
|
||||
|
||||
```bash
|
||||
ssh -F ~/.ssh-local/config -o ControlPath=none mypeer 'echo $SSH_CLIENT'
|
||||
```
|
||||
|
||||
Use one key per machine (`devbox-mbp`, `devbox-studio`, …) so a single
|
||||
`authorized_keys` line can be revoked without locking out the others.
|
||||
|
||||
### Smoke-testing a local build
|
||||
|
||||
@@ -776,10 +1096,21 @@ After `docker compose up -d --force-recreate`, run the **runtime** peer of
|
||||
persisted volumes survived, and pi runtime wiring is intact:
|
||||
|
||||
```bash
|
||||
./scripts/recreate-sanity-check.sh # auto-detects variant
|
||||
./scripts/recreate-sanity-check.sh --expected-version 0.79.4 # assert pi version
|
||||
./scripts/recreate-sanity-check.sh # auto-detects variant
|
||||
./scripts/recreate-sanity-check.sh --expected-image-version 1.8.9 # assert the pi-devbox release tag
|
||||
./scripts/recreate-sanity-check.sh --expected-version 0.85.1 # assert the pi coding agent version
|
||||
```
|
||||
|
||||
Those are **two different versions**, and the flags are not interchangeable:
|
||||
`--expected-image-version` takes the pi-devbox release tag (`v` optional),
|
||||
`--expected-version` takes `pi --version`. Hand one the other's value and it
|
||||
says so by name instead of reporting a mismatch against the wrong component.
|
||||
With neither flag, both values are read from the image's own build manifest
|
||||
(`/etc/pi-devbox/build-manifest.json`): the live pi version is asserted against
|
||||
the one recorded at build time — which catches a stale `pi` in the
|
||||
`~/.pi/npm-global` volume shadowing the baked one — and the release tag is
|
||||
reported informationally.
|
||||
|
||||
If `cli_utils` is on your PATH, the `pi-devbox-sanity` wrapper runs the same
|
||||
check by short name and locates the repo automatically (override with
|
||||
`PI_DEVBOX_REPO=/path/to/pi-devbox`). Like `smoke-test.sh`, this script is
|
||||
@@ -794,9 +1125,72 @@ pi-devbox follows semver-ish:
|
||||
- **Minor** — new variants, significant base additions.
|
||||
- **Patch** — pi version bumps, smaller fixes.
|
||||
|
||||
The `pi --version` inside the image is asserted by smoke tests to
|
||||
match the release tag's pi component, so version drift between the
|
||||
image and the tag is caught at CI time.
|
||||
The `pi --version` inside the image is asserted by smoke tests to match the
|
||||
version CI resolved (since v1.7.0, the pin below), so drift between what was
|
||||
intended and what actually got baked is caught at CI time rather than on a
|
||||
user's pull.
|
||||
|
||||
### Version pins: pi, pi-atelier, mempalace
|
||||
|
||||
Three components are pinned to an exact version **in the repo** instead of being
|
||||
resolved to `latest` at build time:
|
||||
|
||||
| Component | Pin | Where |
|
||||
|---|---|---|
|
||||
| pi | `0.85.1` | `ARG PI_VERSION` — `Dockerfile.variant` |
|
||||
| pi-atelier | `v0.10.1` | `ARG PI_ATELIER_REF` — `Dockerfile.variant` |
|
||||
| mempalace | `3.9.0` | `ARG MEMPALACE_VERSION` — `Dockerfile.base` |
|
||||
|
||||
The objective is **not** to freeze versions. Bumping is routine — usually one
|
||||
line plus a changelog note. The objective is that adopting a new upstream
|
||||
version is a deliberate, reviewable act, not a side effect of whatever happened
|
||||
to be published the morning CI ran. Each of these has already drawn blood:
|
||||
|
||||
- **pi** — a minor release can move the private TUI/renderer internals that
|
||||
pi-atelier wraps, or the session `.jsonl` format `pi-session-repair` parses.
|
||||
- **pi-atelier** — 0.6.0/0.7.0 hang pi 0.84 **at startup**, burning CPU with no
|
||||
error (fixed in 0.7.1/0.7.2). Its `peerDependencies` still say `>=0.80.7`, so
|
||||
nothing in the npm metadata expresses the real floor.
|
||||
- **mempalace** — an unpinned install once swept in the broken `diary_write` MCP
|
||||
tool schema of 3.3.x/3.4.0, which is why that pin's comment requires a
|
||||
tool-schema review before every bump.
|
||||
|
||||
CI enforces this rather than trusting it:
|
||||
|
||||
- `resolve-versions` reads the pins **out of the Dockerfiles** — single source of
|
||||
truth, so a local `docker build` and a CI release ship the same versions — and
|
||||
fails the build if a pin is not concrete, not a semver tag, or not actually
|
||||
published on npm.
|
||||
- When npm has a newer pi than the pin, CI emits a `::warning::` naming it. That
|
||||
warning is the prompt to audit and bump; it never adopts the version.
|
||||
- `smoke-test.sh` asserts the image's `pi --version` equals the pin, and
|
||||
separately asserts the pairing rule **pi ≥ 0.84 ⇒ pi-atelier ≥ 0.7.1**, so a
|
||||
bad combination fails the build instead of publishing a TUI that never starts.
|
||||
|
||||
To bump pi: read the upstream CHANGELOG for every intervening version (TUI/theme
|
||||
API, session format, extension loader, Node engine floor), re-check pi-atelier's
|
||||
CHANGELOG for the pi version it claims to track, then edit the one `ARG` line and
|
||||
record what you checked in `CHANGELOG.md`.
|
||||
|
||||
#### If you previously hand-installed pi-atelier
|
||||
|
||||
A hand-installed `pi install npm:pi-atelier` lands in `~/.pi/npm-global`, which
|
||||
is on the `devbox-pi-config` **volume** — so it outlives image upgrades and stays
|
||||
at whatever version you installed, unpinned and unaudited. Since the image now
|
||||
vendors an audited pi-atelier at `/opt/pi-atelier`, the entrypoint removes a
|
||||
lingering `npm:pi-atelier` entry from `packages[]` (after backing
|
||||
`settings.json` up to `settings.json.bak.atelier.<timestamp>`) and registers the
|
||||
pinned `/opt` copy instead. Nothing else in your settings is touched, and the
|
||||
npm-global copy itself is left on disk — only the registration changes.
|
||||
|
||||
This matters more than it sounds: leaving a 0.6.x npm copy registered alongside
|
||||
pi 0.84 is precisely the combination that hangs at startup.
|
||||
|
||||
To opt out of pi-atelier entirely, set `DEVBOX_ATELIER=0` in `.env`. The
|
||||
entrypoint then removes any pi-atelier entry from `packages[]` on start. That
|
||||
switch lives in the entrypoint — not in a pi command — deliberately: this
|
||||
component's failure mode is "pi will not start", which you cannot repair with
|
||||
`pi uninstall`.
|
||||
|
||||
## Acknowledgements
|
||||
|
||||
|
||||
@@ -18,8 +18,23 @@ for OS packages, the per-package copyright files inside the image at
|
||||
| pi-fork | github.com/elpapi42/pi-fork | MIT |
|
||||
| pi-observational-memory | github.com/elpapi42/pi-observational-memory | MIT |
|
||||
| pi-studio *(`-studio` variant only)* | github.com/omaclaren/pi-studio | MIT |
|
||||
| pi-atelier | github.com/michaelmjhhhh/pi-atelier | MIT |
|
||||
| pi-toolkit, pi-extensions, mempalace-toolkit | authored by the maintainer (Joakim Persson) | MIT |
|
||||
|
||||
## MemPalace (AI memory)
|
||||
|
||||
| Component | Upstream | License |
|
||||
| --- | --- | --- |
|
||||
| mempalace (core, MCP server) | github.com/MemPalace/mempalace (PyPI: `mempalace`) | MIT — the GitHub repo declares MIT; the PyPI package's own metadata omits a license classifier, so if you need clearance from the package artifact alone, verify against the repo's `LICENSE` file rather than the sdist/wheel metadata |
|
||||
|
||||
## Browser automation
|
||||
|
||||
| Component | Upstream | License |
|
||||
| --- | --- | --- |
|
||||
| agent-browser | github.com/vercel-labs/agent-browser (npm: `agent-browser`) | Apache-2.0 |
|
||||
| Playwright | github.com/microsoft/playwright (npm: `playwright`) | Apache-2.0 |
|
||||
| Chromium | chromium.googlesource.com/chromium/src | BSD-3-Clause for Chromium's own code, plus a large set of bundled third-party components each under their own license (see Chromium's own `LICENSE`/`about:credits`). The binary in this image is **not compiled here** — it is the build Playwright downloads for its pinned version ("Chrome for Testing"), installed via `playwright install --with-deps chromium` at `/usr/local/share/ms-playwright/`. Treat Playwright's own distribution terms for that build as authoritative over any summary here. |
|
||||
|
||||
## Tooling baked into the base image
|
||||
|
||||
| Component | Upstream | License (best effort) |
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
# Point every client at it by setting, in that client's .env:
|
||||
#
|
||||
# MEMPALACE_REMOTE_URL=http://<reachable-host>:8765/mcp
|
||||
# MEMPALACE_REMOTE_TOKEN=<the shared bearer token>
|
||||
#
|
||||
# (see .env.example). When set, the client connects over HTTP and does NOT
|
||||
# spawn its own local mempalace-mcp.
|
||||
@@ -18,12 +19,21 @@
|
||||
# (both are pinned by the same image build). Override with a slimmer image via
|
||||
# MEMPALACE_SERVER_IMAGE if you prefer (it must provide `mempalace-mcp`).
|
||||
#
|
||||
# ⚠ SECURITY: mempalace-mcp's HTTP transport has NO authentication of its own.
|
||||
# Do NOT expose port 8765 to an untrusted network. The default below binds to
|
||||
# 127.0.0.1 (host loopback) only. To let sibling containers reach it, either
|
||||
# attach them to the shared `mempalace-net` network (container-to-container, no
|
||||
# host port needed — use http://mempalace-server:8765/mcp), or front it with a
|
||||
# reverse proxy that enforces MEMPALACE_REMOTE_TOKEN as `Authorization: Bearer`.
|
||||
# ⚠ SECURITY: the HTTP transport IS authenticated as of mempalace 3.6.0 — an
|
||||
# earlier version of this comment said otherwise and was wrong. The server
|
||||
# compares `Authorization: Bearer <token>` with hmac.compare_digest and
|
||||
# **refuses to start on a non-loopback bind without a token**, so
|
||||
# MEMPALACE_REMOTE_TOKEN below is required, not optional: without it this
|
||||
# service crash-loops. It also pins `Host` and allowlists `Origin`.
|
||||
#
|
||||
# Still do not publish port 8765 to an untrusted network. The default binds to
|
||||
# 127.0.0.1 (host loopback) only. To let sibling containers reach it, attach
|
||||
# them to the shared `mempalace-net` network (container-to-container, no host
|
||||
# port needed — use http://mempalace-server:8765/mcp). To reach it from
|
||||
# elsewhere, terminate TLS in a tunnel/reverse proxy and let the bearer token be
|
||||
# the authentication — do NOT add browser-shaped auth (SSO/PIN/password) in
|
||||
# front, because every MCP client here is a headless JSON-RPC POST and would
|
||||
# receive a login page where JSON should be.
|
||||
|
||||
name: mempalace-server
|
||||
|
||||
@@ -40,6 +50,11 @@ services:
|
||||
user: "0:0"
|
||||
environment:
|
||||
- HOME=/data
|
||||
# Required: mempalace refuses a non-loopback bind without a token (it
|
||||
# would exit at startup and, with restart:unless-stopped, crash-loop).
|
||||
# `:?` fails fast at `docker compose up` with a readable message instead.
|
||||
# Clients send the same value as MEMPALACE_REMOTE_TOKEN.
|
||||
- MEMPALACE_MCP_HTTP_TOKEN=${MEMPALACE_REMOTE_TOKEN:?set MEMPALACE_REMOTE_TOKEN in .env — the shared palace requires a bearer token}
|
||||
command:
|
||||
- mempalace-mcp
|
||||
- --transport
|
||||
@@ -60,16 +75,28 @@ services:
|
||||
- mempalace-shared:/data/.mempalace
|
||||
# Embedding-model cache (~79 MB, disposable) so search does not re-download.
|
||||
- mempalace-shared-chroma:/data/.cache/chroma
|
||||
# Transcript inbox. Clients cannot mine into a remote palace directly:
|
||||
# `mempalace_mine` expands its source path in THIS process, so it can only
|
||||
# see paths inside this container. Each client rsyncs its staged session
|
||||
# exports to a per-device subdirectory on the host (see
|
||||
# MEMPALACE_PI_SSH_TARGET in .env.example) and then calls mempalace_mine
|
||||
# with the container-side path below (MEMPALACE_PI_REMOTE_PATH=/data/feed).
|
||||
# Read-only: mining only reads sources, and all locks live palace-side.
|
||||
- ${MEMPALACE_FEED_DIR:-./feed}:/data/feed:ro
|
||||
networks:
|
||||
- mempalace-net
|
||||
healthcheck:
|
||||
# A tools/list round-trip proves the server is answering MCP (python3 is
|
||||
# always present — mempalace itself is a python tool in the image).
|
||||
# GET /healthz, which is Host/Origin-gated but deliberately token-free —
|
||||
# so this probe needs no credentials. Do NOT go back to POSTing
|
||||
# `tools/list` here: that carries no Authorization header and now 401s,
|
||||
# marking a perfectly healthy server unhealthy forever. The Host pin is
|
||||
# only enforced on loopback *binds* (this one is 0.0.0.0), so a request to
|
||||
# 127.0.0.1 inside the container passes.
|
||||
test:
|
||||
- CMD
|
||||
- python3
|
||||
- -c
|
||||
- "import urllib.request,json; d=json.dumps({'jsonrpc':'2.0','id':1,'method':'tools/list','params':{}}).encode(); r=urllib.request.Request('http://127.0.0.1:8765/mcp',data=d,headers={'Content-Type':'application/json','Accept':'application/json'}); urllib.request.urlopen(r,timeout=5).read()"
|
||||
- "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8765/healthz',timeout=5).status==200 else 1)"
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
@@ -0,0 +1,379 @@
|
||||
# Observational memory — why this image has it, and what it does for you
|
||||
|
||||
**Audience:** anyone using this container for long pi sessions who has wondered
|
||||
what `recall`, `/om:status` and "compacted memory" are, or whether they should
|
||||
leave any of it switched on.
|
||||
|
||||
**Companion documents:** the extension ships its own reference docs at
|
||||
`/opt/pi-observational-memory/docs/` —
|
||||
[`concepts.md`](https://github.com/elpapi42/pi-observational-memory/blob/main/docs/concepts.md)
|
||||
(the model),
|
||||
[`how-it-works.md`](https://github.com/elpapi42/pi-observational-memory/blob/main/docs/how-it-works.md)
|
||||
(hooks and internals) and
|
||||
[`configuration.md`](https://github.com/elpapi42/pi-observational-memory/blob/main/docs/configuration.md)
|
||||
(every setting). Pi's own compaction mechanics are in
|
||||
`/usr/lib/node_modules/@earendil-works/pi-coding-agent/docs/compaction.md`.
|
||||
Those are normative; this document is the **deployment** view — what is pinned
|
||||
here, how it is wired, what it costs, and how it differs from MemPalace. For the
|
||||
palace, see
|
||||
[`mempalace-toolkit/docs/fleet-memory.md`](https://gitea.jordbo.se/joakimp/mempalace-toolkit/src/branch/main/docs/fleet-memory.md).
|
||||
|
||||
> Verified on pi-devbox **v1.8.9** (`release_tag v1.8.9`, source `aac4a1c`),
|
||||
> which bakes pi-observational-memory **v3.0.4** at commit `ce9fc98` — the value
|
||||
> in `/etc/pi-devbox/build-manifest.json` → `components.pi-observational-memory`.
|
||||
> Every number below was read from that tree, from pi's own docs, or from the
|
||||
> live container. The pi-side mechanics were first read at pi **0.84.3** and
|
||||
> re-checked at **0.84.4** (v1.8.12), which moved one of them — see §3.
|
||||
|
||||
---
|
||||
|
||||
## 1. The problem it solves
|
||||
|
||||
A long pi session outgrows the model's context window. Pi's answer is
|
||||
**compaction**: fold the older part of the conversation into a summary and keep
|
||||
recent messages verbatim. That is unavoidable, and it is where sessions go
|
||||
wrong — the summary is produced *at the moment of pressure*, by a model, about a
|
||||
transcript that is about to leave the context.
|
||||
|
||||
Observational memory changes *when* the remembering happens. Instead of
|
||||
summarising in a panic at the end, it keeps a small **ledger** up to date while
|
||||
the session runs, and compaction then just folds that ledger.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A0["plain compaction"] --> A1["context fills"]
|
||||
A1 --> A2["a model summarises<br/>under pressure"]
|
||||
A2 --> A3["prose summary,<br/>no way back"]
|
||||
B0["with observational<br/>memory"] --> B1["context fills"]
|
||||
B1 --> B2["ledger written<br/>as you work"]
|
||||
B2 --> B3["compaction folds<br/>the ledger"]
|
||||
B3 --> B4["ids you can<br/>recall"]
|
||||
```
|
||||
|
||||
Top row is pi on its own: one model call at the worst possible moment, detail
|
||||
chosen in a hurry, and the original wording gone from view. Bottom row is this
|
||||
image's default: the thinking happened earlier on a cheap model, the fold is
|
||||
deterministic, and every line in the result carries an id that resolves back to
|
||||
the exact source.
|
||||
|
||||
## 2. The mental model: three layers and a ledger
|
||||
|
||||
| Layer | What it is | Example |
|
||||
|---|---|---|
|
||||
| **Observation** | a timestamped, source-backed event from the conversation | "user rejected option B because it needs a base rebuild" |
|
||||
| **Reflection** | a durable conclusion *backed by* observations | "the user optimises for avoiding 67-minute rebuilds" |
|
||||
| **Drop** | a tombstone retiring an observation from active memory | the superseded detail of a bug that is now fixed |
|
||||
|
||||
These are appended to the session as silent ledger entries
|
||||
(`om.observations.recorded`, `om.reflections.recorded`,
|
||||
`om.observations.dropped`) and **folded** — replayed in order — to produce the
|
||||
memory state. The ledger is the source of truth; what you see in a compacted
|
||||
session is a rendering of it.
|
||||
|
||||
Two properties follow, and both matter later:
|
||||
|
||||
- **The ledger itself costs no context.** Those entries are pi `custom` entries,
|
||||
which *"do not participate in LLM context"* (pi `docs/session-format.md`). They
|
||||
sit in the session file and reach the model only via the fold at compaction.
|
||||
- **Memory is branch-local.** A pi session is a tree (resume, fork), and the fold
|
||||
follows the current branch only, so a forked branch does not inherit another
|
||||
branch's view.
|
||||
|
||||
## 3. The lifecycle
|
||||
|
||||
Three background workers and one compaction hook, driven by *raw token
|
||||
progress* rather than wall-clock time. Defaults in brackets.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
T(["turn_end"]) --> O{"10k raw tokens<br/>since observing?"}
|
||||
O -- yes --> OBS["<b>observer</b> runs"]
|
||||
O -- "no" --> R{"20k tokens<br/>since reflecting?"}
|
||||
R -- yes --> REF["<b>reflector</b> runs"]
|
||||
REF -- "if pool over 10k" --> DR["<b>dropper</b> prunes"]
|
||||
S(["agent_settled"]) --> C{"81k tokens<br/>since compacting?"}
|
||||
C -- yes --> CP["ctx.compact()"]
|
||||
CP --> H(["session_before_compact"])
|
||||
A(["pi autoCompact<br/>idle, or mid-run<br/>after a tool batch"]) --> H
|
||||
H --> F["fold the ledger<br/>no model call"]
|
||||
F --> VIS["compacted memory"]
|
||||
```
|
||||
|
||||
- **observer** — `observeAfterTokens` [10000]: writes observations for the
|
||||
conversation it has not covered yet.
|
||||
- **reflector** — `reflectAfterTokens` [20000]: promotes patterns across
|
||||
observations into durable reflections.
|
||||
- **dropper** — no clock of its own. It is post-reflection maintenance, gated on
|
||||
a *successful same-turn* reflection **and** an active pool above
|
||||
`observationsPoolTargetTokens` [10000]. Not a third worker on a third
|
||||
threshold.
|
||||
- **compaction** — `compactAfterTokens` [81000], checked at `agent_settled`, so
|
||||
*this* trigger never interrupts a turn. Pi will also compact on its own when
|
||||
the context is nearly full (`contextTokens > contextWindow - reserveTokens`,
|
||||
`reserveTokens` [16384]), and **from pi 0.84.4 that check also runs mid-run** —
|
||||
after a tool batch's results are appended, before the next assistant response,
|
||||
skipped only when the batch ends the run and no queued message needs another
|
||||
response. So `session_before_compact` has **two** entry points and the second
|
||||
one can fire *inside* a turn. Harmless for the fold itself, which makes no
|
||||
model call, but worth stating plainly: "never interrupts a turn" was only ever
|
||||
true of the observational-memory trigger, and reads as a promise about pi's.
|
||||
|
||||
## 4. What compaction actually does to your context
|
||||
|
||||
This is the question the rest of the document used to leave hanging: if the old
|
||||
conversation is folded away, is the session back to knowing nothing?
|
||||
|
||||
**No.** Compaction replaces *part* of the context, not all of it, and it deletes
|
||||
nothing at all from disk.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
SYS["system prompt<br/>+ AGENTS.md"] --> CTX["what the model sees<br/>on the next turn"]
|
||||
SUM["folded memory:<br/>reflections + observations"] --> CTX
|
||||
TAIL["recent turns,<br/>verbatim"] --> CTX
|
||||
DISK[("session .jsonl: all of it")] -. "recall(id)" .-> CTX
|
||||
```
|
||||
|
||||
Where each piece comes from:
|
||||
|
||||
- **System prompt and `AGENTS.md` — never compacted, because they were never
|
||||
conversation.** Pi rebuilds them from disk on every request
|
||||
(`loadContextFileFromDir`), so they cannot be lost by compaction.
|
||||
- **The verbatim tail — sized by a token budget, not a message count.** Pi walks
|
||||
backwards from the newest entry accumulating token estimates until
|
||||
`keepRecentTokens` [20000] is reached; that entry becomes `firstKeptEntryId`,
|
||||
and *everything from there on is kept unchanged*. Cut points land on turn
|
||||
boundaries, never mid-tool-call. So the most recent ~20k tokens of real work —
|
||||
your last instructions, the diffs, the test output — survive word for word.
|
||||
- **The folded memory — replaces only what came before that cut.** Rendered from
|
||||
the ledger's records: reflections and observations, each with its 12-hex id.
|
||||
- **The session file — untouched.** Compaction *appends* a `compaction` entry
|
||||
(`{"type":"compaction", summary, firstKeptEntryId, tokensBefore, …}`) and
|
||||
rebuilds context from it on later turns. Nothing is rewritten in place; the
|
||||
only documented way to remove session content is deleting the whole `.jsonl`.
|
||||
|
||||
That last point is what makes the answer to "is the detail gone?" *no* rather
|
||||
than *mostly*: `recall` does not read the context window at all. It calls
|
||||
`sessionManager.getBranch()` — the full branch from the root — and resolves an
|
||||
observation id back to the original entries. Detail that left the model's view
|
||||
an hour ago is still one `recall` away.
|
||||
|
||||
**Repeated compaction does not summarise the summary.** The rendered text is
|
||||
always built from live observation/reflection *records*, never from the previous
|
||||
compaction's prose, so there is no generation-loss spiral. (Mechanically the
|
||||
projection is incremental — it re-derives back to the last full-fold boundary and
|
||||
carries the rest forward, escalating to a genuine re-fold from the branch root
|
||||
when the observation pool reaches `observationsPoolMaxTokens` [20000].)
|
||||
|
||||
So the honest summary of the state after compaction: **the model keeps its
|
||||
instructions, keeps recent work verbatim, trades older turns for a dense
|
||||
id-carrying digest of them, and can pull any of it back on demand.** Not a fresh
|
||||
start — a smaller, cheaper, still-navigable one.
|
||||
|
||||
### One caveat about "no model call"
|
||||
|
||||
If the ledger is empty — compaction fires before the observer has ever run — the
|
||||
hook returns nothing and *declines ownership*, and pi's own model-based
|
||||
summariser runs instead:
|
||||
|
||||
```ts
|
||||
const summary = renderSummary(projection.reflections, projection.observations);
|
||||
if (summary.length === 0) {
|
||||
// Decline ownership so Pi's native summarizer preserves the pre-cut context.
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
In steady state (any session old enough to have produced one observation) om's
|
||||
hook wins and compaction is model-free. "Never calls a model" is true in practice
|
||||
and false in principle; the fallback is deliberate, so an empty ledger degrades
|
||||
to normal pi rather than to no summary at all.
|
||||
|
||||
## 5. What you actually get
|
||||
|
||||
- **Compaction stops being a stall.** In steady state the latency path is
|
||||
deterministic work over ledger entries, not a summarisation call.
|
||||
- **Nothing important vanishes silently.** Compaction is lossy by design, but
|
||||
every item keeps a 12-character id, and `recall(<id>)` returns the exact
|
||||
evidence — original wording, reasoning, file path, error text.
|
||||
- **The bookkeeping runs on a cheaper model than your session.** In this image
|
||||
that is deliberate and visible (§7): background workers on Haiku, session on
|
||||
Opus.
|
||||
- **It is automatic.** No habit to maintain, unlike the palace protocol — which is
|
||||
exactly why the two complement each other (§11).
|
||||
- **Forks stay clean.** Branch-local memory means a `fork` sub-agent's noise does
|
||||
not leak into the parent's folded memory.
|
||||
|
||||
## 6. `recall` is not a search tool
|
||||
|
||||
`recall` takes **one specific 12-hex id** that already appears in compacted
|
||||
memory or in `/om:view`. It cannot be given a topic. It can return an observation
|
||||
(marked `active` or `dropped`), or a reflection together with the observations
|
||||
supporting it.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant M as compacted memory
|
||||
participant A as agent
|
||||
participant L as ledger
|
||||
M->>A: "[high] user rejected option B (a1b2c3d4e5f6)"
|
||||
A->>L: recall("a1b2c3d4e5f6")
|
||||
L-->>A: exact observation + source ids
|
||||
Note over A: acts on the original wording
|
||||
```
|
||||
|
||||
The rule of thumb the agent skill uses: recall **before a load-bearing action**
|
||||
that rests on a compressed memory — shipping a change, asserting a fact,
|
||||
answering "why do you believe that". One recall is cheap; redoing finished work
|
||||
is not.
|
||||
|
||||
## 7. How it is wired in this image
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
IMG["baked in the image:<br/>v3.0.4 @ ce9fc98"] --> REG["settings.json<br/>packages[]"]
|
||||
REG --> SESS["your pi session"]
|
||||
SESS -- "your turns" --> SM["session model:<br/>Opus"]
|
||||
SESS -- "observer, reflector,<br/>dropper" --> WM["memory model:<br/>Haiku"]
|
||||
SESS -- "ledger entries" --> JL["session .jsonl"]
|
||||
JL --> VOL[("devbox-pi-config<br/>volume")]
|
||||
```
|
||||
|
||||
Four consequences of that wiring:
|
||||
|
||||
1. **There is no separate database.** Memory *is* entries inside the ordinary pi
|
||||
session file (`~/.pi/agent/sessions/<project>/<timestamp>_<uuid>.jsonl`).
|
||||
Nothing extra to back up, nothing to migrate.
|
||||
2. **It survives container recreate**, because `~/.pi` is the `devbox-pi-config`
|
||||
named volume (`docker-compose.yml`) — the same one holding your pi config and
|
||||
session history.
|
||||
3. **`packages[]` is the only source of truth for which copy is loaded.** A clone
|
||||
at `/workspace/pi-observational-memory` may exist (and today matches `/opt`
|
||||
byte-for-byte at `ce9fc98`) — its presence proves nothing. To run a patched
|
||||
build you point `packages[]` at it explicitly and start a new session.
|
||||
4. **The worker model is a deliberate choice, and it is yours to change.** The
|
||||
seeded config sends background work to Haiku while your session runs Opus:
|
||||
|
||||
```json
|
||||
"observational-memory": {
|
||||
"model": { "provider": "amazon-bedrock", "id": "eu.anthropic.claude-haiku-4-5-20251001-v1:0" },
|
||||
"debugLog": false
|
||||
}
|
||||
```
|
||||
|
||||
## 8. What it costs
|
||||
|
||||
| Resource | Cost |
|
||||
|---|---|
|
||||
| Model calls | up to **three** background calls per consolidation pass (observer, reflector, dropper), each capped at `agentMaxTurns` [16], on the configured memory model — not your session model |
|
||||
| Latency in your turns | none by construction: workers run from `turn_end`, compaction runs when pi is idle, and the fold itself does no model work |
|
||||
| Disk | negligible — JSON lines inside a session file that would exist anyway (measured here: `~/.pi/agent/sessions` = 30 MB total, tens of `om.*` entries per session) |
|
||||
| Context window | **zero until compaction.** `custom` entries do not enter LLM context; only the folded summary does |
|
||||
| Attention | none once configured; there is no protocol for you or the agent to remember |
|
||||
|
||||
If that is still more than you want on a given run, §9's `passive` switch turns
|
||||
off all proactive work while keeping `recall` and `/om:*` usable.
|
||||
|
||||
## 9. Configuration
|
||||
|
||||
Global: `~/.pi/agent/settings.json` (persisted in the volume). Per project:
|
||||
`<project>/.pi/settings.json`, which overrides global. Precedence is
|
||||
project → global → environment, and the environment can only override `passive`.
|
||||
|
||||
| Key | Default | What it changes |
|
||||
|---|---|---|
|
||||
| `observeAfterTokens` | `10000` | observer cadence — lower means smaller chunks and more calls |
|
||||
| `reflectAfterTokens` | `20000` | reflector cadence (and thereby dropper opportunities) |
|
||||
| `observerChunkMaxTokens` | 20% of the memory model's context window, else `60000` | cap on one observer run's input |
|
||||
| `compactAfterTokens` | `81000` | when proactive auto-compaction fires |
|
||||
| `observationsPoolMaxTokens` | `20000` | pool size at which compaction does a full re-fold from the branch root |
|
||||
| `observationsPoolTargetTokens` | half of max (`10000`) | what the dropper aims back down to |
|
||||
| `agentMaxTurns` | `16` | shared turn cap for the three workers |
|
||||
| `model` | unset → session model | send background work to a cheaper/faster model |
|
||||
| `showWorkerNotifications` | `true` | routine "observer ran" notices |
|
||||
| `passive` | `false` | **kill switch** for all proactive background work; `recall` and `/om:*` still work |
|
||||
| `debugLog` | `false` | per-session NDJSON trace at `~/.pi/agent/observational-memory/debug/<session-id>.ndjson` |
|
||||
|
||||
Pi's own compaction knobs live under a separate `compaction` key —
|
||||
`keepRecentTokens` [20000] sets the verbatim tail from §4, `reserveTokens`
|
||||
[16384] the headroom that triggers pi's own compaction.
|
||||
|
||||
One-off passive run, no config edit:
|
||||
|
||||
```bash
|
||||
PI_OBSERVATIONAL_MEMORY_PASSIVE=1 pi
|
||||
```
|
||||
|
||||
Invalid values are ignored rather than fatal, so a typo degrades to the default
|
||||
instead of breaking your session — which also means a typo is silent. Check with
|
||||
`/om:status`.
|
||||
|
||||
## 10. Confirming it is actually working
|
||||
|
||||
Do not infer health from the absence of a warning; look:
|
||||
|
||||
```bash
|
||||
# 1. inside pi — the authoritative view
|
||||
/om:status # visible-vs-full drift, thresholds, worker state
|
||||
/om:view # what the agent currently sees
|
||||
/om:view full # full ledger truth at the branch tip
|
||||
|
||||
# 2. from a shell — are ledger entries being written, and has it compacted?
|
||||
grep -o '"customType":"om\.[a-z.]*"' \
|
||||
"$(ls -t ~/.pi/agent/sessions/*/*.jsonl | head -1)" | sort | uniq -c
|
||||
grep -c '"type":"compaction"' "$(ls -t ~/.pi/agent/sessions/*/*.jsonl | head -1)"
|
||||
|
||||
# 3. which copy is loaded, and at what commit
|
||||
python3 -c "import json;print(json.load(open('$HOME/.pi/agent/settings.json'))['packages'])"
|
||||
git -c safe.directory=/opt/pi-observational-memory -C /opt/pi-observational-memory rev-parse HEAD
|
||||
```
|
||||
|
||||
Ledger entries are `"type":"custom"` with `"customType":"om.…"`. Do not grep for
|
||||
`custom_message` — that is a *different* pi API for entries that **do** enter LLM
|
||||
context, used here by the MemPalace mailbox (`customType: "mempalace-mailbox"`),
|
||||
not by om.
|
||||
|
||||
## 11. It is not the same thing as MemPalace
|
||||
|
||||
Both are called "memory" and they solve different problems. Nothing is wrong with
|
||||
running both — this image does, and they cover each other's failure modes.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
O0["observational<br/>memory"] --> O1["horizon:<br/>this session"]
|
||||
O1 --> O2["scope: one branch,<br/>one machine"]
|
||||
O2 --> O3["automatic"]
|
||||
O3 --> O4["retrieval:<br/>recall(id)"]
|
||||
P0["MemPalace"] --> P1["horizon: months,<br/>machines"]
|
||||
P1 --> P2["scope:<br/>the fleet"]
|
||||
P2 --> P3["protocol-driven"]
|
||||
P3 --> P4["retrieval:<br/>search, KG, mailbox"]
|
||||
```
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| "What did we decide 200 turns ago in *this* session?" | observational memory (and `recall` for the exact wording) |
|
||||
| "What did we decide last month, or on another machine?" | MemPalace (`mempalace_search`, diaries) |
|
||||
| "What is true *right now* about version X?" | MemPalace knowledge graph |
|
||||
| "Does another machine need something from me?" | MemPalace coordination log — see [Cross-machine agent coordination](../README.md#cross-machine-agent-coordination) |
|
||||
| "Why is compaction not losing my session?" | observational memory |
|
||||
|
||||
The crisp version: **observational memory keeps a session coherent; the palace
|
||||
keeps the fleet coherent.** A container recreate wipes neither — but only because
|
||||
`~/.pi` and the palace both live outside the container filesystem.
|
||||
|
||||
## 12. Gotchas
|
||||
|
||||
- **Branch-local means branch-local.** Resuming or forking changes which ledger
|
||||
is folded. Memory that "disappeared" is usually on another branch.
|
||||
- **`recall` needs an id, not a topic.** If you only have a topic, that is a
|
||||
palace search, not a recall.
|
||||
- **A `/workspace` clone is not evidence of what is loaded** — see §7.3.
|
||||
- **`showWorkerNotifications: true` is not proof of work**; it reports runs, and
|
||||
an observer that deliberately emits nothing writes no ledger entry and simply
|
||||
retries after another `observeAfterTokens`.
|
||||
- **A turn bigger than `keepRecentTokens` splits.** The cut then lands mid-turn at
|
||||
an assistant message and pi merges two summaries — rare, but it is why a very
|
||||
large single turn can lose more verbatim detail than you would expect.
|
||||
- **`git log` in the baked tree needs `safe.directory`** (`/opt` is root-owned):
|
||||
`git -c safe.directory=/opt/pi-observational-memory -C /opt/pi-observational-memory log`.
|
||||
+304
-10
@@ -7,7 +7,11 @@ set -euo pipefail
|
||||
# so this reaches the same stream as the interactive shell the user lands
|
||||
# in). Reads the ground-truth manifest baked in Dockerfile.variant; a no-op
|
||||
# with a short stderr notice on images built before it existed.
|
||||
command -v pi-devbox-version >/dev/null 2>&1 && pi-devbox-version || true
|
||||
# `--no-skills`: this runs FIRST, before the baked skill links are created
|
||||
# below and long before the skillset deploy + devbox-skill-reconcile run at the
|
||||
# end of this script, so the skill-source section would report a pre-reconcile
|
||||
# state that is about to change. Wrong-but-plausible is worse than absent.
|
||||
command -v pi-devbox-version >/dev/null 2>&1 && pi-devbox-version --no-skills || true
|
||||
|
||||
# ── SSH ControlMaster socket dir ────────────────────────────────
|
||||
# Companion to /etc/ssh/ssh_config.d/00-devbox-controlmaster.conf in the
|
||||
@@ -58,10 +62,17 @@ fi
|
||||
# the runtime skill-link assertion. Pointing at the image path (/usr/local/...)
|
||||
# keeps the skill fresh from the image and surviving volume recreate (unlike
|
||||
# anything baked under a home dir, which a named volume would shadow). Created
|
||||
# only when absent, so a same-named skillset skill (deployed later, at the end
|
||||
# of this script) or a user override is never clobbered; the skillset deploy
|
||||
# classifies these as foreign-links and its --prune-stale pass leaves them
|
||||
# alone (only dangling symlinks are pruned).
|
||||
# only when absent, so a user override is never clobbered.
|
||||
#
|
||||
# NB: "created only when absent" does NOT hand a same-named skillset skill
|
||||
# priority — the opposite. The skillset deploy runs at the end of this script
|
||||
# and classifies these links as foreign, so through v1.8.4 the BAKED copy
|
||||
# always won and an edit pushed to a skillset-owned skill was invisible until
|
||||
# the next image build. The links below are therefore the FALLBACK only;
|
||||
# devbox-skill-reconcile (invoked right after the skillset deploy) hands the
|
||||
# skillset-OWNED skills back to the live clone. Ownership is per-skill, listed
|
||||
# in skills/skillset-owned.txt — see VENDORED.md for why pi-extensions must
|
||||
# keep losing to the baked copy.
|
||||
DEVBOX_SKILLS_SRC=/usr/local/share/pi-devbox/skills
|
||||
if [ -d "$DEVBOX_SKILLS_SRC" ]; then
|
||||
mkdir -p "$HOME/.agents/skills"
|
||||
@@ -69,7 +80,16 @@ if [ -d "$DEVBOX_SKILLS_SRC" ]; then
|
||||
[ -d "$_sk" ] || continue
|
||||
_skname=$(basename "$_sk")
|
||||
if [ ! -e "$HOME/.agents/skills/$_skname" ]; then
|
||||
ln -s "${_sk%/}" "$HOME/.agents/skills/$_skname"
|
||||
# -sfn, not -s: `[ ! -e ]` is TRUE for a DANGLING symlink (-e follows the
|
||||
# link), and since v1.8.5 these links can point into /workspace/skillset
|
||||
# (see devbox-skill-reconcile, invoked after the skillset deploy). If that
|
||||
# mount vanishes while the writable layer survives — a `docker restart` or
|
||||
# a host reboot under restart: unless-stopped, as opposed to a recreate —
|
||||
# plain `ln -s` fails with "File exists" and, under `set -e`, aborts
|
||||
# container start before `exec "$@"`. With -f the broken link heals back to
|
||||
# the baked fallback, and the reconciler re-points it in the same boot if
|
||||
# the clone is back.
|
||||
ln -sfn "${_sk%/}" "$HOME/.agents/skills/$_skname"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
@@ -92,6 +112,187 @@ if command -v mempalace &>/dev/null && [ -d /workspace ]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── MemPalace: pi transcript feeder ─────────────────────────────────
|
||||
# mempalace-toolkit ships `mempalace-pi-session`, which mines pi's own JSONL
|
||||
# session transcripts into the palace. pi's mempalace extension drives it on
|
||||
# session_shutdown and on a debounced agent_settled; this is the catch-up for
|
||||
# the one case no handler can cover — a hard kill (docker kill, OOM, host
|
||||
# reboot) runs nothing at all, so without this the previous life's transcripts
|
||||
# are never mined.
|
||||
#
|
||||
# No MEMPALACE_PI_STAGE override here on purpose: the feeder stages next to the
|
||||
# palace it feeds (<palace-root>/pi-stage), so the stage and the dedup keys
|
||||
# referencing it share one lifetime — whatever persistence the palace has, the
|
||||
# stage inherits. Pinning it elsewhere (e.g. into the ~/.pi volume) would
|
||||
# re-introduce the very split that design prevents: palace volume kept, stage
|
||||
# volume dropped, and `mempalace sync` then prunes every conversation drawer.
|
||||
#
|
||||
# Backgrounded: a cold mine can take tens of seconds and must never delay the
|
||||
# shell. Contention with a live session is handled by the tool itself (it exits
|
||||
# 0 and lets the palace holder do the mine).
|
||||
|
||||
# Self-heal onto PATH for images whose base predates the toolkit symlink.
|
||||
# ~/.local/bin is already ahead of /usr/local/bin on PATH (Dockerfile.base sets
|
||||
# it in ENV PATH) and is writable by this (non-root) user, unlike /usr/local/bin.
|
||||
if [ -x /opt/mempalace-toolkit/bin/mempalace-pi-session ] && \
|
||||
! command -v mempalace-pi-session >/dev/null 2>&1; then
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
ln -sf /opt/mempalace-toolkit/bin/mempalace-pi-session "$HOME/.local/bin/mempalace-pi-session"
|
||||
fi
|
||||
|
||||
# Resolve the feeder explicitly rather than trusting PATH: this runs before any
|
||||
# login shell, and a silently-skipped catch-up is exactly the failure we are
|
||||
# here to prevent.
|
||||
MEMPALACE_FEEDER=""
|
||||
if command -v mempalace-pi-session >/dev/null 2>&1; then
|
||||
MEMPALACE_FEEDER="mempalace-pi-session"
|
||||
elif [ -x /opt/mempalace-toolkit/bin/mempalace-pi-session ]; then
|
||||
MEMPALACE_FEEDER="/opt/mempalace-toolkit/bin/mempalace-pi-session"
|
||||
fi
|
||||
|
||||
if [ "${MEMPALACE_FEED:-1}" != "0" ] && [ -n "$MEMPALACE_FEEDER" ]; then
|
||||
if [ -n "${MEMPALACE_REMOTE_URL:-}" ] && [ -z "${MEMPALACE_PI_SSH_TARGET:-}" ]; then
|
||||
# Remote palace, but no inbox to ship transcripts to — the feeder genuinely
|
||||
# cannot do anything here, so skipping is right. Saying so is the point:
|
||||
# this branch used to be a bare `:`, and the skip happens *before* the
|
||||
# subshell below that writes mempalace-catchup.log, so a container in this
|
||||
# state contributed nothing to the palace and left no artifact at all — not
|
||||
# even an empty log — to explain why. That is indistinguishable from a
|
||||
# healthy run that simply had nothing to file. `tee` puts the notice both in
|
||||
# the container's start output (docker logs) and at the path anyone
|
||||
# debugging "why is nothing from this container in the palace?" looks first.
|
||||
# This is an entrypoint: a notice must never be able to stop a container
|
||||
# from starting. An unwritable ~/.pi (root-owned volume — a classic Docker
|
||||
# permission accident) makes `mkdir -p` fail, and under `set -e` that would
|
||||
# abort startup entirely: a brand-new failure mode in precisely the branch
|
||||
# that used to do nothing at all. Degrade to stdout-only instead.
|
||||
_mp_log="$HOME/.pi/agent/mempalace-catchup.log"
|
||||
mkdir -p "$HOME/.pi/agent" 2>/dev/null || _mp_log=/dev/null
|
||||
{
|
||||
echo "MemPalace catch-up skipped: remote palace with no transcript inbox."
|
||||
echo " MEMPALACE_REMOTE_URL is set (${MEMPALACE_REMOTE_URL})"
|
||||
echo " but MEMPALACE_PI_SSH_TARGET is not, so there is nowhere to ship this"
|
||||
echo " container's staged sessions. MCP tools still read and write the shared"
|
||||
echo " palace — but this container's own conversations are mined nowhere."
|
||||
echo " Fix: set MEMPALACE_PI_SSH_TARGET (and MEMPALACE_PI_DEVICE) in .env,"
|
||||
echo " or unset MEMPALACE_REMOTE_URL to keep the palace local."
|
||||
echo " Deliberate? MEMPALACE_FEED=0 turns the feed off and silences this."
|
||||
} | tee "$_mp_log" 2>/dev/null || true
|
||||
unset _mp_log
|
||||
else
|
||||
mkdir -p "$HOME/.pi/agent"
|
||||
(
|
||||
"$MEMPALACE_FEEDER" --reason container-start \
|
||||
>"$HOME/.pi/agent/mempalace-catchup.log" 2>&1 || true
|
||||
) &
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── cli_utils: link workspace bin/ commands onto PATH ────────────────
|
||||
# Standalone commands from a mounted cli_utils checkout (git-status-all,
|
||||
# git-pull-all, devbox-sanity, pi-session-repair, ...) live in <repo>/bin. On a
|
||||
# host they reach PATH via cli_utils' own install.sh, whose install_bin step
|
||||
# symlinks them into ~/.local/bin — but that home is on the container's WRITABLE
|
||||
# LAYER, so every recreate loses them and the human is back to typing
|
||||
# /workspace/cli_utils/bin/git-status-all. This is the container equivalent of
|
||||
# that install step, re-run at every start.
|
||||
#
|
||||
# WHY SYMLINKS RATHER THAN A PATH EDIT IN AN rc FILE: ~/.local/bin is already
|
||||
# ahead of /usr/local/bin in ENV PATH (Dockerfile.base), so links here resolve in
|
||||
# NON-interactive shells too — `docker exec <c> git-status-all`, agent tool
|
||||
# shells, scripts. An rc-file PATH edit cannot reach those, because ~/.bashrc
|
||||
# returns early when the shell is not interactive. Measured 2026-08-27 on
|
||||
# tor-ms22: `command -v git-status-all` failed in a non-interactive shell while
|
||||
# working in an interactive one, from exactly that asymmetry.
|
||||
#
|
||||
# Detection order (first hit wins):
|
||||
# 1. CLI_UTILS_CONTAINER_PATH explicit, for non-standard layouts
|
||||
# 2. /workspace/cli_utils repo directly in the workspace root
|
||||
# 3. $HOME/cli_utils dedicated mount
|
||||
# 4. /workspace/*/cli_utils workspace root holds several repo groups
|
||||
# CLI_UTILS_LINK=0 disables. Absent repo = silent no-op, which is the common
|
||||
# case for anyone who does not use cli_utils.
|
||||
if [ "${CLI_UTILS_LINK:-1}" != "0" ]; then
|
||||
CLI_UTILS_BIN=""
|
||||
if [ -n "${CLI_UTILS_CONTAINER_PATH:-}" ] && [ -d "${CLI_UTILS_CONTAINER_PATH}/bin" ]; then
|
||||
CLI_UTILS_BIN="${CLI_UTILS_CONTAINER_PATH}/bin"
|
||||
elif [ -d /workspace/cli_utils/bin ]; then
|
||||
CLI_UTILS_BIN=/workspace/cli_utils/bin
|
||||
elif [ -d "$HOME/cli_utils/bin" ]; then
|
||||
CLI_UTILS_BIN="$HOME/cli_utils/bin"
|
||||
else
|
||||
# `if` bodies, not `&&` chains: under `set -e` a loop whose LAST command is a
|
||||
# false test exits non-zero and would abort the entrypoint. With no match the
|
||||
# glob stays literal, so that is the normal case on any machine without this
|
||||
# repo — i.e. the bug would have been "container will not start", not "links
|
||||
# missing".
|
||||
for _cu in /workspace/*/cli_utils/bin; do
|
||||
if [ -d "$_cu" ]; then
|
||||
CLI_UTILS_BIN="$_cu"
|
||||
break
|
||||
fi
|
||||
done
|
||||
unset _cu
|
||||
fi
|
||||
|
||||
if [ -n "$CLI_UTILS_BIN" ]; then
|
||||
mkdir -p "$HOME/.local/bin" 2>/dev/null || true
|
||||
# Never clobber a real file, and never steal a link that points elsewhere: a
|
||||
# deliberate user override in ~/.local/bin must win, and silently shadowing
|
||||
# an image-provided command is worse than the missing command.
|
||||
for _f in "$CLI_UTILS_BIN"/*; do
|
||||
if [ ! -f "$_f" ] || [ ! -x "$_f" ]; then
|
||||
continue
|
||||
fi
|
||||
_link="$HOME/.local/bin/$(basename "$_f")"
|
||||
if [ -e "$_link" ] && [ ! -L "$_link" ]; then
|
||||
continue
|
||||
fi
|
||||
if [ -L "$_link" ]; then
|
||||
case "$(readlink "$_link")" in
|
||||
"$CLI_UTILS_BIN"/*) ;;
|
||||
*) continue ;;
|
||||
esac
|
||||
fi
|
||||
ln -sf "$_f" "$_link" 2>/dev/null || true
|
||||
done
|
||||
# Prune links we own whose target vanished (command renamed, repo moved),
|
||||
# mirroring the skillset deploy's --prune-stale. A dangling link on PATH
|
||||
# reports "No such file or directory" for a command that simply no longer
|
||||
# exists, which reads as a broken container rather than a removed script.
|
||||
for _link in "$HOME/.local/bin"/*; do
|
||||
[ -L "$_link" ] || continue
|
||||
case "$(readlink "$_link")" in
|
||||
*/cli_utils/bin/*) [ -e "$_link" ] || rm -f "$_link" ;;
|
||||
esac
|
||||
done
|
||||
unset _f _link
|
||||
fi
|
||||
unset CLI_UTILS_BIN
|
||||
fi
|
||||
|
||||
# ── Per-device boot hook ─────────────────────────────────────────────
|
||||
# Runs ~/.config/devbox-shell/init.sh if the host provides one. That directory is
|
||||
# the host-owned, bind-mounted shell-sharing dir (see "Volumes and persistence"),
|
||||
# so a hook placed there survives every recreate WITHOUT an image change — the
|
||||
# boot-time twin of the interactive bridge in /etc/skel-devbox/.bash_aliases,
|
||||
# which sources ~/.config/devbox-shell/bash_aliases for every interactive shell.
|
||||
#
|
||||
# NO NEW TRUST BOUNDARY: that same directory is already sourced into every
|
||||
# interactive shell, i.e. it is already arbitrary code from the same owner. What
|
||||
# is new is only WHEN it runs — once at start, before any shell — which is what
|
||||
# non-interactive fixups (symlinks, dirs, one-off migrations) need.
|
||||
#
|
||||
# Deliberately `bash <file>`, not `.` — a hook must not be able to mutate this
|
||||
# entrypoint's own shell state, and its exit status must not matter. Output goes
|
||||
# to a log rather than the container's start output, so a chatty hook cannot
|
||||
# masquerade as a startup error.
|
||||
if [ -r "$HOME/.config/devbox-shell/init.sh" ]; then
|
||||
mkdir -p "$HOME/.pi/agent" 2>/dev/null || true
|
||||
bash "$HOME/.config/devbox-shell/init.sh" \
|
||||
>"$HOME/.pi/agent/devbox-init.log" 2>&1 || true
|
||||
fi
|
||||
|
||||
# ── Git config defaults ──────────────────────────────────────────────
|
||||
if [ -n "${GIT_USER_NAME:-}" ] && ! git config --global user.name &>/dev/null; then
|
||||
git config --global user.name "$GIT_USER_NAME"
|
||||
@@ -169,9 +370,9 @@ if command -v pi &>/dev/null; then
|
||||
"$HOME/.pi/agent/extensions/mempalace.ts"
|
||||
fi
|
||||
|
||||
# pi-fork (fork tool) + pi-observational-memory (recall tool) + (in the
|
||||
# :latest-studio variant only) pi-studio (/studio command + studio_*
|
||||
# tools + theme). These are pi packages (not symlink-style extensions):
|
||||
# pi-fork (fork tool) + pi-observational-memory (recall tool) + pi-atelier
|
||||
# (TUI sidebar panels/split-pane) + (in the :latest-studio variant only)
|
||||
# pi-studio (/studio command + studio_* tools + theme). These are pi packages (not symlink-style extensions):
|
||||
# they're cloned to /opt with node_modules baked at BUILD time, then
|
||||
# registered here via `pi install <local-path>`. A local-path install is
|
||||
# instant + in-place (pi loads the extension directly from /opt) +
|
||||
@@ -206,9 +407,63 @@ if command -v pi &>/dev/null; then
|
||||
fi
|
||||
}
|
||||
|
||||
for _pkg in /opt/pi-fork /opt/pi-observational-memory /opt/pi-studio; do
|
||||
# ── pi-atelier: retire a stale `npm:pi-atelier`, plus an opt-out ──────
|
||||
# The image now vendors pi-atelier at a pinned, audited tag (PI_ATELIER_REF
|
||||
# in Dockerfile.variant). A leftover `npm:pi-atelier` entry from a
|
||||
# hand-install resolves through ~/.pi/npm-global, which lives on the
|
||||
# devbox-pi-config VOLUME — so it survives image upgrades and keeps whatever
|
||||
# version was installed by hand, unpinned and unaudited. That is not
|
||||
# academic: pi-atelier < 0.7.1 makes pi >= 0.84 hang at startup with
|
||||
# sustained CPU, so leaving it in place turns a pi bump into a TUI that will
|
||||
# not start. And `_pi_pkg_registered` deliberately counts `npm:<name>` as
|
||||
# registered (it respects a user's own npm install), so the loop below would
|
||||
# never replace it.
|
||||
#
|
||||
# We only DELETE the exact `npm:pi-atelier` string; the loop then registers
|
||||
# /opt/pi-atelier in pi's own canonical serialization, so this code never has
|
||||
# to guess the stored relative-path form. Idempotent — after the rewrite
|
||||
# there is no npm entry left to match.
|
||||
#
|
||||
# DEVBOX_ATELIER=0 goes further and removes pi-atelier from `packages`
|
||||
# altogether. That escape hatch lives HERE, in the entrypoint, precisely
|
||||
# because this component's known failure mode is "pi will not start" — which
|
||||
# you cannot repair with `pi uninstall`.
|
||||
_pi_atelier_drop() {
|
||||
# $1 = jq predicate over one `packages` entry, selecting what to REMOVE.
|
||||
# Returns 0 only when the file was actually rewritten (caller logs), 1 for
|
||||
# "nothing to do" — including missing jq or unparseable JSON, which must
|
||||
# never clobber user settings. Backs up first, same convention as the
|
||||
# template merge above.
|
||||
_ad_settings="$HOME/.pi/agent/settings.json"
|
||||
[ -f "$_ad_settings" ] || return 1
|
||||
command -v jq >/dev/null 2>&1 || return 1
|
||||
_ad_new=$(jq "(.packages // []) |= map(select(($1) | not))" "$_ad_settings" 2>/dev/null) || return 1
|
||||
[ -n "$_ad_new" ] || return 1
|
||||
if printf '%s' "$_ad_new" | jq -e --slurpfile cur "$_ad_settings" '. == $cur[0]' >/dev/null 2>&1; then
|
||||
return 1
|
||||
fi
|
||||
# `.bak.atelier.` rather than the merge's plain `.bak.` prefix: both can
|
||||
# fire in the same startup, and a bare seconds-resolution timestamp would
|
||||
# make the second cp overwrite the first one's backup.
|
||||
cp "$_ad_settings" "${_ad_settings}.bak.atelier.$(date +%Y%m%d-%H%M%S)"
|
||||
printf '%s\n' "$_ad_new" > "$_ad_settings"
|
||||
return 0
|
||||
}
|
||||
if [ "${DEVBOX_ATELIER:-1}" = "0" ]; then
|
||||
if _pi_atelier_drop '(. == "npm:pi-atelier") or ((type == "string") and endswith("/pi-atelier"))'; then
|
||||
echo "pi-atelier: unregistered per DEVBOX_ATELIER=0 (settings backup saved)"
|
||||
fi
|
||||
elif [ -d /opt/pi-atelier ]; then
|
||||
if _pi_atelier_drop '. == "npm:pi-atelier"'; then
|
||||
echo "pi-atelier: dropped stale npm: registration — the pinned /opt copy takes over (settings backup saved)"
|
||||
fi
|
||||
fi
|
||||
|
||||
for _pkg in /opt/pi-fork /opt/pi-observational-memory /opt/pi-studio /opt/pi-atelier; do
|
||||
[ -d "$_pkg" ] || continue
|
||||
_name=$(basename "$_pkg")
|
||||
# DEVBOX_ATELIER=0 → leave pi-atelier unregistered (handled just above).
|
||||
if [ "$_name" = "pi-atelier" ] && [ "${DEVBOX_ATELIER:-1}" = "0" ]; then continue; fi
|
||||
if ! _pi_pkg_registered "$_name"; then
|
||||
pi install "$_pkg" >/dev/null 2>&1 || \
|
||||
echo "WARN: pi install $_name failed (continuing)"
|
||||
@@ -216,6 +471,38 @@ if command -v pi &>/dev/null; then
|
||||
done
|
||||
fi
|
||||
|
||||
# ── agent-browser: retire a stale volume copy that shadows the image ───
|
||||
# Same hazard class as the pi-atelier retirement above, different delivery
|
||||
# path — and this block exists because that guard did not generalise.
|
||||
# ~/.pi/npm-global lives on the devbox-pi-config VOLUME, so anything ever
|
||||
# installed there with `npm i -g` survives every image upgrade, and PATH puts
|
||||
# it AHEAD of /usr/bin (position 2 vs 8).
|
||||
#
|
||||
# Measured on mbp-m1-2020, 2026-09-06: a 2026-07-17 hand-install pinned
|
||||
# agent-browser 0.27.0 in the volume while the image shipped 0.35.2, so every
|
||||
# session for ~7 weeks ran a stale CLI. The damaging part was not the binary
|
||||
# but its BUNDLED SKILL, which is what the agent actually reads: 3 skillsets /
|
||||
# 17.6 KB core in 0.27.0 vs 8 skillsets / 31.5 KB core in 0.35.2, with ten
|
||||
# subcommands present in the image and undocumented to the agent (a11y,
|
||||
# browser, data, mcp, page, plugin, read, selectors, to, webmcp). A stale tool
|
||||
# announces itself; a stale skill quietly teaches the wrong commands.
|
||||
#
|
||||
# MOVE rather than delete (reversible, same instinct as the settings backups
|
||||
# above), and only when the image ships its own copy — a machine that
|
||||
# deliberately hand-installs agent-browser on an image WITHOUT one keeps it.
|
||||
_ab_vol="$HOME/.pi/npm-global/lib/node_modules/agent-browser"
|
||||
if [ -d "$_ab_vol" ] && [ -d /usr/lib/node_modules/agent-browser ]; then
|
||||
_ab_park="$HOME/.pi/npm-global/.retired-agent-browser-$(date +%Y%m%d-%H%M%S)"
|
||||
if mkdir -p "$_ab_park" 2>/dev/null && mv "$_ab_vol" "$_ab_park/" 2>/dev/null; then
|
||||
# The bin shim is what PATH actually hits; leaving it behind would give a
|
||||
# dangling symlink, which is a worse failure than a stale version.
|
||||
rm -f "$HOME/.pi/npm-global/bin/agent-browser" 2>/dev/null || true
|
||||
echo "agent-browser: retired stale volume copy -> ${_ab_park} (image copy now wins; delete the parked dir when satisfied)"
|
||||
else
|
||||
echo "WARN: agent-browser: stale volume copy at $_ab_vol shadows the image copy and could not be moved; retire it by hand"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── pi-studio: optional loopback bridge (opt-in) ──────────────────────
|
||||
# pi-studio binds its server to 127.0.0.1 inside the container, which a
|
||||
# published Docker port cannot reach. When STUDIO_EXPOSE is truthy (set in
|
||||
@@ -255,6 +542,13 @@ elif [ -x /workspace/skillset/deploy-skills.sh ]; then
|
||||
fi
|
||||
if [ -n "$SKILLSET_DEPLOY" ]; then
|
||||
"$SKILLSET_DEPLOY" --bootstrap --prune-stale >/dev/null 2>&1 || true
|
||||
# The deploy leaves the early baked links (above) in place as foreign links,
|
||||
# which silently shadows the live clone for skills the skillset OWNS. Repoint
|
||||
# just those; baked stays the fallback, user overrides still win. `|| true`:
|
||||
# a skill-link refinement must never break container start.
|
||||
if command -v devbox-skill-reconcile >/dev/null 2>&1; then
|
||||
devbox-skill-reconcile "$(dirname "$SKILLSET_DEPLOY")" || true
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Execute command ──────────────────────────────────────────────────
|
||||
|
||||
Executable
+64
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env bash
|
||||
# Pre-push gate for pi-devbox: shellcheck every shell script before it leaves
|
||||
# this clone. Thin wrapper — all logic lives in scripts/lint-shell.sh, which is
|
||||
# the SAME script the CI release gate runs. One copy, not two: a duplicated
|
||||
# check that drifts is the failure this repo keeps paying for.
|
||||
#
|
||||
# Install per clone: git config core.hooksPath hooks
|
||||
# Bypass this gate: git push --no-verify (a guard, not a wall)
|
||||
#
|
||||
# WHY THIS HOOK EXISTS
|
||||
# v1.8.14's first release attempt died at scripts/smoke-test.sh:770 after
|
||||
# build-base had already spent ~46 minutes. shellcheck had ALREADY caught the
|
||||
# defect — SC2289 at severity error, on the very push that introduced it — and
|
||||
# the lint job stayed red for 24 hours, unread, across three runs. The fix at
|
||||
# the time was to gate the release on the same script (the `lint-gate` job).
|
||||
# This hook is the cheaper end of that: the same finding, before the push,
|
||||
# in seconds rather than after a 40 s CI gate or a 46 min build.
|
||||
#
|
||||
# WHY IT COULD NOT EXIST UNTIL NOW
|
||||
# Measured on v1.8.14 (2026-09-09): shellcheck was absent from the devbox
|
||||
# image by all three routes — PATH, dpkg and a filesystem search. So
|
||||
# lint-shell.sh exited 2 in every container, and a hook calling it would have
|
||||
# refused EVERY push rather than gating anything. `shellcheck` was added to
|
||||
# Dockerfile.base in the same change that added this file; on an image built
|
||||
# before that, enable this hook and you will simply be told the gate cannot
|
||||
# run. That is the correct behaviour, but it is not a working hook — so do not
|
||||
# set core.hooksPath on a container older than the release that bakes it.
|
||||
#
|
||||
# NOTE ON SCOPE: this lints the WORKING TREE, not the exact commit range being
|
||||
# pushed. That is deliberate and matches what the CI gate does to the tagged
|
||||
# tree. It means a defect you have staged-but-not-committed is also reported,
|
||||
# which is noisy in the safe direction.
|
||||
set -euo pipefail
|
||||
|
||||
HOOK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd "$HOOK_DIR/.." && pwd)"
|
||||
LINTER="$REPO_ROOT/scripts/lint-shell.sh"
|
||||
tag="[lint-shell]"
|
||||
|
||||
# Same rule the gate itself applies, applied one level up: a missing check is
|
||||
# not a pass. If the script is gone, the push is refused rather than waved
|
||||
# through on the assumption that CI will catch it.
|
||||
if [ ! -r "$LINTER" ]; then
|
||||
echo "$tag refusing the push: $LINTER is missing, so the gate cannot" >&2
|
||||
echo "$tag run. A gate that cannot run must not pass." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Point the message at the actual remedy when the binary is absent, because the
|
||||
# linter's own message ("install it or run this in CI") is written for a CI
|
||||
# runner and is misleading inside a container the developer cannot apt-install
|
||||
# into persistently.
|
||||
if ! command -v shellcheck >/dev/null 2>&1; then
|
||||
echo "$tag refusing the push: shellcheck is not installed, so the gate" >&2
|
||||
echo "$tag cannot run. A gate that cannot run must not pass." >&2
|
||||
echo "$tag" >&2
|
||||
echo "$tag This container predates the image that bakes shellcheck." >&2
|
||||
echo "$tag Either recreate onto an image that has it, or unset the hook:" >&2
|
||||
echo "$tag git config --unset core.hooksPath" >&2
|
||||
echo "$tag To push this once without the gate: git push --no-verify" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
exec bash "$LINTER" "$REPO_ROOT"
|
||||
@@ -116,6 +116,50 @@ if command -v fzf >/dev/null 2>&1; then
|
||||
eval "$(fzf --bash)" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# cli_utils — shell FUNCTIONS (fgit, fhist, fssh, portcheck, up, mkcd, extract,
|
||||
# agents-sync, …). This is the OTHER HALF of the cli_utils wiring, and until
|
||||
# v1.8.11 the image shipped only one half. entrypoint-user.sh symlinks the repo's
|
||||
# bin/ COMMANDS into ~/.local/bin, which is what makes them resolve in
|
||||
# NON-interactive shells (docker exec, agent tool shells, scripts). A symlink
|
||||
# cannot carry a shell function, and a function cannot be reached from a
|
||||
# non-interactive shell, so the two mechanisms are disjoint and both are
|
||||
# required. Nothing sourced the loader: measured 2026-08-30 on v1.8.11, all 14
|
||||
# functions were simply missing on a device whose $HOME has no zsh rc — which is
|
||||
# the normal case, since the container's interactive shell is bash and zsh is not
|
||||
# installed in the image. The image was already paying this layer's dependency
|
||||
# cost (fzf, bat, fd, rg, jq are all baked partly FOR these functions) while
|
||||
# delivering none of its benefit.
|
||||
#
|
||||
# Detection order deliberately mirrors the symlink block in entrypoint-user.sh so
|
||||
# that commands and functions can never come from two different clones.
|
||||
# CLI_UTILS_SOURCE=0 opts out. That is independent of CLI_UTILS_LINK=0 on purpose:
|
||||
# they disable independent mechanisms, and someone who wants PATH commands
|
||||
# without 14 extra functions in every prompt (or vice versa) should be able to
|
||||
# say so.
|
||||
#
|
||||
# THE LOADER IS BASH-SAFE, MEASURED, NOT ASSUMED: despite every function file
|
||||
# being named *.zsh, sourcing cli_utils.sh under `bash --noprofile --norc` exits
|
||||
# 0 with no errors and defines all 14, and they run (pathls, mkcd, up, extract,
|
||||
# agents-sync, fhist all verified). The single zsh-only construct in the tree
|
||||
# (`print -z` in fzf/fhist.zsh) is already guarded by [[ -n $ZSH_VERSION ]] with
|
||||
# a bash fallback, and the loader's own header states "bash & zsh compatible".
|
||||
# ACCEPTED RISK, stated plainly: /workspace/cli_utils is a HOST BIND MOUNT, so
|
||||
# unlike a pinned git ref this content floats outside the image's control. A
|
||||
# future cli_utils commit that adds a genuinely zsh-only file would surface as
|
||||
# parse errors at every prompt on every device. Errors are left VISIBLE rather
|
||||
# than sent to /dev/null so that failure is diagnosable instead of mysterious,
|
||||
# and CLI_UTILS_SOURCE=0 is the documented one-line escape hatch.
|
||||
if [ "${CLI_UTILS_SOURCE:-1}" != "0" ]; then
|
||||
for _cu in "${CLI_UTILS_CONTAINER_PATH:-}" /workspace/cli_utils "$HOME/cli_utils" /workspace/*/cli_utils; do
|
||||
[ -n "$_cu" ] || continue
|
||||
if [ -r "$_cu/cli_utils.sh" ]; then
|
||||
. "$_cu/cli_utils.sh" || true
|
||||
break
|
||||
fi
|
||||
done
|
||||
unset _cu
|
||||
fi
|
||||
|
||||
# ── PROMPT_COMMAND: flush history every prompt ───────────────────────
|
||||
# Installed AFTER zoxide init so zoxide's hook is already in place;
|
||||
# we append with a newline separator to avoid the ';;' parse error
|
||||
|
||||
Executable
+91
@@ -0,0 +1,91 @@
|
||||
#!/bin/sh
|
||||
# devbox-skill-reconcile — hand skillset-OWNED skills back to the live clone.
|
||||
#
|
||||
# WHY THIS EXISTS
|
||||
# ---------------
|
||||
# entrypoint-user.sh links the image-baked skills into ~/.agents/skills/ EARLY
|
||||
# (before pi-deploy), because the smoke readiness probe gates on markers that
|
||||
# only land later, and a link created after that gate produced a flaky
|
||||
# assertion. Those links are created with a `[ ! -e ]` guard — "only when
|
||||
# absent" — and the skillset deploy runs LAST, treating already-present links
|
||||
# as foreign and leaving them alone. Net effect through v1.8.4: the baked copy
|
||||
# always won, so an edit pushed to a skillset-owned skill was invisible in
|
||||
# every container until the next image build (measured on two hosts: live
|
||||
# skillset md5 129bcc4752 vs baked 5236024fef, the new section absent).
|
||||
#
|
||||
# The fix is NOT "the skillset always wins". Ownership is per-skill (see
|
||||
# rootfs/usr/local/share/pi-devbox/skills/VENDORED.md):
|
||||
#
|
||||
# pi-devbox-environment authored in pi-devbox → baked IS canonical
|
||||
# pi-extensions owned by the package repo, copied over the snapshot
|
||||
# at build time; skillset carries a DOWNSTREAM copy
|
||||
# that can lag → baked must keep winning
|
||||
# mempalace owned by the skillset repo; baked is a snapshot
|
||||
# fallback for containers with no skillset mounted
|
||||
# → the live clone must win when it is present
|
||||
#
|
||||
# So only skills listed in skills/skillset-owned.txt are handed over. Baked
|
||||
# links stay as the fallback (the early-link race fix is untouched), and a user
|
||||
# override always beats both: a real directory is never replaced, and neither is
|
||||
# a symlink that already points somewhere other than the baked tree.
|
||||
#
|
||||
# Usage: devbox-skill-reconcile <skillset-root> [skills-dir] [baked-src]
|
||||
# skillset-root the mounted skillset repo (contains skills/<name>/)
|
||||
# skills-dir default $HOME/.agents/skills
|
||||
# baked-src default /usr/local/share/pi-devbox/skills
|
||||
#
|
||||
# Idempotent, and silent unless it changes something. Exits 0 when there is
|
||||
# nothing to do (no skillset, no list) so the entrypoint never fails on it.
|
||||
set -eu
|
||||
|
||||
SKILLSET_ROOT="${1:-}"
|
||||
SKILLS_DIR="${2:-$HOME/.agents/skills}"
|
||||
BAKED_SRC="${3:-/usr/local/share/pi-devbox/skills}"
|
||||
BAKED_SRC="${BAKED_SRC%/}" # a trailing slash would make the prefix
|
||||
# match below ("$BAKED_SRC"/*) match nothing
|
||||
|
||||
[ -n "$SKILLSET_ROOT" ] || exit 0
|
||||
[ -d "$SKILLSET_ROOT/skills" ] || exit 0
|
||||
[ -d "$SKILLS_DIR" ] || exit 0
|
||||
|
||||
# Absolutise BOTH roots before they are used, because each has its own way of
|
||||
# failing silently when relative: a relative symlink TARGET is resolved against
|
||||
# the link's directory (~/.agents/skills), not $PWD, so it would dangle on
|
||||
# creation; and a relative BAKED_SRC would never prefix-match the absolute
|
||||
# target that `readlink` reports, so every skill would be skipped and the fix
|
||||
# would look like it had simply done nothing.
|
||||
SKILLSET_ROOT=$(CDPATH= cd -- "$SKILLSET_ROOT" 2>/dev/null && pwd) || exit 0
|
||||
BAKED_SRC=$(CDPATH= cd -- "$BAKED_SRC" 2>/dev/null && pwd) || exit 0
|
||||
OWNED_LIST="$BAKED_SRC/skillset-owned.txt"
|
||||
[ -f "$OWNED_LIST" ] || exit 0
|
||||
|
||||
while IFS= read -r _line || [ -n "$_line" ]; do
|
||||
# strip comments and surrounding whitespace; skip blanks
|
||||
_name=$(printf '%s\n' "$_line" | sed -e 's/#.*$//' -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
|
||||
[ -n "$_name" ] || continue
|
||||
# defensive: a list entry must be a plain skill name, never a path
|
||||
case "$_name" in */*|.*) continue ;; esac
|
||||
|
||||
_live="$SKILLSET_ROOT/skills/$_name"
|
||||
_link="$SKILLS_DIR/$_name"
|
||||
|
||||
# the skillset does not ship it → the baked fallback is all there is
|
||||
[ -d "$_live" ] || continue
|
||||
# a real directory is a user override → never touch
|
||||
[ -L "$_link" ] || continue
|
||||
|
||||
# only ever replace OUR OWN link. readlink is deliberate: `readlink -f`
|
||||
# would resolve a link that already points into the skillset clone and,
|
||||
# since both trees hold a same-named skill, could not tell them apart.
|
||||
_target=$(readlink "$_link" 2>/dev/null || true)
|
||||
case "$_target" in
|
||||
"$BAKED_SRC"/*|"$BAKED_SRC") ;; # baked link → ours to replace
|
||||
*) continue ;; # user/foreign target → leave alone
|
||||
esac
|
||||
|
||||
# -n so an existing symlink-to-directory is replaced rather than followed
|
||||
# (without it, ln would create $_link/$_name inside the baked tree).
|
||||
if ln -sfn "$_live" "$_link" 2>/dev/null; then
|
||||
printf 'skill %s: baked snapshot -> live skillset (%s)\n' "$_name" "$_live"
|
||||
fi
|
||||
done < "$OWNED_LIST"
|
||||
@@ -14,6 +14,8 @@
|
||||
# pi-devbox-version human-readable summary (default)
|
||||
# pi-devbox-version --json raw manifest JSON (for scripting)
|
||||
# pi-devbox-version --quiet one-line "release_tag (source_revision)" form
|
||||
# pi-devbox-version --no-skills skip the skill-source section (used at
|
||||
# container start, where it would be premature)
|
||||
#
|
||||
# EXIT STATUS
|
||||
# 0 on success. 1 if the manifest is missing (e.g. an image built before
|
||||
@@ -24,15 +26,35 @@ set -euo pipefail
|
||||
|
||||
MANIFEST=/etc/pi-devbox/build-manifest.json
|
||||
MODE="human"
|
||||
SHOW_SKILLS="yes"
|
||||
|
||||
case "${1:-}" in
|
||||
--json) MODE="json" ;;
|
||||
--quiet|-q) MODE="quiet" ;;
|
||||
--help|-h)
|
||||
sed -n '2,20p' "$0" | sed 's/^# \?//'
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
# A `case "${1:-}"` here only ever looked at the FIRST argument, so
|
||||
# `--no-skills --json` matched --no-skills, silently dropped --json, and
|
||||
# printed human text to a caller expecting JSON (a real failure: a jq
|
||||
# consumer piping that output gets a parse error, not a wrong-but-parseable
|
||||
# answer). Loop over every argument instead, and reject anything unknown
|
||||
# rather than silently ignoring it the same way.
|
||||
for _arg in "$@"; do
|
||||
case "$_arg" in
|
||||
--json) MODE="json" ;;
|
||||
--quiet|-q) MODE="quiet" ;;
|
||||
--no-skills) SHOW_SKILLS="no" ;;
|
||||
--help|-h)
|
||||
# Print the leading `#`-comment block verbatim, stopping at the first
|
||||
# non-comment line, rather than a hardcoded line range: `sed -n
|
||||
# '2,22p'` was silently truncating --help because this file has grown
|
||||
# usage lines since that range was written, and a fixed range will
|
||||
# drift again the next time a comment is added above it.
|
||||
awk 'NR==1{next} /^#/{sub(/^# ?/,""); print; next} {exit}' "$0"
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "pi-devbox-version: unknown option: $_arg" >&2
|
||||
echo " try --help" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ ! -f "$MANIFEST" ]; then
|
||||
echo "pi-devbox-version: no build manifest at $MANIFEST" >&2
|
||||
@@ -55,6 +77,10 @@ release_tag=$(jq -r '.release_tag' "$MANIFEST")
|
||||
build_date=$(jq -r '.build_date' "$MANIFEST")
|
||||
source_rev=$(jq -r '.source_revision' "$MANIFEST")
|
||||
pi_version_baked=$(jq -r '.pi_version' "$MANIFEST")
|
||||
# `// empty` matters: images built before v1.8.6 have no such field, and
|
||||
# `jq -r` renders a JSON null as the 4-char string "null" — which would
|
||||
# print as a bogus version rather than being treated as absent.
|
||||
mp_version_baked=$(jq -r '.mempalace_version // empty' "$MANIFEST")
|
||||
|
||||
if [ "$MODE" = "quiet" ]; then
|
||||
printf '%s (%s)\n' "$release_tag" "${source_rev:0:7}"
|
||||
@@ -71,6 +97,16 @@ if command -v pi >/dev/null 2>&1; then
|
||||
pi_version_live=$(pi --version 2>/dev/null | head -n1 | tr -d '\r\n')
|
||||
fi
|
||||
|
||||
# Same check for the palace, which matters more than it looks: mempalace is
|
||||
# the one component that is BOTH client (here) and server (synlig runs this
|
||||
# same image), so a skew between the two is a real failure mode rather than
|
||||
# cosmetic. `mempalace --version` prints "MemPalace 3.8.0" — name-prefixed,
|
||||
# unlike pi's bare "0.84.3" — hence $NF rather than reading the whole line.
|
||||
mp_version_live=""
|
||||
if command -v mempalace >/dev/null 2>&1; then
|
||||
mp_version_live=$(mempalace --version 2>/dev/null | head -n1 | awk '{print $NF}' | tr -d '\r\n')
|
||||
fi
|
||||
|
||||
printf 'pi-devbox %s\n' "$release_tag"
|
||||
printf ' built: %s (source %s)\n' "$build_date" "${source_rev:0:12}"
|
||||
if [ -n "$pi_version_live" ] && [ "$pi_version_live" != "$pi_version_baked" ]; then
|
||||
@@ -79,5 +115,146 @@ else
|
||||
printf ' pi: %s\n' "${pi_version_live:-$pi_version_baked}"
|
||||
fi
|
||||
|
||||
# Printed only when known, so this degrades quietly on pre-v1.8.6 images
|
||||
# instead of showing an empty or "null" palace line.
|
||||
if [ -n "$mp_version_live" ] || [ -n "$mp_version_baked" ]; then
|
||||
if [ -n "$mp_version_live" ] && [ -n "$mp_version_baked" ] && [ "$mp_version_live" != "$mp_version_baked" ]; then
|
||||
printf ' palace: %s \033[33m(baked as %s — drift detected)\033[0m\n' "$mp_version_live" "$mp_version_baked"
|
||||
else
|
||||
printf ' palace: %s\n' "${mp_version_live:-$mp_version_baked}"
|
||||
fi
|
||||
fi
|
||||
|
||||
printf ' components:\n'
|
||||
jq -r '.components | to_entries[] | select(.value != null) | " \(.key): \(.value[0:12])"' "$MANIFEST"
|
||||
|
||||
# ── Which copy of each vendored skill is actually being read? ─────────
|
||||
# The image bakes fallback skills under /usr/local/share/pi-devbox/skills/,
|
||||
# but for skills the skillset repo OWNS (skillset-owned.txt) a mounted live
|
||||
# clone takes over at container start via devbox-skill-reconcile. Nothing
|
||||
# reported which copy won, so a stale baked snapshot and a current live clone
|
||||
# looked identical from inside — and on this fleet the baked mempalace copy is
|
||||
# read by NOBODY (all four compose stacks mount a workspace containing the
|
||||
# skillset), which is exactly the sort of fact that should be visible rather
|
||||
# than reasoned about. Same "drift detected" shape as the pi/palace lines
|
||||
# above: what is live, annotated with what was baked, when they disagree.
|
||||
#
|
||||
# Skipped with --no-skills at container start (entrypoint-user.sh calls this
|
||||
# FIRST, before the baked links exist and long before the skillset deploy and
|
||||
# reconcile run last), because a section that is accurate only after boot
|
||||
# finishes is worse than no section at all.
|
||||
BAKED_SKILLS=/usr/local/share/pi-devbox/skills
|
||||
SKILLS_DIR="${HOME:-/home/developer}/.agents/skills"
|
||||
|
||||
if [ "$SHOW_SKILLS" = "yes" ] && [ -d "$BAKED_SKILLS" ] && [ -d "$SKILLS_DIR" ]; then
|
||||
# Recorded provenance of the vendored mempalace snapshot (absent on images
|
||||
# built before this existed — `// empty` so a JSON null never prints as the
|
||||
# 4-char string "null", the same trap noted for mempalace_version above).
|
||||
# `_tree_sha256`, not `_sha256`: it is a hash over every file in the
|
||||
# vendored skill DIRECTORY (see tree_sha256() below), not one file, because
|
||||
# a single-file hash reports "identical" against a live checkout that added
|
||||
# or edited a sibling file — pi-extensions already ships two files, so this
|
||||
# is not hypothetical.
|
||||
snap_ref=$(jq -r '.skillset_snapshot_ref // empty' "$MANIFEST")
|
||||
snap_sha=$(jq -r '.skillset_snapshot_tree_sha256 // empty' "$MANIFEST")
|
||||
# Which pi-extensions copy the BUILD baked. Distinct from everything else in
|
||||
# this section, which reports which copy is being READ at runtime: for
|
||||
# pi-extensions the baked tree is itself one of two possible copies, and that
|
||||
# choice was made at build time and is not recoverable by inspection.
|
||||
px_src=$(jq -r '.pi_extensions_skill_source // empty' "$MANIFEST")
|
||||
# Same pipeline Dockerfile.variant uses to measure the baked directory at
|
||||
# build time: relative paths in `find | sort` order, each hashed, the whole
|
||||
# listing folded into one sha256. Keep the two definitions identical — they
|
||||
# run in different processes (image build vs. this container) and are
|
||||
# meaningless to compare unless they agree byte-for-byte on the algorithm.
|
||||
tree_sha256() {
|
||||
( cd "$1" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum ) 2>/dev/null | sha256sum | cut -d' ' -f1
|
||||
}
|
||||
|
||||
# Iterate the baked tree rather than a hardcoded name list, so vendoring a
|
||||
# fourth skill needs no edit here. The header prints only if the tree is
|
||||
# non-empty, so this can never emit a dangling "skills:" label.
|
||||
_printed_header="no"
|
||||
for _dir in "$BAKED_SKILLS"/*/; do
|
||||
[ -d "$_dir" ] || continue
|
||||
if [ "$_printed_header" = "no" ]; then
|
||||
printf ' skills:\n'
|
||||
_printed_header="yes"
|
||||
fi
|
||||
_name=$(basename "$_dir")
|
||||
_link="$SKILLS_DIR/$_name"
|
||||
|
||||
if [ ! -e "$_link" ]; then
|
||||
printf ' %-22s not linked\n' "$_name"
|
||||
continue
|
||||
fi
|
||||
|
||||
_target=$(readlink -f "$_link" 2>/dev/null || echo "$_link")
|
||||
case "$_target" in
|
||||
"$BAKED_SKILLS"/*|"$BAKED_SKILLS")
|
||||
# "baked" alone used to be the whole story. For pi-extensions it is not:
|
||||
# the baked tree holds EITHER the package copy that Dockerfile.variant
|
||||
# lays over the snapshot, OR the vendored floor, when the clone had no
|
||||
# skill/ at that ref. The two are indistinguishable by inspection — same
|
||||
# path, same filenames, same permissions — so the build records which one
|
||||
# it used and this reports it. Without this line a six-week-stale
|
||||
# fallback skill looks exactly like a current one, which is precisely how
|
||||
# the floor went unnoticed from 2026-07-30 to 2026-09-10.
|
||||
if [ "$_name" = "pi-extensions" ] && [ -n "$px_src" ]; then
|
||||
case "$px_src" in
|
||||
package)
|
||||
printf ' %-22s baked (package copy)\n' "$_name" ;;
|
||||
vendored-floor)
|
||||
printf ' %-22s baked \033[33m(FALLBACK: vendored floor — clone had no skill/)\033[0m\n' "$_name" ;;
|
||||
divergent)
|
||||
printf ' %-22s baked \033[33m(MIXED: part package, part floor)\033[0m\n' "$_name" ;;
|
||||
*)
|
||||
printf ' %-22s baked\n' "$_name" ;;
|
||||
esac
|
||||
else
|
||||
printf ' %-22s baked\n' "$_name"
|
||||
fi
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
|
||||
# Outside the baked tree: a mounted skillset clone, or a user override.
|
||||
# The link target is <repo>/skills/<name>, so the repo root is two up.
|
||||
# Everything here is guarded: this script runs on the container-start path
|
||||
# and must never fail, and `set -e` is in force.
|
||||
_root=$(cd "$_target/../.." 2>/dev/null && pwd) || _root=""
|
||||
_head=""
|
||||
if [ -n "$_root" ]; then
|
||||
_head=$(git -C "$_root" rev-parse HEAD 2>/dev/null || echo "")
|
||||
fi
|
||||
_where="live ${_root:-$_target}"
|
||||
[ -n "$_head" ] && _where="$_where @ ${_head:0:7}"
|
||||
|
||||
# For the one skill whose baked fingerprint we recorded, say plainly
|
||||
# whether the live copy differs from what shipped. This is the check CI
|
||||
# cannot perform (the skillset is private) and the container can, free.
|
||||
# Hash the whole live DIRECTORY with the same tree_sha256() used to
|
||||
# measure the baked one in Dockerfile.variant — a SKILL.md-only compare
|
||||
# would silently ignore a changed or added sibling file.
|
||||
_live_sha=""
|
||||
if [ -n "$snap_sha" ] && [ "$_name" = "mempalace" ] && [ -d "$_target" ]; then
|
||||
_live_sha=$(tree_sha256 "$_target")
|
||||
fi
|
||||
if [ -z "$_live_sha" ]; then
|
||||
printf ' %-22s %s\n' "$_name" "$_where"
|
||||
elif [ "$_live_sha" = "$snap_sha" ]; then
|
||||
printf ' %-22s %s (identical to baked snapshot)\n' "$_name" "$_where"
|
||||
elif [ -n "$_head" ] && [ "$_head" = "$snap_ref" ]; then
|
||||
# Same commit, different bytes — i.e. uncommitted edits in the live
|
||||
# checkout. Distinguished from plain drift because otherwise the line
|
||||
# reads as a self-contradiction ("@ c04cd15 ... baked snapshot c04cd15
|
||||
# — live copy differs") and a reader would suspect the tool, not the
|
||||
# working tree.
|
||||
printf ' %-22s %s \033[33m(baked snapshot %s + uncommitted edits)\033[0m\n' \
|
||||
"$_name" "$_where" "${snap_ref:0:7}"
|
||||
else
|
||||
printf ' %-22s %s \033[33m(baked snapshot %s — live copy differs)\033[0m\n' \
|
||||
"$_name" "$_where" "${snap_ref:0:7}"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
@@ -197,6 +197,45 @@ EOF
|
||||
)
|
||||
fi
|
||||
|
||||
# ── Multiplexing default, deliberately LAST ───────────────────────────
|
||||
# Why this block exists: ControlPath above is forced, but ControlMaster is not
|
||||
# set anywhere for targets that come from the user's own ~/.ssh/config. A target
|
||||
# whose entry omits ControlMaster therefore opens a NEW TCP connection per ssh
|
||||
# call, and an agent doing a dozen calls in a few minutes can trip fail2ban or a
|
||||
# CGNAT flow-table cap on the far end — observed 2026-08-25: ~12 connections in
|
||||
# 15 min and port 22 stopped answering while HTTPS to the same estate stayed fine.
|
||||
#
|
||||
# WHY IT IS AT THE BOTTOM, and ControlPath is at the top. ssh_config is
|
||||
# first-value-wins, so position encodes intent:
|
||||
# * BEFORE the Include = an OVERRIDE. Correct for ControlPath, whose value in
|
||||
# the user's config points at read-only ~/.ssh and simply cannot work here.
|
||||
# * AFTER the Include = a DEFAULT. Correct for ControlMaster, because an
|
||||
# explicit per-host 'ControlMaster no' (or 'auto', or any value) in the
|
||||
# user's own config must keep winning. We are supplying an opinion only
|
||||
# where the user expressed none.
|
||||
# That asymmetry is the whole design: force what is broken, default what is
|
||||
# merely absent. It also means this needs no audit of anyone's ~/.ssh/config —
|
||||
# which matters because that file is per-machine, differs across the fleet, and
|
||||
# future machines' versions do not exist yet to be audited.
|
||||
#
|
||||
# Caveat worth knowing (and documented in the pi-devbox-environment skill): a
|
||||
# stale master socket — file present, daemon gone, e.g. after the host suspends
|
||||
# or changes network — makes every later ssh to that host hang. Recovery is
|
||||
# 'ssh -F ~/.ssh-local/config -O exit <host>'. ControlPersist is deliberately
|
||||
# short (10m idle, and each new session resets the idle timer) so an abandoned
|
||||
# socket ages out on its own rather than lingering for hours.
|
||||
MULTIPLEX_DEFAULT_BLOCK=$(cat <<'EOF'
|
||||
|
||||
# Multiplexing DEFAULT — intentionally after the Include above, so any explicit
|
||||
# per-host ControlMaster in your own ~/.ssh/config still wins (first-value-wins).
|
||||
# Applies only to targets that never mentioned ControlMaster at all.
|
||||
# Stale socket after a suspend/network change? ssh -O exit <host>.
|
||||
Host *
|
||||
ControlMaster auto
|
||||
ControlPersist 10m
|
||||
EOF
|
||||
)
|
||||
|
||||
cat > "$CONFIG" <<EOF
|
||||
# AUTO-GENERATED by setup-lan-access.sh on every container start. Do not edit
|
||||
# by hand — edits are overwritten. Used via: ssh -F ~/.ssh-local/config <host>
|
||||
@@ -216,6 +255,7 @@ ${JUMP_BLOCK}
|
||||
${LAN_CONF_BLOCK}
|
||||
${AUTOJUMP_BLOCK}
|
||||
${INCLUDE_BLOCK}
|
||||
${MULTIPLEX_DEFAULT_BLOCK}
|
||||
EOF
|
||||
chmod 600 "$CONFIG" 2>/dev/null || true
|
||||
|
||||
|
||||
@@ -41,3 +41,70 @@ especially load-bearing here — a pi-devbox container is frequently recreated,
|
||||
the palace is your only memory across recreates. Without the habit it is just
|
||||
storage, not memory. (The skill is the consumer side; feeding the palace is the
|
||||
separate `opencode-mempalace-bridge` skill, if present.)
|
||||
|
||||
### If the palace is central, it is shared — three rules
|
||||
|
||||
If `MEMPALACE_REMOTE_URL` is set, the MCP tools write to a **central palace
|
||||
shared with other machines**, not to a local one. Your drawers are not the only
|
||||
ones in there, and most drawers' `source_file` paths do not exist on this host.
|
||||
The skill covers the orientation side (provenance, chronology, whose diary is
|
||||
whose); these three are here instead because getting them wrong does *damage*
|
||||
rather than merely confusing you:
|
||||
|
||||
- **Never run `mempalace sync` / `mempalace_sync` against a shared palace.** It
|
||||
prunes drawers whose source files look gitignored, deleted, or moved — and on
|
||||
a shared palace that describes most of the content, including every other
|
||||
machine's. Compounding it (RFC-001 §7.2): feeders now stage *inside* the
|
||||
palace root, so a scoped sync can delete the very drawers it just filed.
|
||||
`mempalace_delete_by_source` is exact-match rather than existence-based, but
|
||||
its blast radius is now the whole fleet's palace — leave it on its default
|
||||
`dry_run=true` and confirm the match count before committing.
|
||||
- **A timeout is not a failure.** The palace is single-writer, and one large
|
||||
mine can block every client for minutes, so a write or mine that exceeds the
|
||||
client's deadline has usually *completed* server-side. Verify with
|
||||
`mempalace_get_drawer` or `mempalace_search` before retrying — a blind retry
|
||||
files a duplicate. `[mempalace ext] feed (tick) failed: mine timed out after
|
||||
30000ms` is the common benign instance: the transcript is already in the
|
||||
server's inbox and the mine is idempotent, so nothing is lost either way.
|
||||
- **The `mempalace` CLI is not remote-aware.** It always opens a palace on
|
||||
local disk, so `mempalace search` can return older and different results than
|
||||
the MCP tools while both look correct. Use the MCP tools for the central
|
||||
palace; the CLI only for a local one.
|
||||
|
||||
## Before you file a finding: second measurement, different route
|
||||
|
||||
This is here rather than in a skill because it has to fire *without* a matching
|
||||
task description, and because the version of it that lived only in a skill was
|
||||
violated five times in one session by an agent that had the skill available.
|
||||
|
||||
**Any claim you are about to record as fact — in a drawer, a diary entry, a
|
||||
coordination event, or a report to the user — needs a second measurement taken
|
||||
by a different route.** Not a re-read of your reasoning: re-reading has caught
|
||||
zero of these. A disagreeing measurement has caught all of them.
|
||||
|
||||
The two shapes that get filed as fact and are not:
|
||||
|
||||
- **A negative result** (`401`, connection refused, zero rows, "not found") is
|
||||
first a claim about *your filter*, not about the world. Wrong host, wrong port,
|
||||
wrong table, capped output.
|
||||
- **A positive result** proves only what your command *actually asked*. An SSH
|
||||
handshake can succeed against the wrong host (`ssh -G` tells you which rule
|
||||
captured the name); a `401` can be a real answer from an issuer that never
|
||||
minted the credential.
|
||||
|
||||
Cheapest habit that works: **write the expected result next to each check before
|
||||
running it**, then diff. Expectations declared up front turn a silent wrong
|
||||
assumption into a visible mismatch. And if you cannot think of a second route to
|
||||
the same fact, you do not have a finding — you have a hypothesis, so label it as
|
||||
one.
|
||||
|
||||
## Handling an exposed credential
|
||||
|
||||
If a task touches a leaked secret, a token rotation, "is this credential still
|
||||
live?", whether to delete stored content, or which scopes a new token needs:
|
||||
**read `~/.agents/skills/credential-incident-response/SKILL.md` first.** One rule
|
||||
is load-bearing enough to state here: **probe the issuing provider before doing
|
||||
anything else** — most "exposed" credentials in a long-lived fleet are already
|
||||
dead, and the ones that are live are often far more privileged than assumed.
|
||||
Severity first, cleanup second, and prefer **revocation over deletion** for
|
||||
anything already replicated.
|
||||
|
||||
@@ -1,13 +1,15 @@
|
||||
# Vendored fallback skills
|
||||
|
||||
Most directories here are **image-baked skills** that `entrypoint-user.sh`
|
||||
symlinks into `~/.agents/skills/` on container start (only when a skill of the
|
||||
same name is not already present, so a mounted `skillset` repo or a user
|
||||
override always wins).
|
||||
symlinks into `~/.agents/skills/` on container start. They are the **fallback**
|
||||
layer: see *Runtime precedence* below for which copy actually wins when a
|
||||
`skillset` repo is mounted (through v1.8.4 the answer was "always the baked
|
||||
one", which was a bug).
|
||||
|
||||
| skill | owner | how it gets here |
|
||||
|-------|-------|------------------|
|
||||
| `pi-devbox-environment` | pi-devbox (this repo) | authored here; the canonical copy |
|
||||
| `credential-incident-response` | pi-devbox (this repo) | authored here; the canonical copy |
|
||||
| `pi-extensions` | the `pi-extensions` package repo (`skill/`) | **vendored fallback** + refreshed at build |
|
||||
| `mempalace` | the `skillset` repo | **vendored fallback** (snapshot only) |
|
||||
|
||||
@@ -38,16 +40,108 @@ its skill file needed baking.
|
||||
*different* skill, `opencode-mempalace-bridge`), so there is no public
|
||||
package source to copy from. This snapshot is refreshed manually per release.
|
||||
|
||||
**Refresh it with `scripts/vendor-mempalace-skill.sh <skillset-root>`, not
|
||||
`cp`.** Because the image cannot clone the private upstream, the snapshot used
|
||||
to be *anonymous* — nothing recorded which skillset commit the bytes came
|
||||
from, so the only staleness check possible was a hand-maintained phrase canary
|
||||
in `scripts/smoke-test.sh`, which by construction detects "older than the
|
||||
phrase I remembered to pin", never "older than skillset main". Two facts now
|
||||
travel with the file:
|
||||
|
||||
| Fact | Where | Kind |
|
||||
|---|---|---|
|
||||
| `ARG SKILLSET_SNAPSHOT_REF` in `Dockerfile.variant` | manifest `skillset_snapshot_ref` + OCI label `se.jordbo.pi-devbox.skillset-snapshot-ref` | a **claim** about which commit these bytes are |
|
||||
| `sha256sum` of this file, measured in the manifest layer | manifest `skillset_snapshot_sha256` | the bytes that **actually shipped** |
|
||||
|
||||
The script writes both together, refuses when the upstream file has
|
||||
uncommitted modifications (no commit describes those bytes), and
|
||||
`--check` verifies the claim against a real clone. Deliberately an `ARG`
|
||||
default rather than a CI-resolved value: no credential for a private repo, no
|
||||
change at any of the four `Dockerfile.variant` build call sites, and a local
|
||||
`docker build` records the same thing CI does.
|
||||
|
||||
Verifying "is this snapshot current?" is **not** a CI job and was deliberately
|
||||
not made one — see the Unreleased CHANGELOG entry for why (private repo;
|
||||
another repo's branch must not be able to fail this build; and the artefact it
|
||||
would guard is read by no host on this fleet). The check belongs where the
|
||||
skillset actually is: `vendor-mempalace-skill.sh --check` for a maintainer,
|
||||
and `pi-devbox-version`'s `skills:` section for an agent inside a container.
|
||||
|
||||
## Runtime precedence (v1.8.5+)
|
||||
|
||||
The baked links are created **early** in `entrypoint-user.sh` (before pi-deploy,
|
||||
to close a smoke readiness race) with a create-only-when-absent guard, and the
|
||||
skillset deploy runs **last** and treats them as foreign links. Through v1.8.4
|
||||
that combination meant the baked snapshot always won: an edit pushed to
|
||||
`skillset/skills/mempalace/SKILL.md` was invisible in every container until the
|
||||
next image build (measured on two hosts — live `md5 129bcc4752` vs baked
|
||||
`5236024fef`, new section absent). Editing those skills *appeared* to work.
|
||||
|
||||
`devbox-skill-reconcile` now runs immediately after the skillset deploy and
|
||||
repoints the links for skills the **skillset owns**, listed one per line in
|
||||
`skillset-owned.txt`. Precedence, highest first:
|
||||
|
||||
1. **user override** — a real directory, or a symlink pointing outside the baked
|
||||
tree; never touched by anything
|
||||
2. **live skillset clone** — but only for names in `skillset-owned.txt`
|
||||
3. **baked snapshot** — everything else, and every skill when no skillset is
|
||||
mounted
|
||||
|
||||
**Which one won is now reportable from inside the container:**
|
||||
`pi-devbox-version` prints a `skills:` section naming, per vendored skill,
|
||||
`baked` or `live <repo> @ <sha>` — and for `mempalace` whether that live copy is
|
||||
identical to the baked fingerprint, at the same commit but with uncommitted
|
||||
edits, or genuinely divergent. Before that, a stale baked snapshot and a current
|
||||
live clone were indistinguishable from inside, which is how the freshness of
|
||||
this file went unexamined for three releases. The section is suppressed with
|
||||
`--no-skills` on the container-start banner, because `entrypoint-user.sh` prints
|
||||
the version *before* the links exist and long before the reconcile below runs.
|
||||
|
||||
On this fleet, precedence 2 wins for `mempalace` on **every** host — all four
|
||||
compose stacks mount a workspace containing the skillset — so the baked copy is
|
||||
exercised only by CI and by a hypothetical no-mount container. Worth
|
||||
remembering before spending effort on its freshness.
|
||||
|
||||
Ownership is per-skill on purpose: `pi-extensions`' authoritative source is the
|
||||
package repo (copied over the snapshot at build), and `skillset` carries a
|
||||
downstream copy that can lag, so handing it to the clone would *regress* the
|
||||
skill. Only `mempalace` is skillset-owned today.
|
||||
|
||||
Verify with `readlink -f ~/.agents/skills/<skill>` — not by reading the
|
||||
entrypoint. Smoke covers both directions (baked resolution with no skillset
|
||||
mounted, plus a fabricated-skillset run of the reconciler).
|
||||
|
||||
## Refreshing the snapshots
|
||||
|
||||
cp <pi-extensions-pkg>/skill/SKILL.md pi-extensions/SKILL.md
|
||||
cp <pi-extensions-pkg>/skill/evaluate-extension-usage.py pi-extensions/
|
||||
cp <skillset>/skills/mempalace/SKILL.md mempalace/SKILL.md
|
||||
|
||||
Copy each snapshot **from its owner in the table above** — `pi-extensions` from
|
||||
the package repo's `skill/` (since `a7f3044` co-located it there; `skillset`
|
||||
also carries a copy, but it is a downstream duplicate and can lag), and
|
||||
`mempalace` from `skillset`. Copying `pi-extensions` from `skillset` would
|
||||
regress the snapshot to whatever that repo last mirrored.
|
||||
Copy `pi-extensions` **from its owner in the table above** — the package
|
||||
repo's `skill/` (since `a7f3044` co-located it there; `skillset` also carries a
|
||||
copy, but it is a downstream duplicate and can lag). Copying `pi-extensions`
|
||||
from `skillset` would regress the snapshot to whatever that repo last mirrored.
|
||||
|
||||
Snapshot provenance at last refresh: skillset `63f3bf5`, pi-extensions pkg `e73cb9f`.
|
||||
`mempalace` is **not** refreshed by `cp` — see the *Freshness model* section
|
||||
above: `scripts/vendor-mempalace-skill.sh <skillset-root>` is the only thing
|
||||
that should ever touch that snapshot, because a bare copy can update the bytes
|
||||
without updating the ref that claims to describe them, which produces a
|
||||
manifest that confidently lies.
|
||||
|
||||
Neither vendored skill has a hand-maintained "last refreshed at" line here on
|
||||
purpose — one previously existed (skillset `670f7f1`, pi-extensions pkg
|
||||
`e73cb9f`) and went stale within hours, because nothing forced it to move
|
||||
when the ARGs did. `670f7f1` is now a cautionary example rather than a fact
|
||||
worth recording: it is the commit that told agents to hand-stamp `added_by`,
|
||||
which a later skillset commit (and the pi-devbox edge stamper) withdrew — so a
|
||||
reader trusting that line would have been pointed at superseded guidance.
|
||||
Both facts it tried to capture now live somewhere that cannot drift by hand:
|
||||
|
||||
| Fact | Where |
|
||||
|---|---|
|
||||
| which skillset commit `mempalace`'s bytes came from | `ARG SKILLSET_SNAPSHOT_REF` (Dockerfile.variant) + `skillset_snapshot_ref` in `build-manifest.json`, written *only* by `vendor-mempalace-skill.sh` |
|
||||
| which pi-extensions package commit was vendored | `ARG PI_EXTENSIONS_REF` (Dockerfile.variant, CI-resolved to a 40-hex commit) → OCI label `se.jordbo.pi-devbox.pi-extensions-ref` and `build-manifest.json`'s `components.pi-extensions`, both read from the actual `/opt/pi-extensions` checkout, not from intent |
|
||||
|
||||
When you refresh the `mempalace` snapshot, also update the phrase asserted by
|
||||
the "mempalace skill snapshot is current" smoke test — it deliberately pins the
|
||||
**newest** section, because the previous canary grepped a phrase that survived
|
||||
the very edit that made the snapshot stale, and so passed on stale content.
|
||||
|
||||
@@ -0,0 +1,271 @@
|
||||
---
|
||||
name: credential-incident-response
|
||||
description: >-
|
||||
Respond correctly when a live credential is found where it should not be —
|
||||
in a chat transcript, a MemPalace drawer, a log, a git-tracked config, or an
|
||||
agent-authored note. Load this whenever a task involves a leaked/exposed
|
||||
secret, a token rotation, a "is this credential still live?" question, deciding
|
||||
whether to delete or scrub stored content, proving a corpus is clean, or
|
||||
choosing scopes for a new API token. Covers the mandatory order of operations
|
||||
(probe the issuer FIRST — severity before cleanliness), leak-free identity via
|
||||
sha256[:8] fingerprints and when publishing one is safe,
|
||||
why revocation beats deletion for anything already replicated, scopes derived
|
||||
from measured consumers, the three places a secret hides in a Chroma palace, how to prove ABSENCE rather than assume it (instrument strength,
|
||||
census vs class passes, the tokenisation trap where quoting decides detectability, why git filters never run on symlinks, self-tests that abort),
|
||||
where this fleet's secrets live, and what rotation does NOT fix.
|
||||
---
|
||||
|
||||
# Credential incident response
|
||||
|
||||
A leaked credential is a **severity** question before it is a cleanliness
|
||||
question. Two days of scrubbing, redaction plumbing and deletion planning were
|
||||
once spent on a set of 13 credentials of which **11 were already dead at the
|
||||
provider** — a fact that cost five HTTP requests to establish and was never
|
||||
checked. Meanwhile the two live ones turned out to be instance-owner **admin**
|
||||
tokens, which nobody had looked at either.
|
||||
|
||||
## 1. Order of operations — do not reorder this
|
||||
|
||||
1. **Is it still accepted?** Probe the issuing provider. Dead credential →
|
||||
hygiene item, stop panicking. Live → incident, continue.
|
||||
2. **What can it do?** Read the identity back. `is_admin`, `id=1`, scopes,
|
||||
which account. A read-only repo token and an instance-owner admin token are
|
||||
not the same finding.
|
||||
3. **What consumes it?** Grep for real consumers before assuming breakage.
|
||||
4. **Where does it live?** Enumerate copies (store, palace, transcripts, git).
|
||||
5. **Then** rotate/revoke, and only then consider cleanup.
|
||||
|
||||
Doing 4→3→1 in reverse produces confident, wrong severity calls and wasted
|
||||
cleanup. If you only have time for one step, do step 1.
|
||||
|
||||
## 2. Leak-free identity: fingerprint, never the value
|
||||
|
||||
Publishing an 8-hex fingerprint lets you compare a credential across machines,
|
||||
files, drawers and peers without ever materialising the secret. Same formula as
|
||||
`mempalace_redact.py`:
|
||||
|
||||
```sh
|
||||
printf '%s' "$SECRET" | sha256sum | cut -c1-8 # printf, NOT echo (no newline)
|
||||
printf '%s' 'test' | sha256sum | cut -c1-8 # self-test -> 9f86d081
|
||||
```
|
||||
|
||||
Report as `(variable, fp, length)`. Equal fingerprints across hosts prove a
|
||||
shared credential; that is usually the important part. **Never** paste a live
|
||||
value into a search query, a palace drawer, an event body, or a chat message —
|
||||
in an agent context your own tool output is itself captured and re-filed.
|
||||
|
||||
**Precondition — only fingerprint what an adversary cannot enumerate.** An 8-hex
|
||||
fingerprint is 32 bits over its *input space*, so publishing `fp8(x)` hands
|
||||
anyone a **membership oracle**: they can test `x == v` for every candidate `v`
|
||||
they can generate. For a 40-char random token that space is unreachable. For a
|
||||
hostname, username, e-mail, port, path, commit SHA or weak password it is a
|
||||
wordlist. **If you can imagine writing the wordlist, you cannot publish the
|
||||
fingerprint** — reference those by name and location instead. "High entropy" is
|
||||
the usual *sufficient condition*, not the test: a commit SHA is 160-bit and still
|
||||
fully enumerable from the repo. `sha256("")` = `e3b0c442` is the degenerate case,
|
||||
recognisable on sight precisely because its input space has one member.
|
||||
|
||||
**Candidate fingerprints are working memory, never output.** A scanner that hashes
|
||||
every token in a file also hashes hostnames, paths and e-mails. Print only
|
||||
fingerprints that *matched* a known entry — the tempting debug step when a scan
|
||||
returns zero ("print what it saw") publishes low-entropy fingerprints wholesale.
|
||||
|
||||
And say plainly what a fingerprint register *is*, so nobody rediscovers it later
|
||||
as an alarm: even for an unguessable secret, a published fingerprint is a
|
||||
**confirmation oracle** for anyone who already holds a candidate corpus. That is
|
||||
exactly how a long-retired token gets identified in old transcripts — and it works
|
||||
identically for someone else holding those same files. Net positive, since they
|
||||
would already hold the value; state it rather than leaving it implicit.
|
||||
|
||||
## 3. Liveness probes, and the trap that scoping creates
|
||||
|
||||
```sh
|
||||
# Gitea
|
||||
curl -sS -m 10 -o /dev/null -w '%{http_code}\n' -H "Authorization: token $T" \
|
||||
"$GITEA_HOST/api/v1/repos/<owner>/<repo>/actions/runs?limit=1"
|
||||
# GitHub
|
||||
curl -sS -m 10 -o /dev/null -w '%{http_code}\n' -H "Authorization: token $T" \
|
||||
https://api.github.com/user
|
||||
```
|
||||
|
||||
- `200` live · `401` revoked/invalid · **`403` = wrong question, not a dead token**
|
||||
- **Probe the issuer that minted it.** A 401 from an unrelated instance says
|
||||
nothing. Resolve the host from config (`GITEA_EGL_HOST` etc.), do not assume.
|
||||
- **Under scoped tokens, `/api/v1/user` returns 403 for a perfectly live token**
|
||||
unless `user` scope was granted. So it cannot distinguish *revoked* from
|
||||
*merely scoped*. Use a **repository route the token is authorised for**.
|
||||
- Verify **both directions** after a rotation: old → 401, new → 200. The second
|
||||
check is what catches "deleted the wrong token".
|
||||
- Port/scheme come from config, not habit: one instance here is
|
||||
`http://gitea.egl.lan:3000` — plain HTTP, with 443 refused.
|
||||
|
||||
## 4. Revocation beats deletion — the load-bearing rule
|
||||
|
||||
Once revoked, stored copies are **inert**; you may leave them. Deleting them is
|
||||
best-effort over an *unbounded* copy set: FTS shadow rows, feed inbox `.jsonl`
|
||||
files on every host, sqlite free pages after the delete, mesh replicas that
|
||||
already synced, and backups. **Revocation invalidates every copy everywhere at
|
||||
once, including copies nobody enumerated.**
|
||||
|
||||
So: **rotate + revoke first.** Treat drawer deletion as optional hygiene, never
|
||||
as the remedy. Then record the retired fingerprints as *known-dead* so the next
|
||||
census recognises them instead of reopening the investigation.
|
||||
|
||||
Corollary: never reach for `mempalace_sync` or a bulk `delete_by_source` on a
|
||||
shared palace as incident response. High blast radius, low actual benefit.
|
||||
|
||||
## 5. Finding a secret in a Chroma palace — three targets, in this order
|
||||
|
||||
1. `embedding_fulltext_search_content.c0` — document text
|
||||
2. `embedding_metadata.string_value` — metadata fields, **and a second copy of
|
||||
the document text** under key `chroma:document`
|
||||
3. raw byte scan of every `*.sqlite3` — backstop, covers FTS pages and free space
|
||||
|
||||
**Correction, measured on chroma 1.5.9 with a sentinel drawer:** one row in (1)
|
||||
AND one row in (2) for the same drawer, so **(2) is not structurally
|
||||
content-blind** — an earlier version of this section said it held "metadata
|
||||
fields only", and that was wrong. Scan (1) and (3) regardless: (1) is the direct
|
||||
target. But if a `string_value` query returns zero for a value you know is in a
|
||||
drawer, the cause is a key filter, a query shape or escaping — *not* structural
|
||||
absence, and the difference matters because the false explanation is what makes
|
||||
the zero feel safe. See §6: do not explain a zero with a mechanism you have not
|
||||
read from source.
|
||||
|
||||
Semantic search proves nothing about absence — it returns top-k. For
|
||||
completeness, enumerate by filing window (`list_drawers(since=T, before=T+1m)`),
|
||||
since one mine shares a minute.
|
||||
|
||||
Value-agnostic sweeps (uuid / 40-hex / `NAME=VALUE`) drown in false positives at
|
||||
fleet scale — 608 candidates, mostly session UUIDs and git SHAs. Name-anchoring
|
||||
plus entropy plus provenance, applied to **document text**, is what works.
|
||||
|
||||
## 6. Proving absence: instrument strength, and four ways a scan lies clean
|
||||
|
||||
Section 5's warning is about false *positives* — name-anchoring and provenance are
|
||||
what stop a triage sweep drowning in session UUIDs. **A gate is the opposite job.**
|
||||
Triage optimises precision; proving absence optimises recall. Every failure below
|
||||
reported a reassuring zero over a secret that was really there.
|
||||
|
||||
**Rank the instrument, and state which one produced your zero.**
|
||||
|
||||
| Instrument | Needs | Blind to |
|
||||
|---|---|---|
|
||||
| exact-byte value search | you hold the value | nothing — no tokeniser to fool |
|
||||
| class/structure pass | a header pattern | anything without a recognisable shape |
|
||||
| fingerprint census | a fingerprint list | any secret not listed; tokenisation |
|
||||
|
||||
A census is deliberately value-free, so it must *extract candidates and hash them*
|
||||
— which makes its sensitivity a property of the tokeniser, not of the corpus. If
|
||||
you hold the value, search the bytes instead, and search the value's JSON-escaped
|
||||
rendering too when the corpus is `.jsonl`.
|
||||
|
||||
**1. Census and class answer different questions; neither substitutes.** A census
|
||||
answers *"has a KNOWN secret leaked?"*, a class pass *"is there secret-SHAPED
|
||||
material here?"* Both failure modes were measured on this fleet: a class-only
|
||||
pre-commit hook passed plaintext UUID API credentials to a shared repo twice,
|
||||
because a UUID carries no key header — while a census-only gate reported 0 hits
|
||||
with freshly-synced SSH private keys and an age identity in the tree, because no
|
||||
key is in the census. Run both passes.
|
||||
|
||||
**2. Tokenisation — quoting alone can decide detectability.** Maximal-run
|
||||
extraction swallows the value of an *unquoted* assignment:
|
||||
|
||||
```
|
||||
PROXMOX_SECRET=<uuid> # ONE run; the uuid is never hashed alone -> MISS
|
||||
export SECRET="<uuid>" # the quote ends the run; bare uuid hashed -> HIT
|
||||
```
|
||||
|
||||
Take the **union** of three strategies, because each fails in a different
|
||||
direction — (2) is the one that recovers the unquoted case:
|
||||
|
||||
~~~python
|
||||
runs = re.findall(r'[^\s"\'`]{12,}', text) # 1. maximal runs
|
||||
split = [p for r in runs for p in re.split(r'[=!,;:@|()\[\]{}<>]', r) if len(p) >= 12]
|
||||
shape = re.findall(UUID_RE, text) + re.findall(r'[0-9a-f]{32,64}', text)
|
||||
candidates = set(runs) | set(split) | set(shape)
|
||||
~~~
|
||||
|
||||
**3. Scan the index or the pushed tree, never the working tree.** The working tree
|
||||
is not what gets published. And for an rsync-published mirror a repo-only fix is
|
||||
not weaker, it is *temporary*: the next sync re-publishes the live disk. Fix the
|
||||
live file first, verify it clean **by fingerprint**, then sync. Read blobs with
|
||||
`git ls-tree -r <sha>` plus one `git cat-file --batch` (thousands of `git show`
|
||||
calls is the slow way).
|
||||
|
||||
**4. Git filters never run on symlinks — and `check-attr` will not tell you.** A
|
||||
symlink's blob is the *target path*, so `filter=git-crypt` can never encrypt it,
|
||||
yet `git check-attr filter` cheerfully answers `git-crypt` for that path. **A
|
||||
symlinked secret stays plaintext no matter what `.gitattributes` says.** Join the
|
||||
attribute against the **file mode** (`git ls-files -s`, mode `120000`) and verify
|
||||
the index blob really begins `\0GITCRYPT\0`. Report encrypted / symlinked /
|
||||
scanned as three separate numbers and assert they sum — encrypted and symlinked
|
||||
blobs are *skipped*, not certified clean.
|
||||
|
||||
**Self-test two-sided, and abort if it cannot discriminate.** Require a synthetic
|
||||
positive to fire AND a negative to stay silent before trusting any zero. Keep the
|
||||
fixtures in *structurally separate buffers*: put a quoted and an unquoted probe in
|
||||
one buffer and the quote terminates the run, handing the bare token to the weak
|
||||
extractor and making it look as strong as the union — a self-test artifact that
|
||||
has already fooled an agent here. And never gate on `$?` when the tool has a
|
||||
lock-skip or no-op path that also exits 0; judge the reported line.
|
||||
|
||||
**Row-gone is not bytes-gone.** Measured, same sentinel drawer: after
|
||||
`delete_by_source` the row count went 1 -> 0 in *both* the FTS content table and
|
||||
`embedding_metadata`, while the raw byte count stayed 4 -> 4 — sqlite does not
|
||||
zero freed pages, so the payload sits in free space until `VACUUM`. Deletion
|
||||
effectiveness is therefore *two* numbers, and each direction has a trap: one
|
||||
aggregate figure reported as "erased" has only measured "unretrievable", while a
|
||||
raw byte scan used as the acceptance gate reads a CORRECT, complete deletion as a
|
||||
failure. (Note how this was measured: the blocker was never a better instrument,
|
||||
it was the subject — file your own disposable sentinel and delete that, instead
|
||||
of testing deletion on real data.)
|
||||
|
||||
## 7. Choosing scopes: derive them from measured consumers
|
||||
|
||||
Before creating a replacement token, find out what actually uses it:
|
||||
|
||||
```sh
|
||||
git -C <repo> remote get-url origin # ssh:// ? then git needs NO token
|
||||
git config --global --list | grep -iE 'credential|insteadof' # and no helper?
|
||||
grep -rhoE 'api/v1/[A-Za-z0-9/{}$_.-]+' <consumers> | sort -u # exact routes
|
||||
grep -rhoE '\-X [A-Z]+' <consumers> # any writes?
|
||||
```
|
||||
|
||||
Real outcome here: git used SSH keys throughout, and the token's only consumer
|
||||
read three CI-run routes with `GET`. So `repository: Read` and nothing else
|
||||
replaced two admin tokens. **Scoping shrinks the blast radius of the next leak
|
||||
far more than any redaction pipeline does** — a read-only token in a transcript
|
||||
is a hygiene event, not an instance compromise.
|
||||
|
||||
Then prove the scope with an acceptance suite that declares expectations first:
|
||||
must-work routes → `200`; `/admin/*`, `/user`, `/user/repos` → `403`.
|
||||
|
||||
## 8. What rotation does *not* fix
|
||||
|
||||
- **A cleartext channel.** If the endpoint is `http://`, the *new* token is
|
||||
exposed identically from first use. Raise TLS separately.
|
||||
- **Git history.** A secret committed and pushed cannot be fixed by any store or
|
||||
palace operation — it needs rotation *and* history surgery.
|
||||
- **Agent-authored content.** Stage-write redactors see transcripts only, never
|
||||
`add_drawer` / `checkpoint` / `diary_write` output. Never type a secret into
|
||||
the palace yourself; nothing downstream will catch it.
|
||||
- **Plaintext/encrypted drift.** Gitignored plaintext `.env` files go stale while
|
||||
`.env.age` moves on, so old values linger on disk (and in backups) long after
|
||||
rotation. They are a common source of "mystery" fingerprints in a census.
|
||||
|
||||
## 9. This fleet's secret store (verify, do not assume)
|
||||
|
||||
- All `*.env.age` live in **one** repo: `joakimp/docker-compose-repo`. `myconfigs`
|
||||
has none.
|
||||
- Every `.age` file has **one X25519 recipient** — a single key tracked in
|
||||
`myconfigs` under git-crypt. Unlocking git-crypt therefore decrypts the entire
|
||||
fleet's secrets, including hosts you have no access to. The age layer adds no
|
||||
isolation beyond git-crypt.
|
||||
- Flow: `./fetch-secrets.sh <host>` (decrypt → `.env`) → edit → `./encrypt-secrets.sh <host>`
|
||||
→ commit → push → `docker compose up -d --force-recreate`.
|
||||
- **Always pass the host argument** to `encrypt-secrets.sh`. Bare, it walks the
|
||||
whole tree and re-encrypts every `.env` it finds, re-nonced, including stale
|
||||
ones — silently rolling back other hosts' secrets.
|
||||
- After any re-encrypt, check the header still shows exactly **one X25519
|
||||
recipient**; a hand-rolled `age -r` locks the rest of the fleet out, and the
|
||||
failure only appears on another machine, later.
|
||||
@@ -41,6 +41,21 @@ Run these immediately when a session begins, before responding to the user:
|
||||
mempalace_kg_query(entity="<project_or_person>")
|
||||
```
|
||||
|
||||
4. **Check your mailbox.** Just run it — an empty result is a fine answer and
|
||||
costs one call. Do not try to decide first whether coordination "applies to
|
||||
you"; that test is what used to be wrong here (see *Cross-Machine
|
||||
Coordination* below):
|
||||
```
|
||||
mempalace_event_list(to_agent="<harness>@<device>", status="open")
|
||||
```
|
||||
This is a candidate list, not a to-do list — `status` never changes after an
|
||||
event is written, so finished asks keep matching. Subtract the ones you have
|
||||
already answered using the rule in *What you actually owe*, below.
|
||||
Another machine may have asked you something, or corrected something you are
|
||||
about to rely on. This costs one call and is the only way you will find out:
|
||||
nothing pushes an event into your session unless your bridge delivers it for
|
||||
you, and if it does you will already have seen it before reading this.
|
||||
|
||||
Do NOT announce this to the user. Just do it silently to orient yourself.
|
||||
|
||||
### Temporal grounding — compute time deltas, don't guess
|
||||
@@ -79,6 +94,79 @@ mempalace_search(query="<keywords>", wing="<project>")
|
||||
|
||||
**Never guess about facts that might be in the palace.** Wrong is worse than slow. Say "let me check" and query.
|
||||
|
||||
#### Search Before You *Probe*
|
||||
|
||||
The rule above covers **questions**. This one covers **actions** — and it is the one
|
||||
that actually gets skipped, because mid-task the impulse is to go and *look* rather
|
||||
than to remember. The palace is a **fleet** record: another machine's agent has
|
||||
usually already paid the cost of discovering how this environment is wired, and its
|
||||
notes include the corrections that came afterwards, which a fresh probe cannot show
|
||||
you.
|
||||
|
||||
**Before you SSH somewhere to find out how it is set up, enumerate infrastructure,
|
||||
or derive a deployment — search.** Concrete triggers, all meaning *search first*:
|
||||
|
||||
- about to run `ssh <host> …`, `docker ps`, `systemctl list-units`, `ip addr` to
|
||||
discover how something is deployed or connected
|
||||
- about to establish topology: which hosts/runners/services exist, where they live,
|
||||
which of them can reach which
|
||||
- about to conclude "this isn't documented anywhere" or "there's no way to know"
|
||||
- about to assert an environment fact you learned **earlier in this same session**
|
||||
|
||||
**That last trigger is the sharp edge.** A compacted session summary is lossy by
|
||||
design, and a belief you formed 40 turns ago may already be *retracted* in the
|
||||
palace by another machine. Trusting your own context over the shared record is how a
|
||||
withdrawn claim gets re-published as fact.
|
||||
|
||||
Search broadly before narrowing — fleet knowledge often sits in another machine's
|
||||
wing, or inside a mined conversation, not where you would file it yourself:
|
||||
|
||||
```
|
||||
mempalace_search(query="<topic> <host> <mechanism>") # no wing filter first
|
||||
mempalace_search(query="…", wing="<likely-wing>") # then narrow
|
||||
```
|
||||
|
||||
Two or three searches cost seconds. Re-deriving infrastructure costs minutes **and
|
||||
can be wrong**: a probe shows one host's present state, while the palace records
|
||||
intent, history, and what was already disproved.
|
||||
|
||||
> **Worked example (real, 2026-08-25).** An agent evaluating whether to add an ARM
|
||||
> CI runner probed hosts directly instead of searching. It concluded "the runner
|
||||
> lives on synlig" — there are **four** — and that "synlig is on the home LAN" —
|
||||
> it is an OpenStack VM with a public floating IP that cannot reach the home LAN at
|
||||
> all. Both facts were already in the palace, the second one as an **explicit
|
||||
> retraction of the very same mistake** made weeks earlier. The palace also held
|
||||
> the runner labels and the deliberate `capacity: 1` setting, which the probe never
|
||||
> revealed. Cost: a wrong recommendation written into the palace twice, then
|
||||
> corrected twice.
|
||||
|
||||
|
||||
**A search that comes back empty is not an answer — least of all about recent work.**
|
||||
Semantic search is weakest exactly where the fleet record is freshest: a drawer filed
|
||||
minutes ago is unranked against a keyword-shaped query, and the drawer you most need
|
||||
is *by construction* the newest one, because the other machine files its release,
|
||||
handoff and correction drawers at the **end** of its session. So a single miss proves
|
||||
nothing. **If the work is 0-2 days old and the first search looks stale or empty,
|
||||
enumerate before concluding:**
|
||||
|
||||
```
|
||||
mempalace_list_drawers(wing="<wing>", since="<today>") # or room=, or no filter
|
||||
mempalace_diary_read(agent_name="<you>", wing="<wing>") # the other machine's handoff
|
||||
```
|
||||
|
||||
Enumeration is exact where embeddings are probabilistic. Treat "I searched and found
|
||||
nothing" as a hypothesis you have not yet tested, and never as licence to go probing.
|
||||
|
||||
> **Worked example (real, 2026-08-25, same fleet as above).** An agent asked to
|
||||
> orient on an in-flight release *did* search first — `"v1.8.6 release run 579
|
||||
> Docker Hub verification"` — and got back only v1.6.4 / v0.78.0 era hits, because
|
||||
> the release drawer it needed was **58 seconds old**. It accepted the miss and went
|
||||
> off to probe Docker Hub and the Gitea API. The user had to prompt "maybe there is a
|
||||
> note in mempalace"; `list_drawers(wing="pi-devbox", since=<today>)` then returned
|
||||
> the drawer immediately, along with the diary entry naming the exact open item. The
|
||||
> rule above was present and correct in this very file at the time — the failure was
|
||||
> not knowing to *retry differently* after a bad first hit.
|
||||
|
||||
#### Mine New Projects
|
||||
|
||||
When working on a new codebase for the first time:
|
||||
@@ -267,26 +355,289 @@ mempalace_kg_invalidate(subject="...", predicate="...", object="...", ended="<to
|
||||
mempalace_kg_add(subject="...", predicate="...", object="...", valid_from="<today>")
|
||||
```
|
||||
|
||||
## Cross-Machine Coordination — the logstream
|
||||
|
||||
The palace stores what you *know*. The logstream (`mempalace_event_*`,
|
||||
`mempalace_artifact_*`) carries what you want to *say to another agent* —
|
||||
delegation, review, patch handoff, retraction. It is the only channel on which
|
||||
another machine can reach you.
|
||||
|
||||
**Does this apply to you at all? Do not use `mempalace_mesh_peers` to decide.**
|
||||
It answers a different question than it appears to. A shared palace can be
|
||||
*hub-and-spoke* — many machines as thin clients of one central replica — and
|
||||
then `mesh_peers` reports `peers: []` because there are no peer *replicas*,
|
||||
even while four machines are actively writing to the same log. Measured on this
|
||||
fleet: `peers: []`, one replica authoring every event from every machine. An
|
||||
earlier version of this section told you to read `mesh_peers` and skip the
|
||||
mailbox when it came back empty, which disabled the mailbox on precisely the
|
||||
fleet it was written for.
|
||||
|
||||
The honest discriminators, cheapest first: **just run the mailbox query** (empty
|
||||
is a fine answer); check whether `MEMPALACE_REMOTE_URL` is set, which is what
|
||||
actually selects a shared palace; or look for any event whose `from_agent` is
|
||||
not you. On a solitary palace the event tools still work — you are writing to
|
||||
yourself and your mailbox stays empty. That is not a fault to debug.
|
||||
|
||||
**It is a durable log, not a bus — nobody is "listening".** Events are appended
|
||||
and persist; there is no subscription, no delivery window, and nothing is lost
|
||||
by being offline when one is written. A message waits indefinitely for you, and
|
||||
your reply waits just as patiently for a sender who has since gone away. Machines
|
||||
in a fleet are rarely awake at the same time, which is exactly why this is a log
|
||||
and not a chat.
|
||||
|
||||
**Agent name is the only identity the log has.** Depending on deployment, every
|
||||
client may share one `origin_replica` — on the fleet this skill was written for,
|
||||
all machines are thin MCP clients of a single central replica, so `origin_replica`
|
||||
is identical for every event and cannot tell two machines apart. `from_agent` /
|
||||
`to_agent` carry the whole distinction, which is why the `<harness>@<device>`
|
||||
stamping in *Provenance is stamped for you* is load-bearing here and not mere
|
||||
tidiness.
|
||||
|
||||
### Reading your mailbox
|
||||
|
||||
```
|
||||
mempalace_event_list(to_agent="<harness>@<device>", status="open")
|
||||
```
|
||||
|
||||
- `to_agent=<you>` **also matches `*` broadcasts**, so one call covers both. No
|
||||
second query needed.
|
||||
- `status="open"` narrows the mailbox to what a sender *said was an ask at the
|
||||
time of writing* — that is all it can do. It is a good first filter (on a real
|
||||
stream it cut 5 events to 2), but it is **not** a list of what you owe, and it
|
||||
never shrinks as you work. Treating it as owed-ness is the mistake this
|
||||
section previously made: an earlier draft cited "5 unfiltered, exactly 1
|
||||
filtered — the one that needed a reply" as proof the filter tracked
|
||||
obligation. It did not. That single result was an event which had *already
|
||||
been acked* half an hour earlier; the filter looked decisive only because the
|
||||
stream happened to contain one directed `open` event. **Unfiltered mailboxes
|
||||
train you to ignore them — and so does a filter that keeps showing you
|
||||
finished work.**
|
||||
- To resume where you left off, use `since_event_id`, **never**
|
||||
`since_created_at`. A timestamp cursor permanently skips an event that synced
|
||||
in late — it is a time window ("what happened today"), not a cursor.
|
||||
- Read `metadata` before acting: senders put the load-bearing specifics there
|
||||
(which host verified what, which run failed, what a change retracts).
|
||||
|
||||
### The ack contract — the sender declares whether a reply is owed
|
||||
|
||||
An obligation you never agreed to is noise, so the sender states it:
|
||||
|
||||
| Sender writes | Means | Recipient owes |
|
||||
|---|---|---|
|
||||
| `to_agent="<specific agent>"` + `status="open"` | an ask | an ack or a reply (the event itself keeps matching forever — see below) |
|
||||
| `to_agent="*"` (any status) | broadcast FYI | nothing |
|
||||
| any other status (`ready`, `applied`, `blocked`, …) | a statement of fact | nothing |
|
||||
|
||||
**That table says what you *owe*. Delivery is stricter, and the difference bites:
|
||||
the mailbox is an obligation channel, not a news channel.** Mailbox candidates are
|
||||
drawn with `status="open"`, so an event carrying any **terminal** status
|
||||
(`applied`, `superseded`, `failed`, `blocked`) is never a candidate — *whoever it
|
||||
is addressed to*. A `task.reply` written to a named machine to share a finding is
|
||||
delivered to nobody, ever, and neither is any `event_ack`. It sits in the log
|
||||
until somebody reads the log.
|
||||
|
||||
So the most natural inter-machine message — *"here is something you should
|
||||
know"* — is exactly the shape that gets no delivery. Pick deliberately:
|
||||
|
||||
| You want the peer to… | Write |
|
||||
|---|---|
|
||||
| **do something**, and you need it tracked until done | directed `status="open"` ask, with a `correlation_id` |
|
||||
| **know something**, no response needed | terminal-status event **plus a drawer** — the drawer is what actually reaches them, via search |
|
||||
|
||||
What does **not** work is a terminal report plus an expectation of attention.
|
||||
Measured 2026-08-26: a detailed report addressed to `pi@<peer>` with
|
||||
`status="applied"` went unread for two and a half hours until the operator quoted
|
||||
the event id by hand, with the mailbox working correctly the whole time. Full
|
||||
mechanism in the toolkit's `docs/rfc-003-coordination-log.md` §7.12.
|
||||
|
||||
One more timing fact, because it looks like negligence and is not: a delivered
|
||||
ask is queued into the agent's **next turn** (`deliverAs: "steer"`, deliberately
|
||||
no `triggerTurn`), and the poll fires when the agent is *idle*. Between delivery
|
||||
and the next turn no inference runs, so **a human starting a turn is the
|
||||
trigger** (§7.11). An agent that "has not reacted" has usually not been running.
|
||||
|
||||
Ack with `mempalace_event_ack(event_id=…, from_agent="<you>", status=…)`. It
|
||||
**appends a new event** and never mutates the original; the correlation id is
|
||||
copied for you, and `metadata.ack_of` is set to the event you answered.
|
||||
|
||||
**Claiming, and what it does not do.** `status="claimed"` announces that you have
|
||||
picked work up. Nothing requires it — a directed open ask owes "an ack *or* a
|
||||
reply", and finishing the work is a complete answer. Do it anyway when the work is
|
||||
long or the machine is unreliable, because it is the only thing that later
|
||||
distinguishes *nobody started this* from *someone started and their container
|
||||
died mid-task*. Be clear about its limits, both of which follow from candidacy
|
||||
requiring exactly `status="open"`:
|
||||
|
||||
- **It does not notify the requester.** `claimed` is not `open`, so a claim is no
|
||||
more deliverable than a finished report is (see the delivery table above). Its
|
||||
reader is whoever pulls the log.
|
||||
- **It does not quiet your own mailbox.** The ask stays owed until a *terminal*
|
||||
event of yours joins it, so a claimed-then-silent thread keeps resurfacing —
|
||||
correctly.
|
||||
|
||||
Prefer a prompt terminal reply over a claim plus a long silence; claim *in
|
||||
addition*, when the gap between pickup and finish is where a machine might die.
|
||||
|
||||
#### What you actually owe — derive it, do not read it off `status`
|
||||
|
||||
The log is append-only and `status` is written **once**, so it is an honest
|
||||
statement about an item *at the moment it was written* and nothing more. It is
|
||||
not mutable state, and asking it to carry mutable state is what breaks:
|
||||
acking appends a new event and changes nothing about the old one, so **a
|
||||
directed `open` event matches your mailbox query forever, answered or not.**
|
||||
Nothing is ever "dismissed" — which also means a deferred ask cannot be
|
||||
accidentally lost, only that you must compute what is outstanding:
|
||||
|
||||
```
|
||||
candidates = mempalace_event_list(to_agent="<you>", status="open")
|
||||
mine = mempalace_event_list(from_agent="<you>")
|
||||
```
|
||||
|
||||
A candidate is **answered** when one of your own events
|
||||
|
||||
1. has a **higher `seq`** than the candidate, and
|
||||
2. joins to it — `metadata.ack_of == candidate.id` (exact, written for you by
|
||||
`event_ack`) or the same `correlation_id` (the fallback), and
|
||||
3. carries a **terminal** status: `applied`, `superseded`, `failed`, `blocked`.
|
||||
|
||||
Everything else is still owed. Two calls, constant cost.
|
||||
|
||||
**Compare `seq`, never `created_at`** — the same reason you resume with
|
||||
`since_event_id`. Without the ordering test, one terminal reply would suppress
|
||||
every later ask on the same `correlation_id` for good; verified on a live thread
|
||||
where a `ready` reply at `seq` 16 sits *before* the request at `seq` 17 that it
|
||||
obviously cannot have answered.
|
||||
|
||||
**On a real mesh, compare `hlc` instead.** `seq` is *replica-local*: it equals
|
||||
`origin_seq` today only because a single replica authors events for every
|
||||
machine. Enrol a second replica and a late-syncing peer event gets a late local
|
||||
`seq`, so two replicas can order the same pair differently and derive different
|
||||
owed-sets from the same log. Every event already carries `hlc`
|
||||
(`<millis>-<counter>-<replica_id>`), which is total and causally consistent.
|
||||
So: compare `seq` while `mempalace_mesh_peers` reports no peers, `hlc` once it
|
||||
reports any, and `created_at` never. (This is a legitimate use of `mesh_peers` —
|
||||
choosing an ordering key — not the discredited gate on *whether* to read your
|
||||
mailbox at all.)
|
||||
|
||||
**The failure directions are not symmetric, which is why this is safe to get
|
||||
slightly wrong.** Local-`seq` skew can make an already-answered item *resurface*
|
||||
as owed: noise, self-correcting, and visible. A timestamp comparison can
|
||||
*suppress an unanswered ask forever*: silent and permanent. So if you ever see an
|
||||
item you know you answered come back, do **not** "fix" it by reaching for
|
||||
`created_at` — you would be trading the safe failure for the dangerous one.
|
||||
|
||||
This also supplies the "taken, not finished" state that looked missing:
|
||||
`claimed` and `ready` are deliberately **not** terminal, so work you have picked
|
||||
up keeps resurfacing until you close it out. No extra convention, no new field.
|
||||
|
||||
Two consequences worth internalising:
|
||||
|
||||
- **"Seen, not doing it" is a legitimate ack** — `status="blocked"` or
|
||||
`"superseded"` plus the reason. Silence is not, and it is not merely rude:
|
||||
with no terminal event of yours to join to, the ask stays in the owed set
|
||||
indefinitely and there is nothing anyone can do about it from the other end.
|
||||
- **Nothing expires, and it should not.** An `open` with no terminal reply is
|
||||
still live by definition, and the finished threads are valuable history. If
|
||||
content is genuinely perishable ("do not push to main for the next hour"), say
|
||||
so in `metadata.expires_at` — metadata is stored verbatim — and honour it as a
|
||||
hint when reading. An old `open` that the derivation still counts as owed is a
|
||||
signal, not garbage: it means somebody asked and nobody answered.
|
||||
|
||||
### Writing to another machine
|
||||
|
||||
- **Address the stamped name you actually saw** in a `from_agent` field, e.g.
|
||||
`pi@tor-ms22`. A bare `pi` reaches nobody's mailbox once stamping is live, and
|
||||
older events in the log still carry bare names — do not copy them.
|
||||
- **The rule runs in reverse too: what you put in YOUR OWN `from_agent` decides
|
||||
where every reply to your event goes.** Nothing stops you writing a synthetic
|
||||
or borrowed identity there, and a reply is always addressed back to exactly
|
||||
that string — so if no live session ever runs as it, the reply is stored,
|
||||
searchable, and delivered to no one. Measured cost: a directed ask sent under
|
||||
a synthetic sender got two correct replies, one of them an urgent security
|
||||
finding, and both sat unread for ~2h20m because nobody's mailbox was that
|
||||
identity (RFC 003 §7.13). Authoring under a synthetic name is fine for a
|
||||
deliberate control experiment — this fleet does it on purpose — but then
|
||||
**name the real identity to reply to inside the body**, because the address
|
||||
line is not a safe place to also carry provenance.
|
||||
- **Use `status="open"` only when you truly need an answer.** It places an
|
||||
obligation on another machine.
|
||||
- **Never broadcast an ask.** `to_agent="*"` + `status="open"` obliges everyone
|
||||
and therefore no one.
|
||||
- **Always set a `correlation_id` on a directed `open`,** and reply with the
|
||||
same one. It is not just for reconstructing a conversation later: it is the
|
||||
join the owed-set derivation depends on. An uncorrelated ask can only ever be
|
||||
closed by an `event_ack` (which sets `ack_of` for you) — a plain reply cannot
|
||||
be matched to it at all.
|
||||
- **Corrections are new events, never edits.** Say explicitly what you retract
|
||||
and name the id — drawer or event — that carried the withdrawn claim.
|
||||
- **Put a retraction where the reader will look.** A *directed open ask* reaches a
|
||||
live agent's mailbox; a **terminal-status event reaches no mailbox at all**, and
|
||||
a *drawer* is what a future semantic search finds. If you filed advice as a
|
||||
drawer and later withdraw it, file the withdrawal as a drawer too — otherwise
|
||||
the next agent finds your original confident advice and no trace of the
|
||||
correction. (This is a real incident, not a hypothetical.)
|
||||
- **Hand over exact content as an artifact**, not prose: `mempalace_artifact_put`
|
||||
or `mempalace_patch_submit` store bytes with a sha256, and the event references
|
||||
the id. Never paste a diff into a body and hope it survives.
|
||||
- **Waiting on a specific reply?** `mempalace_event_wait` blocks with backoff —
|
||||
do not poll `event_list` in a loop. A timeout there is a normal result, not an
|
||||
error.
|
||||
|
||||
## Palace Structure
|
||||
|
||||
### Wings
|
||||
|
||||
Wings are top-level categories, typically one per project or domain:
|
||||
- Named after the project directory (e.g., `cli_utils`, `opencode_devbox`)
|
||||
- Agent diaries live in `wing_<agent_name>` (e.g., `wing_orchestrator`, `wing_pi`)
|
||||
Wings are top-level categories, typically one per project or domain.
|
||||
|
||||
#### Multi-harness palace
|
||||
**NAMING CONVENTION — decided 2026-09-06 by Joakim: bare project names, no `wing_`
|
||||
prefix.** `home-network`, `pi-devbox`, `mempalace-toolkit` — *not* `wing_pi-devbox`. The
|
||||
mass is already there (`pi-devbox` 2061 drawers vs `wing_pi-devbox` 25), and a prefix
|
||||
present on some wings and absent on others turns every read into a guess about which
|
||||
spelling holds the content.
|
||||
|
||||
A single palace can be fed by multiple coding-agent harnesses. On this machine the palace is shared between **opencode** and **pi** (Mario Zechner's pi-coding-agent). Implications:
|
||||
- Named after the project directory or domain (e.g., `cli_utils`, `home-network`)
|
||||
- **Always pass `wing` explicitly to `diary_write`.** Omitting it defaults to
|
||||
`wing_{agent_name}`, which mints or feeds a *parallel* wing — this tool default, not
|
||||
anyone's sloppiness, is the mechanism that produced the drift. Measured harm
|
||||
(2026-09-06, `pi@mbp-m1-2020`): a diary entry written with `agent_name=pi` and no
|
||||
`wing` landed in `wing_pi` while that agent's history lives in `pi-devbox`, so a
|
||||
`diary_read` scoped to `pi-devbox` showed **no trace of it**. A wing-scoped read that
|
||||
silently returns an incomplete history is the worst failure mode a memory store has.
|
||||
- **Legacy `wing_*` wings are frozen and documented, not renamed.** `wing_conversations`
|
||||
(written by the session feeders), `wing_pi`, `wing_pi-devbox`, `wing_pi-tor-ms22`,
|
||||
`wing_pi-devbox-emb7kj`, `wing_mempalace`, `wing_orchestrator`, `wing_code` all still
|
||||
hold real content. **When searching for history, check both spellings** — this is the
|
||||
practical cost of the drift and it does not go away by decree.
|
||||
- If a migration is ever done, the acceptance criterion must be at the **relationship**
|
||||
level: chunk ids still resolve to their parent, and `diary_read` returns the same entry
|
||||
set before and after. Per-wing drawer counts can look correct while the relationships
|
||||
underneath are broken, because a count query never touches them.
|
||||
|
||||
#### Shared palace: multiple harnesses, and possibly multiple machines
|
||||
|
||||
A single palace can be fed by multiple coding-agent harnesses, and — when
|
||||
`MEMPALACE_REMOTE_URL` points at a central palace — by multiple *machines*. On
|
||||
this machine the palace is shared between **opencode** and **pi** (Mario
|
||||
Zechner's pi-coding-agent). Implications:
|
||||
|
||||
- **`wing_conversations` mixes sources.** Both harnesses' session feeders write into the same wing. To tell them apart, look at the `source_file` metadata on each drawer:
|
||||
- `pi_<uuid>.jsonl` → pi session
|
||||
- `<slug>_ses_<id>.jsonl` → opencode session
|
||||
- The first chunk of each session also carries a `| source: opencode` or `| source: pi` marker in the synthetic header line.
|
||||
- **Other wings may belong to other harnesses.** For example `wing_pi` is pi's diary, not opencode's. Don't assume every diary entry was written by you — check `agent_name` on the entry.
|
||||
- **Session feeders run on different schedules.** Pi sessions are fed Tue 03:00, opencode sessions Mon 03:00. Recent sessions from either harness can lag the palace by up to a week, so absence-of-evidence in `wing_conversations` is not evidence-of-absence for recent work.
|
||||
- **Session feeders run on different schedules.** Pi sessions are fed Tue 03:00, opencode sessions Mon 03:00 (launchd `Weekday`: `0`/`7`=Sunday, `1`=Monday, `2`=Tuesday — misreading this by one day is easy). Recent sessions from either harness can lag the palace by up to a week, so absence-of-evidence in `wing_conversations` is not evidence-of-absence for recent work.
|
||||
- **Reading another harness's diary is useful.** When orienting after a gap, `mempalace_diary_read agent_name=pi` (or whichever sibling agent has been active) often gives a fresher picture than waiting for the conversations feeder to catch up.
|
||||
|
||||
When the palace is **central** (shared across machines), these further things apply:
|
||||
|
||||
- **Check which machine a conversation came from.** Transcripts are fed per device, so `source_path` reads `…/mempalace-feed/<device>/pi_<uuid>.jsonl` while the displayed `source_file` is only the basename. One search can legitimately return hits from several machines at once — look at the device segment before attributing a decision to *this* project.
|
||||
- **Provenance is stamped for you — leave it alone.** Drawers carry `device` and `agent_kind` metadata (plus `device_source`/`agent_kind_source` recording *how* each was determined, so an inference is never mistaken for a fact). You do **not** set these, and you no longer set `added_by` either: the pi bridge defaults the writer field to `<harness>@<device>` on `add_drawer`/`checkpoint`/`mine`/`event_append`/`artifact_put`, and prefixes diary entries with `HOST:<device>|`, from host-supplied `$MEMPALACE_PI_DEVICE`. RFC 001 §7.3.2 ranks "agent stamps it via a skill instruction" as the *worst possible* place for exactly the reason you would expect — it is per-call boilerplate that gets forgotten, and it did: the agent who wrote the previous version of this bullet then filed its own provenance drawer as `added_by=checkpoint`. **Confirm the bridge in your image actually stamps before trusting it:** the extension is baked at image build time, so a container on an image older than the stamping commit (pi-devbox < v1.8.7) stamps nothing while still satisfying both gates — the env vars are set and the code is simply absent. Check with `grep -c MEMPALACE_PI_DEVICE "$(readlink -f ~/.pi/agent/extensions/mempalace.ts)"`; zero means keep passing `added_by="<harness>@<device>"` and a manual `HOST:<device>|` diary prefix until the container is recreated on a newer image. Two things remain yours: pass `source_drawer_id` on `kg_add` (triples have no provenance field, so that pointer is the only path back to a device), and pass an explicit `added_by` **only** when deliberately filing on behalf of another device — and when you do, it **must** be `<harness>@<device>`. A bare nickname (`pi-devbox-claude`) has no `@device` to parse, so `agent_at_device` cannot attribute it and the drawer is unattributable *by rule*, not by lag: it survives every future stamp run with no `device`, and on a shared palace a device-less drawer is one nobody can later scope, audit or clean up per machine. Measured 2026-09-06: 11 drawers on `tor-ms22` were filed this way — including the credential rows, i.e. exactly where "which machine measured this?" matters most — by an agent that had passed its own chosen nickname on every call. Its *diary* entries escaped, because `HOST:<device>|` in the AAAK text recovers the device. **Diaries self-heal; plain drawers do not.** The safest habit is the one above: pass nothing and let the bridge stamp. Never invent values for `device`/`agent_kind`/`origin_device` — a fabricated value is worse than a blank, because it silently corrupts a future merge.
|
||||
- **Metadata is invisible to search — so check the text, not the fields.** `search` results are built from a fixed key list and `diary_read` returns content, so neither ever shows `device`/`added_by`. Only `mempalace_get_drawer` reveals them. This is why diary entries carry an in-text `HOST:<device>` marker: it is the only attribution a reader actually sees. **A diary entry with no `HOST:` marker predates the convention and may be from any machine — do not assume it is this one's history.**
|
||||
- **Mined drawers carry the MINE date, not the session date.** When history is imported, or re-mined on the palace host, `filed_at`/`created_at` is the *import* time — so sorting by them does not give chronological order. Real session time is recoverable from the UUIDv7 in `pi_<uuid>.jsonl`: the first 12 hex digits are milliseconds since the epoch (and UUIDv7 sorts lexicographically in time order, so a plain filename sort is already chronological). Agent-authored drawers and diaries have no such backdoor — for those `filed_at` is the only chronology, which is why it must never be restamped.
|
||||
- **Beware the timezone mismatch when you combine those.** Palace `filed_at`/`created_at` are naive timestamps in the palace host's local time, while a UUIDv7 decodes to UTC. Comparing them directly introduces a silent offset (2 h for a CEST host). Normalise before drawing conclusions about ordering.
|
||||
- **`agent_name` is not device-scoped.** `mempalace_diary_read(agent_name="pi")` returns *every* machine's `pi` diary, interleaved. Read the entry before assuming it is your own history — and note that a container cannot tell you which machine it is on (`hostname` is a docker hash, `$DEVBOX_HOST_ALIAS` is generic). `$MEMPALACE_PI_DEVICE` is the cheap answer; `ssh -F ~/.ssh-local/config host hostname` is the independent one.
|
||||
- **One writer, no queue.** A concurrent mine returns a structured `already-running` error rather than waiting its turn, and one large mine can make the palace unresponsive to every client for minutes. After another client's mine, call `mempalace_reconnect` to see the new drawers. A client-side timeout is not evidence of failure — verify before retrying, or you file a duplicate.
|
||||
|
||||
### Rooms
|
||||
|
||||
Rooms are aspects within a wing:
|
||||
@@ -318,9 +669,16 @@ Entity-relationship triples with temporal validity. Query with `mempalace_kg_que
|
||||
## Anti-Patterns
|
||||
|
||||
- **Don't guess when you can search.** If a question touches past work, search first.
|
||||
- **Don't probe what the fleet already knows.** Before SSH-ing into a host, enumerating infrastructure, or deriving how something is deployed, search the palace. A probe reveals one host's present state; the palace holds intent, history and prior corrections — including the ones that contradict what you are about to conclude.
|
||||
- **Don't trust this session's context over the palace.** A compacted summary is lossy, and another machine may have corrected the fact since. Verify load-bearing environment claims against the shared record before acting on them.
|
||||
- **Don't take one empty search as proof the palace is silent.** Fresh drawers rank worst, and the drawer that matters is usually the newest one. For anything 0-2 days old, enumerate with `mempalace_list_drawers(since=…)` and read the other machine's diary before you go and probe.
|
||||
- **Don't infer elapsed time from session or container boundaries.** A restart isn't a new day. Compare the actual timestamp (`timestamp` / `created_at`) against the current date/time before saying "yesterday", "last week", etc.
|
||||
- **Don't skip the diary.** A session without a diary entry is a session forgotten.
|
||||
- **Don't summarize drawer content.** File verbatim — the embedding model needs the original words.
|
||||
- **Don't mine .git directories or node_modules.** The CLI miner respects .gitignore by default.
|
||||
- **Don't create duplicate drawers.** Use `mempalace_check_duplicate` before adding manually.
|
||||
- **Don't treat the palace as a task list.** It's for knowledge and context, not todos.
|
||||
- **Don't broadcast an ask, and don't leave one unanswered.** On a shared palace, `to_agent="*"` + `status="open"` obliges every machine and therefore none of them. And don't expect acking to tidy your mailbox: `status` is immutable, so the event keeps matching either way — what a terminal reply buys you is that the *derived* owed set (see *What you actually owe*) stops counting it. Leave asks unanswered and that set only grows, until everyone learns to stop looking. "Seen, not doing it" is a complete answer — silence is not.
|
||||
- **Don't assume you would have heard.** Nothing pushes another machine's message into your session. If you did not run the mailbox query at wake-up, a correction addressed to you by name can sit unread while you confidently rebuild the thing it warned you about.
|
||||
- **Don't author an ask under an identity nobody runs as, including your own throwaway labels.** The failure is symmetric to the one above: it is not that you missed a message, it is that nothing could ever have delivered the reply to you, because you addressed it at a name instead of an agent. If you must use a synthetic sender for a control or an experiment, say inside the body who should actually receive the reply.
|
||||
- **Don't invent provenance metadata, and don't hand-stamp it either.** An earlier version of this list told you to set `added_by="<harness>@<device>"` by hand; that instruction has been withdrawn, because RFC 001 §7.3.2 places provenance at the client/server boundary and the pi bridge now does it uniformly (see *Provenance is stamped for you* above) — but the withdrawal only holds where the bridge is live, so run the one-line check in that bullet first; on an older image hand-stamping is still the only signal a hand-filed drawer gets. DO NOT invent values for the palace's own metadata fields (`device`, `agent_kind`, `origin_device`): those are stamped by infrastructure that also records *how* each was determined, and a fabricated value is worse than none because it silently corrupts a future merge. DO pass `source_drawer_id` on `kg_add`. And never put a machine name in a diary's `agent_name` — it becomes the wing name and hides your entries from `diary_read`.
|
||||
|
||||
@@ -130,6 +130,85 @@ are "command not found" there — you must spell out the underlying command.
|
||||
If a command "works in my terminal but not when the agent runs it," this alias
|
||||
gap is the first thing to suspect.
|
||||
|
||||
### A negative result is usually your own filter
|
||||
|
||||
**When you are about to report that something is absent, unreachable, or not
|
||||
running, the filter you wrote is the prime suspect — not the thing.** This
|
||||
environment produces false negatives cheaply, and they are convincing because
|
||||
the command "succeeded". Three real instances from one session, all wrong, all
|
||||
mine:
|
||||
|
||||
| Claim I made | Why it was false |
|
||||
|---|---|
|
||||
| "`tor-ms22` is not in the SSH config" | `grep … \| head -20` — the entry was at **line 454**. `~/.ssh/config` here is ~500 lines. |
|
||||
| "the Docker host has no `docker`" | non-interactive SSH `PATH` lacks `/usr/local/bin` (§2, §3). It was at `/usr/local/bin/docker`. |
|
||||
| "no ControlMaster is running" | pattern `ssh ` (trailing space) cannot match a master: those processes **rename themselves** to `ssh: <controlpath> [mux]`. |
|
||||
| "the credential is not in the palace" | scanned `embedding_metadata.string_value` only. Drawer **text** lives in `embedding_fulltext_search_content.c0`; 554k metadata rows proved nothing. |
|
||||
| "this token is dead — 401" | probed it against the **wrong issuer**. A 401 from an instance that never issued the credential is not evidence about the credential. |
|
||||
| "that host is unreachable, can't test" | tried ports 443 and 80. It was on **3000**, and the env var I already held (`GITEA_EGL_HOST`) stated the scheme and port. |
|
||||
| "this repo has no `## Unreleased` convention" | read `CHANGELOG.md` **once**, minutes after a release commit had renamed that section to a version heading. 33 commits touch `## Unreleased`. A snapshot cannot show you a cycle. |
|
||||
|
||||
Habits that would have caught all three:
|
||||
|
||||
```sh
|
||||
# don't cap the output of a search whose answer you don't already know
|
||||
grep -n -i -A6 'tor-ms22' ~/.ssh/config # not | head -20
|
||||
|
||||
# on the host, resolve the binary instead of trusting PATH
|
||||
ssh -F "$HOME/.ssh-local/config" mac 'command -v docker || ls /usr/local/bin/docker'
|
||||
|
||||
# match a process's ACTUAL argv, not the name you imagine
|
||||
ps -eo pid,etime,args | grep -Ei 'mux|mosh|ssh'
|
||||
|
||||
# to learn a repeating PROCESS or convention, read history, not the file. A
|
||||
# file's current content is one frame of a cycle, and the frame you happen to
|
||||
# catch may be the one where the thing you are looking for was just consumed.
|
||||
git log -S'## Unreleased' -- CHANGELOG.md # not `head -60 CHANGELOG.md`
|
||||
```
|
||||
|
||||
Absence has to be *earned*, so spend the extra command there.
|
||||
|
||||
### …and a positive result only proves what you *actually asked*
|
||||
|
||||
An earlier version of this section claimed "a positive result needs no such
|
||||
scepticism — it carries its own evidence." **That is false, and believing it
|
||||
cost a later session three more wrong findings.** A positive result is evidence
|
||||
about the question your command really posed, which may not be the question you
|
||||
meant. The failure is invisible precisely *because* the command succeeded.
|
||||
|
||||
| Claim | The command succeeded — at answering something else |
|
||||
|---|---|
|
||||
| "EGL git over SSH works" | `ssh git@gitea.egl.lan` greeted me as `joakimp`. `~/.ssh/config` had `Host gitea*` → `HostName gitea.jordbo.se`, so I authenticated **to the wrong instance**. The real EGL account is `ecsjper`. |
|
||||
| "the port config regressed" | compared `ssh -G` output against `2222` — a value produced by **my own earlier `-p 2222` flag**, not by the config. I reported the user's edit as a regression it never caused. |
|
||||
| "the CI runners authenticate with this token" | pure fabrication, contradicted by my own scan output already on screen. The runners use per-runner `REGISTRATION_TOKEN`. |
|
||||
|
||||
Two habits that actually catch this class, both cheap:
|
||||
|
||||
```sh
|
||||
# 1. ask which RULE captured your hostname before trusting any ssh result.
|
||||
# ssh_config is first-obtained-value-wins PER KEYWORD, not per block: a
|
||||
# specific block only wins the keywords it declares, so a later `Host gitea*`
|
||||
# still supplies HostName unless the specific block restates it.
|
||||
ssh -G git@thehost | grep -E '^(hostname|port|user|identityfile)'
|
||||
|
||||
# 2. state the expected result BEFORE running the check, and diff against it.
|
||||
# This is the single technique that separated the one verification that went
|
||||
# right (10/10, expectations declared per probe) from five that went wrong
|
||||
# (results interpreted after the fact, each time in the direction I expected).
|
||||
probe "/repos/.../actions/runs" 200 # must work
|
||||
probe "/admin/users" 403 # must be denied
|
||||
```
|
||||
|
||||
And the meta-observation, which is the reason this subsection exists: across all
|
||||
five errors, **not one was caught by re-reading my own reasoning.** Every one was
|
||||
caught by a second measurement that disagreed — the SSH lie surfaced only because
|
||||
the greeting said `joakimp` while a token probe minutes earlier had said
|
||||
`ecsjper`; the fabrication surfaced only because the user read my own output back
|
||||
to me. So the operational rule is not "be careful". It is: **for a load-bearing
|
||||
claim, produce a second measurement by a different route, and expect it to
|
||||
disagree.** If you cannot think of a second route, you do not yet have a finding
|
||||
— you have a hypothesis.
|
||||
|
||||
**`dscp`/`scp` with accented filenames on a macOS host.** macOS stores filenames
|
||||
in Unicode **NFD** (decomposed — e.g. `ä` is `a` + combining U+0308), while the
|
||||
string you type or paste is usually **NFC** (precomposed `ä`, U+00E4). The bytes
|
||||
@@ -155,6 +234,16 @@ entrypoint's `setup-lan-access.sh` writes a **writable SSH sidecar** at
|
||||
- A `Host *` block redirecting `ControlPath` into the writable `~/.ssh-local/cm`
|
||||
(because `~/.ssh` is typically bind-mounted **read-only**, so a master socket
|
||||
can't be created under it), plus `Include ~/.ssh/config`.
|
||||
- A **trailing** `Host *` block supplying `ControlMaster auto` + `ControlPersist
|
||||
10m` as a *default*. Position is the design: `ControlPath` sits **before** the
|
||||
`Include` (an override — the value in your own config points at read-only
|
||||
`~/.ssh` and cannot work here), while `ControlMaster` sits **after** it (a
|
||||
default — an explicit per-host `ControlMaster no`/`auto` in your own config
|
||||
still wins, because ssh_config is first-value-wins). **Force what is broken,
|
||||
default what is merely absent.** Without this, a target whose entry never
|
||||
mentioned `ControlMaster` opens a fresh TCP connection per `ssh` call, and an
|
||||
agent making a dozen calls in a few minutes can trip fail2ban or a CGNAT
|
||||
flow-table cap on the far end.
|
||||
- Aliases **`host` / `mac`** → `host.docker.internal` (user comes from
|
||||
`HOST_SSH_USER`) — i.e. SSH back into the Docker host.
|
||||
- On VM-backed hosts only: an **SSH-jump-via-host** block so the container can
|
||||
@@ -169,12 +258,48 @@ ssh -F "$HOME/.ssh-local/config" mac 'hostname; whoami' # reach the host
|
||||
ssh -F "$HOME/.ssh-local/config" <lan-peer> '…' # reach a LAN peer (if configured)
|
||||
```
|
||||
|
||||
**Always go through the sidecar, never `-F ~/.ssh/config`.** This is the single
|
||||
easiest way to break SSH from inside the container, and the failure actively
|
||||
misleads: the read-only path makes the master socket uncreatable, so
|
||||
multiplexing appears *impossible* rather than misconfigured. What follows is a
|
||||
burst of fresh connections and, on a rate-limiting peer, a block that looks like
|
||||
an outage. The tell that it is rate-limiting and not an outage: HTTPS to the same
|
||||
estate keeps working while port 22 stops answering. (Recorded 2026-08-25 — an
|
||||
agent hit exactly this, concluded "ControlMaster is impossible here", disabled
|
||||
multiplexing, and filed that as a lesson. The sidecar had solved it since v1.4.)
|
||||
|
||||
If every `ssh` to one host suddenly hangs, suspect a **stale master** — socket
|
||||
file present, daemon gone, typically after the host suspended or changed
|
||||
network. Check and clear it:
|
||||
|
||||
```sh
|
||||
ssh -F "$HOME/.ssh-local/config" -O check <host> # "Master running (pid=…)" or no master
|
||||
ssh -F "$HOME/.ssh-local/config" -O exit <host> # tear down a stale one
|
||||
```
|
||||
|
||||
Two related mechanisms (don't reinvent them):
|
||||
|
||||
- **ControlMaster multiplexing** is preconfigured (`/tmp/sshcm/`) to survive
|
||||
CGNAT per-destination flow caps on residential ISPs. If `~/.ssh/config` pins
|
||||
a `ControlPath` under the read-only `~/.ssh`, override with
|
||||
`-o ControlPath=none` (or use the sidecar, which already redirects it).
|
||||
- **A live master socket MASKS auth and config changes on the far end.** Once
|
||||
`~/.ssh-local/cm/<user>@<host>:22` exists, later commands ride it and
|
||||
authenticate **not at all** — so after editing remote `authorized_keys`,
|
||||
`sshd_config`, host keys, or firewall rules, "it still works" proves nothing.
|
||||
A corrupted `authorized_keys` then bites on the next *cold* connect, likely in
|
||||
a future session with no memory of the edit. Prove it immediately instead:
|
||||
|
||||
```sh
|
||||
ssh -F "$HOME/.ssh-local/config" -O check <host> # 'Master running (pid=N)'
|
||||
ssh -F "$HOME/.ssh-local/config" -o ControlPath=none -o ControlMaster=no \
|
||||
-o BatchMode=yes <host> 'echo COLD AUTH OK'
|
||||
```
|
||||
|
||||
To attribute a socket rather than guess whose it is: `ps -p <pid> -o
|
||||
pid,ppid,lstart,etime,args`. A `mosh` the *user* started on the host
|
||||
bootstraps with the **host's** `~/.ssh/cm/` and is invisible from in here;
|
||||
only a mosh started *inside* the container shares `~/.ssh-local/cm/`.
|
||||
- **`pi --ssh <host>`** rewires pi's own read/write/edit/bash tools to run on a
|
||||
remote host; it has its own writable-socket fallback. See the `pi-extensions`
|
||||
skill for that path.
|
||||
@@ -257,6 +382,10 @@ hardcode. Details are in the `mempalace` skill.
|
||||
- [ ] Writing durable output? → `/workspace`, not the ephemeral layer.
|
||||
- [ ] Using `dssh`/`dscp`/`ll` in the bash tool? → spell out the real command.
|
||||
- [ ] Assuming a hostname / domain / nameserver / host OS? → stop, detect it.
|
||||
- [ ] About to report something **absent / unreachable / not running**? → re-run
|
||||
without your own `head`/pattern/`PATH` assumptions first (§2).
|
||||
- [ ] Changed remote `authorized_keys` / `sshd_config`? → prove it with a **cold**
|
||||
connect; a live master socket hides breakage (§3).
|
||||
- [ ] "Resolves but won't connect"? → check route *and* DNS (§3 + §4).
|
||||
- [ ] `apt`/toolchain install? → tell the user it's ephemeral unless imaged.
|
||||
- [ ] Editing a skill? → `readlink -f ~/.agents/skills/<name>` first (§1).
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: pi-extensions
|
||||
description: >-
|
||||
Use the pi extensions (pi-fork, pi-observational-memory, ssh-controlmaster) effectively in the pi coding agent harness. Load this skill only when running inside pi (detection - `fork` and `recall` are present in your tool list, or `pi --ssh` was used to start the session). pi-fork dispatches focused subtasks to forked agents at fast/balanced/deep effort tiers; pi-observational-memory compacts long sessions into recallable observations + reflections; ssh-controlmaster rewires pi's read/write/edit/bash tools to execute on a remote host over a multiplexed SSH connection. This skill covers tier selection, task design, boundary discipline, when to use recall, and remote-pi mechanics.
|
||||
Use the pi extensions (pi-fork, pi-observational-memory, ssh-controlmaster) effectively in the pi coding agent harness. Load this skill only when running inside pi (detection - `fork` and `recall` are present in your tool list, or `pi --ssh` was used to start the session). pi-fork dispatches focused subtasks to forked agents at fast/balanced/deep effort tiers; pi-observational-memory compacts long sessions into recallable observations + reflections; ssh-controlmaster rewires pi's read/write/edit/bash tools to execute on a remote host over a multiplexed SSH connection. Also covers the context ladder L0-L4 and when to reach for the separate `pi-task` CLI instead of `fork` - isolated child, immutable spec, machine-checked envelope, write-boundary diff. This skill covers tier selection, task design, boundary discipline, when to use recall, and remote-pi mechanics.
|
||||
---
|
||||
|
||||
# Pi Extensions: pi-fork, pi-observational-memory, ssh-controlmaster
|
||||
@@ -161,6 +161,68 @@ The "three" things it completed were exactly the main thread's pending todos, vi
|
||||
- Distrust **quantities** and **provenance claims** in fork prose specifically ("all N sessions", "shipped with the image", "as expected") — those are the slots confabulation fills.
|
||||
- The fact that the fork was "right anyway" is not the same as the fork having followed instructions.
|
||||
|
||||
### The context ladder — and the second dispatch mechanism (`pi-task`)
|
||||
|
||||
Everything above describes a child that inherits everything. That is not a fixed
|
||||
cost of delegation — **how much context a child gets is a choice**, and `fork`
|
||||
sits at one extreme of it. Five rungs:
|
||||
|
||||
| rung | what the child sees | mechanism | built? |
|
||||
|---|---|---|---|
|
||||
| **L0** | nothing but the goal | `pi-task` default: fresh `--session-id pitask-<id>-<stamp>` in a private `--session-dir` | yes |
|
||||
| **L1** | goal + **names** of files/commands to read itself | `pi-task` spec `context.files` / `context.commands` (`bin/pi-task:154,157`) | yes |
|
||||
| **L2** | goal + an **excerpt the parent curated** | `pi-task` spec `context.facts`, pasted verbatim (`bin/pi-task:151`) | yes |
|
||||
| **L3** | a **truncated tail** of the parent branch | *nothing implements this* — would need a new spec key plus `--session <trimmed snapshot>` | **no** |
|
||||
| **L4** | the **entire** parent branch | `fork(task=…)` — `getHeader()+getBranch()`, no offset or limit anywhere in the call chain | yes |
|
||||
|
||||
**`pi-task` is a CLI, not an extension — it will never appear in your tool list.**
|
||||
Invoke it with `bash`: `/opt/pi-toolkit/bin/pi-task run <spec.json>` (source at
|
||||
`/workspace/pi-toolkit/bin/pi-task`, `schema` subcommand prints the spec fields).
|
||||
It reads an immutable JSON spec, and "inherit the session" is not expressible in
|
||||
that schema — the isolation is structural, not a request.
|
||||
|
||||
**Choose the lowest rung that can do the job:**
|
||||
|
||||
- **`fork` (L4)** when the subtask only makes sense against this conversation,
|
||||
when you want several independent opinions in parallel from one message, or for
|
||||
read-only exploration whose detail you will discard. Everything in "Boundary
|
||||
discipline" above applies in full.
|
||||
- **`pi-task` (L0–L2)** when the brief contains a **prohibition** (the inherited
|
||||
transcript is exactly what overrides those), when you want a **pass/fail**
|
||||
result instead of prose, when you need an **audit trail**, or when writes
|
||||
outside an authorised set must be caught.
|
||||
- **Neither** for trivial work, iterative work (both are one-shot), or judgement
|
||||
that needs context only you have.
|
||||
|
||||
**What `pi-task` gets you that no brief can.** The envelope must parse or the run
|
||||
FAILED, however fluent the prose. `roots[]` is the WATCHED set and
|
||||
`write_allowed` the CHANGEABLE subset, diffed before and after with git
|
||||
`--porcelain --ignored`. That `--ignored` flag is load-bearing: in the T4 test the
|
||||
child obeyed its brief perfectly and still tripped the detector, because
|
||||
`py_compile` wrote `__pycache__` into a watched-but-not-writable root — a
|
||||
gitignored path that plain `--porcelain` reports as clean. Note the structural
|
||||
point that test exposed: under `read_only: true` a write is *defiance*, so a
|
||||
well-behaved child never produces a delta and the detector is never exercised.
|
||||
Splitting WATCHED from WRITABLE is what lets an **obedient** child reveal a
|
||||
violation, which is the realistic hazard.
|
||||
|
||||
**What it does not fix.** `--no-extensions` removes extensions, not the core
|
||||
`read`/`write`/`edit`/`bash` tools — exactly as described above — so the boundary
|
||||
diff is post-hoc **detection, not prevention**. And a fresh L0 context removes the
|
||||
*narrative* failures (parent voice, invented continuity) without removing
|
||||
confabulation: given an under-specified spec built on a false premise, the child
|
||||
still filled the `deliverable` slot with a confident shape. The envelope's own
|
||||
structure creates that pressure. Verify decisive claims from the filesystem
|
||||
regardless of which rung you used.
|
||||
|
||||
**Trap — the capability floor is inverted from intuition.** `runner.ts:188` reads
|
||||
`if (extensions !== null) args.push("--no-extensions")`. So `pi-fork.extensions:
|
||||
[]` passes the flag and the floor is **on**; setting it to `null` — documented in
|
||||
`settings.json` as the way to "restore normal extension loading" — passes nothing
|
||||
and the floor is **off**, restoring palace writes inside every fork child.
|
||||
Changing `[]` to `null` as a tidy-up re-arms what was deliberately disarmed.
|
||||
`pi-task` hardcodes the flag and cannot drift this way.
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- **Forking trivial work.** A fork has overhead. If the task takes < 30 seconds in your main thread, just do it.
|
||||
@@ -230,7 +292,7 @@ When entries conflict, **the most recent observation reflects the latest known s
|
||||
## Quick Reference
|
||||
|
||||
```
|
||||
fork(task=..., effort=fast|balanced|deep)
|
||||
fork(task=..., effort=fast|balanced|deep) # L4: child inherits your WHOLE branch
|
||||
- state decision authority explicitly
|
||||
- pass verified context up front
|
||||
- specify deliverable shape
|
||||
@@ -240,6 +302,13 @@ fork(task=..., effort=fast|balanced|deep)
|
||||
- write-capable? demand "What I did NOT do", then verify from git/fs, not the report
|
||||
- prohibition in the brief => not a `fast` task
|
||||
|
||||
bash: /opt/pi-toolkit/bin/pi-task run <spec> # L0-L2: isolated child, NOT a tool
|
||||
- schema | selftest | run [--dry-run]
|
||||
- context.facts (pasted) / .files (names only) / .commands
|
||||
- roots[] = WATCHED, write_allowed[] = CHANGEABLE subset
|
||||
- envelope must parse or the run FAILED
|
||||
- audit + cost: ~/.pi/agent/pi-task/<stamp>-<id>/result.json
|
||||
|
||||
recall(id=<12-char-hex>)
|
||||
- only when stakes justify the cost
|
||||
- id must already be visible in your context
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
# Skills in this directory whose OWNER is the skillset repo.
|
||||
#
|
||||
# Read by devbox-skill-reconcile, which runs after the skillset deploy in
|
||||
# entrypoint-user.sh: for each name below, if the mounted skillset ships a
|
||||
# skill of that name, the baked link in ~/.agents/skills/ is repointed at the
|
||||
# live clone. The baked copy remains the fallback for containers started
|
||||
# WITHOUT a skillset mount, and a user override always wins over both.
|
||||
#
|
||||
# Add a name here ONLY if the skillset repo is the authoritative source (see
|
||||
# the ownership table in VENDORED.md). Do NOT add:
|
||||
# pi-devbox-environment — authored in this repo; baked IS canonical
|
||||
# pi-extensions — owned by the pi-extensions package repo and copied
|
||||
# over the snapshot at build time; the skillset copy
|
||||
# is a downstream duplicate that can lag, so letting
|
||||
# it win would regress the skill.
|
||||
mempalace
|
||||
Executable
+246
@@ -0,0 +1,246 @@
|
||||
#!/usr/bin/env bash
|
||||
# check-doc-drift.sh — fail when a hand-maintained doc claim contradicts the
|
||||
# build files it describes.
|
||||
#
|
||||
# THE DEFECT CLASS THIS EXISTS TO CATCH, measured 2026-09-10 while preparing
|
||||
# v1.9.0. Five separate claims had rotted, all of them the same shape: a fact
|
||||
# written once by hand, in a file nothing verifies, about a value that lives
|
||||
# somewhere else and moved.
|
||||
#
|
||||
# 1..3. README.md's "Version pins" table was wrong on EVERY row — pi `0.84.4`
|
||||
# vs ARG PI_VERSION=0.85.1, pi-atelier `v0.10.0` vs v0.10.1, mempalace
|
||||
# `3.8.0` vs 3.9.0. That table is the worst possible place for this: it
|
||||
# exists precisely to be the reviewable record of what is deliberately
|
||||
# frozen, so when it lies, the review it enables is worthless.
|
||||
# 4. README.md carried a "Planned for an upcoming minor release" section
|
||||
# listing typst PDF export, which had ALREADY SHIPPED, tagged with a
|
||||
# self-contradicting "(shipped in Unreleased/base)" marker. The
|
||||
# CHANGELOG had already documented three earlier instances of exactly
|
||||
# this stale-"Unreleased"-pointer class (see its v1.8.7 notes).
|
||||
# 5. DOCKER_HUB.md claimed "Node.js v22" while this release ships Node 24.
|
||||
# This one is the reason the gate exists at all: DOCKER_HUB.md is
|
||||
# PUBLISHED. `update-description` in docker-publish.yml POSTs it to Hub
|
||||
# as full_description on every tag, so unlike README.md — which no
|
||||
# workflow or gate reads — a stale claim here is what users see.
|
||||
#
|
||||
# WHY A GATE AND NOT "REMEMBER TO CHECK". DOCKER_HUB.md had gone eight releases
|
||||
# (v1.8.6 → v1.9.0) without a touch. Nothing generates it and nothing verifies
|
||||
# it; the only mechanism keeping it true was whoever remembered. That is the
|
||||
# same failure mode check-skill-floor.sh was written for, and the same fix:
|
||||
# convert "someone remembers" into "CI refuses".
|
||||
#
|
||||
# WHY THESE FIVE CHECKS AND NOT MORE. Every check here compares a doc string to
|
||||
# a value that EXISTS IN THIS REPO, so it can never be wrong about the world and
|
||||
# needs no network, no token, and no built image. Claims that require a running
|
||||
# container to verify (image sizes, the "N mempalace_* tools" count) are
|
||||
# deliberately NOT gated: a check that cannot be evaluated honestly at lint time
|
||||
# would either be skipped or guessed, and a guessing gate is worse than none.
|
||||
# If you want those, assert them in scripts/smoke-test.sh where a real image is
|
||||
# available.
|
||||
#
|
||||
# DELIBERATELY NOT GATED: Dockerfile.base's `# BASE_REBUILD_DATE:` comment, which
|
||||
# is also stale (2026-07-13, three base rebuilds ago). base_tag is a hash of
|
||||
# Dockerfile.base's CONTENT plus rootfs/, comments included, so a gate that
|
||||
# demanded that comment be current would force a ~60 min base rebuild on any
|
||||
# release that touched no base files at all. Fix it when you are already
|
||||
# rebuilding the base — then it is free. This is a real cost asymmetry, not
|
||||
# laziness.
|
||||
#
|
||||
# EXIT CODES (same contract as lint-shell.sh and check-skill-floor.sh):
|
||||
# 0 every checked claim matches
|
||||
# 1 at least one claim has drifted
|
||||
# 2 cannot run (a file or ARG this gate reads is missing/unparseable)
|
||||
# A gate that cannot run must not pass, so a missing input is 2, never 0.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$REPO_ROOT"
|
||||
|
||||
README="README.md"
|
||||
HUB="DOCKER_HUB.md"
|
||||
DF_VARIANT="Dockerfile.variant"
|
||||
DF_BASE="Dockerfile.base"
|
||||
|
||||
# Docker Hub rejects a full_description longer than this. docker-publish.yml has
|
||||
# no size check of its own; it only notices via a non-200 from the API, i.e.
|
||||
# after paying the whole build. Catching it here makes it a 2-second failure.
|
||||
HUB_MAX_CHARS=25000
|
||||
|
||||
WARN_ONLY=0
|
||||
FAILURES=0
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: check-doc-drift.sh [--warn-only] [-h|--help]
|
||||
|
||||
Compares hand-written claims in README.md and DOCKER_HUB.md against the build
|
||||
files they describe (Dockerfile.base, Dockerfile.variant).
|
||||
|
||||
--warn-only Report drift but exit 0 (advisory use, e.g. a local pre-push hook).
|
||||
|
||||
Exit: 0 = in sync, 1 = drift, 2 = cannot run.
|
||||
EOF
|
||||
}
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--warn-only) WARN_ONLY=1; shift ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "::error::unknown argument: $1" >&2; usage >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
for f in "$README" "$HUB" "$DF_VARIANT" "$DF_BASE"; do
|
||||
if [ ! -f "$f" ]; then
|
||||
echo "::error::$f not found (cwd $PWD). Cannot evaluate doc drift, so this is exit 2, not a pass."
|
||||
exit 2
|
||||
fi
|
||||
done
|
||||
|
||||
# Read `ARG NAME=value` from a Dockerfile. Exit 2 when absent: if the ARG this
|
||||
# gate is built around has been renamed, the gate is measuring nothing and must
|
||||
# say so rather than silently comparing against an empty string.
|
||||
read_arg() {
|
||||
local file="$1" name="$2" value
|
||||
value="$(sed -n "s/^ARG ${name}=\\(.*\\)\$/\\1/p" "$file" | head -1)"
|
||||
if [ -z "$value" ]; then
|
||||
echo "::error::ARG ${name} not found in ${file}. It was probably renamed;" >&2
|
||||
echo "::error::update check-doc-drift.sh to match, because this gate is now blind." >&2
|
||||
exit 2
|
||||
fi
|
||||
printf '%s' "$value"
|
||||
}
|
||||
|
||||
# One row of README's "Version pins" table: `| pi | `0.85.1` | ... |`
|
||||
read_pin_row() {
|
||||
sed -n "s/^| $1 | \`\\([^\`]*\`*\\)\` |.*/\\1/p" "$README" | head -1
|
||||
}
|
||||
|
||||
fail() {
|
||||
FAILURES=$((FAILURES + 1))
|
||||
echo "::error::$1"
|
||||
}
|
||||
|
||||
ok() { printf ' OK %s\n' "$1"; }
|
||||
|
||||
echo "Checking hand-maintained doc claims against the build files they describe."
|
||||
echo
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1-3. README's version-pin table vs the ARGs it names by name.
|
||||
# ---------------------------------------------------------------------------
|
||||
check_pin() {
|
||||
local label="$1" documented="$2" actual="$3" where="$4"
|
||||
if [ -z "$documented" ]; then
|
||||
fail "README.md: no '| $label |' row found in the version-pin table. Either the
|
||||
table was restructured (update this gate) or the row was dropped (restore it)."
|
||||
return
|
||||
fi
|
||||
if [ "$documented" != "$actual" ]; then
|
||||
fail "README.md version-pin table is stale for $label: says '$documented',
|
||||
$where says '$actual'. Fix the table — it is the reviewable record of what
|
||||
this repo deliberately freezes, so a wrong row defeats its only purpose."
|
||||
return
|
||||
fi
|
||||
ok "README pin $label = $actual"
|
||||
}
|
||||
|
||||
PI_ACTUAL="$(read_arg "$DF_VARIANT" PI_VERSION)"
|
||||
ATELIER_ACTUAL="$(read_arg "$DF_VARIANT" PI_ATELIER_REF)"
|
||||
MEMPALACE_ACTUAL="$(read_arg "$DF_BASE" MEMPALACE_VERSION)"
|
||||
|
||||
check_pin pi "$(read_pin_row pi)" "$PI_ACTUAL" "ARG PI_VERSION in $DF_VARIANT"
|
||||
check_pin pi-atelier "$(read_pin_row pi-atelier)" "$ATELIER_ACTUAL" "ARG PI_ATELIER_REF in $DF_VARIANT"
|
||||
check_pin mempalace "$(read_pin_row mempalace)" "$MEMPALACE_ACTUAL" "ARG MEMPALACE_VERSION in $DF_BASE"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. DOCKER_HUB.md's Node claim vs ARG NODE_VERSION. This is the published page,
|
||||
# so it is the one whose staleness reaches users.
|
||||
# ---------------------------------------------------------------------------
|
||||
NODE_ACTUAL="$(read_arg "$DF_BASE" NODE_VERSION)"
|
||||
NODE_DOCUMENTED="$(sed -n 's/.*\*\*Node\.js\*\* v\([0-9][0-9]*\).*/\1/p' "$HUB" | head -1)"
|
||||
if [ -z "$NODE_DOCUMENTED" ]; then
|
||||
fail "$HUB: could not find a '**Node.js** vNN' claim. If the wording changed,
|
||||
update this gate; do not leave the published page unverified."
|
||||
elif [ "$NODE_DOCUMENTED" != "$NODE_ACTUAL" ]; then
|
||||
fail "$HUB claims Node v$NODE_DOCUMENTED but ARG NODE_VERSION=$NODE_ACTUAL.
|
||||
This file is PUBLISHED to Docker Hub by update-description on every tag,
|
||||
and it is read from the TAG — so fix it before tagging, not after."
|
||||
else
|
||||
ok "$HUB Node claim = v$NODE_ACTUAL"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. Placeholders CI will not substitute. docker-publish.yml substitutes exactly
|
||||
# {{PI_VERSION}} and then greps for leftovers of that ONE token, so any other
|
||||
# {{...}} sails through the guard and is published literally.
|
||||
# ---------------------------------------------------------------------------
|
||||
UNKNOWN_PLACEHOLDERS="$(grep -o '{{[A-Za-z0-9_]*}}' "$HUB" | sort -u | grep -v '^{{PI_VERSION}}$' || true)"
|
||||
if [ -n "$UNKNOWN_PLACEHOLDERS" ]; then
|
||||
fail "$HUB contains placeholders CI does not substitute, which would be
|
||||
published verbatim: $(echo "$UNKNOWN_PLACEHOLDERS" | tr '\n' ' ')
|
||||
docker-publish.yml only fills {{PI_VERSION}}; add substitution there first."
|
||||
else
|
||||
ok "$HUB has no placeholders beyond {{PI_VERSION}}"
|
||||
fi
|
||||
|
||||
# Match only the UPPER_SNAKE placeholder convention CI uses. A bare '{{' search
|
||||
# is WRONG here, and the first version of this check proved it by failing on
|
||||
# README.md:900 — `docker inspect --format '{{json .Config.Labels}}'`, a Go
|
||||
# template in a legitimate example, not a placeholder. The gate was wrong, not
|
||||
# the doc. Keep this anchored to [A-Z] so Go/Jinja/Handlebars examples pass.
|
||||
README_PLACEHOLDERS="$(grep -o '{{[A-Z][A-Z0-9_]*}}' "$README" | sort -u || true)"
|
||||
if [ -n "$README_PLACEHOLDERS" ]; then
|
||||
fail "$README contains placeholder(s) nothing substitutes, so they would render
|
||||
literally for every reader: $(echo "$README_PLACEHOLDERS" | tr '\n' ' ')
|
||||
Only DOCKER_HUB.md gets substitution, and only for {{PI_VERSION}}."
|
||||
else
|
||||
ok "$README has no unsubstituted placeholders"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. Hub full_description length.
|
||||
# ---------------------------------------------------------------------------
|
||||
HUB_CHARS="$(wc -c < "$HUB" | tr -d ' ')"
|
||||
if [ "$HUB_CHARS" -gt "$HUB_MAX_CHARS" ]; then
|
||||
fail "$HUB is $HUB_CHARS chars, over Docker Hub's $HUB_MAX_CHARS-char
|
||||
full_description limit. update-description would fail with a non-200 AFTER
|
||||
the full build. Trim it — this file is the essentials-only page, and
|
||||
README.md is the long form on purpose."
|
||||
else
|
||||
ok "$HUB is $HUB_CHARS chars (limit $HUB_MAX_CHARS)"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. Stale "Unreleased" pointers. "Unreleased" is a CHANGELOG-only concept; in
|
||||
# a user-facing doc it is always a pointer that outlived what it pointed at.
|
||||
# This class has now bitten five times, hence a gate rather than vigilance.
|
||||
# ---------------------------------------------------------------------------
|
||||
STALE_MARKERS="$(grep -n 'Unreleased' "$README" "$HUB" || true)"
|
||||
if [ -n "$STALE_MARKERS" ]; then
|
||||
fail "'Unreleased' appears in a user-facing doc, which is always a stale
|
||||
pointer once the thing ships (it has happened five times here):
|
||||
${STALE_MARKERS//$'\n'/$'\n' }
|
||||
State the fact directly, or move it to CHANGELOG.md where 'Unreleased' means something."
|
||||
else
|
||||
ok "no stale 'Unreleased' pointers in $README or $HUB"
|
||||
fi
|
||||
|
||||
echo
|
||||
if [ "$FAILURES" -eq 0 ]; then
|
||||
echo "OK: every checked doc claim matches the build files."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "::error::$FAILURES doc claim(s) have drifted from the build files."
|
||||
echo
|
||||
echo "Docs are read from the TAG, not from main: docker-publish.yml checks out"
|
||||
echo "github.ref, so a fix pushed after tagging does not reach the release or the"
|
||||
echo "Hub page. Update the docs BEFORE you tag."
|
||||
|
||||
if [ "$WARN_ONLY" -eq 1 ]; then
|
||||
echo "(--warn-only: exiting 0 anyway)"
|
||||
exit 0
|
||||
fi
|
||||
exit 1
|
||||
Executable
+166
@@ -0,0 +1,166 @@
|
||||
#!/usr/bin/env bash
|
||||
# check-skill-floor.sh — fail when the vendored pi-extensions skill snapshot in
|
||||
# rootfs/ ("the floor") has drifted from the package repo it is a snapshot of.
|
||||
#
|
||||
# THE DEFECT THIS EXISTS TO CATCH, measured 2026-09-10.
|
||||
# rootfs/usr/local/share/pi-devbox/skills/pi-extensions/ ships a vendored copy
|
||||
# of the pi-extensions skill so the skill is ALWAYS present in the image.
|
||||
# Dockerfile.variant then copies the freshly-cloned package copy OVER the served
|
||||
# path at /usr/local/share/... — but it never writes back to the repo floor. So
|
||||
# the floor only silently rots, and it had: 34284 B, untouched since fa04d20
|
||||
# (2026-07-30), while the package copy was 38973 B. Four copies existed with
|
||||
# three different sizes.
|
||||
#
|
||||
# Why that is worse than ordinary staleness: the floor is a FALLBACK. The copy
|
||||
# step is guarded by `if [ -f /opt/pi-extensions/skill/SKILL.md ]`, so a build
|
||||
# where the package clone yields no skill/ keeps the vendored snapshot and still
|
||||
# succeeds — green, with no manifest flag and no label saying which copy was
|
||||
# served. The image would ship a July skill and nothing would say so. Keeping
|
||||
# the floor fresh means that fallback is harmless instead of a silent regression.
|
||||
#
|
||||
# WHY A DIRECTORY HASH AND NOT `sha256sum SKILL.md`.
|
||||
# The same pipeline Dockerfile.variant uses for skillset_snapshot_tree_sha256,
|
||||
# and for the same documented reason: a file-only compare answers "did this one
|
||||
# file change", not "is this the same skill". pi-extensions ships TWO files
|
||||
# (SKILL.md + evaluate-extension-usage.py), so a sibling-file edit would pass a
|
||||
# file-only check. If you change the pipeline here, change it there too.
|
||||
#
|
||||
# WHY GATING ON ANOTHER REPO IS PROPORTIONATE HERE, since that is normally a
|
||||
# smell: this fires only when the package's skill/ DIRECTORY HASH changes, which
|
||||
# is exactly and only when the floor has genuinely gone stale. pi-extensions
|
||||
# commits that do not touch skill/ leave the hash alone and cannot turn this red.
|
||||
# The repo is also anonymously clonable (verified 2026-09-10 with `git ls-remote`
|
||||
# and no credentials), so this needs no secret and cannot break on token expiry.
|
||||
#
|
||||
# Exit codes — deliberately three, matching scripts/lint-shell.sh's philosophy
|
||||
# that a gate which cannot run must not pass:
|
||||
# 0 in sync (or the package legitimately has no skill/ at this ref)
|
||||
# 1 DRIFT — the floor differs from the package
|
||||
# 2 cannot run — no package copy could be obtained
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
||||
FLOOR_DIR="${REPO_ROOT}/rootfs/usr/local/share/pi-devbox/skills/pi-extensions"
|
||||
|
||||
# Defaults mirror Dockerfile.variant's ARGs so this checks what the build builds.
|
||||
PI_EXTENSIONS_REPO="${PI_EXTENSIONS_REPO:-https://gitea.jordbo.se/joakimp/pi-extensions.git}"
|
||||
PI_EXTENSIONS_REF="${PI_EXTENSIONS_REF:-main}"
|
||||
|
||||
PACKAGE_DIR=""
|
||||
WARN_ONLY=0
|
||||
TMPDIR_CLONE=""
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/check-skill-floor.sh [options]
|
||||
|
||||
--package-dir DIR Compare against an existing skill directory instead of
|
||||
cloning. In a devbox container use /opt/pi-extensions/skill
|
||||
for a fully offline run.
|
||||
--warn-only Report drift but exit 0 (advisory use, e.g. a local hook).
|
||||
-h, --help This text.
|
||||
|
||||
Environment: PI_EXTENSIONS_REPO, PI_EXTENSIONS_REF (default main) — both mirror
|
||||
the Dockerfile.variant ARGs of the same name.
|
||||
EOF
|
||||
}
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--package-dir) PACKAGE_DIR="${2:-}"; shift 2 ;;
|
||||
--warn-only) WARN_ONLY=1; shift ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "::error::unknown argument: $1" >&2; usage >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
cleanup() {
|
||||
if [ -n "$TMPDIR_CLONE" ]; then rm -rf "$TMPDIR_CLONE"; fi
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# Identical to Dockerfile.variant's tree_sha256(): relative paths + per-file
|
||||
# sha256 over a sorted `find`, folded into one digest. Deterministic, never
|
||||
# readdir order.
|
||||
tree_sha256() {
|
||||
( cd "$1" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum ) \
|
||||
2>/dev/null | sha256sum | cut -d' ' -f1
|
||||
}
|
||||
|
||||
if [ ! -d "$FLOOR_DIR" ]; then
|
||||
echo "::error::floor directory is missing: ${FLOOR_DIR}"
|
||||
echo "::error::rootfs/ is supposed to guarantee the skill is always in the image."
|
||||
exit 2
|
||||
fi
|
||||
|
||||
SOURCE_DESC=""
|
||||
if [ -n "$PACKAGE_DIR" ]; then
|
||||
if [ ! -d "$PACKAGE_DIR" ]; then
|
||||
echo "::error::--package-dir does not exist: ${PACKAGE_DIR}"
|
||||
exit 2
|
||||
fi
|
||||
SOURCE_DESC="local directory ${PACKAGE_DIR}"
|
||||
else
|
||||
command -v git >/dev/null 2>&1 || { echo "::error::git not found; cannot obtain the package copy."; exit 2; }
|
||||
TMPDIR_CLONE=$(mktemp -d)
|
||||
# Fetch the single ref shallowly. `git fetch <ref>` accepts a branch, a tag
|
||||
# and (on Gitea) a reachable commit, which is why this is not `clone --branch`
|
||||
# — CI resolves PI_EXTENSIONS_REF to a 40-hex SHA before the build.
|
||||
if ! ( cd "$TMPDIR_CLONE" \
|
||||
&& git init -q . \
|
||||
&& git remote add origin "$PI_EXTENSIONS_REPO" \
|
||||
&& git fetch -q --depth 1 origin "$PI_EXTENSIONS_REF" \
|
||||
&& git checkout -q FETCH_HEAD ) 2>/dev/null; then
|
||||
echo "::error::could not fetch ${PI_EXTENSIONS_REF} from ${PI_EXTENSIONS_REPO}"
|
||||
echo "::error::Cannot determine whether the floor is stale, so this is exit 2, not a pass."
|
||||
echo "::error::For an offline run, pass --package-dir /opt/pi-extensions/skill"
|
||||
exit 2
|
||||
fi
|
||||
PACKAGE_SHA=$( cd "$TMPDIR_CLONE" && git rev-parse --short HEAD )
|
||||
PACKAGE_DIR="${TMPDIR_CLONE}/skill"
|
||||
SOURCE_DESC="${PI_EXTENSIONS_REPO} @ ${PI_EXTENSIONS_REF} (${PACKAGE_SHA})"
|
||||
fi
|
||||
|
||||
# A ref with no skill/ is the documented fallback case: Dockerfile.variant keeps
|
||||
# the vendored snapshot and the build succeeds. Nothing to compare, so this is
|
||||
# not drift — but it IS the exact condition under which the floor ships, so say
|
||||
# so loudly rather than printing a silent green tick.
|
||||
if [ ! -d "$PACKAGE_DIR" ]; then
|
||||
echo "::warning::package has no skill/ at this ref — the vendored floor is what will ship."
|
||||
echo " source : ${SOURCE_DESC}"
|
||||
echo " floor : $(tree_sha256 "$FLOOR_DIR")"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
FLOOR_HASH=$(tree_sha256 "$FLOOR_DIR")
|
||||
PKG_HASH=$(tree_sha256 "$PACKAGE_DIR")
|
||||
|
||||
if [ "$FLOOR_HASH" = "$PKG_HASH" ]; then
|
||||
echo "OK: vendored pi-extensions floor matches the package."
|
||||
echo " source : ${SOURCE_DESC}"
|
||||
echo " tree_sha256: ${FLOOR_HASH}"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# `set -e` interacts badly with `[ … ] && x` as a bare statement, so both of
|
||||
# these are explicit if-blocks rather than AND-lists.
|
||||
LEVEL="error"
|
||||
if [ "$WARN_ONLY" -eq 1 ]; then LEVEL="warning"; fi
|
||||
|
||||
echo "::${LEVEL}::vendored pi-extensions skill floor has DRIFTED from the package."
|
||||
echo " source : ${SOURCE_DESC}"
|
||||
echo " floor tree_sha256 : ${FLOOR_HASH}"
|
||||
echo " pkg tree_sha256 : ${PKG_HASH}"
|
||||
echo ""
|
||||
echo " per-file differences:"
|
||||
diff -rq "$FLOOR_DIR" "$PACKAGE_DIR" 2>&1 | sed 's/^/ /' || true
|
||||
echo ""
|
||||
echo " Remedy — re-sync the floor and commit it:"
|
||||
echo " cp -a <pi-extensions>/skill/. ${FLOOR_DIR}/"
|
||||
echo " git add ${FLOOR_DIR#"${REPO_ROOT}/"} && git commit"
|
||||
echo ""
|
||||
echo " NOTE this forces one full base rebuild: base_tag hashes Dockerfile.base"
|
||||
echo " + rootfs/, and that rebuild is what re-bakes the refreshed floor."
|
||||
|
||||
if [ "$WARN_ONLY" -eq 1 ]; then exit 0; fi
|
||||
exit 1
|
||||
@@ -0,0 +1,91 @@
|
||||
#!/usr/bin/env bash
|
||||
# Shellcheck + syntax-check every shell script in this repo. Severity: error.
|
||||
#
|
||||
# SINGLE SOURCE OF TRUTH for two callers:
|
||||
# .gitea/workflows/lint.yml — advisory, every branch push and PR
|
||||
# .gitea/workflows/docker-publish.yml — the release GATE (lint-gate job)
|
||||
# Extracted from lint.yml on 2026-09-08 rather than copied, because a second
|
||||
# copy is exactly the drift this repo has been bitten by (see skillset's
|
||||
# pi-extensions mirror, refreshed the same evening after sitting 9579 B behind).
|
||||
#
|
||||
# WHY THIS CHECK EXISTS AT ALL
|
||||
# actionlint shellchecks workflow `run:` steps only. The repo's own scripts —
|
||||
# entrypoint.sh, scripts/*.sh, and the extensionless tools under
|
||||
# rootfs/usr/local/bin/ — were never shellchecked. A sibling repo with the same
|
||||
# gap shipped a broken `echo "$json" | python3 <<'EOF' ... json.load(sys.stdin)`
|
||||
# for two months: with no script argument python reads its SCRIPT from stdin,
|
||||
# so the heredoc IS stdin and json.load hits EOF. shellcheck flags that at
|
||||
# severity error (SC2259); nothing ever ran it.
|
||||
#
|
||||
# WHY THE RELEASE GATES ON IT (added 2026-09-08, the expensive way round)
|
||||
# v1.8.14's first attempt failed after build-base had already spent ~46 min:
|
||||
# scripts/smoke-test.sh had an apostrophe inside a single-quoted exec_test body
|
||||
# ("the fleet\'s"), which CLOSES the string, so the body truncated and its tail
|
||||
# ran on the CI runner instead of inside the image. shellcheck had already
|
||||
# caught it as SC2289 at severity error — the lint job went red on the very
|
||||
# push that introduced it and stayed red for 24 hours, unread. lint.yml
|
||||
# deliberately does not run on tag pushes (sound: the tagged tree was linted on
|
||||
# main, and a tag-ref lint run sorts above the publish run and makes a release
|
||||
# look finished early). The gap was never "lint the tag" — it was that a tree
|
||||
# whose lint FAILED could still be released. Hence a gate inside the publish
|
||||
# workflow, ~40 s, ahead of everything expensive.
|
||||
#
|
||||
# SEVERITY CHOICE
|
||||
# -S error is 0 findings across this repo when clean, so it is free to add.
|
||||
# -S warning is NOT free here (19x SC2088 tilde-in-quotes in
|
||||
# recreate-sanity-check.sh, plus assorted SC2016 — both intentional), and a
|
||||
# noisy gate trains people to ignore it. Error-only, matching the
|
||||
# SHELLCHECK_OPTS philosophy in lint.yml.
|
||||
#
|
||||
# Usage: bash scripts/lint-shell.sh [root] (default root: repo top level)
|
||||
set -uo pipefail
|
||||
|
||||
root="${1:-}"
|
||||
if [ -z "$root" ]; then
|
||||
root="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
|
||||
fi
|
||||
cd "$root" || { echo "::error::cannot cd to $root"; exit 2; }
|
||||
|
||||
# A gate that cannot run must not pass. Without this, a machine (or a CI job
|
||||
# whose install step was reordered away) without shellcheck would sail through
|
||||
# printing nothing, which is the failure mode this whole file exists to prevent.
|
||||
if ! command -v shellcheck >/dev/null 2>&1; then
|
||||
echo "::error::shellcheck not found — the gate cannot run, so it must not pass" >&2
|
||||
echo " install it (apt-get install -y shellcheck) or run this in CI" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Union of two signals, because either alone misses a real case: a shebang scan
|
||||
# misses a sourced fragment with no shebang, and a *.sh glob misses the
|
||||
# extensionless tools in rootfs/usr/local/bin/. Silent skipping is precisely the
|
||||
# failure mode this gate exists to prevent, so err toward over-collecting.
|
||||
# -print0/mapfile -d '' so a path containing a space cannot silently split.
|
||||
mapfile -d '' -t all_files < <(find . -not -path './.git/*' -type f -print0)
|
||||
sh_files=()
|
||||
for f in "${all_files[@]}"; do
|
||||
case "$f" in *.sh) sh_files+=("$f"); continue;; esac
|
||||
if head -n1 "$f" 2>/dev/null | grep -qE '^#!.*\b(bash|sh)\b'; then
|
||||
sh_files+=("$f")
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Checking ${#sh_files[@]} shell file(s) with $(shellcheck --version | awk '/version:/{print $2}')"
|
||||
# A green tick over an empty file set is not a check.
|
||||
if [ "${#sh_files[@]}" -eq 0 ]; then
|
||||
echo "::error::no shell files found — the shebang scan or the checkout is wrong"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
rc=0
|
||||
shellcheck -S error -f gcc "${sh_files[@]}" || rc=1
|
||||
|
||||
# bash -n catches a different class than shellcheck (unbalanced constructs it
|
||||
# declines to parse), so both run and both count.
|
||||
for f in "${sh_files[@]}"; do
|
||||
bash -n "$f" || { echo "::error file=$f::bash -n failed"; rc=1; }
|
||||
done
|
||||
|
||||
if [ "$rc" -eq 0 ]; then
|
||||
echo "OK: ${#sh_files[@]} shell file(s) clean at severity error"
|
||||
fi
|
||||
exit "$rc"
|
||||
@@ -2,8 +2,8 @@
|
||||
# Runtime post-recreate verification for pi-devbox.
|
||||
#
|
||||
# Verifies that after `docker compose up -d --force-recreate`:
|
||||
# - The new image is actually live (pi version matches, when an expected
|
||||
# version is supplied — see the version note below)
|
||||
# - The new image is actually live (both the pi version and — when asked —
|
||||
# the pi-devbox image release tag; see the two version notes below)
|
||||
# - Persisted named volumes survived (~/.pi config, shell history, zoxide,
|
||||
# nvim data, uv cache, ssh-local)
|
||||
# - pi runtime wiring is intact: keybindings symlink, AGENTS.md symlink,
|
||||
@@ -25,13 +25,33 @@
|
||||
# the pi-devbox repo (which a maintainer already has for CI builds). A plain
|
||||
# `docker pull` consumer is not the audience and will not have this file.
|
||||
#
|
||||
# Version note: pi's version is resolved from `latest` at CI build time and is
|
||||
# NOT pinned to a concrete value in Dockerfile.variant (ARG PI_VERSION=latest).
|
||||
# So unlike opencode-devbox, this script cannot self-derive an expected version
|
||||
# from the Dockerfile. Pass --expected-version to assert a match; without it the
|
||||
# live pi version is reported as an informational WARN, not a failure.
|
||||
# TWO DIFFERENT VERSIONS, TWO DIFFERENT FLAGS. This distinction has already
|
||||
# cost a release day, so it is spelled out here and in AGENTS.md step 4:
|
||||
#
|
||||
# Usage: ./scripts/recreate-sanity-check.sh [--expected-version X.Y.Z] [--variant studio|plain]
|
||||
# --expected-version the PI CODING AGENT version, e.g. 0.84.3
|
||||
# (`pi --version`; pinned as ARG PI_VERSION in
|
||||
# Dockerfile.variant, which CI reads as the source
|
||||
# of truth)
|
||||
# --expected-image-version the PI-DEVBOX IMAGE release tag, e.g. 1.8.9 or
|
||||
# v1.8.9 (the `release_tag` baked into
|
||||
# /etc/pi-devbox/build-manifest.json)
|
||||
#
|
||||
# Passing a release tag to --expected-version used to report
|
||||
# "pi version mismatch: expected 1.8.8, got 0.84.3" — an accusation aimed at
|
||||
# the wrong component, on the last gate of a release. Both flags now detect
|
||||
# being handed the other one's value and say so instead.
|
||||
#
|
||||
# Neither flag is required. Both values are derivable from the image's own
|
||||
# build manifest, so by default the script asserts the LIVE pi version against
|
||||
# the version recorded at build time — which is not a tautology: a stale
|
||||
# `pi` in the ~/.pi/npm-global volume can shadow the baked one, exactly the
|
||||
# way a stale npm:pi-atelier can (see the packages[] check below). Pass the
|
||||
# flags when you want an assertion against a value you name yourself, which
|
||||
# is what a release checklist wants.
|
||||
#
|
||||
# Usage: ./scripts/recreate-sanity-check.sh [--expected-version X.Y.Z]
|
||||
# [--expected-image-version X.Y.Z]
|
||||
# [--variant studio|plain]
|
||||
#
|
||||
# Exit codes:
|
||||
# 0 all checks passed
|
||||
@@ -41,22 +61,61 @@
|
||||
set -euo pipefail
|
||||
|
||||
EXPECTED_VERSION=""
|
||||
EXPECTED_IMAGE_VERSION=""
|
||||
VARIANT=""
|
||||
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
MANIFEST=/etc/pi-devbox/build-manifest.json
|
||||
|
||||
# Parse arguments
|
||||
usage() {
|
||||
cat >&2 <<'EOF'
|
||||
usage: recreate-sanity-check.sh [--expected-version X.Y.Z]
|
||||
[--expected-image-version X.Y.Z]
|
||||
[--variant studio|plain]
|
||||
|
||||
--expected-version pi coding agent version, e.g. 0.84.3 (`pi --version`)
|
||||
--expected-image-version pi-devbox image release tag, e.g. 1.8.9 or v1.8.9
|
||||
--variant studio|plain (auto-detected when omitted)
|
||||
|
||||
These are two different versions. Both are read from the image's own build
|
||||
manifest when the corresponding flag is omitted.
|
||||
EOF
|
||||
}
|
||||
|
||||
# Parse arguments. Every flag takes a value, so reject a missing one rather
|
||||
# than swallowing the next flag as if it were the value.
|
||||
need_value() {
|
||||
case "${2:-}" in
|
||||
""|-*)
|
||||
echo "$1 requires a value" >&2
|
||||
usage
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
}
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--expected-version)
|
||||
need_value "$@"
|
||||
EXPECTED_VERSION="$2"
|
||||
shift 2
|
||||
;;
|
||||
--expected-image-version)
|
||||
need_value "$@"
|
||||
EXPECTED_IMAGE_VERSION="$2"
|
||||
shift 2
|
||||
;;
|
||||
--variant)
|
||||
need_value "$@"
|
||||
VARIANT="$2"
|
||||
shift 2
|
||||
;;
|
||||
--help|-h)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "usage: $0 [--expected-version X.Y.Z] [--variant studio|plain]" >&2
|
||||
echo "unknown option: $1" >&2
|
||||
usage
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
@@ -67,6 +126,19 @@ pass() { echo " ✓ $1"; }
|
||||
fail() { echo " ✗ $1" >&2; FAILED=$((FAILED + 1)); }
|
||||
warn() { echo " ⚠ $1" >&2; }
|
||||
|
||||
# Read one top-level field from the build manifest, or print nothing. The
|
||||
# manifest is the image's own ground truth (written at `docker build` time by
|
||||
# Dockerfile.variant), so it needs no checkout and no network. Absent on an
|
||||
# image built before it existed, hence every caller treats "" as unknown.
|
||||
manifest_field() {
|
||||
[ -f "$MANIFEST" ] || return 0
|
||||
command -v jq >/dev/null 2>&1 || return 0
|
||||
jq -r --arg k "$1" '.[$k] // empty' "$MANIFEST" 2>/dev/null || true
|
||||
}
|
||||
# Release tags are written with a leading v in the manifest and quoted without
|
||||
# one in checklists; compare on the bare number so both spellings work.
|
||||
strip_v() { printf '%s' "${1#v}"; }
|
||||
|
||||
# Auto-detect variant if not provided. The studio variant vendors pi-studio to
|
||||
# /opt/pi-studio; the plain variant does not.
|
||||
if [ -z "$VARIANT" ]; then
|
||||
@@ -86,21 +158,59 @@ else
|
||||
fi
|
||||
echo
|
||||
|
||||
echo "-- pi version --"
|
||||
MANIFEST_PI_VERSION=$(manifest_field pi_version)
|
||||
MANIFEST_RELEASE_TAG=$(manifest_field release_tag)
|
||||
|
||||
echo "-- pi (coding agent) version --"
|
||||
if ACTUAL_VERSION=$(pi --version 2>&1 | head -1); then
|
||||
if [ -n "$EXPECTED_VERSION" ]; then
|
||||
if [ "$ACTUAL_VERSION" = "$EXPECTED_VERSION" ]; then
|
||||
pass "pi version $ACTUAL_VERSION"
|
||||
if [ "$(strip_v "$EXPECTED_VERSION")" = "$(strip_v "$ACTUAL_VERSION")" ]; then
|
||||
pass "pi version $ACTUAL_VERSION (matches --expected-version)"
|
||||
elif [ -n "$MANIFEST_RELEASE_TAG" ] &&
|
||||
[ "$(strip_v "$EXPECTED_VERSION")" = "$(strip_v "$MANIFEST_RELEASE_TAG")" ]; then
|
||||
# Exact, not heuristic: the value handed over IS this image's release
|
||||
# tag, so it cannot be a pi version anyone meant.
|
||||
fail "--expected-version $EXPECTED_VERSION is the pi-devbox IMAGE version, not the pi version — use --expected-image-version $EXPECTED_VERSION (live pi is $ACTUAL_VERSION)"
|
||||
else
|
||||
fail "pi version mismatch: expected $EXPECTED_VERSION, got $ACTUAL_VERSION"
|
||||
fail "pi version mismatch: expected $EXPECTED_VERSION, got $ACTUAL_VERSION (this flag asserts the pi coding agent version; for the image release tag use --expected-image-version)"
|
||||
fi
|
||||
elif [ -n "$MANIFEST_PI_VERSION" ]; then
|
||||
# Not a tautology: the manifest records what pi reported at BUILD time,
|
||||
# while `pi --version` resolves through PATH, which a stale npm-global
|
||||
# volume install can shadow.
|
||||
if [ "$MANIFEST_PI_VERSION" = "$ACTUAL_VERSION" ]; then
|
||||
pass "pi version $ACTUAL_VERSION (matches this image's build manifest)"
|
||||
else
|
||||
fail "live pi $ACTUAL_VERSION != $MANIFEST_PI_VERSION recorded in $MANIFEST — a stale pi in the ~/.pi/npm-global volume is shadowing the baked one"
|
||||
fi
|
||||
else
|
||||
warn "pi version $ACTUAL_VERSION (no --expected-version given; pi is built from 'latest', cannot self-derive — informational only)"
|
||||
warn "pi version $ACTUAL_VERSION (no --expected-version and no build manifest to compare against — informational only)"
|
||||
fi
|
||||
else
|
||||
fail "pi --version failed"
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "-- pi-devbox image version --"
|
||||
if [ -z "$MANIFEST_RELEASE_TAG" ]; then
|
||||
if [ -n "$EXPECTED_IMAGE_VERSION" ]; then
|
||||
fail "cannot verify --expected-image-version $EXPECTED_IMAGE_VERSION: no readable release_tag in $MANIFEST (image built before the manifest existed, or jq missing)"
|
||||
else
|
||||
warn "image release tag unknown (no readable $MANIFEST) — pi-devbox-version would say the same"
|
||||
fi
|
||||
elif [ -n "$EXPECTED_IMAGE_VERSION" ]; then
|
||||
if [ "$(strip_v "$EXPECTED_IMAGE_VERSION")" = "$(strip_v "$MANIFEST_RELEASE_TAG")" ]; then
|
||||
pass "image version $MANIFEST_RELEASE_TAG (matches --expected-image-version)"
|
||||
elif [ -n "$MANIFEST_PI_VERSION" ] &&
|
||||
[ "$(strip_v "$EXPECTED_IMAGE_VERSION")" = "$MANIFEST_PI_VERSION" ]; then
|
||||
fail "--expected-image-version $EXPECTED_IMAGE_VERSION is the pi version, not the image release tag — use --expected-version $EXPECTED_IMAGE_VERSION (this image is $MANIFEST_RELEASE_TAG)"
|
||||
else
|
||||
fail "image version mismatch: expected $EXPECTED_IMAGE_VERSION, got $MANIFEST_RELEASE_TAG — the recreate did not pick up the intended image"
|
||||
fi
|
||||
else
|
||||
warn "image version $MANIFEST_RELEASE_TAG (no --expected-image-version given — informational only)"
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "-- Persisted named volumes (must survive --force-recreate) --"
|
||||
|
||||
@@ -218,6 +328,15 @@ _pkg_registered() {
|
||||
fi
|
||||
}
|
||||
|
||||
# True when a literal `npm:pi-atelier` entry is still present — the
|
||||
# volume-resident registration the entrypoint migrates away from.
|
||||
_npm_atelier_present() {
|
||||
_s="$HOME/.pi/agent/settings.json"
|
||||
[ -f "$_s" ] || return 1
|
||||
command -v jq >/dev/null 2>&1 || return 1
|
||||
jq -e '(.packages // []) | any(. == "npm:pi-atelier")' "$_s" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
if [ -f "$HOME/.pi/agent/settings.json" ]; then
|
||||
for pkg in pi-fork pi-observational-memory; do
|
||||
if _pkg_registered "$pkg"; then
|
||||
@@ -234,6 +353,75 @@ if [ -f "$HOME/.pi/agent/settings.json" ]; then
|
||||
fail "pi-studio NOT in settings.json packages[] (studio variant)"
|
||||
fi
|
||||
fi
|
||||
|
||||
# pi-atelier — vendored from v1.7.0 on. Absent on older images, and
|
||||
# deliberately unregistered when DEVBOX_ATELIER=0; neither is a failure.
|
||||
if [ -d /opt/pi-atelier ]; then
|
||||
if [ "${DEVBOX_ATELIER:-1}" = "0" ]; then
|
||||
if _pkg_registered pi-atelier; then
|
||||
fail "pi-atelier still in packages[] despite DEVBOX_ATELIER=0"
|
||||
else
|
||||
pass "pi-atelier unregistered (DEVBOX_ATELIER=0, as requested)"
|
||||
fi
|
||||
elif _pkg_registered pi-atelier; then
|
||||
pass "pi-atelier registered in settings.json packages[]"
|
||||
else
|
||||
fail "pi-atelier NOT in settings.json packages[] (sidebar will not load)"
|
||||
fi
|
||||
if _npm_atelier_present; then
|
||||
fail "stale npm:pi-atelier still in packages[] — it resolves through the ~/.pi/npm-global VOLUME and shadows the pinned /opt copy (entrypoint migration did not run)"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── agent-browser must resolve to the image, not the config volume ────
|
||||
# The same volume-shadowing hazard already asserted for pi (above) and
|
||||
# pi-atelier (just now), for the third package it has bitten. This check
|
||||
# belongs HERE rather than only in smoke-test.sh: a build-time container has an
|
||||
# empty ~/.pi/npm-global, so smoke-test can never see the stale copy that a
|
||||
# real recreate inherits. Measured instance: 0.27.0 from 2026-07-17 shadowed
|
||||
# the image's 0.35.2 for ~7 weeks on mbp-m1-2020, silently supplying an older
|
||||
# BUNDLED SKILL (3 skillsets vs 8) — the agent read the stale instructions
|
||||
# without any version mismatch ever being surfaced.
|
||||
AB_PATH=$(command -v agent-browser 2>/dev/null || true)
|
||||
if [ -z "$AB_PATH" ]; then
|
||||
warn "agent-browser not on PATH (expected in v1.6.0+ images; skipping shadow check)"
|
||||
else
|
||||
AB_REAL=$(readlink -f "$AB_PATH" 2>/dev/null || echo "$AB_PATH")
|
||||
AB_VER=$(agent-browser --version 2>/dev/null | head -n1)
|
||||
case "$AB_REAL" in
|
||||
/usr/*)
|
||||
pass "agent-browser resolves to the image copy (${AB_VER:-version unknown})"
|
||||
;;
|
||||
*)
|
||||
fail "agent-browser resolves to $AB_REAL (${AB_VER:-version unknown}) — a ~/.pi/npm-global VOLUME copy is shadowing the image; the entrypoint retirement guard did not run or could not move it"
|
||||
;;
|
||||
esac
|
||||
if [ -d "$HOME/.pi/npm-global/lib/node_modules/agent-browser" ]; then
|
||||
fail "stale agent-browser still present in the ~/.pi/npm-global volume (entrypoint guard did not retire it)"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── pi <-> pi-atelier compatibility floor ─────────────────────────────
|
||||
# atelier < 0.7.1 wraps pi's private TUI renderer in a way that recurses under
|
||||
# pi >= 0.84: pi hangs at startup burning CPU, with no error message. atelier's
|
||||
# own peerDependencies (>=0.80.7) do not encode this. Assert it here too, not
|
||||
# just in the build-time smoke test: this script runs after a real
|
||||
# `--force-recreate` on a live box, where a volume-resident old copy is exactly
|
||||
# what could bite.
|
||||
if [ -d /opt/pi-atelier ] && command -v jq >/dev/null 2>&1; then
|
||||
_ge() { [ "$(printf '%s\n%s\n' "$1" "$2" | sort -V | head -n1)" = "$2" ]; }
|
||||
_av=$(jq -r '.version // empty' /opt/pi-atelier/package.json 2>/dev/null || true)
|
||||
_pv=$(pi --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -n1 || true)
|
||||
if [ -n "$_av" ] && [ -n "$_pv" ]; then
|
||||
if _ge "$_pv" 0.84.0 && ! _ge "$_av" 0.7.1; then
|
||||
fail "pi $_pv with pi-atelier $_av — atelier < 0.7.1 hangs pi >= 0.84 at startup (bump PI_ATELIER_REF in Dockerfile.variant)"
|
||||
else
|
||||
pass "pi $_pv + pi-atelier $_av (compatibility floor OK)"
|
||||
fi
|
||||
else
|
||||
warn "could not compare pi/pi-atelier versions (pi='$_pv' atelier='$_av')"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo
|
||||
|
||||
+594
-14
@@ -5,14 +5,18 @@
|
||||
#
|
||||
# Verifies:
|
||||
# - pi binary present and (if EXPECTED_PI_VERSION set) matches CI's resolved version
|
||||
# - node MAJOR matches Dockerfile.base's ARG NODE_VERSION (if EXPECTED_NODE_MAJOR set)
|
||||
# - mempalace core matches the audited pin (if EXPECTED_MEMPALACE_VERSION set)
|
||||
# - new v1.0.0 base additions (pandoc, graphviz, imagemagick, yq, tealdeer)
|
||||
# - typst PDF engine for pandoc (Unreleased) — `pandoc --pdf-engine=typst`
|
||||
# - typst PDF engine for pandoc (v1.4.0) — `pandoc --pdf-engine=typst`
|
||||
# - non-modal editors nano + micro (alongside nvim)
|
||||
# - terminfo for modern emulators: xterm-kitty, xterm-ghostty, wezterm,
|
||||
# alacritty, foot (kitty-terminfo + ncurses-term + compiled ghostty alias)
|
||||
# - tmux 0-indexing baked in /etc/tmux.conf (required for pi-studio variants)
|
||||
# - pi-toolkit cloned at /opt/pi-toolkit
|
||||
# - pi-extensions cloned at /opt/pi-extensions
|
||||
# - pi-atelier vendored at /opt/pi-atelier, registered from /opt (not npm:),
|
||||
# and >= the version floor pi's TUI requires (see the floor test)
|
||||
# - pi-fork + pi-observational-memory cloned with node_modules baked
|
||||
# - entrypoint deploys pi-toolkit keybindings symlink
|
||||
# - entrypoint deploys ≥4 extensions
|
||||
@@ -41,12 +45,23 @@ PASS=0; FAIL=0
|
||||
# catching an unexpected +GB regression.
|
||||
SIZE_THRESHOLD_MB=3800
|
||||
|
||||
# On failure, surface the last few lines the command produced. This used to
|
||||
# discard output entirely (`>/dev/null 2>&1`), which made a red ❌ carry zero
|
||||
# diagnostic weight: explaining the single v1.8.0 stage-default failure took a
|
||||
# full CI-log dig plus a registry-config inspection, when the container had
|
||||
# already printed the answer and thrown it away. Assertions that want a
|
||||
# diagnostic just echo it to stderr — it stays hidden while they pass.
|
||||
run() {
|
||||
local label="$1"; local cmd="$2"
|
||||
if docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" >/dev/null 2>&1; then
|
||||
local out
|
||||
if out=$(docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" 2>&1); then
|
||||
printf " ✅ %s\n" "$label"; PASS=$((PASS+1))
|
||||
else
|
||||
printf " ❌ %s\n" "$label"; FAIL=$((FAIL+1))
|
||||
# `if`, not `&&` — a trailing false under `set -e` would abort the script.
|
||||
if [ -n "$out" ]; then
|
||||
printf " └─ %s\n" "$(printf '%s' "$out" | tail -3 | tr '\n' ' ' | cut -c1-300)"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
@@ -77,8 +92,31 @@ if [ -n "${EXPECTED_PI_VERSION:-}" ]; then
|
||||
else
|
||||
run "pi" "pi --version"
|
||||
fi
|
||||
run "node" "node --version"
|
||||
# Until 2026-09-07 this was a bare `run "node" "node --version"`, which asserts
|
||||
# only that the binary exists and exits 0 — the printed version was never
|
||||
# compared to anything. A node major bump would therefore have passed this suite
|
||||
# SILENTLY, while a reader skimming it would reasonably assume node regressions
|
||||
# were covered. EXPECTED_NODE_MAJOR closes that: CI derives it from
|
||||
# Dockerfile.base's ARG NODE_VERSION (the single source of truth), so this also
|
||||
# catches a stale cached layer whose node does not match the declared ARG.
|
||||
if [ -n "${EXPECTED_NODE_MAJOR:-}" ]; then
|
||||
run_expect "node major matches Dockerfile ARG" "node --version" "v${EXPECTED_NODE_MAJOR}."
|
||||
else
|
||||
run "node" "node --version"
|
||||
fi
|
||||
run "git" "git --version"
|
||||
# NOTE: the shellcheck binary is a GATE DEPENDENCY, not a convenience.
|
||||
# scripts/lint-shell.sh is the release gate (the lint-gate job resolve-versions
|
||||
# depends on) and it exits 2 when the binary is missing, by design — "a gate that
|
||||
# cannot run must not pass". Measured on v1.8.14: it was absent from the image, so
|
||||
# that gate could not be run by a developer in ANY container, only in CI.
|
||||
# Asserted here so its absence fails a build instead of being discovered by a hook
|
||||
# that then refuses every push (hooks/pre-push).
|
||||
#
|
||||
# This comment must not BEGIN with the tool's name: a line starting with
|
||||
# `# shellcheck` is parsed as a DIRECTIVE, not a comment (SC1073/SC1072). The
|
||||
# gate added in this same change caught that here, before the push.
|
||||
run "shellcheck (lint gate dependency)" "shellcheck --version | grep -qE '^version: [0-9]'"
|
||||
run "aws" "aws --version"
|
||||
run "uv" "uv --version"
|
||||
run "nvim" "nvim --version"
|
||||
@@ -89,6 +127,136 @@ run "terminfo: modern emulators (ncurses-term)" 'for t in wezterm alacritty foot
|
||||
run "terminfo: xterm-ghostty alias (tic)" "infocmp -x xterm-ghostty >/dev/null 2>&1"
|
||||
run "nvim true-colour default (sysinit.vim)" "nvim --headless -c 'lua os.exit(vim.o.termguicolors and 0 or 1)'"
|
||||
run "mempalace-mcp" "mempalace-mcp --help"
|
||||
run "mempalace-pi-session on PATH" "mempalace-pi-session --help"
|
||||
# The staging dir must sit next to the palace, not in a disposable cache: the
|
||||
# palace keys per-source dedup on the STAGED path, so a stage that can be wiped
|
||||
# while the palace survives lets `mempalace sync` prune every drawer mined from
|
||||
# it. Assert the resolved default, not an env var — the guarantee is "stage
|
||||
# shares the palace's lifetime", which an ENV pin would quietly break.
|
||||
# NOTE: --sessions-dir gets an EMPTY temp dir, never /tmp. The stage banner is
|
||||
# printed before any export, so nothing needs to be found — and pointing a
|
||||
# default-staged run at a populated dir would export whatever transcripts it
|
||||
# finds into the real stage, which is how a synthetic test session ends up
|
||||
# staged for mining as if it were a real conversation.
|
||||
#
|
||||
# Asserted $HOME-RELATIVE, not against a literal /home/developer. `run` invokes
|
||||
# `docker run --entrypoint=""`, and neither Dockerfile sets USER or ENV HOME
|
||||
# (HOME is set by entrypoint-user.sh, which --entrypoint="" deliberately skips),
|
||||
# so these assertions execute as root with HOME=/root. The original literal
|
||||
# /home/developer form could therefore never match and failed the v1.8.0
|
||||
# release — a test bug, not a product one: the stage resolution was correct all
|
||||
# along, it just follows $HOME. The invariant under test ("the stage sits beside
|
||||
# the palace, sharing its lifetime") is user-independent, so pinning the user
|
||||
# was never part of it. A cache-dir default still fails the pattern below, which
|
||||
# is the regression this guards.
|
||||
#
|
||||
# It went unnoticed for three days because this workflow only triggers on
|
||||
# `push: tags: v*` — the assertion was added on a main push, so v1.8.0 was its
|
||||
# first execution ever. Use the `smoke_only` workflow_dispatch input to run
|
||||
# smoke against HEAD without cutting a tag.
|
||||
run "pi stage defaults next to the palace (not a cache dir)" '
|
||||
out=$(mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||||
stage=$(echo "$out" | grep -oE "stage=[^ ]+" | head -1)
|
||||
echo "resolved ${stage:-<no stage= line>} with HOME=$HOME" >&2
|
||||
case "$stage" in
|
||||
"stage=$HOME/.mempalace/pi-stage/"*) exit 0 ;;
|
||||
*) exit 1 ;;
|
||||
esac
|
||||
'
|
||||
# Companion to the above: the deployment-specific case the literal assertion was
|
||||
# reaching for, done properly by supplying the HOME the container actually runs
|
||||
# with instead of assuming it.
|
||||
run "pi stage is palace-adjacent for the developer user" '
|
||||
out=$(HOME=/home/developer mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||||
echo "$out" | grep -oE "stage=[^ ]+" | head -1 >&2
|
||||
echo "$out" | grep -q "stage=/home/developer/.mempalace/pi-stage/"
|
||||
'
|
||||
run "pi stage follows MEMPALACE_PALACE_PATH" '
|
||||
out=$(MEMPALACE_PALACE_PATH=/tmp/alt/.mempalace/palace \
|
||||
mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||||
echo "$out" | grep -q "stage=/tmp/alt/.mempalace/pi-stage/"
|
||||
'
|
||||
# The feeder's --agent default is WHO a drawer is attributed to. mempalace core
|
||||
# records neither the machine nor the harness on a write, and one shared bearer
|
||||
# token means the server cannot tell clients apart, so toolkit c64ffa1 changed
|
||||
# this default from $USER to pi@$MEMPALACE_PI_DEVICE — the one string that makes
|
||||
# a write attributable to both. Nothing ever PRINTED the resolved value (the
|
||||
# banner shows mode= and stage= only), so an image built from a pre-c64ffa1
|
||||
# toolkit ref would ship unattributed writes with every check still green.
|
||||
#
|
||||
# `--help` assigns AGENT (script top) before it parses args, then exits 0 with
|
||||
# no side effects — so `bash -x` observes the REAL resolution, env interpolation
|
||||
# and fallback included, rather than grepping the source for a literal line that
|
||||
# any reformat would break. Two-sided on purpose: device set => pi@<device>;
|
||||
# device UNSET => must not be pi@anything. The second half is what fails against
|
||||
# the old unconditional $USER default, which ignored the device entirely.
|
||||
#
|
||||
# Probes the PATH entry (a symlink into the /opt clone) rather than that clone
|
||||
# path directly: this is the invocation the systemd/launchd timers and
|
||||
# entrypoint-user.sh actually use, so it is the default that reaches the palace.
|
||||
run "feeder resolves --agent to pi@<device> (drawer attribution)" '
|
||||
f=$(command -v mempalace-pi-session) || { echo "feeder not on PATH" >&2; exit 1; }
|
||||
with=$(MEMPALACE_PI_DEVICE=smoke-device bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
|
||||
without=$(env -u MEMPALACE_PI_DEVICE bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
|
||||
echo "resolved with-device=[$with] without-device=[$without]" >&2
|
||||
[ "$with" = "pi@smoke-device" ] || exit 1
|
||||
case "$without" in pi@*) exit 1 ;; esac
|
||||
echo ok
|
||||
'
|
||||
# Regression guard for the pi transcript exporter. If pi ever changes its
|
||||
# session JSONL shape, the exporter stops recognising sessions and the palace
|
||||
# silently gets nothing (or, worse, raw JSON chunked as prose). Feed it a
|
||||
# synthetic session and assert it is actually exported. Uses --dry-run so no
|
||||
# palace is touched, and a temp stage so nothing real is written.
|
||||
run "pi transcript exporter recognises a pi session" '
|
||||
set -e
|
||||
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
|
||||
{
|
||||
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
|
||||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question one\"}}"
|
||||
a=$(printf "a%.0s" $(seq 1 1200))
|
||||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"$a\"}]}}"
|
||||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question two\"}}"
|
||||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"short reply\"}]}}"
|
||||
} > "$s/2026-01-01T00-00-00-000Z_smoke.jsonl"
|
||||
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
|
||||
echo "$out" | grep -q "Exported 1 session"
|
||||
'
|
||||
# The same guard from the other side: a session with no real assistant output
|
||||
# (an abandoned prompt, whose bulk is injected skill text) must NOT be filed.
|
||||
run "pi transcript exporter rejects an abandoned session" '
|
||||
set -e
|
||||
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
|
||||
{
|
||||
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke2\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
|
||||
u=$(printf "u%.0s" $(seq 1 13000))
|
||||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"$u\"}}"
|
||||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"Ready. What would you like to work on?\"}]}}"
|
||||
} > "$s/2026-01-01T00-00-00-000Z_smoke2.jsonl"
|
||||
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
|
||||
echo "$out" | grep -q "no sessions qualified"
|
||||
'
|
||||
# The remote-palace-without-inbox skip must ANNOUNCE itself, not vanish. This
|
||||
# branch of entrypoint-user.sh runs at container start (not reachable from a
|
||||
# `docker run` one-shot), so assert against the entrypoint that actually shipped
|
||||
# in the image. Guards a silent regression back to the bare `:` no-op, which
|
||||
# left a container contributing nothing to the palace with no artifact saying
|
||||
# why — the log it would normally leave is written by the other branch.
|
||||
run_expect "remote-palace-without-inbox skip is announced, not silent" \
|
||||
"grep -o 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | head -1" \
|
||||
"MemPalace catch-up skipped"
|
||||
run "...and the skip notice names the variable that fixes it" \
|
||||
"grep -A6 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | grep -q 'MEMPALACE_PI_SSH_TARGET'"
|
||||
# A remote mine that FAILS must not report success. MCP answers a hard tool
|
||||
# failure with HTTP 200 and the tool's own JSON escaped inside
|
||||
# result.content[].text, so the feeder's old `'\"error\"' in body` check could
|
||||
# never see it: on 2026-08-15 a mine that died with "source directory not found:
|
||||
# '/data/feed/...'" logged "Done. Wing updated." and exited 0, and this
|
||||
# container's transcripts were filed nowhere for a whole session. The feeder
|
||||
# carries fixtures for that exact body; run them against the baked toolkit so a
|
||||
# stale/reverted toolkit ref can't reintroduce a silent feed.
|
||||
run "baked feeder detects a failed remote mine (no silent false success)" \
|
||||
"mempalace-pi-session --self-test"
|
||||
# v1.0.0 base additions — verify presence and basic functionality.
|
||||
run "pandoc" "pandoc --version"
|
||||
run "typst" "typst --version"
|
||||
@@ -101,6 +269,8 @@ run "socat" "socat -V"
|
||||
run "studio-expose helper" "test -x /usr/local/bin/studio-expose"
|
||||
run "image-baked pi-devbox-environment skill" \
|
||||
"test -f /usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md"
|
||||
run "image-baked credential-incident-response skill" \
|
||||
"test -f /usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md"
|
||||
run "global-AGENTS append snippet present" \
|
||||
"test -f /usr/local/share/pi-devbox/pi-global-AGENTS.append.md"
|
||||
run "pi-devbox block merged into pi-global-AGENTS.md" \
|
||||
@@ -119,6 +289,16 @@ run "image-baked mempalace fallback skill" \
|
||||
# baked copy must be the fresh package copy (Option 1), not the stale snapshot.
|
||||
run "pi-extensions skill refreshed from package when present" \
|
||||
"if [ -f /opt/pi-extensions/skill/SKILL.md ]; then cmp -s /opt/pi-extensions/skill/SKILL.md /usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md; else true; fi"
|
||||
# Runtime ownership handover (v1.8.5): the baked links are a FALLBACK, and
|
||||
# skillset-OWNED skills must be repointed at the live clone when one is mounted.
|
||||
# The list is data, so assert its content, not just its presence: mempalace in,
|
||||
# pi-extensions deliberately out (its skillset copy is a lagging duplicate).
|
||||
run "devbox-skill-reconcile helper present + executable" \
|
||||
"test -x /usr/local/bin/devbox-skill-reconcile"
|
||||
run "skillset-owned list ships and names mempalace" \
|
||||
"grep -qx 'mempalace' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
|
||||
run "skillset-owned list excludes pi-extensions (ownership)" \
|
||||
"! grep -qx 'pi-extensions' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
|
||||
|
||||
# ── tmux 0-indexing (required for pi-studio variants) ─────────────────
|
||||
echo ""
|
||||
@@ -137,6 +317,44 @@ run "pi-fork clone + node_modules" \
|
||||
"test -f /opt/pi-fork/package.json && test -d /opt/pi-fork/node_modules"
|
||||
run "pi-observational-memory clone + node_modules" \
|
||||
"test -f /opt/pi-observational-memory/package.json && test -d /opt/pi-observational-memory/node_modules"
|
||||
# ...and that the clone carries the AUTH FIX, not merely that it exists. om's
|
||||
# pre-flight hasUsableAuth() check silently disabled `recall` for ~8 weeks once
|
||||
# pi moved to request-time SigV4 signing and stopped exposing a static Bedrock
|
||||
# key; upstream fixed it in ce9fc98, adopted in v1.8.4. PI_OBSMEM_REF tracks
|
||||
# master, so an upstream revert or force-push would ship a dead `recall` with
|
||||
# the clone assertion above still green — the exact gap flagged as open in the
|
||||
# v1.8.5 changelog.
|
||||
#
|
||||
# Pin the markers to src/runtime.ts, the fix SITE, rather than grepping the
|
||||
# repo: two of these three strings also appear under tests/, so a repo-wide
|
||||
# grep stays green with runtime.ts itself reverted. That is a false green of the
|
||||
# same family as the old skill-snapshot canary.
|
||||
run "pi-observational-memory carries the ce9fc98 auth fix (recall stays alive)" '
|
||||
f=/opt/pi-observational-memory/src/runtime.ts
|
||||
test -f "$f" || { echo "fix site missing: $f" >&2; exit 1; }
|
||||
for m in availability_recheck providerCredentialConfigured hasConfiguredAuth; do
|
||||
grep -q "$m" "$f" || { echo "marker absent from runtime.ts: $m" >&2; exit 1; }
|
||||
done
|
||||
echo ok
|
||||
'
|
||||
# pi-atelier: deliberately NO node_modules assertion, unlike its siblings —
|
||||
# it declares zero runtime dependencies (only peerDeps, satisfied by the baked
|
||||
# pi) and has no build step, so Dockerfile.variant skips `npm install` for it.
|
||||
# Assert what pi actually loads instead: the entry point named by its
|
||||
# package.json `pi.extensions` key.
|
||||
run "pi-atelier clone + entry point" \
|
||||
"test -f /opt/pi-atelier/package.json && test -f /opt/pi-atelier/extensions/index.ts"
|
||||
|
||||
# ── pi <-> pi-atelier compatibility floor (executable, not a comment) ──
|
||||
# pi-atelier < 0.7.1 wraps pi's PRIVATE TUI renderer in a way that recurses
|
||||
# under pi >= 0.84: pi hangs at startup burning CPU, with no error. Upstream
|
||||
# fixed it in 0.7.1/0.7.2, but atelier's peerDependencies still say
|
||||
# `>=0.80.7`, so neither npm nor pi can warn about the real floor. Both
|
||||
# versions are pinned in Dockerfile.variant; this makes a bad PAIRING fail the
|
||||
# build instead of publishing an image whose TUI never starts.
|
||||
run_expect "pi-atelier >= 0.7.1 floor for pi >= 0.84 (startup-hang guard)" \
|
||||
'ge() { [ "$(printf "%s\n%s\n" "$1" "$2" | sort -V | head -n1)" = "$2" ]; }; AV=$(jq -r ".version // empty" /opt/pi-atelier/package.json 2>/dev/null); PV=$(pi --version 2>/dev/null | grep -oE "[0-9]+\.[0-9]+\.[0-9]+" | head -n1); if [ -z "$AV" ] || [ -z "$PV" ]; then echo "unreadable versions (atelier=$AV pi=$PV)"; elif ge "$PV" 0.84.0 && ! ge "$AV" 0.7.1; then echo "VIOLATION: pi $PV with pi-atelier $AV"; else echo "compatible: pi $PV + pi-atelier $AV"; fi' \
|
||||
"compatible:"
|
||||
|
||||
# pi-studio is present only in the :latest-studio variant. Auto-detect by
|
||||
# probing /opt/pi-studio so this one script covers both variants.
|
||||
@@ -157,24 +375,201 @@ echo ""
|
||||
echo "── Build provenance ──"
|
||||
run "/etc/pi-devbox/build-manifest.json present" \
|
||||
"test -f /etc/pi-devbox/build-manifest.json"
|
||||
run_expect "manifest records pi-extensions component" \
|
||||
"cat /etc/pi-devbox/build-manifest.json" '"pi-extensions"'
|
||||
run_expect "manifest records pi_version" \
|
||||
"cat /etc/pi-devbox/build-manifest.json" '"pi_version"'
|
||||
# These next checks replace three that grepped the manifest for the FIELD NAME
|
||||
# and never looked at the value:
|
||||
#
|
||||
# run_expect "manifest records pi_version" "cat …manifest.json" '"pi_version"'
|
||||
#
|
||||
# which passes on {"pi_version": ""} and on {"pi_version": null}. The tell was
|
||||
# visible in its own passing output — `✅ manifest records pi_version (got
|
||||
# "pi_version")` echoes the key back as the thing it claims to have found.
|
||||
# Two failure modes were therefore invisible: a key that survives with an empty
|
||||
# or garbage value, and a key that vanishes from the manifest while every
|
||||
# remaining value still looks fine.
|
||||
#
|
||||
# Those two need SEPARATE assertions, and the reason is a trap worth keeping in
|
||||
# writing: an "every component value is a valid SHA" loop passes VACUOUSLY on
|
||||
# components:{} — jq's all() over an empty list is true — so the value check
|
||||
# alone would go green on a manifest that lost every component. Mutation-tested
|
||||
# 2026-08-25 across nine fabricated manifests (empty map, deleted key, "",
|
||||
# null, "unknown", 12-hex truncation, 40 non-hex chars, legit null pi-studio).
|
||||
run "manifest declares every required component key" '
|
||||
req="pi-toolkit pi-extensions pi-fork pi-observational-memory pi-atelier mempalace-toolkit pi-studio"
|
||||
for k in $req; do
|
||||
jq -e --arg k "$k" "(.components|has(\$k))" /etc/pi-devbox/build-manifest.json >/dev/null \
|
||||
|| { echo "manifest lost component key: $k" >&2; exit 1; }
|
||||
done
|
||||
'
|
||||
# Subsumes the old `! grep -q \"unknown\"` check ("unknown" is not 40-hex), and
|
||||
# also catches "", null and truncated SHAs, which that grep let through. null is
|
||||
# legitimate for pi-studio alone: the non-studio variant has no such clone.
|
||||
run "manifest component values are resolved 40-hex commits" '
|
||||
jq -e "
|
||||
.components
|
||||
| to_entries
|
||||
| all(if .key == \"pi-studio\" and .value == null then true
|
||||
else (.value|type) == \"string\" and (.value|test(\"^[0-9a-f]{40}\$\")) end)
|
||||
" /etc/pi-devbox/build-manifest.json >/dev/null
|
||||
'
|
||||
# pi_version against ground truth, same shape as the mempalace check below.
|
||||
# Chains with the "pi version matches build arg" assertion earlier in this file:
|
||||
# together they tie build arg -> installed binary -> recorded manifest, so a
|
||||
# manifest written from a stale variable cannot pass by agreeing with itself.
|
||||
run "manifest pi_version matches the installed pi" '
|
||||
m=$(jq -r ".pi_version // empty" /etc/pi-devbox/build-manifest.json)
|
||||
b=$(pi --version 2>/dev/null | head -n1 | tr -d "\r")
|
||||
echo "manifest=[$m] installed=[$b]" >&2
|
||||
[ -n "$m" ] && [ "$m" = "$b" ]
|
||||
'
|
||||
# Top-level provenance fields: assert the SHAPE of each value, and only when the
|
||||
# field is populated. source_revision and build_date legitimately default to
|
||||
# empty (Dockerfile.variant ARGs) on a plain local `docker build`, so demanding
|
||||
# them would fail honest local smoke runs; a populated-but-malformed value is
|
||||
# the actual defect. release_tag defaults to "dev", so empty means a broken write.
|
||||
run "manifest top-level fields are well-formed, not merely present" '
|
||||
j=/etc/pi-devbox/build-manifest.json
|
||||
t=$(jq -r ".release_tag // empty" $j)
|
||||
r=$(jq -r ".source_revision // empty" $j)
|
||||
d=$(jq -r ".build_date // empty" $j)
|
||||
echo "release_tag=[$t] source_revision=[$r] build_date=[$d]" >&2
|
||||
[ -n "$t" ] || { echo "release_tag empty (ARG default is dev)" >&2; exit 1; }
|
||||
if [ -n "$r" ]; then
|
||||
printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" || { echo "source_revision not a 40-hex commit" >&2; exit 1; }
|
||||
fi
|
||||
if [ -n "$d" ]; then
|
||||
printf "%s" "$d" | grep -qE "^[0-9]{4}-[0-9]{2}-[0-9]{2}T" || { echo "build_date not ISO-8601" >&2; exit 1; }
|
||||
fi
|
||||
'
|
||||
# mempalace CORE was absent from the manifest through v1.8.5: the toolkit SHA
|
||||
# was recorded but the palace version behind the MCP tools was not, so a palace
|
||||
# bug could not be correlated to an image version. Assert the field exists AND
|
||||
# equals the installed binary — recording it from ARG MEMPALACE_VERSION instead
|
||||
# would look identical here yet drift silently the first time an install
|
||||
# resolved to something other than the pin, which is the whole reason this file
|
||||
# is built from ground truth. `// empty` matters: jq -r prints the 4-char
|
||||
# string "null" for a JSON null, which would satisfy a naive -n test.
|
||||
run "manifest mempalace_version matches the installed core" '
|
||||
m=$(jq -r ".mempalace_version // empty" /etc/pi-devbox/build-manifest.json)
|
||||
b=$(mempalace --version 2>/dev/null | head -n1 | tr -d "\r"); b=${b##* }
|
||||
echo "manifest=[$m] installed=[$b]" >&2
|
||||
[ -n "$m" ] && [ "$m" = "$b" ]
|
||||
'
|
||||
# ... and, when CI supplies it, that the installed core is the version CI
|
||||
# actually AUDITED (published + not yanked on PyPI, in resolve-versions). This
|
||||
# does NOT duplicate the check above, which compares two properties of one
|
||||
# image and so cannot notice that BOTH are the wrong version. The live failure
|
||||
# mode it covers: the variant builds `FROM` a base tag chosen by base-decide's
|
||||
# content hash, so a bug in that hashing (the reason scripts/check-base-hash.sh
|
||||
# exists) could reuse a cached base built from an OLDER MEMPALACE_VERSION pin —
|
||||
# internally consistent, silently stale, invisible to every other assertion.
|
||||
if [ -n "${EXPECTED_MEMPALACE_VERSION:-}" ]; then
|
||||
run "installed mempalace matches CI's audited pin (${EXPECTED_MEMPALACE_VERSION})" "
|
||||
b=\$(mempalace --version 2>/dev/null | head -n1 | tr -d '\r'); b=\${b##* }
|
||||
echo \"installed=[\$b] audited_pin=[${EXPECTED_MEMPALACE_VERSION}]\" >&2
|
||||
[ \"\$b\" = \"${EXPECTED_MEMPALACE_VERSION}\" ]
|
||||
"
|
||||
fi
|
||||
# Every component must be a resolved commit (or null for pi-studio in the
|
||||
# non-studio variant) — 'unknown' means a clone silently failed to resolve.
|
||||
run "manifest has no unresolved ('unknown') components" \
|
||||
"! grep -q '\"unknown\"' /etc/pi-devbox/build-manifest.json"
|
||||
# pi-devbox-version wraps the manifest into a human-first command (this
|
||||
# PR); verify the binary is present, executable, and both output modes work.
|
||||
# non-studio variant) — now enforced by the 40-hex value check above, which
|
||||
# strictly subsumes the old whole-file grep for '"unknown"'. Only rev() ever
|
||||
# emits "unknown" and rev() feeds components only, so nothing is lost.
|
||||
# pi-devbox-version wraps the manifest into a human-first command; verify the
|
||||
# binary is present, executable, and that all three output modes work.
|
||||
run "pi-devbox-version binary present + executable" \
|
||||
"test -x /usr/local/bin/pi-devbox-version"
|
||||
run_expect "pi-devbox-version human output shows release tag" \
|
||||
"pi-devbox-version" "pi-devbox "
|
||||
run_expect "pi-devbox-version --json round-trips the manifest" \
|
||||
"pi-devbox-version --json" '"release_tag"'
|
||||
# --json is a verbatim `cat` of the manifest, so "round-trips" is assertable
|
||||
# literally. The old form grepped the output for the string "release_tag" — the
|
||||
# key name again — which would pass on a truncated or re-serialised dump.
|
||||
run "pi-devbox-version --json round-trips the manifest byte-for-byte" '
|
||||
a=$(cat /etc/pi-devbox/build-manifest.json)
|
||||
b=$(pi-devbox-version --json)
|
||||
[ "$a" = "$b" ] || { echo "--json output differs from the manifest on disk" >&2; exit 1; }
|
||||
'
|
||||
run_expect "pi-devbox-version --quiet is a compact one-liner" \
|
||||
"pi-devbox-version --quiet | wc -l" "1"
|
||||
# ── Vendored skill snapshot provenance ─────────────────────────────────
|
||||
# The vendored mempalace skill is the one baked artefact with no /opt clone
|
||||
# behind it (private upstream — see VENDORED.md), so until now the manifest
|
||||
# could not say which skillset commit it came from. Two fields now travel with
|
||||
# it: the CLAIMED ref (ARG default in Dockerfile.variant) and the MEASURED
|
||||
# sha256 of the shipped bytes. Assert both are well-formed, and — separately —
|
||||
# that the measurement still describes the file in the image.
|
||||
#
|
||||
# Kept as two assertions for the same reason the component checks are: one
|
||||
# proves the fields are not empty/garbage, the other proves they are not merely
|
||||
# self-consistent. A single combined check could pass on a manifest whose hash
|
||||
# was computed from a file that was later overwritten (the pi-extensions skill
|
||||
# copy at Dockerfile.variant:165 does exactly that kind of overwrite, one stage
|
||||
# earlier), which is the failure this second one exists to catch.
|
||||
run "manifest records the vendored skill snapshot provenance" '
|
||||
j=/etc/pi-devbox/build-manifest.json
|
||||
r=$(jq -r ".skillset_snapshot_ref // empty" $j)
|
||||
s=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
|
||||
echo "ref=[$r] tree_sha256=[$s]" >&2
|
||||
printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" \
|
||||
|| { echo "skillset_snapshot_ref is not a 40-hex commit" >&2; exit 1; }
|
||||
printf "%s" "$s" | grep -qxE "[0-9a-f]{64}" \
|
||||
|| { echo "skillset_snapshot_tree_sha256 is not a 64-hex digest" >&2; exit 1; }
|
||||
'
|
||||
# Recomputes over the whole DIRECTORY with the same tree_sha256() pipeline
|
||||
# Dockerfile.variant used to measure it, not a plain `sha256sum SKILL.md` —
|
||||
# a file-only compare here would pass even if the manifest recorded a
|
||||
# fingerprint over a directory that has since grown a second file (this is
|
||||
# not hypothetical: pi-extensions already ships two files for its skill).
|
||||
run "manifest skill fingerprint matches the baked snapshot" '
|
||||
j=/etc/pi-devbox/build-manifest.json
|
||||
d=/usr/local/share/pi-devbox/skills/mempalace
|
||||
m=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
|
||||
a=$( (cd "$d" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum) | sha256sum | cut -d" " -f1)
|
||||
echo "manifest=[$m] actual=[$a]" >&2
|
||||
[ -n "$m" ] && [ "$m" = "$a" ]
|
||||
'
|
||||
|
||||
# ── Which pi-extensions skill copy shipped ──────────────────────────────
|
||||
# Closes the silent-fallback hole. The refresh in Dockerfile.variant is guarded
|
||||
# by `[ -f /opt/pi-extensions/skill/SKILL.md ]`, so a build whose clone predates
|
||||
# the co-located skill keeps the vendored floor and still succeeds GREEN, with
|
||||
# nothing recording that a snapshot shipped instead of the package copy. Measured
|
||||
# 2026-09-10: the floor had been stale since 2026-07-30, so that path would have
|
||||
# shipped a six-week-old skill in silence. The floor is fresh now and gated by the
|
||||
# skill-floor lint job, but "the fallback is currently harmless" is a fact with a
|
||||
# shelf life, whereas "the image says which copy it got" keeps working.
|
||||
#
|
||||
# vendored-floor FAILS here rather than merely warning: these images track main,
|
||||
# where the package has co-located skill/ since fa04d20, so a fallback means the
|
||||
# clone did not resolve as intended and that is a defect to investigate. A fork
|
||||
# deliberately pointing at a mirror without skill/ is the one case that should
|
||||
# edit this assertion — which is the honest place for that decision to surface.
|
||||
run "manifest names which pi-extensions skill copy shipped" '
|
||||
j=/etc/pi-devbox/build-manifest.json
|
||||
s=$(jq -r ".pi_extensions_skill_source // empty" $j)
|
||||
h=$(jq -r ".pi_extensions_skill_tree_sha256 // empty" $j)
|
||||
echo "source=[$s] tree_sha256=[$h]" >&2
|
||||
printf "%s" "$h" | grep -qxE "[0-9a-f]{64}" || {
|
||||
echo "pi_extensions_skill_tree_sha256 is not a 64-hex digest" >&2; exit 1; }
|
||||
case "$s" in
|
||||
package) ;;
|
||||
vendored-floor)
|
||||
echo "FALLBACK: clone had no skill/ at this ref, so the image ships the committed floor" >&2; exit 1 ;;
|
||||
divergent)
|
||||
echo "MIXED: served directory is part package and part floor" >&2; exit 1 ;;
|
||||
*)
|
||||
echo "pi_extensions_skill_source absent or unrecognised" >&2; exit 1 ;;
|
||||
esac
|
||||
'
|
||||
|
||||
# Same shape as the mempalace fingerprint check above, and for the same reason: a
|
||||
# recorded hash that is never recomputed is a claim, not a measurement.
|
||||
run "recorded pi-extensions skill hash matches the served bytes" '
|
||||
j=/etc/pi-devbox/build-manifest.json
|
||||
d=/usr/local/share/pi-devbox/skills/pi-extensions
|
||||
m=$(jq -r ".pi_extensions_skill_tree_sha256 // empty" $j)
|
||||
a=$( (cd "$d" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum) | sha256sum | cut -d" " -f1)
|
||||
echo "manifest=[$m] actual=[$a]" >&2
|
||||
[ -n "$m" ] && [ "$m" = "$a" ]
|
||||
'
|
||||
# OCI labels live in the image config, not the container fs — inspect them
|
||||
# from the host docker rather than via `docker run`.
|
||||
LBL=$(docker inspect --format '{{ index .Config.Labels "se.jordbo.pi-devbox.pi-extensions-ref" }}' "$IMAGE" 2>/dev/null || true)
|
||||
@@ -232,6 +627,122 @@ exec_test "settings.json bootstrapped" 'test -f $HOME/.pi/agent/sett
|
||||
exec_test "pi-devbox-environment skill linked" 'test -L $HOME/.agents/skills/pi-devbox-environment && test -f $HOME/.agents/skills/pi-devbox-environment/SKILL.md && echo ok'
|
||||
exec_test "pi-extensions skill linked (fallback)" 'test -L $HOME/.agents/skills/pi-extensions && test -f $HOME/.agents/skills/pi-extensions/SKILL.md && echo ok'
|
||||
exec_test "mempalace skill linked (fallback)" 'test -L $HOME/.agents/skills/mempalace && test -f $HOME/.agents/skills/mempalace/SKILL.md && echo ok'
|
||||
# The vendored mempalace snapshot is refreshed MANUALLY per release (see
|
||||
# rootfs/usr/local/share/pi-devbox/skills/VENDORED.md). Through v1.8.4 it also
|
||||
# silently SHADOWED the live skillset copy, so staleness was invisible — and the
|
||||
# canary that was supposed to catch it could not: it grepped "Shared palace:
|
||||
# multiple harnesses", a phrase present in BOTH the stale and the fresh copy.
|
||||
# A snapshot canary must pin the NEWEST section, so update this string whenever
|
||||
# the snapshot is refreshed — that is the point of it.
|
||||
#
|
||||
# v1.8.7: this fired for real, and on the release that changed the snapshot. The
|
||||
# pinned phrase was "Attribute what you file yourself", the heading of the
|
||||
# instruction telling agents to hand-stamp added_by — which that same release
|
||||
# WITHDREW (RFC 001 §7.3.2 ranks agent-side stamping worst-possible; the bridge
|
||||
# now does it). So the canary correctly reported "snapshot changed, expectation
|
||||
# did not", and blocked publication of an otherwise-green build (81 passed, 1
|
||||
# failed, twice). Two lessons kept in the assertion itself:
|
||||
# * it is now BIDIRECTIONAL — the new phrase must be present AND the withdrawn
|
||||
# one absent, so a re-vendored stale snapshot fails just as loudly as a
|
||||
# forgotten bump. A one-way canary only catches half the drift.
|
||||
# * a phrase canary can only ever detect "older than what I remembered to pin",
|
||||
# never "older than skillset main".
|
||||
#
|
||||
# That structural limit is now addressed, but NOT by the "CI job diffing this
|
||||
# file against the skillset repo" this comment used to point at (that pointer
|
||||
# also dangled: it referenced an Unreleased changelog note that had become the
|
||||
# v1.8.7 heading). A CI diff cannot be done without granting CI a credential
|
||||
# for the PRIVATE skillset repo, and it would guard a file that on this fleet
|
||||
# NO host reads — all four compose stacks mount a workspace containing the
|
||||
# skillset, so devbox-skill-reconcile repoints this link at the live clone and
|
||||
# the baked copy is a CI/no-mount fallback only. Instead the snapshot now
|
||||
# carries its provenance (skillset_snapshot_ref + a measured
|
||||
# skillset_snapshot_sha256 in build-manifest.json, written by
|
||||
# scripts/vendor-mempalace-skill.sh), which moves the check to where the
|
||||
# skillset actually IS: `scripts/vendor-mempalace-skill.sh --check` for a
|
||||
# maintainer, and `pi-devbox-version` for an agent inside any container.
|
||||
# This assertion is kept because it is orthogonal and free: it pins content,
|
||||
# not provenance, so it still catches a re-vendored snapshot whose ref was
|
||||
# bumped correctly but whose bytes came from the wrong place.
|
||||
#
|
||||
# v1.8.13: RE-PINNED on refresh a12fe5e -> e9e09d9, which is the whole point of
|
||||
# the mechanism — the previous pair ("Provenance is stamped for you" present /
|
||||
# "Attribute what you file yourself" absent) still passed against the NEW
|
||||
# snapshot, so leaving it would have produced a canary that is green on both the
|
||||
# old and the new bytes, i.e. blind to precisely the refresh it exists to
|
||||
# witness. Same false-green family as the pre-v1.8.5 canary this comment warns
|
||||
# about. The replacement pair was chosen by MEASURING direction against both
|
||||
# files rather than by reading the diff: "Diaries self-heal; plain drawers do
|
||||
# not" is new=1/old=0, "Agent diaries live in" is new=0/old=1 — so each string
|
||||
# discriminates on its own and the pair still fails loudly in BOTH directions
|
||||
# (forgotten bump AND re-vendored stale snapshot). Upstream content behind this
|
||||
# refresh: the bare project-name wing convention and the <harness>@<device>
|
||||
# added_by rule.
|
||||
exec_test "mempalace skill snapshot is current" 'f=$HOME/.agents/skills/mempalace/SKILL.md; grep -q "Diaries self-heal; plain drawers do not" "$f" && ! grep -q "Agent diaries live in" "$f" && echo ok'
|
||||
# Link TARGETS, not just link existence: with no skillset mounted (as here) the
|
||||
# baked tree must be what resolves, for all four vendored skills.
|
||||
exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
|
||||
'for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
|
||||
case "$(readlink -f $HOME/.agents/skills/$s)" in
|
||||
/usr/local/share/pi-devbox/skills/$s) ;;
|
||||
*) echo "$s resolves to $(readlink -f $HOME/.agents/skills/$s)" >&2; exit 1 ;;
|
||||
esac
|
||||
done; echo ok'
|
||||
# ... and that the tool REPORTS that resolution, which is the half that was
|
||||
# missing: a stale baked snapshot and a current live clone were
|
||||
# indistinguishable from inside the container. CI mounts no skillset, so every
|
||||
# vendored skill must report "baked" here — which also makes this a real test of
|
||||
# the fallback path rather than of the environment it happens to run in.
|
||||
exec_test "pi-devbox-version reports skill sources (all baked, no skillset here)" \
|
||||
'out=$(pi-devbox-version)
|
||||
echo "$out" | grep -q "skills:" || { echo "no skills section" >&2; exit 1; }
|
||||
for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
|
||||
echo "$out" | grep -qE "^ $s +baked$" \
|
||||
|| { echo "$s not reported as baked" >&2; exit 1; }
|
||||
done; echo ok'
|
||||
# The boot banner must NOT carry the section: entrypoint-user.sh prints the
|
||||
# version FIRST, before the baked links exist and long before the skillset
|
||||
# deploy + reconcile run last, so anything it said about skill sources would be
|
||||
# a pre-reconcile state that is about to change.
|
||||
# A bare negative (`! grep -q "skills:"`) passes if the tool crashes or
|
||||
# prints nothing at all — it cannot tell "correctly omitted the section"
|
||||
# apart from "the binary is broken". Anchor it positively: the command must
|
||||
# still succeed and still print its normal release-tag line.
|
||||
exec_test "pi-devbox-version --no-skills omits the skills section" \
|
||||
'out=$(pi-devbox-version --no-skills) && echo "$out" | grep -q "^pi-devbox " && ! echo "$out" | grep -q "skills:"'
|
||||
exec_test "entrypoint prints the version banner with --no-skills" \
|
||||
'grep -q "pi-devbox-version --no-skills" /usr/local/bin/entrypoint-user.sh'
|
||||
# The handover path itself. CI never mounts a skillset, so without this the
|
||||
# v1.8.5 fix would ship untested: fabricate a skillset + a skills dir holding
|
||||
# baked-style links, run the reconciler, and assert all three outcomes —
|
||||
# owned skill repointed, unowned skill left baked, user override untouched.
|
||||
exec_test "reconciler: owned skill handed to live clone, others untouched" \
|
||||
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/ss/skills/pi-extensions $t/skills
|
||||
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo LIVE > $t/ss/skills/pi-extensions/SKILL.md
|
||||
ln -s /usr/local/share/pi-devbox/skills/mempalace $t/skills/mempalace
|
||||
ln -s /usr/local/share/pi-devbox/skills/pi-extensions $t/skills/pi-extensions
|
||||
mkdir -p $t/skills/mine; echo MINE > $t/skills/mine/SKILL.md
|
||||
devbox-skill-reconcile $t/ss $t/skills >/dev/null
|
||||
devbox-skill-reconcile $t/ss $t/skills >/dev/null # idempotent
|
||||
[ "$(readlink $t/skills/mempalace)" = "$t/ss/skills/mempalace" ] || { echo "owned skill NOT repointed" >&2; exit 1; }
|
||||
[ "$(readlink $t/skills/pi-extensions)" = /usr/local/share/pi-devbox/skills/pi-extensions ] || { echo "unowned skill was repointed" >&2; exit 1; }
|
||||
[ "$(cat $t/skills/mine/SKILL.md)" = MINE ] || { echo "user override clobbered" >&2; exit 1; }
|
||||
rm -rf $t; echo ok'
|
||||
# The case above cannot fail if the reconciler stops checking WHERE a link
|
||||
# points — a mutation test showed all three of its assertions still passing with
|
||||
# that guard deleted, which is the same false-green shape as the old snapshot
|
||||
# canary. This one discriminates: an OWNED name (so it is considered) whose link
|
||||
# is a user override pointing outside the baked tree (so it must be left alone).
|
||||
exec_test "reconciler: user override on an owned name is left alone" \
|
||||
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/skills $t/mine-skill
|
||||
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo USERLINK > $t/mine-skill/SKILL.md
|
||||
ln -sfn $t/mine-skill $t/skills/mempalace
|
||||
devbox-skill-reconcile $t/ss $t/skills >/dev/null
|
||||
[ "$(cat $t/skills/mempalace/SKILL.md)" = USERLINK ] || { echo "user symlink override clobbered" >&2; exit 1; }
|
||||
rm -rf $t; echo ok'
|
||||
# mempalace-census gained a /usr/local/bin symlink in v1.8.3; its three siblings
|
||||
# had one since they were added, so this asserts the set stays complete.
|
||||
exec_test "mempalace-census on PATH" 'command -v mempalace-census >/dev/null && mempalace-census --help >/dev/null && echo ok'
|
||||
|
||||
# pi-fork + pi-observational-memory are registered by entrypoint-user.sh via
|
||||
# `pi install /opt/<pkg>`, which runs slightly after the keybindings marker.
|
||||
@@ -271,6 +782,75 @@ if [ "${STUDIO_VARIANT:-0}" = "1" ]; then
|
||||
"$(pkg_registered_cmd pi-studio)"
|
||||
fi
|
||||
|
||||
# pi-atelier registration. It is LAST in the entrypoint's install loop, so a
|
||||
# pass here also means that loop ran to completion rather than dying midway.
|
||||
for i in $(seq 1 15); do
|
||||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-atelier)" \
|
||||
>/dev/null 2>&1; then
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
exec_test "pi-atelier registered in packages[] (TUI sidebar)" \
|
||||
"$(pkg_registered_cmd pi-atelier)"
|
||||
# ...and registered from the vendored /opt copy, NOT as `npm:pi-atelier`: an
|
||||
# npm: entry resolves through ~/.pi/npm-global on the config VOLUME, which
|
||||
# outlives image upgrades and would silently keep an old, unaudited atelier —
|
||||
# exactly the shape that pairs a stale 0.6.x with a new pi and hangs at startup.
|
||||
exec_test "pi-atelier registered from /opt, not npm: (volume-shadowing guard)" \
|
||||
'jq -e "((.packages // []) | any((type == \"string\") and endswith(\"/pi-atelier\"))) and (((.packages // []) | any(. == \"npm:pi-atelier\")) | not)" $HOME/.pi/agent/settings.json'
|
||||
|
||||
# agent-browser: the third package hit by ~/.pi/npm-global volume shadowing
|
||||
# (after pi itself and pi-atelier). This build-time check is deliberately WEAK
|
||||
# and says so: a `docker run` container has an EMPTY config volume, so it can
|
||||
# only prove the image ships a sane copy and nothing in the image itself
|
||||
# shadows it. The check that actually bites lives in
|
||||
# recreate-sanity-check.sh, which runs where the volume is real — that is
|
||||
# where a 7-week-old 0.27.0 was caught shadowing 0.35.2 on 2026-09-06.
|
||||
# EXECUTION is ASSERTED here, not printed. Until 2026-09-07 the version was
|
||||
# captured inside an echo with 2>/dev/null, so a binary that could not run at all
|
||||
# still PASSED and simply printed version=[] -- the same failure class as the bare
|
||||
# `node --version` two hundred lines up: a value displayed rather than compared.
|
||||
#
|
||||
# Why this exit code matters more than most: smoke runs `platforms: linux/amd64`
|
||||
# on an x86 runner, i.e. NATIVE amd64, so this is the fleet's only recurring
|
||||
# amd64 runtime proof for the linux-x64 ELF. No devbox can supply one -- every
|
||||
# machine in the pi fleet is an Apple Silicon Mac (mbp-m1-2020; tor-ms22 = Mac
|
||||
# Studio Mac13,1 M1 Max, verified 2026-08-17 by system_profiler; emb-7kj4vr4g =
|
||||
# Apple Silicon, 4 routes 2026-09-07). Asking a device for that proof is asking
|
||||
# for the impossible; CI already had it and was discarding it.
|
||||
#
|
||||
# KEEP PROSE OUT OF THE QUOTED BODY BELOW. On 2026-09-07 this explanation lived
|
||||
# INSIDE the single-quoted argument and contained an apostrophe ("the fleet's").
|
||||
# Inside '...' bash treats a backslash literally, so \' does not escape -- it
|
||||
# CLOSES the string. The body silently truncated, the remaining lines were parsed
|
||||
# by the RUNNER's shell instead of the container's, and `agent-browser --version`
|
||||
# ran on a host that has no agent-browser: "line 770: command not found", release
|
||||
# v1.8.14's smoke job failed after the base had already built. shellcheck caught
|
||||
# it as SC2289 the same day and the red lint job went unread for 24h.
|
||||
exec_test "agent-browser resolves under /usr (volume-shadowing guard, build-time half)" '
|
||||
p=$(command -v agent-browser) || { echo "agent-browser not on PATH" >&2; exit 1; }
|
||||
r=$(readlink -f "$p")
|
||||
v=$(agent-browser --version) || { echo "agent-browser did not EXECUTE" >&2; exit 1; }
|
||||
test -n "$v" || { echo "agent-browser --version produced no output" >&2; exit 1; }
|
||||
echo "resolved=[$r] version=[$(printf %s "$v" | head -n1)]" >&2
|
||||
case "$r" in /usr/*) ;; *) exit 1 ;; esac
|
||||
test ! -d "$HOME/.pi/npm-global/lib/node_modules/agent-browser" || exit 1
|
||||
echo ok
|
||||
'
|
||||
|
||||
# pi-fork capability floor. `extensions: []` makes a fork child run with
|
||||
# --no-extensions, which is the only MECHANICAL guarantee that a fork cannot
|
||||
# file drawers or diary entries under the parent's identity — the mempalace
|
||||
# bridge is an extension, so removing extensions removes the write path.
|
||||
# Asserted because it is a security-shaped default that a settings merge or a
|
||||
# hand-edit could silently drop, and its absence is invisible until a fork
|
||||
# writes to the shared palace as you (measured twice: 2026-09-01, 2026-09-06).
|
||||
# Deliberately compares to [] and not "is falsy": null means "load normal
|
||||
# extensions", i.e. exactly the unguarded state this asserts against.
|
||||
exec_test "pi-fork extensions floor is [] (forks cannot write to the palace)" \
|
||||
'jq -e ".[\"pi-fork\"].extensions == []" $HOME/.pi/agent/settings.json'
|
||||
|
||||
# ── /tmp/sshcm directory created by entrypoint ────────────────────────
|
||||
exec_test "/tmp/sshcm dir mode 700 (ssh ControlMaster)" \
|
||||
'test -d /tmp/sshcm && [ "$(stat -c %a /tmp/sshcm)" = "700" ] && echo ok'
|
||||
|
||||
Executable
+293
@@ -0,0 +1,293 @@
|
||||
#!/usr/bin/env bash
|
||||
# vendor-mempalace-skill.sh — refresh the vendored mempalace skill snapshot
|
||||
# AND its recorded provenance, together, so the two cannot drift apart.
|
||||
#
|
||||
# WHY THIS EXISTS
|
||||
# ---------------
|
||||
# rootfs/usr/local/share/pi-devbox/skills/mempalace/SKILL.md is a snapshot of a
|
||||
# file owned by the PRIVATE skillset repo (see VENDORED.md). Because the image
|
||||
# cannot clone that repo, refreshing the snapshot was a manual `cp` — and the
|
||||
# result was anonymous: nothing recorded WHICH skillset commit the bytes came
|
||||
# from. The only staleness check available was a hand-maintained phrase canary
|
||||
# in scripts/smoke-test.sh, which by construction detects "older than the phrase
|
||||
# I remembered to pin", never "older than skillset main".
|
||||
#
|
||||
# Two facts now travel with the snapshot: the skillset commit it was taken from
|
||||
# (ARG SKILLSET_SNAPSHOT_REF in Dockerfile.variant) and the sha256 of the bytes
|
||||
# themselves (measured at build time into build-manifest.json). This script is
|
||||
# the only thing that should ever write the first one, because a `cp` without a
|
||||
# matching ARG bump produces a manifest that CONFIDENTLY LIES — worse than the
|
||||
# anonymous snapshot it replaced.
|
||||
#
|
||||
# HARDENED after peer review (pi@emb-7kj4vr4g, logstream correlation
|
||||
# skills-provenance-review, 2026-08-26) proved the original --check could print
|
||||
# OK and exit 0 without actually verifying anything: `git show <ref>:<path>`
|
||||
# emits NOTHING when the ref/path doesn't resolve, and `sha256sum` still hashes
|
||||
# that empty stdin, so "ref not found" silently collided with "the file really
|
||||
# is 0 bytes". Depending on which side of the comparison hit the collision this
|
||||
# fell through as either a false MISMATCH (blaming provenance for what was
|
||||
# really an incomplete clone) or, worse, a false OK. See EXIT STATUS below —
|
||||
# "cannot determine" is now its own outcome, distinct from "confirmed wrong",
|
||||
# which is the same distinction the phrase canary this script replaced lacked.
|
||||
#
|
||||
# USAGE
|
||||
# scripts/vendor-mempalace-skill.sh [skillset-root] [--force]
|
||||
# refresh: rewrite the snapshot and the ARG together.
|
||||
# scripts/vendor-mempalace-skill.sh --check [skillset-root]
|
||||
# verify only, writes nothing. The root path and any flag may appear in
|
||||
# either order — a positional-only parser previously made `<root>
|
||||
# --check` silently run a refresh instead of the verification asked for.
|
||||
#
|
||||
# skillset-root defaults to /workspace/skillset, then $HOME/skillset.
|
||||
#
|
||||
# --force (refresh mode only) proceed even when the recorded ref cannot be
|
||||
# proven to be an ancestor of the skillset's current HEAD — i.e.
|
||||
# skip the guard against silently REWINDING provenance, which a
|
||||
# detached HEAD, an older checkout, or a shallow clone lacking the
|
||||
# recorded commit can all trigger. Meant to be used deliberately,
|
||||
# not habitually: each use is a human deciding a rewind is fine.
|
||||
#
|
||||
# --check answers "is the committed snapshot really skillset@<recorded ref>?"
|
||||
# — the question CI cannot answer without a credential for the private repo,
|
||||
# and which anyone with the skillset checked out can answer for free.
|
||||
#
|
||||
# EXIT STATUS (same three codes in both modes)
|
||||
# 0 the operation succeeded, or (--check) the record is verified truthful.
|
||||
# This INCLUDES a truthful record that is merely stale — upstream has
|
||||
# moved on since the recorded ref, or the local working tree has since
|
||||
# diverged. A NOTICE is printed to stderr, but the snapshot is not being
|
||||
# accused of lying, so this is not a release-blocking failure. Skipping a
|
||||
# refresh is a legitimate release-day choice (see AGENTS.md); this exit
|
||||
# code is what makes that choice checkable rather than merely asserted.
|
||||
# 1 refused: a CONFIRMED problem. Dirty upstream file; a refresh that would
|
||||
# rewind past the recorded ref; or (--check) the vendored bytes provably
|
||||
# do NOT match the file at the recorded ref — a lying record.
|
||||
# 2 cannot determine: the recorded ref, or the path at that ref, is not
|
||||
# resolvable in this clone. Commonly a shallow clone missing history, or
|
||||
# a ref that was rewritten or never pushed. Deliberately NOT the same as
|
||||
# 1 — "I can't tell" must never be reported as "it's wrong".
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
DOCKERFILE="Dockerfile.variant"
|
||||
VENDORED="rootfs/usr/local/share/pi-devbox/skills/mempalace/SKILL.md"
|
||||
ARG_NAME="SKILLSET_SNAPSHOT_REF"
|
||||
REL_PATH="skills/mempalace/SKILL.md"
|
||||
|
||||
die() { printf '%s: %s\n' "$(basename "$0")" "$1" >&2; exit 1; }
|
||||
|
||||
# Parse flags and the optional root path in either order, and reject anything
|
||||
# unrecognised rather than silently absorbing it.
|
||||
MODE="refresh"
|
||||
FORCE=0
|
||||
ROOT=""
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--check) MODE="check" ;;
|
||||
--force) FORCE=1 ;;
|
||||
# This is the one script whose argument ORDER was itself a landmine, so the
|
||||
# path that documents the trap must not be the path that errors.
|
||||
-h|--help)
|
||||
awk 'NR>1 && /^#/ { sub(/^# ?/, ""); print; next } NR>1 { exit }' "$0"
|
||||
exit 0
|
||||
;;
|
||||
--*) die "unknown option: $arg (try --help)" ;;
|
||||
*)
|
||||
[ -z "$ROOT" ] || die "unexpected extra argument: $arg (root already set to $ROOT)"
|
||||
ROOT="$arg"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
if [ "$MODE" = "check" ] && [ "$FORCE" = 1 ]; then
|
||||
die "--force has no effect with --check (nothing is written); remove it"
|
||||
fi
|
||||
|
||||
if [ -z "$ROOT" ]; then
|
||||
for candidate in /workspace/skillset "$HOME/skillset"; do
|
||||
if [ -d "$candidate/.git" ]; then
|
||||
ROOT="$candidate"
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
[ -n "$ROOT" ] || die "no skillset clone found (pass one: $(basename "$0") /path/to/skillset)"
|
||||
[ -d "$ROOT/.git" ] || die "not a git clone: $ROOT"
|
||||
[ -f "$ROOT/$REL_PATH" ] || die "no $REL_PATH in $ROOT"
|
||||
[ -f "$VENDORED" ] || die "vendored snapshot missing: $VENDORED"
|
||||
# LOAD-BEARING, DO NOT DELETE AS "REDUNDANT WITH THE EXISTENCE PROBES": -f
|
||||
# accepts an empty file, and sha256 of an empty file equals sha256 of a failed
|
||||
# pipeline's empty stdin. Guarding it HERE, before mode dispatch, makes that
|
||||
# collision unreachable by construction rather than by a probe further down --
|
||||
# which also means no test below exercises the collision any more. Remove this
|
||||
# line and the false "OK" for a nonexistent ref returns with nothing failing.
|
||||
[ -s "$VENDORED" ] || die "vendored snapshot is empty: $VENDORED"
|
||||
|
||||
head_sha=$(git -C "$ROOT" rev-parse HEAD 2>/dev/null) || die "cannot read HEAD of $ROOT"
|
||||
recorded=$(grep -oE "^ARG ${ARG_NAME}=[0-9a-f]{40}$" "$DOCKERFILE" | cut -d= -f2 || true)
|
||||
[ -n "$recorded" ] || die "no 'ARG ${ARG_NAME}=<40-hex>' line in $DOCKERFILE"
|
||||
|
||||
sha_of() { sha256sum "$1" | cut -d' ' -f1; }
|
||||
vendored_sha=$(sha_of "$VENDORED")
|
||||
upstream_sha=$(sha_of "$ROOT/$REL_PATH")
|
||||
|
||||
# Does $REL_PATH exist at HEAD at all? Proven with `cat-file -e` BEFORE
|
||||
# hashing anything. Piping a failed `git show` straight into sha256sum, as
|
||||
# this script used to, hashes an EMPTY stream and produces sha256(""): a real,
|
||||
# collidable value — not a representation of absence. That collapsed "doesn't
|
||||
# exist" and "exists and happens to be empty" into the same signal, which is
|
||||
# exactly the defect class the peer review found in --check's at_ref, below.
|
||||
blob_sha=""
|
||||
if git -C "$ROOT" cat-file -e "HEAD:$REL_PATH" 2>/dev/null; then
|
||||
blob_sha=$(git -C "$ROOT" show "HEAD:$REL_PATH" | sha256sum | cut -d' ' -f1)
|
||||
fi
|
||||
|
||||
upstream_dirty=""
|
||||
if [ -z "$blob_sha" ]; then
|
||||
upstream_dirty="not present at HEAD (untracked, or absent at this commit)"
|
||||
elif [ "$blob_sha" != "$upstream_sha" ]; then
|
||||
if ! git -C "$ROOT" diff --quiet -- "$REL_PATH" 2>/dev/null; then
|
||||
upstream_dirty="modified but not committed"
|
||||
elif ! git -C "$ROOT" diff --cached --quiet -- "$REL_PATH" 2>/dev/null; then
|
||||
upstream_dirty="staged but not committed"
|
||||
else
|
||||
upstream_dirty="different at HEAD than in the working tree"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$MODE" = "check" ]; then
|
||||
# Resolve the recorded ref the same careful way: existence is proven with
|
||||
# `cat-file -e` before anything is hashed, and "the ref itself is missing"
|
||||
# is reported distinctly from "the ref resolves but the path isn't there
|
||||
# at it" — both used to be silently swallowed into a plausible sha256("").
|
||||
ref_exists=0
|
||||
path_at_ref_exists=0
|
||||
at_ref=""
|
||||
if git -C "$ROOT" cat-file -e "${recorded}^{commit}" 2>/dev/null; then
|
||||
ref_exists=1
|
||||
if git -C "$ROOT" cat-file -e "${recorded}:${REL_PATH}" 2>/dev/null; then
|
||||
path_at_ref_exists=1
|
||||
at_ref=$(git -C "$ROOT" show "${recorded}:${REL_PATH}" | sha256sum | cut -d' ' -f1)
|
||||
fi
|
||||
fi
|
||||
|
||||
printf 'recorded ref: %s\n' "$recorded"
|
||||
printf 'vendored sha256: %s\n' "$vendored_sha"
|
||||
if [ "$path_at_ref_exists" = 1 ]; then
|
||||
printf 'sha256 at ref: %s\n' "$at_ref"
|
||||
elif [ "$ref_exists" = 1 ]; then
|
||||
printf 'sha256 at ref: <%s not present at %s>\n' "$REL_PATH" "${recorded:0:7}"
|
||||
else
|
||||
printf 'sha256 at ref: <%s not present in this clone>\n' "${recorded:0:7}"
|
||||
fi
|
||||
printf 'skillset HEAD: %s (%s)\n' "$head_sha" "$upstream_sha"
|
||||
if [ -n "$upstream_dirty" ]; then
|
||||
printf 'live working tree: %s\n' "$upstream_dirty"
|
||||
fi
|
||||
|
||||
rc=0
|
||||
if [ "$ref_exists" != 1 ]; then
|
||||
printf 'CANNOT-DETERMINE: %s is not present in %s — fetch, or check against a complete clone\n' "$recorded" "$ROOT" >&2
|
||||
rc=2
|
||||
elif [ "$path_at_ref_exists" != 1 ]; then
|
||||
printf 'MISMATCH: %s does not exist at %s in this clone — the recorded ref cannot be describing these bytes\n' "$REL_PATH" "$recorded" >&2
|
||||
rc=1
|
||||
elif [ "$at_ref" != "$vendored_sha" ]; then
|
||||
printf 'MISMATCH: the vendored snapshot is NOT the file at the recorded ref\n' >&2
|
||||
rc=1
|
||||
else
|
||||
printf 'OK: the vendored snapshot is exactly skillset@%s:%s\n' "${recorded:0:7}" "$REL_PATH"
|
||||
fi
|
||||
|
||||
# Staleness is orthogonal to truthfulness: a record can correctly describe
|
||||
# an old commit even after upstream has moved on, and a dirty local working
|
||||
# tree in $ROOT doesn't rewrite git history either — it says nothing about
|
||||
# whether the RECORDED, committed ref describes the RECORDED, committed
|
||||
# bytes. Only worth reporting once we already know rc=0 (truthful) — a
|
||||
# MISMATCH or CANNOT-DETERMINE is the dominant fact and a staleness note
|
||||
# would only muddy it.
|
||||
if [ "$rc" = 0 ] && [ "$vendored_sha" != "$upstream_sha" ]; then
|
||||
# Name the ACTUAL cause. "working tree differs" is wrong when the tree is
|
||||
# clean and the ref simply moved on — a message that names the wrong cause
|
||||
# is the same defect class as a canary pinned to a deleted phrase.
|
||||
if [ "$recorded" != "$head_sha" ] && [ "$blob_sha" = "$upstream_sha" ]; then
|
||||
# Do not ASSERT which side is newer — test it. Asserting that HEAD is the
|
||||
# newer side points the operator at a refresh (which costs a ~67-minute
|
||||
# base rebuild) when the real remedy may be `git pull` in this clone. The
|
||||
# refresh path below already uses this primitive; reuse it here.
|
||||
if git -C "$ROOT" merge-base --is-ancestor "$recorded" "$head_sha" 2>/dev/null; then
|
||||
printf 'NOTICE: %s has moved on to %s; the snapshot describes the older %s (stale, not untruthful — refresh to catch up)\n' \
|
||||
"$ROOT" "${head_sha:0:7}" "${recorded:0:7}" >&2
|
||||
elif git -C "$ROOT" merge-base --is-ancestor "$head_sha" "$recorded" 2>/dev/null; then
|
||||
printf 'NOTICE: %s is BEHIND at %s; the snapshot describes the newer %s — pull this clone, do NOT refresh the snapshot\n' \
|
||||
"$ROOT" "${head_sha:0:7}" "${recorded:0:7}" >&2
|
||||
else
|
||||
printf 'NOTICE: %s (HEAD %s) and the recorded %s have DIVERGED — neither is an ancestor of the other; reconcile the clone before refreshing\n' \
|
||||
"$ROOT" "${head_sha:0:7}" "${recorded:0:7}" >&2
|
||||
fi
|
||||
else
|
||||
printf 'NOTICE: the working tree of %s differs from the snapshot (HEAD %s)\n' \
|
||||
"$ROOT" "${head_sha:0:7}" >&2
|
||||
fi
|
||||
fi
|
||||
|
||||
exit "$rc"
|
||||
fi
|
||||
|
||||
[ -z "$upstream_dirty" ] || die "$ROOT/$REL_PATH is $upstream_dirty — commit it first, or the recorded ref would not describe these bytes"
|
||||
|
||||
if [ "$vendored_sha" = "$upstream_sha" ] && [ "$recorded" = "$head_sha" ]; then
|
||||
printf 'already current: snapshot == skillset@%s\n' "${head_sha:0:7}"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Refuse to silently REWIND provenance. `git checkout <tag>`, a detached HEAD,
|
||||
# or an older checkout can all leave $ROOT's HEAD behind the already-recorded
|
||||
# ref; without this guard a refresh there would happily rewrite both the ARG
|
||||
# and the bytes backwards and report it as an ordinary update.
|
||||
if [ "$recorded" != "$head_sha" ]; then
|
||||
if git -C "$ROOT" cat-file -e "${recorded}^{commit}" 2>/dev/null; then
|
||||
if ! git -C "$ROOT" merge-base --is-ancestor "$recorded" "$head_sha" 2>/dev/null; then
|
||||
if [ "$FORCE" != 1 ]; then
|
||||
die "refusing: $ROOT's HEAD ($head_sha) is not a descendant of the recorded ref ($recorded) — this looks like a rewind. Pass --force if this is intentional."
|
||||
fi
|
||||
printf 'WARNING: --force set; %s is not an ancestor of HEAD %s — proceeding anyway\n' "${recorded:0:7}" "${head_sha:0:7}" >&2
|
||||
fi
|
||||
else
|
||||
if [ "$FORCE" != 1 ]; then
|
||||
printf 'CANNOT-DETERMINE: %s is not present in %s (shallow clone?) — fetch full history to verify this refresh moves forward, or pass --force to proceed without that guarantee\n' "$recorded" "$ROOT" >&2
|
||||
exit 2
|
||||
fi
|
||||
printf 'WARNING: --force set; %s could not be resolved in %s — proceeding without verifying forward motion\n' "${recorded:0:7}" "$ROOT" >&2
|
||||
fi
|
||||
fi
|
||||
|
||||
# Written FROM THE REF, not copied from the working tree, so the pair cannot
|
||||
# be a lie by construction. Via a temp file so a failed write cannot leave a
|
||||
# half-vendored snapshot behind.
|
||||
snap_tmp=$(mktemp)
|
||||
if ! git -C "$ROOT" show "HEAD:$REL_PATH" > "$snap_tmp" 2>/dev/null; then
|
||||
rm -f -- "$snap_tmp"
|
||||
die "cannot read HEAD:$REL_PATH from $ROOT"
|
||||
fi
|
||||
chmod 0644 -- "$snap_tmp"
|
||||
mv -- "$snap_tmp" "$VENDORED"
|
||||
[ "$(sha_of "$VENDORED")" = "$blob_sha" ] \
|
||||
|| die "internal: written snapshot does not match HEAD:$REL_PATH"
|
||||
|
||||
# In-place, and only the exact pinned line: a broad sed on this Dockerfile
|
||||
# could rewrite one of the other *_REF ARGs.
|
||||
tmp=$(mktemp)
|
||||
sed "s|^ARG ${ARG_NAME}=.*\$|ARG ${ARG_NAME}=${head_sha}|" "$DOCKERFILE" > "$tmp"
|
||||
chmod 0644 -- "$tmp"
|
||||
mv -- "$tmp" "$DOCKERFILE"
|
||||
|
||||
new_recorded=$(grep -oE "^ARG ${ARG_NAME}=[0-9a-f]{40}$" "$DOCKERFILE" | cut -d= -f2 || true)
|
||||
[ "$new_recorded" = "$head_sha" ] || die "failed to rewrite ${ARG_NAME} in $DOCKERFILE"
|
||||
|
||||
printf 'snapshot: %s -> %s\n' "${vendored_sha:0:12}" "$(sha_of "$VENDORED" | cut -c1-12)"
|
||||
printf 'ref: %s -> %s\n' "${recorded:0:7}" "${head_sha:0:7}"
|
||||
printf '\nNOTE: %s is hashed into base_tag, so this costs a base rebuild\n' "$VENDORED"
|
||||
printf 'on the next tag (~67 min). Also re-pin the phrase canary in\n'
|
||||
printf 'scripts/smoke-test.sh if the section it names changed.\n'
|
||||
Reference in New Issue
Block a user