Files
pi-devbox/scripts/smoke-test.sh
T
Joakim Persson b9057fdc8c
Lint / skill-floor (push) Successful in 7s
Lint / hadolint (push) Successful in 10s
Lint / doc-drift (push) Successful in 16s
Lint / actionlint (push) Successful in 21s
Dockerfile.base: LABEL se.jordbo.pi-devbox.mempalace-version, inherited by both variants
Closes the blind spot check 9 (cb6d9e5) named: no label recorded the palace
pin, so a MEMPALACE_VERSION bump — the one component whose skew against the
shared central palace is fleet-wide — could ship without a CHANGELOG line.

In Dockerfile.base, not Dockerfile.variant, deliberately: the value sits next
to the ARG that defines it (a copy in the variant is one more pin able to
drift); labels are inherited by every image built FROM the base, so no
build-arg to plumb through the variant's four call sites; and inheritance
means the label states the pin of the base the image ACTUALLY built on,
which is the question when base-decide cache-hits an older base. Both
mechanisms measured on the published v1.9.2 config blob rather than assumed:
maintainer + image.source (set only in Dockerfile.base) are present on the
variant image, and pi-version=0.85.1 is an ARG expanded inside a LABEL.

Intent, like every se.jordbo.pi-devbox.* label; the manifest's
mempalace_version (read from the installed binary) stays the ground truth,
and smoke-test.sh now asserts label == installed core — the one way they
diverge is a base built with INSTALL_MEMPALACE=false or an off-pin install,
both invisible to a label-only check.

check-doc-drift check 9 gains the component (literal, against
ARG MEMPALACE_VERSION in Dockerfile.base); the label-key rule generalises to
"names ending in -version are the label itself". Until a release carries the
label it reports a counted SKIP, not OK — measured: "v1.9.2 carries no
se.jordbo.pi-devbox.mempalace-version label", summary says 1 SKIPPED.

Costs nothing extra: this Unreleased already forces a base rebuild
(50153e6 rootfs/ skill floor). check-base-hash unchanged (no new *_REF).
2026-09-19 17:27:34 +02:00

1021 lines
63 KiB
Bash
Executable File
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env bash
# smoke-test.sh — sanity checks for the pi-devbox image
#
# Usage: ./scripts/smoke-test.sh <image>
#
# Verifies:
# - pi binary present and (if EXPECTED_PI_VERSION set) matches CI's resolved version
# - node MAJOR matches Dockerfile.base's ARG NODE_VERSION (if EXPECTED_NODE_MAJOR set)
# - mempalace core matches the audited pin (if EXPECTED_MEMPALACE_VERSION set)
# - new v1.0.0 base additions (pandoc, graphviz, imagemagick, yq, tealdeer)
# - typst PDF engine for pandoc (v1.4.0) — `pandoc --pdf-engine=typst`
# - non-modal editors nano + micro (alongside nvim)
# - terminfo for modern emulators: xterm-kitty, xterm-ghostty, wezterm,
# alacritty, foot (kitty-terminfo + ncurses-term + compiled ghostty alias)
# - tmux 0-indexing baked in /etc/tmux.conf (required for pi-studio variants)
# - pi-toolkit cloned at /opt/pi-toolkit
# - pi-extensions cloned at /opt/pi-extensions
# - pi-atelier vendored at /opt/pi-atelier, registered from /opt (not npm:),
# and >= the version floor pi's TUI requires (see the floor test)
# - pi-fork + pi-observational-memory cloned with node_modules baked
# - entrypoint deploys pi-toolkit keybindings symlink
# - entrypoint deploys ≥4 extensions
# - mempalace bridge symlink present
# - settings.json bootstrapped
# - pi-fork + pi-observational-memory registered in settings.json packages[]
# via `pi install`
# - pi-devbox-version command present + wraps the build manifest correctly
# (human, --json, --quiet)
# - (studio variant only, auto-detected) pi-studio cloned + prebuilt
# client bundle present + registered via `pi install`
# - no foreign npm-11 platform packages (@esbuild, clipboard) beyond the host
# - no build-time npm cache (/root/.npm) shipped in the image
# - esbuild compiles + clipboard native loads at every install site
# - image size within threshold
set -euo pipefail
IMAGE="${1:?usage: $0 <image>}"
PASS=0; FAIL=0
# pi-devbox v1.0.0 (decoupled from opencode-devbox) added pandoc, graphviz,
# imagemagick, yq, tealdeer, a baked /etc/tmux.conf, and the non-modal
# editors nano + micro (~15 MB combined). v1.6.0 baked in agent-browser +
# Playwright Chromium (~291 MB net after dropping the unused headless-shell
# build), which lifted the baseline. CI amd64 actuals observed on run 512
# (v1.6.1): 3411 MB non-studio, 3574 MB studio. Threshold below carries
# ~225 MB margin above the studio number to absorb minor arch/build-cache
# differences and small future growth without false reds, while still
# catching an unexpected +GB regression.
SIZE_THRESHOLD_MB=3800
# On failure, surface the last few lines the command produced. This used to
# discard output entirely (`>/dev/null 2>&1`), which made a red ❌ carry zero
# diagnostic weight: explaining the single v1.8.0 stage-default failure took a
# full CI-log dig plus a registry-config inspection, when the container had
# already printed the answer and thrown it away. Assertions that want a
# diagnostic just echo it to stderr — it stays hidden while they pass.
run() {
local label="$1"; local cmd="$2"
local out
if out=$(docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" 2>&1); then
printf " ✅ %s\n" "$label"; PASS=$((PASS+1))
else
printf " ❌ %s\n" "$label"; FAIL=$((FAIL+1))
# `if`, not `&&` — a trailing false under `set -e` would abort the script.
if [ -n "$out" ]; then
printf " └─ %s\n" "$(printf '%s' "$out" | tail -3 | tr '\n' ' ' | cut -c1-300)"
fi
fi
}
# Stricter version of `run` that asserts an expected substring in stdout.
# Catches the "image bytes silently identical to previous release" class of
# regression — Docker layer cache hit on `npm install -g <pkg>` because the
# bare command string is identical across builds, even when `latest` would
# resolve differently. Discovered 2026-05-23 — every pi-devbox release
# v0.74.0..v0.75.5 had been shipping the same image bytes.
run_expect() {
local label="$1"; local cmd="$2"; local expect="$3"
local out
out=$(docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" 2>&1) || true
if echo "$out" | grep -Fq "$expect"; then
printf " ✅ %s (got %s)\n" "$label" "$expect"; PASS=$((PASS+1))
else
printf " ❌ %s — expected substring %q, got: %s\n" "$label" "$expect" "$out"; FAIL=$((FAIL+1))
fi
}
echo "=== pi-devbox smoke test: $IMAGE ==="
echo ""
# ── Binaries ─────────────────────────────────────────────────────────
echo "── Binaries ──"
if [ -n "${EXPECTED_PI_VERSION:-}" ]; then
run_expect "pi version matches build arg" "pi --version" "$EXPECTED_PI_VERSION"
else
run "pi" "pi --version"
fi
# Until 2026-09-07 this was a bare `run "node" "node --version"`, which asserts
# only that the binary exists and exits 0 — the printed version was never
# compared to anything. A node major bump would therefore have passed this suite
# SILENTLY, while a reader skimming it would reasonably assume node regressions
# were covered. EXPECTED_NODE_MAJOR closes that: CI derives it from
# Dockerfile.base's ARG NODE_VERSION (the single source of truth), so this also
# catches a stale cached layer whose node does not match the declared ARG.
if [ -n "${EXPECTED_NODE_MAJOR:-}" ]; then
run_expect "node major matches Dockerfile ARG" "node --version" "v${EXPECTED_NODE_MAJOR}."
else
run "node" "node --version"
fi
run "git" "git --version"
# NOTE: the shellcheck binary is a GATE DEPENDENCY, not a convenience.
# scripts/lint-shell.sh is the release gate (the lint-gate job resolve-versions
# depends on) and it exits 2 when the binary is missing, by design — "a gate that
# cannot run must not pass". Measured on v1.8.14: it was absent from the image, so
# that gate could not be run by a developer in ANY container, only in CI.
# Asserted here so its absence fails a build instead of being discovered by a hook
# that then refuses every push (hooks/pre-push).
#
# This comment must not BEGIN with the tool's name: a line starting with
# `# shellcheck` is parsed as a DIRECTIVE, not a comment (SC1073/SC1072). The
# gate added in this same change caught that here, before the push.
run "shellcheck (lint gate dependency)" "shellcheck --version | grep -qE '^version: [0-9]'"
run "aws" "aws --version"
run "uv" "uv --version"
run "nvim" "nvim --version"
run "nano" "nano --version"
run "micro" "micro --version"
run "kitty-terminfo" "infocmp -x xterm-kitty >/dev/null 2>&1"
run "terminfo: modern emulators (ncurses-term)" 'for t in wezterm alacritty foot ghostty st-256color; do infocmp -x "$t" >/dev/null 2>&1 || exit 1; done'
run "terminfo: xterm-ghostty alias (tic)" "infocmp -x xterm-ghostty >/dev/null 2>&1"
run "nvim true-colour default (sysinit.vim)" "nvim --headless -c 'lua os.exit(vim.o.termguicolors and 0 or 1)'"
run "mempalace-mcp" "mempalace-mcp --help"
run "mempalace-pi-session on PATH" "mempalace-pi-session --help"
# The staging dir must sit next to the palace, not in a disposable cache: the
# palace keys per-source dedup on the STAGED path, so a stage that can be wiped
# while the palace survives lets `mempalace sync` prune every drawer mined from
# it. Assert the resolved default, not an env var — the guarantee is "stage
# shares the palace's lifetime", which an ENV pin would quietly break.
# NOTE: --sessions-dir gets an EMPTY temp dir, never /tmp. The stage banner is
# printed before any export, so nothing needs to be found — and pointing a
# default-staged run at a populated dir would export whatever transcripts it
# finds into the real stage, which is how a synthetic test session ends up
# staged for mining as if it were a real conversation.
#
# Asserted $HOME-RELATIVE, not against a literal /home/developer. `run` invokes
# `docker run --entrypoint=""`, and neither Dockerfile sets USER or ENV HOME
# (HOME is set by entrypoint-user.sh, which --entrypoint="" deliberately skips),
# so these assertions execute as root with HOME=/root. The original literal
# /home/developer form could therefore never match and failed the v1.8.0
# release — a test bug, not a product one: the stage resolution was correct all
# along, it just follows $HOME. The invariant under test ("the stage sits beside
# the palace, sharing its lifetime") is user-independent, so pinning the user
# was never part of it. A cache-dir default still fails the pattern below, which
# is the regression this guards.
#
# It went unnoticed for three days because this workflow only triggers on
# `push: tags: v*` — the assertion was added on a main push, so v1.8.0 was its
# first execution ever. Use the `smoke_only` workflow_dispatch input to run
# smoke against HEAD without cutting a tag.
run "pi stage defaults next to the palace (not a cache dir)" '
out=$(mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
stage=$(echo "$out" | grep -oE "stage=[^ ]+" | head -1)
echo "resolved ${stage:-<no stage= line>} with HOME=$HOME" >&2
case "$stage" in
"stage=$HOME/.mempalace/pi-stage/"*) exit 0 ;;
*) exit 1 ;;
esac
'
# Companion to the above: the deployment-specific case the literal assertion was
# reaching for, done properly by supplying the HOME the container actually runs
# with instead of assuming it.
run "pi stage is palace-adjacent for the developer user" '
out=$(HOME=/home/developer mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
echo "$out" | grep -oE "stage=[^ ]+" | head -1 >&2
echo "$out" | grep -q "stage=/home/developer/.mempalace/pi-stage/"
'
run "pi stage follows MEMPALACE_PALACE_PATH" '
out=$(MEMPALACE_PALACE_PATH=/tmp/alt/.mempalace/palace \
mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
echo "$out" | grep -q "stage=/tmp/alt/.mempalace/pi-stage/"
'
# The feeder's --agent default is WHO a drawer is attributed to. mempalace core
# records neither the machine nor the harness on a write, and one shared bearer
# token means the server cannot tell clients apart, so toolkit c64ffa1 changed
# this default from $USER to pi@$MEMPALACE_PI_DEVICE — the one string that makes
# a write attributable to both. Nothing ever PRINTED the resolved value (the
# banner shows mode= and stage= only), so an image built from a pre-c64ffa1
# toolkit ref would ship unattributed writes with every check still green.
#
# `--help` assigns AGENT (script top) before it parses args, then exits 0 with
# no side effects — so `bash -x` observes the REAL resolution, env interpolation
# and fallback included, rather than grepping the source for a literal line that
# any reformat would break. Two-sided on purpose: device set => pi@<device>;
# device UNSET => must not be pi@anything. The second half is what fails against
# the old unconditional $USER default, which ignored the device entirely.
#
# Probes the PATH entry (a symlink into the /opt clone) rather than that clone
# path directly: this is the invocation the systemd/launchd timers and
# entrypoint-user.sh actually use, so it is the default that reaches the palace.
run "feeder resolves --agent to pi@<device> (drawer attribution)" '
f=$(command -v mempalace-pi-session) || { echo "feeder not on PATH" >&2; exit 1; }
with=$(MEMPALACE_PI_DEVICE=smoke-device bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
without=$(env -u MEMPALACE_PI_DEVICE bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
echo "resolved with-device=[$with] without-device=[$without]" >&2
[ "$with" = "pi@smoke-device" ] || exit 1
case "$without" in pi@*) exit 1 ;; esac
echo ok
'
# Regression guard for the pi transcript exporter. If pi ever changes its
# session JSONL shape, the exporter stops recognising sessions and the palace
# silently gets nothing (or, worse, raw JSON chunked as prose). Feed it a
# synthetic session and assert it is actually exported. Uses --dry-run so no
# palace is touched, and a temp stage so nothing real is written.
run "pi transcript exporter recognises a pi session" '
set -e
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
{
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question one\"}}"
a=$(printf "a%.0s" $(seq 1 1200))
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"$a\"}]}}"
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question two\"}}"
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"short reply\"}]}}"
} > "$s/2026-01-01T00-00-00-000Z_smoke.jsonl"
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
echo "$out" | grep -q "Exported 1 session"
'
# The same guard from the other side: a session with no real assistant output
# (an abandoned prompt, whose bulk is injected skill text) must NOT be filed.
run "pi transcript exporter rejects an abandoned session" '
set -e
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
{
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke2\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
u=$(printf "u%.0s" $(seq 1 13000))
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"$u\"}}"
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"Ready. What would you like to work on?\"}]}}"
} > "$s/2026-01-01T00-00-00-000Z_smoke2.jsonl"
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
echo "$out" | grep -q "no sessions qualified"
'
# The remote-palace-without-inbox skip must ANNOUNCE itself, not vanish. This
# branch of entrypoint-user.sh runs at container start (not reachable from a
# `docker run` one-shot), so assert against the entrypoint that actually shipped
# in the image. Guards a silent regression back to the bare `:` no-op, which
# left a container contributing nothing to the palace with no artifact saying
# why — the log it would normally leave is written by the other branch.
run_expect "remote-palace-without-inbox skip is announced, not silent" \
"grep -o 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | head -1" \
"MemPalace catch-up skipped"
run "...and the skip notice names the variable that fixes it" \
"grep -A6 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | grep -q 'MEMPALACE_PI_SSH_TARGET'"
# A remote mine that FAILS must not report success. MCP answers a hard tool
# failure with HTTP 200 and the tool's own JSON escaped inside
# result.content[].text, so the feeder's old `'\"error\"' in body` check could
# never see it: on 2026-08-15 a mine that died with "source directory not found:
# '/data/feed/...'" logged "Done. Wing updated." and exited 0, and this
# container's transcripts were filed nowhere for a whole session. The feeder
# carries fixtures for that exact body; run them against the baked toolkit so a
# stale/reverted toolkit ref can't reintroduce a silent feed.
run "baked feeder detects a failed remote mine (no silent false success)" \
"mempalace-pi-session --self-test"
# v1.0.0 base additions — verify presence and basic functionality.
run "pandoc" "pandoc --version"
run "typst" "typst --version"
run "pandoc+typst PDF engine" "printf '# hi\n' | pandoc --pdf-engine=typst -o /tmp/_smoke.pdf - && test -s /tmp/_smoke.pdf; rm -f /tmp/_smoke.pdf"
run "graphviz (dot)" "dot -V"
run "imagemagick" "magick --version"
run "yq (mikefarah v4)" "yq --version | grep -qE 'mikefarah.*version v4'"
run "tldr (tealdeer)" "tldr --version"
run "socat" "socat -V"
run "studio-expose helper" "test -x /usr/local/bin/studio-expose"
run "image-baked pi-devbox-environment skill" \
"test -f /usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md"
run "image-baked credential-incident-response skill" \
"test -f /usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md"
run "global-AGENTS append snippet present" \
"test -f /usr/local/share/pi-devbox/pi-global-AGENTS.append.md"
run "pi-devbox block merged into pi-global-AGENTS.md" \
"grep -q 'pi-devbox:managed-block' /opt/pi-toolkit/pi-global-AGENTS.md"
run "mempalace session-start pointer merged into global AGENTS.md" \
"grep -q 'load the mempalace skill' /opt/pi-toolkit/pi-global-AGENTS.md"
# Vendored fallback skills (so a no-skillset container still resolves the
# AGENTS.md 'read the pi-extensions skill' pointer).
run "image-baked pi-extensions fallback skill" \
"test -f /usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md"
run "pi-extensions skill ships its helper" \
"test -f /usr/local/share/pi-devbox/skills/pi-extensions/evaluate-extension-usage.py"
run "image-baked mempalace fallback skill" \
"test -f /usr/local/share/pi-devbox/skills/mempalace/SKILL.md"
# Layered freshness: when the pinned pi-extensions clone carries the skill, the
# baked copy must be the fresh package copy (Option 1), not the stale snapshot.
run "pi-extensions skill refreshed from package when present" \
"if [ -f /opt/pi-extensions/skill/SKILL.md ]; then cmp -s /opt/pi-extensions/skill/SKILL.md /usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md; else true; fi"
# Runtime ownership handover (v1.8.5): the baked links are a FALLBACK, and
# skillset-OWNED skills must be repointed at the live clone when one is mounted.
# The list is data, so assert its content, not just its presence: mempalace in,
# pi-extensions deliberately out (its skillset copy is a lagging duplicate).
run "devbox-skill-reconcile helper present + executable" \
"test -x /usr/local/bin/devbox-skill-reconcile"
run "skillset-owned list ships and names mempalace" \
"grep -qx 'mempalace' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
run "skillset-owned list excludes pi-extensions (ownership)" \
"! grep -qx 'pi-extensions' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
# ── tmux 0-indexing (required for pi-studio variants) ─────────────────
echo ""
echo "── tmux config ──"
run_expect "/etc/tmux.conf has base-index 0" \
"cat /etc/tmux.conf" "set -g base-index 0"
run_expect "/etc/tmux.conf has pane-base-index 0" \
"cat /etc/tmux.conf" "set -g pane-base-index 0"
# ── Repo clones ───────────────────────────────────────────────────────
echo ""
echo "── Repo clones ──"
run "pi-toolkit clone" "test -d /opt/pi-toolkit && git -C /opt/pi-toolkit rev-parse --short HEAD"
run "pi-extensions clone" "test -d /opt/pi-extensions && git -C /opt/pi-extensions rev-parse --short HEAD"
run "pi-fork clone + node_modules" \
"test -f /opt/pi-fork/package.json && test -d /opt/pi-fork/node_modules"
# om is checked differently from pi-fork ON PURPOSE. It declares ZERO runtime
# dependencies: 8 devDependencies (omitted by --omit=dev) and 4 peerDependencies,
# which pi itself provides. npm 10 still materialised a node_modules for it, but
# that directory held exactly ONE file (.package-lock.json, 4 KB) and no nested
# package.json at all — 20 empty scope dirs. npm 11 stopped creating it, so the
# old `test -d node_modules` assertion went red on v1.9.0 while nothing about om
# had changed or broken. It was asserting an npm artefact, not a property of the
# shipped software. What actually has to hold is that the entry point pi loads
# exists, so assert THAT, straight out of the manifest pi reads
# (package.json -> pi.extensions), rather than a hardcoded path that could drift.
run "pi-observational-memory clone + declared pi entry point" \
"test -f /opt/pi-observational-memory/package.json && \
node -e 'const p=require(\"/opt/pi-observational-memory/package.json\"),f=require(\"fs\"),h=require(\"path\"); \
const l=(p.pi&&p.pi.extensions)||[]; \
if(!l.length){console.error(\"package.json declares no pi.extensions\");process.exit(1)} \
for(const e of l){const t=h.resolve(\"/opt/pi-observational-memory\",e); \
if(!f.existsSync(t)){console.error(\"declared entry missing: \"+t);process.exit(1)}} \
console.log(\"entries ok: \"+l.join(\",\"))'"
# ...and that the clone carries the AUTH FIX, not merely that it exists. om's
# pre-flight hasUsableAuth() check silently disabled `recall` for ~8 weeks once
# pi moved to request-time SigV4 signing and stopped exposing a static Bedrock
# key; upstream fixed it in ce9fc98, adopted in v1.8.4. PI_OBSMEM_REF tracks
# master, so an upstream revert or force-push would ship a dead `recall` with
# the clone assertion above still green — the exact gap flagged as open in the
# v1.8.5 changelog.
#
# Pin the markers to src/runtime.ts, the fix SITE, rather than grepping the
# repo: two of these three strings also appear under tests/, so a repo-wide
# grep stays green with runtime.ts itself reverted. That is a false green of the
# same family as the old skill-snapshot canary.
run "pi-observational-memory carries the ce9fc98 auth fix (recall stays alive)" '
f=/opt/pi-observational-memory/src/runtime.ts
test -f "$f" || { echo "fix site missing: $f" >&2; exit 1; }
for m in availability_recheck providerCredentialConfigured hasConfiguredAuth; do
grep -q "$m" "$f" || { echo "marker absent from runtime.ts: $m" >&2; exit 1; }
done
echo ok
'
# pi-atelier: deliberately NO node_modules assertion, unlike its siblings —
# it declares zero runtime dependencies (only peerDeps, satisfied by the baked
# pi) and has no build step, so Dockerfile.variant skips `npm install` for it.
# Assert what pi actually loads instead: the entry point named by its
# package.json `pi.extensions` key.
run "pi-atelier clone + entry point" \
"test -f /opt/pi-atelier/package.json && test -f /opt/pi-atelier/extensions/index.ts"
# ── pi <-> pi-atelier compatibility floor (executable, not a comment) ──
# pi-atelier < 0.7.1 wraps pi's PRIVATE TUI renderer in a way that recurses
# under pi >= 0.84: pi hangs at startup burning CPU, with no error. Upstream
# fixed it in 0.7.1/0.7.2, but atelier's peerDependencies still say
# `>=0.80.7`, so neither npm nor pi can warn about the real floor. Both
# versions are pinned in Dockerfile.variant; this makes a bad PAIRING fail the
# build instead of publishing an image whose TUI never starts.
run_expect "pi-atelier >= 0.7.1 floor for pi >= 0.84 (startup-hang guard)" \
'ge() { [ "$(printf "%s\n%s\n" "$1" "$2" | sort -V | head -n1)" = "$2" ]; }; AV=$(jq -r ".version // empty" /opt/pi-atelier/package.json 2>/dev/null); PV=$(pi --version 2>/dev/null | grep -oE "[0-9]+\.[0-9]+\.[0-9]+" | head -n1); if [ -z "$AV" ] || [ -z "$PV" ]; then echo "unreadable versions (atelier=$AV pi=$PV)"; elif ge "$PV" 0.84.0 && ! ge "$AV" 0.7.1; then echo "VIOLATION: pi $PV with pi-atelier $AV"; else echo "compatible: pi $PV + pi-atelier $AV"; fi' \
"compatible:"
# pi-studio is present only in the :latest-studio variant. Auto-detect by
# probing /opt/pi-studio so this one script covers both variants.
if docker run --rm --entrypoint="" "$IMAGE" sh -c 'test -d /opt/pi-studio' >/dev/null 2>&1; then
STUDIO_VARIANT=1
echo " ℹ️ pi-studio detected — running studio assertions"
run "pi-studio clone + node_modules" \
"test -f /opt/pi-studio/package.json && test -d /opt/pi-studio/node_modules"
run "pi-studio prebuilt client bundle" \
"test -f /opt/pi-studio/client/studio-client.js"
else
STUDIO_VARIANT=0
echo " ℹ️ pi-studio not present (non-studio variant) — skipping studio clone checks"
fi
# ── Build provenance (manifest + OCI labels) ─────────────────────────
echo ""
echo "── Build provenance ──"
run "/etc/pi-devbox/build-manifest.json present" \
"test -f /etc/pi-devbox/build-manifest.json"
# These next checks replace three that grepped the manifest for the FIELD NAME
# and never looked at the value:
#
# run_expect "manifest records pi_version" "cat …manifest.json" '"pi_version"'
#
# which passes on {"pi_version": ""} and on {"pi_version": null}. The tell was
# visible in its own passing output — `✅ manifest records pi_version (got
# "pi_version")` echoes the key back as the thing it claims to have found.
# Two failure modes were therefore invisible: a key that survives with an empty
# or garbage value, and a key that vanishes from the manifest while every
# remaining value still looks fine.
#
# Those two need SEPARATE assertions, and the reason is a trap worth keeping in
# writing: an "every component value is a valid SHA" loop passes VACUOUSLY on
# components:{} — jq's all() over an empty list is true — so the value check
# alone would go green on a manifest that lost every component. Mutation-tested
# 2026-08-25 across nine fabricated manifests (empty map, deleted key, "",
# null, "unknown", 12-hex truncation, 40 non-hex chars, legit null pi-studio).
run "manifest declares every required component key" '
req="pi-toolkit pi-extensions pi-fork pi-observational-memory pi-atelier mempalace-toolkit pi-studio"
for k in $req; do
jq -e --arg k "$k" "(.components|has(\$k))" /etc/pi-devbox/build-manifest.json >/dev/null \
|| { echo "manifest lost component key: $k" >&2; exit 1; }
done
'
# Subsumes the old `! grep -q \"unknown\"` check ("unknown" is not 40-hex), and
# also catches "", null and truncated SHAs, which that grep let through. null is
# legitimate for pi-studio alone: the non-studio variant has no such clone.
run "manifest component values are resolved 40-hex commits" '
jq -e "
.components
| to_entries
| all(if .key == \"pi-studio\" and .value == null then true
else (.value|type) == \"string\" and (.value|test(\"^[0-9a-f]{40}\$\")) end)
" /etc/pi-devbox/build-manifest.json >/dev/null
'
# pi_version against ground truth, same shape as the mempalace check below.
# Chains with the "pi version matches build arg" assertion earlier in this file:
# together they tie build arg -> installed binary -> recorded manifest, so a
# manifest written from a stale variable cannot pass by agreeing with itself.
run "manifest pi_version matches the installed pi" '
m=$(jq -r ".pi_version // empty" /etc/pi-devbox/build-manifest.json)
b=$(pi --version 2>/dev/null | head -n1 | tr -d "\r")
echo "manifest=[$m] installed=[$b]" >&2
[ -n "$m" ] && [ "$m" = "$b" ]
'
# Top-level provenance fields: assert the SHAPE of each value, and only when the
# field is populated. source_revision and build_date legitimately default to
# empty (Dockerfile.variant ARGs) on a plain local `docker build`, so demanding
# them would fail honest local smoke runs; a populated-but-malformed value is
# the actual defect. release_tag defaults to "dev", so empty means a broken write.
run "manifest top-level fields are well-formed, not merely present" '
j=/etc/pi-devbox/build-manifest.json
t=$(jq -r ".release_tag // empty" $j)
r=$(jq -r ".source_revision // empty" $j)
d=$(jq -r ".build_date // empty" $j)
echo "release_tag=[$t] source_revision=[$r] build_date=[$d]" >&2
[ -n "$t" ] || { echo "release_tag empty (ARG default is dev)" >&2; exit 1; }
if [ -n "$r" ]; then
printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" || { echo "source_revision not a 40-hex commit" >&2; exit 1; }
fi
if [ -n "$d" ]; then
printf "%s" "$d" | grep -qE "^[0-9]{4}-[0-9]{2}-[0-9]{2}T" || { echo "build_date not ISO-8601" >&2; exit 1; }
fi
'
# mempalace CORE was absent from the manifest through v1.8.5: the toolkit SHA
# was recorded but the palace version behind the MCP tools was not, so a palace
# bug could not be correlated to an image version. Assert the field exists AND
# equals the installed binary — recording it from ARG MEMPALACE_VERSION instead
# would look identical here yet drift silently the first time an install
# resolved to something other than the pin, which is the whole reason this file
# is built from ground truth. `// empty` matters: jq -r prints the 4-char
# string "null" for a JSON null, which would satisfy a naive -n test.
run "manifest mempalace_version matches the installed core" '
m=$(jq -r ".mempalace_version // empty" /etc/pi-devbox/build-manifest.json)
b=$(mempalace --version 2>/dev/null | head -n1 | tr -d "\r"); b=${b##* }
echo "manifest=[$m] installed=[$b]" >&2
[ -n "$m" ] && [ "$m" = "$b" ]
'
# ... and, when CI supplies it, that the installed core is the version CI
# actually AUDITED (published + not yanked on PyPI, in resolve-versions). This
# does NOT duplicate the check above, which compares two properties of one
# image and so cannot notice that BOTH are the wrong version. The live failure
# mode it covers: the variant builds `FROM` a base tag chosen by base-decide's
# content hash, so a bug in that hashing (the reason scripts/check-base-hash.sh
# exists) could reuse a cached base built from an OLDER MEMPALACE_VERSION pin —
# internally consistent, silently stale, invisible to every other assertion.
if [ -n "${EXPECTED_MEMPALACE_VERSION:-}" ]; then
run "installed mempalace matches CI's audited pin (${EXPECTED_MEMPALACE_VERSION})" "
b=\$(mempalace --version 2>/dev/null | head -n1 | tr -d '\r'); b=\${b##* }
echo \"installed=[\$b] audited_pin=[${EXPECTED_MEMPALACE_VERSION}]\" >&2
[ \"\$b\" = \"${EXPECTED_MEMPALACE_VERSION}\" ]
"
fi
# Every component must be a resolved commit (or null for pi-studio in the
# non-studio variant) — now enforced by the 40-hex value check above, which
# strictly subsumes the old whole-file grep for '"unknown"'. Only rev() ever
# emits "unknown" and rev() feeds components only, so nothing is lost.
# pi-devbox-version wraps the manifest into a human-first command; verify the
# binary is present, executable, and that all three output modes work.
run "pi-devbox-version binary present + executable" \
"test -x /usr/local/bin/pi-devbox-version"
run_expect "pi-devbox-version human output shows release tag" \
"pi-devbox-version" "pi-devbox "
# --json is a verbatim `cat` of the manifest, so "round-trips" is assertable
# literally. The old form grepped the output for the string "release_tag" — the
# key name again — which would pass on a truncated or re-serialised dump.
run "pi-devbox-version --json round-trips the manifest byte-for-byte" '
a=$(cat /etc/pi-devbox/build-manifest.json)
b=$(pi-devbox-version --json)
[ "$a" = "$b" ] || { echo "--json output differs from the manifest on disk" >&2; exit 1; }
'
run_expect "pi-devbox-version --quiet is a compact one-liner" \
"pi-devbox-version --quiet | wc -l" "1"
# ── Vendored skill snapshot provenance ─────────────────────────────────
# The vendored mempalace skill is the one baked artefact with no /opt clone
# behind it (private upstream — see VENDORED.md), so until now the manifest
# could not say which skillset commit it came from. Two fields now travel with
# it: the CLAIMED ref (ARG default in Dockerfile.variant) and the MEASURED
# sha256 of the shipped bytes. Assert both are well-formed, and — separately —
# that the measurement still describes the file in the image.
#
# Kept as two assertions for the same reason the component checks are: one
# proves the fields are not empty/garbage, the other proves they are not merely
# self-consistent. A single combined check could pass on a manifest whose hash
# was computed from a file that was later overwritten (the pi-extensions skill
# copy at Dockerfile.variant:165 does exactly that kind of overwrite, one stage
# earlier), which is the failure this second one exists to catch.
run "manifest records the vendored skill snapshot provenance" '
j=/etc/pi-devbox/build-manifest.json
r=$(jq -r ".skillset_snapshot_ref // empty" $j)
s=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
echo "ref=[$r] tree_sha256=[$s]" >&2
printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" \
|| { echo "skillset_snapshot_ref is not a 40-hex commit" >&2; exit 1; }
printf "%s" "$s" | grep -qxE "[0-9a-f]{64}" \
|| { echo "skillset_snapshot_tree_sha256 is not a 64-hex digest" >&2; exit 1; }
'
# Recomputes over the whole DIRECTORY with the same tree_sha256() pipeline
# Dockerfile.variant used to measure it, not a plain `sha256sum SKILL.md` —
# a file-only compare here would pass even if the manifest recorded a
# fingerprint over a directory that has since grown a second file (this is
# not hypothetical: pi-extensions already ships two files for its skill).
run "manifest skill fingerprint matches the baked snapshot" '
j=/etc/pi-devbox/build-manifest.json
d=/usr/local/share/pi-devbox/skills/mempalace
m=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
a=$( (cd "$d" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum) | sha256sum | cut -d" " -f1)
echo "manifest=[$m] actual=[$a]" >&2
[ -n "$m" ] && [ "$m" = "$a" ]
'
# ── Which pi-extensions skill copy shipped ──────────────────────────────
# Closes the silent-fallback hole. The refresh in Dockerfile.variant is guarded
# by `[ -f /opt/pi-extensions/skill/SKILL.md ]`, so a build whose clone predates
# the co-located skill keeps the vendored floor and still succeeds GREEN, with
# nothing recording that a snapshot shipped instead of the package copy. Measured
# 2026-09-10: the floor had been stale since 2026-07-30, so that path would have
# shipped a six-week-old skill in silence. The floor is fresh now and gated by the
# skill-floor lint job, but "the fallback is currently harmless" is a fact with a
# shelf life, whereas "the image says which copy it got" keeps working.
#
# vendored-floor FAILS here rather than merely warning: these images track main,
# where the package has co-located skill/ since fa04d20, so a fallback means the
# clone did not resolve as intended and that is a defect to investigate. A fork
# deliberately pointing at a mirror without skill/ is the one case that should
# edit this assertion — which is the honest place for that decision to surface.
run "manifest names which pi-extensions skill copy shipped" '
j=/etc/pi-devbox/build-manifest.json
s=$(jq -r ".pi_extensions_skill_source // empty" $j)
h=$(jq -r ".pi_extensions_skill_tree_sha256 // empty" $j)
echo "source=[$s] tree_sha256=[$h]" >&2
printf "%s" "$h" | grep -qxE "[0-9a-f]{64}" || {
echo "pi_extensions_skill_tree_sha256 is not a 64-hex digest" >&2; exit 1; }
case "$s" in
package) ;;
vendored-floor)
echo "FALLBACK: clone had no skill/ at this ref, so the image ships the committed floor" >&2; exit 1 ;;
divergent)
echo "MIXED: served directory is part package and part floor" >&2; exit 1 ;;
*)
echo "pi_extensions_skill_source absent or unrecognised" >&2; exit 1 ;;
esac
'
# Same shape as the mempalace fingerprint check above, and for the same reason: a
# recorded hash that is never recomputed is a claim, not a measurement.
run "recorded pi-extensions skill hash matches the served bytes" '
j=/etc/pi-devbox/build-manifest.json
d=/usr/local/share/pi-devbox/skills/pi-extensions
m=$(jq -r ".pi_extensions_skill_tree_sha256 // empty" $j)
a=$( (cd "$d" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum) | sha256sum | cut -d" " -f1)
echo "manifest=[$m] actual=[$a]" >&2
[ -n "$m" ] && [ "$m" = "$a" ]
'
# OCI labels live in the image config, not the container fs — inspect them
# from the host docker rather than via `docker run`.
LBL=$(docker inspect --format '{{ index .Config.Labels "se.jordbo.pi-devbox.pi-extensions-ref" }}' "$IMAGE" 2>/dev/null || true)
if [ -n "$LBL" ] && [ "$LBL" != "<no value>" ]; then
printf " ✅ OCI label se.jordbo.pi-devbox.pi-extensions-ref=%s\n" "$LBL"; PASS=$((PASS+1))
else
printf " ❌ OCI label se.jordbo.pi-devbox.pi-extensions-ref missing or empty\n"; FAIL=$((FAIL+1))
fi
# mempalace-version is set in Dockerfile.base and INHERITED by the variant, so
# it states the pin of the base this image actually built on. It must equal the
# installed binary: the one way they diverge is a base built with
# INSTALL_MEMPALACE=false (label says 3.x, nothing installed) or an install that
# resolved to something other than the pin — both invisible to a label-only
# check. Same ground-truth rule as the manifest assertion above.
MP_LBL=$(docker inspect --format '{{ index .Config.Labels "se.jordbo.pi-devbox.mempalace-version" }}' "$IMAGE" 2>/dev/null || true)
MP_BIN=$(docker run --rm --entrypoint= "$IMAGE" sh -c 'mempalace --version 2>/dev/null | head -n1 | tr -d "\r"' 2>/dev/null || true); MP_BIN=${MP_BIN##* }
if [ -n "$MP_LBL" ] && [ "$MP_LBL" != "<no value>" ] && [ "$MP_LBL" = "$MP_BIN" ]; then
printf " ✅ OCI label se.jordbo.pi-devbox.mempalace-version=%s equals the installed core\n" "$MP_LBL"; PASS=$((PASS+1))
else
printf " ❌ OCI label se.jordbo.pi-devbox.mempalace-version=[%s] vs installed mempalace=[%s]\n" "$MP_LBL" "$MP_BIN"; FAIL=$((FAIL+1))
fi
# ── Runtime deployment (needs entrypoint to run) ──────────────────────
echo ""
echo "── Runtime deployment ──"
# Spin up a long-running container WITHOUT overriding the entrypoint, so
# the baked entrypoint chain (entrypoint.sh → entrypoint-user.sh) runs and
# deploys pi-toolkit + pi-extensions to ~/.pi/agent/. Override CMD to
# tail -f /dev/null so the container stays alive while we docker-exec.
CID=$(docker run -d --rm "$IMAGE" tail -f /dev/null)
cleanup() { docker rm -f "$CID" >/dev/null 2>&1 || true; }
trap cleanup EXIT
# Wait for entrypoint-user.sh to finish deploying pi-toolkit + extensions.
# Gate on BOTH the keybindings symlink (deployed by pi-toolkit) AND the
# mempalace.ts bridge (deployed last by entrypoint-user.sh) AND ≥4 *.ts
# extensions present. Parallel build load can otherwise sample the *.ts
# count mid-deploy and produce a flake. See opencode-devbox c6f9d11
# (2026-06-08) — same fix transplanted.
for i in $(seq 1 45); do
if docker exec "$CID" sh -c '
test -L /home/developer/.pi/agent/keybindings.json && \
test -L /home/developer/.pi/agent/extensions/mempalace.ts && \
test -L /home/developer/.agents/skills/pi-devbox-environment && \
test -L /home/developer/.agents/skills/pi-extensions && \
test -L /home/developer/.agents/skills/mempalace && \
count=$(ls -1 /home/developer/.pi/agent/extensions/*.ts 2>/dev/null | wc -l) && \
[ "$count" -ge 4 ]
' >/dev/null 2>&1; then
break
fi
sleep 1
done
exec_test() {
local label="$1"; local cmd="$2"
if docker exec -u developer "$CID" sh -c "$cmd" >/dev/null 2>&1; then
printf " ✅ %s\n" "$label"; PASS=$((PASS+1))
else
printf " ❌ %s\n" "$label"; FAIL=$((FAIL+1))
fi
}
exec_test "keybindings.json (pi-toolkit)" 'test -L $HOME/.pi/agent/keybindings.json && echo ok'
exec_test "extensions ≥ 4 (pi-extensions)" 'count=$(ls -1 $HOME/.pi/agent/extensions/*.ts 2>/dev/null | wc -l); [ $count -ge 4 ] && echo "$count extensions"'
exec_test "mempalace.ts bridge" 'test -L $HOME/.pi/agent/extensions/mempalace.ts && echo ok'
exec_test "settings.json bootstrapped" 'test -f $HOME/.pi/agent/settings.json && echo ok'
exec_test "pi-devbox-environment skill linked" 'test -L $HOME/.agents/skills/pi-devbox-environment && test -f $HOME/.agents/skills/pi-devbox-environment/SKILL.md && echo ok'
exec_test "pi-extensions skill linked (fallback)" 'test -L $HOME/.agents/skills/pi-extensions && test -f $HOME/.agents/skills/pi-extensions/SKILL.md && echo ok'
exec_test "mempalace skill linked (fallback)" 'test -L $HOME/.agents/skills/mempalace && test -f $HOME/.agents/skills/mempalace/SKILL.md && echo ok'
# The vendored mempalace snapshot is refreshed MANUALLY per release (see
# rootfs/usr/local/share/pi-devbox/skills/VENDORED.md). Through v1.8.4 it also
# silently SHADOWED the live skillset copy, so staleness was invisible — and the
# canary that was supposed to catch it could not: it grepped "Shared palace:
# multiple harnesses", a phrase present in BOTH the stale and the fresh copy.
# A snapshot canary must pin the NEWEST section, so update this string whenever
# the snapshot is refreshed — that is the point of it.
#
# v1.8.7: this fired for real, and on the release that changed the snapshot. The
# pinned phrase was "Attribute what you file yourself", the heading of the
# instruction telling agents to hand-stamp added_by — which that same release
# WITHDREW (RFC 001 §7.3.2 ranks agent-side stamping worst-possible; the bridge
# now does it). So the canary correctly reported "snapshot changed, expectation
# did not", and blocked publication of an otherwise-green build (81 passed, 1
# failed, twice). Two lessons kept in the assertion itself:
# * it is now BIDIRECTIONAL — the new phrase must be present AND the withdrawn
# one absent, so a re-vendored stale snapshot fails just as loudly as a
# forgotten bump. A one-way canary only catches half the drift.
# * a phrase canary can only ever detect "older than what I remembered to pin",
# never "older than skillset main".
#
# That structural limit is now addressed, but NOT by the "CI job diffing this
# file against the skillset repo" this comment used to point at (that pointer
# also dangled: it referenced an Unreleased changelog note that had become the
# v1.8.7 heading). A CI diff cannot be done without granting CI a credential
# for the PRIVATE skillset repo, and it would guard a file that on this fleet
# NO host reads — all four compose stacks mount a workspace containing the
# skillset, so devbox-skill-reconcile repoints this link at the live clone and
# the baked copy is a CI/no-mount fallback only. Instead the snapshot now
# carries its provenance (skillset_snapshot_ref + a measured
# skillset_snapshot_sha256 in build-manifest.json, written by
# scripts/vendor-mempalace-skill.sh), which moves the check to where the
# skillset actually IS: `scripts/vendor-mempalace-skill.sh --check` for a
# maintainer, and `pi-devbox-version` for an agent inside any container.
# This assertion is kept because it is orthogonal and free: it pins content,
# not provenance, so it still catches a re-vendored snapshot whose ref was
# bumped correctly but whose bytes came from the wrong place.
#
# v1.8.13: RE-PINNED on refresh a12fe5e -> e9e09d9, which is the whole point of
# the mechanism — the previous pair ("Provenance is stamped for you" present /
# "Attribute what you file yourself" absent) still passed against the NEW
# snapshot, so leaving it would have produced a canary that is green on both the
# old and the new bytes, i.e. blind to precisely the refresh it exists to
# witness. Same false-green family as the pre-v1.8.5 canary this comment warns
# about. The replacement pair was chosen by MEASURING direction against both
# files rather than by reading the diff: "Diaries self-heal; plain drawers do
# not" is new=1/old=0, "Agent diaries live in" is new=0/old=1 — so each string
# discriminates on its own and the pair still fails loudly in BOTH directions
# (forgotten bump AND re-vendored stale snapshot). Upstream content behind this
# refresh: the bare project-name wing convention and the <harness>@<device>
# added_by rule.
#
# Unreleased: RE-PINNED again on refresh e9e09d9 -> e9e45f7. The retired pair was
# still green against the new snapshot (the diaries section was untouched), so it
# was blind to this refresh for the same reason the v1.8.13 pair was blind to
# that one. The replacement pair is unusually strong because BOTH witnesses come
# out of the same upstream commit: skillset e9e45f7 ADDED the "Withdrawing an ask
# you sent" bullet and DELETED the sentence "there is nothing anyone can do about
# it from the other end" that the new bullet contradicts. Directions were
# MEASURED against both files, not read off the diff: "Withdrawing an ask you
# sent" is new=1/old=0, "nothing anyone can do about it from the other end" is
# new=0/old=1. A canary whose negative witness was removed by the very commit it
# pins fails loudly on the OLD bytes instead of merely failing to notice them,
# which is the property every previous pair here lacked. Upstream content:
# requester-side ask withdrawal became DEPLOYED behaviour once v1.9.1 baked
# mempalace-toolkit e68ee20 (>= e2b060a) through the floating MEMPALACE_TOOLKIT_REF.
exec_test "mempalace skill snapshot is current" 'f=$HOME/.agents/skills/mempalace/SKILL.md; grep -q "Withdrawing an ask you sent" "$f" && ! grep -q "nothing anyone can do about it from the other end" "$f" && echo ok'
# Link TARGETS, not just link existence: with no skillset mounted (as here) the
# baked tree must be what resolves, for all four vendored skills.
exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
'for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
case "$(readlink -f $HOME/.agents/skills/$s)" in
/usr/local/share/pi-devbox/skills/$s) ;;
*) echo "$s resolves to $(readlink -f $HOME/.agents/skills/$s)" >&2; exit 1 ;;
esac
done; echo ok'
# ... and that the tool REPORTS that resolution, which is the half that was
# missing: a stale baked snapshot and a current live clone were
# indistinguishable from inside the container. CI mounts no skillset, so every
# vendored skill must report "baked" here — which also makes this a real test of
# the fallback path rather than of the environment it happens to run in.
exec_test "pi-devbox-version reports skill sources (all baked, no skillset here)" \
'out=$(pi-devbox-version)
echo "$out" | grep -q "skills:" || { echo "no skills section" >&2; exit 1; }
for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
echo "$out" | grep -qE "^ $s +baked( \([^)]*\))?$" \
|| { echo "$s not reported as baked" >&2; exit 1; }
done; echo ok'
# The optional " (...)" above is what pi-extensions now appends to say WHICH copy
# shipped — "baked (package copy)", or a loud FALLBACK/MIXED annotation. Without
# allowing it, adding that annotation turned this assertion red on v1.9.0 even
# though the state it reported was the correct one. The suffix is deliberately
# matched loosely rather than pinned to "(package copy)", because WHICH copy
# shipped is already asserted authoritatively above, against the manifest field
# and its measured tree hash, and duplicating that here in a regex would just
# create a second place to update whenever the wording changes.
# The boot banner must NOT carry the section: entrypoint-user.sh prints the
# version FIRST, before the baked links exist and long before the skillset
# deploy + reconcile run last, so anything it said about skill sources would be
# a pre-reconcile state that is about to change.
# A bare negative (`! grep -q "skills:"`) passes if the tool crashes or
# prints nothing at all — it cannot tell "correctly omitted the section"
# apart from "the binary is broken". Anchor it positively: the command must
# still succeed and still print its normal release-tag line.
exec_test "pi-devbox-version --no-skills omits the skills section" \
'out=$(pi-devbox-version --no-skills) && echo "$out" | grep -q "^pi-devbox " && ! echo "$out" | grep -q "skills:"'
exec_test "entrypoint prints the version banner with --no-skills" \
'grep -q "pi-devbox-version --no-skills" /usr/local/bin/entrypoint-user.sh'
# The handover path itself. CI never mounts a skillset, so without this the
# v1.8.5 fix would ship untested: fabricate a skillset + a skills dir holding
# baked-style links, run the reconciler, and assert all three outcomes —
# owned skill repointed, unowned skill left baked, user override untouched.
exec_test "reconciler: owned skill handed to live clone, others untouched" \
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/ss/skills/pi-extensions $t/skills
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo LIVE > $t/ss/skills/pi-extensions/SKILL.md
ln -s /usr/local/share/pi-devbox/skills/mempalace $t/skills/mempalace
ln -s /usr/local/share/pi-devbox/skills/pi-extensions $t/skills/pi-extensions
mkdir -p $t/skills/mine; echo MINE > $t/skills/mine/SKILL.md
devbox-skill-reconcile $t/ss $t/skills >/dev/null
devbox-skill-reconcile $t/ss $t/skills >/dev/null # idempotent
[ "$(readlink $t/skills/mempalace)" = "$t/ss/skills/mempalace" ] || { echo "owned skill NOT repointed" >&2; exit 1; }
[ "$(readlink $t/skills/pi-extensions)" = /usr/local/share/pi-devbox/skills/pi-extensions ] || { echo "unowned skill was repointed" >&2; exit 1; }
[ "$(cat $t/skills/mine/SKILL.md)" = MINE ] || { echo "user override clobbered" >&2; exit 1; }
rm -rf $t; echo ok'
# The case above cannot fail if the reconciler stops checking WHERE a link
# points — a mutation test showed all three of its assertions still passing with
# that guard deleted, which is the same false-green shape as the old snapshot
# canary. This one discriminates: an OWNED name (so it is considered) whose link
# is a user override pointing outside the baked tree (so it must be left alone).
exec_test "reconciler: user override on an owned name is left alone" \
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/skills $t/mine-skill
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo USERLINK > $t/mine-skill/SKILL.md
ln -sfn $t/mine-skill $t/skills/mempalace
devbox-skill-reconcile $t/ss $t/skills >/dev/null
[ "$(cat $t/skills/mempalace/SKILL.md)" = USERLINK ] || { echo "user symlink override clobbered" >&2; exit 1; }
rm -rf $t; echo ok'
# mempalace-census gained a /usr/local/bin symlink in v1.8.3; its three siblings
# had one since they were added, so this asserts the set stays complete.
exec_test "mempalace-census on PATH" 'command -v mempalace-census >/dev/null && mempalace-census --help >/dev/null && echo ok'
# pi-fork + pi-observational-memory are registered by entrypoint-user.sh via
# `pi install /opt/<pkg>`, which runs slightly after the keybindings marker.
#
# Assert against the `packages` ARRAY, never a whole-file grep: the settings
# template ships a top-level "pi-fork" CONFIG block, so `grep -q pi-fork
# settings.json` passes even when `pi install /opt/pi-fork` never ran. That
# false green is exactly why the missing `fork` tool shipped unnoticed from
# v1.0.0 through v1.6.3.
pkg_registered_cmd() {
printf "jq -e --arg n %s '(.packages // []) | any((type == \"string\") and (. == \"npm:\" + \$n or endswith(\"/\" + \$n)))' \$HOME/.pi/agent/settings.json" "$1"
}
for i in $(seq 1 15); do
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-observational-memory)" \
>/dev/null 2>&1; then
break
fi
sleep 1
done
exec_test "pi-fork registered in packages[] (fork tool)" \
"$(pkg_registered_cmd pi-fork)"
exec_test "pi-observational-memory registered in packages[] (recall tool)" \
"$(pkg_registered_cmd pi-observational-memory)"
# pi-studio registration (studio variant only) — registered by the same
# entrypoint-user.sh local-path install loop as fork/obsmem.
if [ "${STUDIO_VARIANT:-0}" = "1" ]; then
for i in $(seq 1 15); do
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-studio)" \
>/dev/null 2>&1; then
break
fi
sleep 1
done
exec_test "pi-studio registered in packages[] (/studio command + studio_* tools)" \
"$(pkg_registered_cmd pi-studio)"
fi
# pi-atelier registration. It is LAST in the entrypoint's install loop, so a
# pass here also means that loop ran to completion rather than dying midway.
for i in $(seq 1 15); do
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-atelier)" \
>/dev/null 2>&1; then
break
fi
sleep 1
done
exec_test "pi-atelier registered in packages[] (TUI sidebar)" \
"$(pkg_registered_cmd pi-atelier)"
# ...and registered from the vendored /opt copy, NOT as `npm:pi-atelier`: an
# npm: entry resolves through ~/.pi/npm-global on the config VOLUME, which
# outlives image upgrades and would silently keep an old, unaudited atelier —
# exactly the shape that pairs a stale 0.6.x with a new pi and hangs at startup.
exec_test "pi-atelier registered from /opt, not npm: (volume-shadowing guard)" \
'jq -e "((.packages // []) | any((type == \"string\") and endswith(\"/pi-atelier\"))) and (((.packages // []) | any(. == \"npm:pi-atelier\")) | not)" $HOME/.pi/agent/settings.json'
# agent-browser: the third package hit by ~/.pi/npm-global volume shadowing
# (after pi itself and pi-atelier). This build-time check is deliberately WEAK
# and says so: a `docker run` container has an EMPTY config volume, so it can
# only prove the image ships a sane copy and nothing in the image itself
# shadows it. The check that actually bites lives in
# recreate-sanity-check.sh, which runs where the volume is real — that is
# where a 7-week-old 0.27.0 was caught shadowing 0.35.2 on 2026-09-06.
# EXECUTION is ASSERTED here, not printed. Until 2026-09-07 the version was
# captured inside an echo with 2>/dev/null, so a binary that could not run at all
# still PASSED and simply printed version=[] -- the same failure class as the bare
# `node --version` two hundred lines up: a value displayed rather than compared.
#
# Why this exit code matters more than most: smoke runs `platforms: linux/amd64`
# on an x86 runner, i.e. NATIVE amd64, so this is the fleet's only recurring
# amd64 runtime proof for the linux-x64 ELF. No devbox can supply one -- every
# machine in the pi fleet is an Apple Silicon Mac (mbp-m1-2020; tor-ms22 = Mac
# Studio Mac13,1 M1 Max, verified 2026-08-17 by system_profiler; emb-7kj4vr4g =
# Apple Silicon, 4 routes 2026-09-07). Asking a device for that proof is asking
# for the impossible; CI already had it and was discarding it.
#
# KEEP PROSE OUT OF THE QUOTED BODY BELOW. On 2026-09-07 this explanation lived
# INSIDE the single-quoted argument and contained an apostrophe ("the fleet's").
# Inside '...' bash treats a backslash literally, so \' does not escape -- it
# CLOSES the string. The body silently truncated, the remaining lines were parsed
# by the RUNNER's shell instead of the container's, and `agent-browser --version`
# ran on a host that has no agent-browser: "line 770: command not found", release
# v1.8.14's smoke job failed after the base had already built. shellcheck caught
# it as SC2289 the same day and the red lint job went unread for 24h.
exec_test "agent-browser resolves under /usr (volume-shadowing guard, build-time half)" '
p=$(command -v agent-browser) || { echo "agent-browser not on PATH" >&2; exit 1; }
r=$(readlink -f "$p")
v=$(agent-browser --version) || { echo "agent-browser did not EXECUTE" >&2; exit 1; }
test -n "$v" || { echo "agent-browser --version produced no output" >&2; exit 1; }
echo "resolved=[$r] version=[$(printf %s "$v" | head -n1)]" >&2
case "$r" in /usr/*) ;; *) exit 1 ;; esac
test ! -d "$HOME/.pi/npm-global/lib/node_modules/agent-browser" || exit 1
echo ok
'
# pi-fork capability floor. `extensions: []` makes a fork child run with
# --no-extensions, which is the only MECHANICAL guarantee that a fork cannot
# file drawers or diary entries under the parent's identity — the mempalace
# bridge is an extension, so removing extensions removes the write path.
# Asserted because it is a security-shaped default that a settings merge or a
# hand-edit could silently drop, and its absence is invisible until a fork
# writes to the shared palace as you (measured twice: 2026-09-01, 2026-09-06).
# Deliberately compares to [] and not "is falsy": null means "load normal
# extensions", i.e. exactly the unguarded state this asserts against.
exec_test "pi-fork extensions floor is [] (forks cannot write to the palace)" \
'jq -e ".[\"pi-fork\"].extensions == []" $HOME/.pi/agent/settings.json'
# ── /tmp/sshcm directory created by entrypoint ────────────────────────
exec_test "/tmp/sshcm dir mode 700 (ssh ControlMaster)" \
'test -d /tmp/sshcm && [ "$(stat -c %a /tmp/sshcm)" = "700" ] && echo ok'
# ── Build-time leftovers (npm 11 bloat sentinels) ─────────────────────
# Both of these are worth a PASS/FAIL assertion rather than a size-gate
# diagnostic, because the size gate has ~225 MB of deliberate margin: v1.9.1
# shipped +131 MB of pure build residue and stayed green. These name the
# residue directly, so a regression is legible instead of merely "bigger".
echo ""
echo "── Build-time leftovers ──"
# npm 11 installs EVERY optional platform package of a native dependency, not
# just the host's (it ignores os/cpu, --os/--cpu and npmrc os=/cpu=). Two
# families are affected and pruned in Dockerfile.variant: @esbuild/<platform>
# and @mariozechner/clipboard-<triple>. Keep-set is the host arch only, plus
# clipboard's gnu AND musl (its napi loader picks between them at runtime).
# Runs as root because the image declares no USER; that is also what lets the
# cache assertion below read /root.
run "no foreign platform packages (npm 11 sentinel)" \
'arch=$(node -p process.arch); bad=$(find /usr/lib/node_modules /opt -type d \( -regex ".*/@esbuild/[^/]+" -o -regex ".*/@mariozechner/clipboard-[^/]+" \) ! -name "linux-$arch" ! -name "clipboard-linux-$arch-gnu" ! -name "clipboard-linux-$arch-musl" -prune -print 2>/dev/null); if [ -n "$bad" ]; then echo "foreign platform dirs shipped:" >&2; echo "$bad" >&2; du -sm $bad 2>/dev/null | sort -rn | head -5 >&2; exit 1; fi; echo ok'
# The build's own npm download cache is not free: it lands in the layer that
# created it. v1.9.1 shipped 145 MB of /root/.npm/_cacache (35 MB in v1.8.14)
# — the largest single item in its +131 MB residual, and invisible to the
# size-gate diagnostics because those only looked under node_modules and /opt.
# Nothing at runtime reads it: root's cache, while the container runs as
# `developer`. NOTE the assertion must run as root or a permission error on
# mode-700 /root would make `test ! -d` pass for the wrong reason.
run "no build-time npm cache shipped (/root/.npm)" \
'test "$(id -u)" = "0" || { echo "assertion needs root to read /root" >&2; exit 1; }; if [ -e /root/.npm ]; then echo "/root/.npm shipped: $(du -sm /root/.npm | cut -f1) MB" >&2; exit 1; fi; echo ok'
# The prune's risk is not "too big" but "removed something needed", and only a
# FUNCTIONAL check covers that. These load the natives from every install site
# found in the image, so they also scale to the studio variant's third site.
#
# NOTE THE PATH-QUALIFIED require(). The obvious form, `node -e
# 'require("esbuild")...'`, resolves by walking up from the CURRENT DIRECTORY —
# so it fails with MODULE_NOT_FOUND from /workspace on a perfectly good image,
# because esbuild lives nested inside the pi trees and global installs are not
# on node's require path (NODE_PATH is unset). That exact command was left in a
# runbook as "if this fails, revert the release", and it duly failed for the
# wrong reason on the first machine that ran it. A check must fail only for the
# thing it is checking.
run "esbuild works at every install site (prune removed weight, not function)" \
'sites=$(find /usr/lib/node_modules /opt -type d -path "*/node_modules/esbuild" -prune 2>/dev/null); if [ -z "$sites" ]; then echo "no esbuild install found at all" >&2; exit 1; fi; for d in $sites; do node -e "require(\"$d\").transformSync(\"const x:number=1\",{loader:\"ts\"})" || { echo "esbuild broken at $d" >&2; exit 1; }; done; echo ok'
# Clipboard is the family pruned second, and its napi-rs loader picks its native
# binding at require() time — so a successful load IS the proof that the kept
# platform package is the one this image needs.
run "clipboard native loads at every install site" \
'sites=$(find /usr/lib/node_modules /opt -type d -path "*/node_modules/@mariozechner/clipboard" -prune 2>/dev/null); if [ -z "$sites" ]; then echo "no @mariozechner/clipboard install found at all" >&2; exit 1; fi; for d in $sites; do node -e "var c=require(\"$d\"); if (typeof c.setText !== \"function\") { throw new Error(\"native binding missing\"); }" || { echo "clipboard native broken at $d" >&2; exit 1; }; done; echo ok'
# ── Image size ────────────────────────────────────────────────────────
echo ""
echo "── Image size ──"
# Sum all layers via `docker history`. Docker's `image inspect --format='{{.Size}}'`
# returns ONLY the variant-unique layer when the base is content-addressed and
# shared (the case in this repo's two-phase build), which understates the
# user-facing image size by 2+ GB. Summing layer sizes from history is the
# metric Hub displays to users and the one we actually want to gate on.
SIZE_MB=$(docker history --format '{{.Size}}' "$IMAGE" | python3 -c '
import sys, re
total=0.0
for line in sys.stdin:
s=line.strip()
if s in ("0B", ""): continue
m=re.match(r"^([0-9.]+)(B|kB|MB|GB)$", s)
if not m: continue
v=float(m.group(1)); u=m.group(2)
mult={"B":1/1048576,"kB":1/1024,"MB":1,"GB":1024}[u]
total+=v*mult
print(int(total))
')
if [ -z "$SIZE_MB" ] || [ "$SIZE_MB" = "0" ]; then
printf " ⚠️ image size: could not parse — skipping check\n"
elif [ "$SIZE_MB" -le "$SIZE_THRESHOLD_MB" ]; then
printf " ✅ size: %d MB (threshold %d MB)\n" "$SIZE_MB" "$SIZE_THRESHOLD_MB"; PASS=$((PASS+1))
else
printf " ❌ size: %d MB exceeds threshold %d MB\n" "$SIZE_MB" "$SIZE_THRESHOLD_MB"; FAIL=$((FAIL+1))
# A bare "too big" verdict cost a full CI-log dig plus a local npm bisect to
# attribute the v1.9.0 overshoot (+431 MB, which turned out to be npm 11
# installing 26 @esbuild platform binaries per pi-coding-agent copy). The
# container already knows where its bytes are, so make it say so: the biggest
# layers, and the biggest directories under the paths that historically grow.
# Same principle as the run() helper above — a red assertion should carry its
# own diagnostic rather than send the next reader spelunking.
echo " ── largest layers (docker history) ──"
docker history --format '{{.Size}}\t{{.CreatedBy}}' "$IMAGE" 2>/dev/null \
| grep -vE '^0B' | head -12 | sed 's/^/ /' | cut -c1-160
echo " ── largest directories in the image ──"
docker run --rm --entrypoint sh "$IMAGE" -c \
'du -sm /usr/lib/node_modules/* /opt/* /usr/local/share/ms-playwright 2>/dev/null | sort -rn | head -12' \
2>/dev/null | sed 's/^/ /' || echo " (could not inspect directories)"
echo " ── build caches that should not be in the image ──"
# v1.9.1's residual was 145 MB of npm cache under /root, and the du list
# above cannot see it: it enumerates node_modules and /opt only. A gate whose
# diagnostic looks only where the bytes were LAST time sends the next reader
# spelunking again, so name the cache paths explicitly.
docker run --rm --entrypoint sh "$IMAGE" -c \
'du -sm /root/.npm /root/.cache /tmp/node-compile-cache /home/developer/.npm 2>/dev/null | sort -rn' \
2>/dev/null | sed 's/^/ /' || true
echo " ── foreign platform dirs (npm 11 regression sentinel) ──"
docker run --rm --entrypoint sh "$IMAGE" -c \
'find /usr/lib/node_modules /opt -type d \( -regex ".*/@esbuild/[^/]+" -o -regex ".*/@mariozechner/clipboard-[^/]+" \) -printf "%f\n" 2>/dev/null | sort | uniq -c | sort -rn | head' \
2>/dev/null | sed 's/^/ /' || true
fi
# ── Summary ───────────────────────────────────────────────────────────
echo ""
echo "=== Results: ${PASS} passed, ${FAIL} failed ==="
[ "$FAIL" -eq 0 ]