Joakim Persson
42bd29d654
Lint / hadolint (push) Successful in 8s
Lint / skill-floor (push) Successful in 9s
Lint / doc-drift (push) Successful in 10s
Lint / actionlint (push) Successful in 19s
Publish Docker Image / lint-gate (push) Successful in 15s
Publish Docker Image / resolve-versions (push) Successful in 10s
Publish Docker Image / base-decide (push) Successful in 14s
Publish Docker Image / build-base (push) Has been skipped
Publish Docker Image / smoke (push) Successful in 5m4s
Publish Docker Image / smoke-studio (push) Successful in 13m17s
Publish Docker Image / build-variant (push) Successful in 33m49s
Publish Docker Image / update-description (push) Successful in 6s
Publish Docker Image / promote-base-latest (push) Successful in 14s
Publish Docker Image / build-variant-studio (push) Successful in 30m58s
v1.9.0 was tagged but never published: smoke failed 90-passed/3-failed and
build-variant needs smoke, so nothing reached the registry. All three are fixed.
1. Size, 431 MB over threshold. Node 24 brings npm 11, which installs EVERY
@esbuild/<platform> optional binary instead of the matching one: 26 dirs,
284 MB per pi-coding-agent copy. Measured on pi-fork: npm 10.9.8 -> 165 MB
(exactly what v1.8.14 shipped), npm 11.19.0 -> 449 MB. npm 11 ignores the
os/cpu constraints AND --os/--cpu AND an npmrc carrying them, all measured,
so Dockerfile.variant prunes explicitly, keeping linux-$(node -p
process.arch) so one line is right on both arches. Pruned in the SAME layer
as each install, or the bytes survive in the earlier layer. Three sites:
global pi, pi-fork, pi-studio. Verified esbuild still transforms TS after.
2. om's node_modules assertion tested an npm artefact. om has zero runtime deps;
its node_modules held ONE file (.package-lock.json) and 20 empty scope dirs.
npm 11 stopped creating it. Now asserts the entry point pi actually loads,
read from package.json -> pi.extensions.
3. The skill-source annotation added in v1.9.0 ("baked (package copy)") broke
the assertion matching "baked$". Pattern now allows an optional suffix; which
copy shipped stays authoritatively asserted against the manifest + tree hash.
Also: a failed size check now prints the largest layers, largest directories and
an @esbuild sentinel, so this class attributes itself next time instead of
costing a CI dig plus a local npm bisect.
Threshold stays 3800 MB: it caught a real regression and raising it would have
thrown the signal away. No Dockerfile.base/rootfs change, so base-0fb1256c7f99
is reused and build-base is skipped.
933 lines
56 KiB
Bash
Executable File
933 lines
56 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
# smoke-test.sh — sanity checks for the pi-devbox image
|
||
#
|
||
# Usage: ./scripts/smoke-test.sh <image>
|
||
#
|
||
# Verifies:
|
||
# - pi binary present and (if EXPECTED_PI_VERSION set) matches CI's resolved version
|
||
# - node MAJOR matches Dockerfile.base's ARG NODE_VERSION (if EXPECTED_NODE_MAJOR set)
|
||
# - mempalace core matches the audited pin (if EXPECTED_MEMPALACE_VERSION set)
|
||
# - new v1.0.0 base additions (pandoc, graphviz, imagemagick, yq, tealdeer)
|
||
# - typst PDF engine for pandoc (v1.4.0) — `pandoc --pdf-engine=typst`
|
||
# - non-modal editors nano + micro (alongside nvim)
|
||
# - terminfo for modern emulators: xterm-kitty, xterm-ghostty, wezterm,
|
||
# alacritty, foot (kitty-terminfo + ncurses-term + compiled ghostty alias)
|
||
# - tmux 0-indexing baked in /etc/tmux.conf (required for pi-studio variants)
|
||
# - pi-toolkit cloned at /opt/pi-toolkit
|
||
# - pi-extensions cloned at /opt/pi-extensions
|
||
# - pi-atelier vendored at /opt/pi-atelier, registered from /opt (not npm:),
|
||
# and >= the version floor pi's TUI requires (see the floor test)
|
||
# - pi-fork + pi-observational-memory cloned with node_modules baked
|
||
# - entrypoint deploys pi-toolkit keybindings symlink
|
||
# - entrypoint deploys ≥4 extensions
|
||
# - mempalace bridge symlink present
|
||
# - settings.json bootstrapped
|
||
# - pi-fork + pi-observational-memory registered in settings.json packages[]
|
||
# via `pi install`
|
||
# - pi-devbox-version command present + wraps the build manifest correctly
|
||
# (human, --json, --quiet)
|
||
# - (studio variant only, auto-detected) pi-studio cloned + prebuilt
|
||
# client bundle present + registered via `pi install`
|
||
# - image size within threshold
|
||
|
||
set -euo pipefail
|
||
|
||
IMAGE="${1:?usage: $0 <image>}"
|
||
PASS=0; FAIL=0
|
||
# pi-devbox v1.0.0 (decoupled from opencode-devbox) added pandoc, graphviz,
|
||
# imagemagick, yq, tealdeer, a baked /etc/tmux.conf, and the non-modal
|
||
# editors nano + micro (~15 MB combined). v1.6.0 baked in agent-browser +
|
||
# Playwright Chromium (~291 MB net after dropping the unused headless-shell
|
||
# build), which lifted the baseline. CI amd64 actuals observed on run 512
|
||
# (v1.6.1): 3411 MB non-studio, 3574 MB studio. Threshold below carries
|
||
# ~225 MB margin above the studio number to absorb minor arch/build-cache
|
||
# differences and small future growth without false reds, while still
|
||
# catching an unexpected +GB regression.
|
||
SIZE_THRESHOLD_MB=3800
|
||
|
||
# On failure, surface the last few lines the command produced. This used to
|
||
# discard output entirely (`>/dev/null 2>&1`), which made a red ❌ carry zero
|
||
# diagnostic weight: explaining the single v1.8.0 stage-default failure took a
|
||
# full CI-log dig plus a registry-config inspection, when the container had
|
||
# already printed the answer and thrown it away. Assertions that want a
|
||
# diagnostic just echo it to stderr — it stays hidden while they pass.
|
||
run() {
|
||
local label="$1"; local cmd="$2"
|
||
local out
|
||
if out=$(docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" 2>&1); then
|
||
printf " ✅ %s\n" "$label"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ %s\n" "$label"; FAIL=$((FAIL+1))
|
||
# `if`, not `&&` — a trailing false under `set -e` would abort the script.
|
||
if [ -n "$out" ]; then
|
||
printf " └─ %s\n" "$(printf '%s' "$out" | tail -3 | tr '\n' ' ' | cut -c1-300)"
|
||
fi
|
||
fi
|
||
}
|
||
|
||
# Stricter version of `run` that asserts an expected substring in stdout.
|
||
# Catches the "image bytes silently identical to previous release" class of
|
||
# regression — Docker layer cache hit on `npm install -g <pkg>` because the
|
||
# bare command string is identical across builds, even when `latest` would
|
||
# resolve differently. Discovered 2026-05-23 — every pi-devbox release
|
||
# v0.74.0..v0.75.5 had been shipping the same image bytes.
|
||
run_expect() {
|
||
local label="$1"; local cmd="$2"; local expect="$3"
|
||
local out
|
||
out=$(docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" 2>&1) || true
|
||
if echo "$out" | grep -Fq "$expect"; then
|
||
printf " ✅ %s (got %s)\n" "$label" "$expect"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ %s — expected substring %q, got: %s\n" "$label" "$expect" "$out"; FAIL=$((FAIL+1))
|
||
fi
|
||
}
|
||
|
||
echo "=== pi-devbox smoke test: $IMAGE ==="
|
||
echo ""
|
||
|
||
# ── Binaries ─────────────────────────────────────────────────────────
|
||
echo "── Binaries ──"
|
||
if [ -n "${EXPECTED_PI_VERSION:-}" ]; then
|
||
run_expect "pi version matches build arg" "pi --version" "$EXPECTED_PI_VERSION"
|
||
else
|
||
run "pi" "pi --version"
|
||
fi
|
||
# Until 2026-09-07 this was a bare `run "node" "node --version"`, which asserts
|
||
# only that the binary exists and exits 0 — the printed version was never
|
||
# compared to anything. A node major bump would therefore have passed this suite
|
||
# SILENTLY, while a reader skimming it would reasonably assume node regressions
|
||
# were covered. EXPECTED_NODE_MAJOR closes that: CI derives it from
|
||
# Dockerfile.base's ARG NODE_VERSION (the single source of truth), so this also
|
||
# catches a stale cached layer whose node does not match the declared ARG.
|
||
if [ -n "${EXPECTED_NODE_MAJOR:-}" ]; then
|
||
run_expect "node major matches Dockerfile ARG" "node --version" "v${EXPECTED_NODE_MAJOR}."
|
||
else
|
||
run "node" "node --version"
|
||
fi
|
||
run "git" "git --version"
|
||
# NOTE: the shellcheck binary is a GATE DEPENDENCY, not a convenience.
|
||
# scripts/lint-shell.sh is the release gate (the lint-gate job resolve-versions
|
||
# depends on) and it exits 2 when the binary is missing, by design — "a gate that
|
||
# cannot run must not pass". Measured on v1.8.14: it was absent from the image, so
|
||
# that gate could not be run by a developer in ANY container, only in CI.
|
||
# Asserted here so its absence fails a build instead of being discovered by a hook
|
||
# that then refuses every push (hooks/pre-push).
|
||
#
|
||
# This comment must not BEGIN with the tool's name: a line starting with
|
||
# `# shellcheck` is parsed as a DIRECTIVE, not a comment (SC1073/SC1072). The
|
||
# gate added in this same change caught that here, before the push.
|
||
run "shellcheck (lint gate dependency)" "shellcheck --version | grep -qE '^version: [0-9]'"
|
||
run "aws" "aws --version"
|
||
run "uv" "uv --version"
|
||
run "nvim" "nvim --version"
|
||
run "nano" "nano --version"
|
||
run "micro" "micro --version"
|
||
run "kitty-terminfo" "infocmp -x xterm-kitty >/dev/null 2>&1"
|
||
run "terminfo: modern emulators (ncurses-term)" 'for t in wezterm alacritty foot ghostty st-256color; do infocmp -x "$t" >/dev/null 2>&1 || exit 1; done'
|
||
run "terminfo: xterm-ghostty alias (tic)" "infocmp -x xterm-ghostty >/dev/null 2>&1"
|
||
run "nvim true-colour default (sysinit.vim)" "nvim --headless -c 'lua os.exit(vim.o.termguicolors and 0 or 1)'"
|
||
run "mempalace-mcp" "mempalace-mcp --help"
|
||
run "mempalace-pi-session on PATH" "mempalace-pi-session --help"
|
||
# The staging dir must sit next to the palace, not in a disposable cache: the
|
||
# palace keys per-source dedup on the STAGED path, so a stage that can be wiped
|
||
# while the palace survives lets `mempalace sync` prune every drawer mined from
|
||
# it. Assert the resolved default, not an env var — the guarantee is "stage
|
||
# shares the palace's lifetime", which an ENV pin would quietly break.
|
||
# NOTE: --sessions-dir gets an EMPTY temp dir, never /tmp. The stage banner is
|
||
# printed before any export, so nothing needs to be found — and pointing a
|
||
# default-staged run at a populated dir would export whatever transcripts it
|
||
# finds into the real stage, which is how a synthetic test session ends up
|
||
# staged for mining as if it were a real conversation.
|
||
#
|
||
# Asserted $HOME-RELATIVE, not against a literal /home/developer. `run` invokes
|
||
# `docker run --entrypoint=""`, and neither Dockerfile sets USER or ENV HOME
|
||
# (HOME is set by entrypoint-user.sh, which --entrypoint="" deliberately skips),
|
||
# so these assertions execute as root with HOME=/root. The original literal
|
||
# /home/developer form could therefore never match and failed the v1.8.0
|
||
# release — a test bug, not a product one: the stage resolution was correct all
|
||
# along, it just follows $HOME. The invariant under test ("the stage sits beside
|
||
# the palace, sharing its lifetime") is user-independent, so pinning the user
|
||
# was never part of it. A cache-dir default still fails the pattern below, which
|
||
# is the regression this guards.
|
||
#
|
||
# It went unnoticed for three days because this workflow only triggers on
|
||
# `push: tags: v*` — the assertion was added on a main push, so v1.8.0 was its
|
||
# first execution ever. Use the `smoke_only` workflow_dispatch input to run
|
||
# smoke against HEAD without cutting a tag.
|
||
run "pi stage defaults next to the palace (not a cache dir)" '
|
||
out=$(mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||
stage=$(echo "$out" | grep -oE "stage=[^ ]+" | head -1)
|
||
echo "resolved ${stage:-<no stage= line>} with HOME=$HOME" >&2
|
||
case "$stage" in
|
||
"stage=$HOME/.mempalace/pi-stage/"*) exit 0 ;;
|
||
*) exit 1 ;;
|
||
esac
|
||
'
|
||
# Companion to the above: the deployment-specific case the literal assertion was
|
||
# reaching for, done properly by supplying the HOME the container actually runs
|
||
# with instead of assuming it.
|
||
run "pi stage is palace-adjacent for the developer user" '
|
||
out=$(HOME=/home/developer mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||
echo "$out" | grep -oE "stage=[^ ]+" | head -1 >&2
|
||
echo "$out" | grep -q "stage=/home/developer/.mempalace/pi-stage/"
|
||
'
|
||
run "pi stage follows MEMPALACE_PALACE_PATH" '
|
||
out=$(MEMPALACE_PALACE_PATH=/tmp/alt/.mempalace/palace \
|
||
mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||
echo "$out" | grep -q "stage=/tmp/alt/.mempalace/pi-stage/"
|
||
'
|
||
# The feeder's --agent default is WHO a drawer is attributed to. mempalace core
|
||
# records neither the machine nor the harness on a write, and one shared bearer
|
||
# token means the server cannot tell clients apart, so toolkit c64ffa1 changed
|
||
# this default from $USER to pi@$MEMPALACE_PI_DEVICE — the one string that makes
|
||
# a write attributable to both. Nothing ever PRINTED the resolved value (the
|
||
# banner shows mode= and stage= only), so an image built from a pre-c64ffa1
|
||
# toolkit ref would ship unattributed writes with every check still green.
|
||
#
|
||
# `--help` assigns AGENT (script top) before it parses args, then exits 0 with
|
||
# no side effects — so `bash -x` observes the REAL resolution, env interpolation
|
||
# and fallback included, rather than grepping the source for a literal line that
|
||
# any reformat would break. Two-sided on purpose: device set => pi@<device>;
|
||
# device UNSET => must not be pi@anything. The second half is what fails against
|
||
# the old unconditional $USER default, which ignored the device entirely.
|
||
#
|
||
# Probes the PATH entry (a symlink into the /opt clone) rather than that clone
|
||
# path directly: this is the invocation the systemd/launchd timers and
|
||
# entrypoint-user.sh actually use, so it is the default that reaches the palace.
|
||
run "feeder resolves --agent to pi@<device> (drawer attribution)" '
|
||
f=$(command -v mempalace-pi-session) || { echo "feeder not on PATH" >&2; exit 1; }
|
||
with=$(MEMPALACE_PI_DEVICE=smoke-device bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
|
||
without=$(env -u MEMPALACE_PI_DEVICE bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
|
||
echo "resolved with-device=[$with] without-device=[$without]" >&2
|
||
[ "$with" = "pi@smoke-device" ] || exit 1
|
||
case "$without" in pi@*) exit 1 ;; esac
|
||
echo ok
|
||
'
|
||
# Regression guard for the pi transcript exporter. If pi ever changes its
|
||
# session JSONL shape, the exporter stops recognising sessions and the palace
|
||
# silently gets nothing (or, worse, raw JSON chunked as prose). Feed it a
|
||
# synthetic session and assert it is actually exported. Uses --dry-run so no
|
||
# palace is touched, and a temp stage so nothing real is written.
|
||
run "pi transcript exporter recognises a pi session" '
|
||
set -e
|
||
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
|
||
{
|
||
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question one\"}}"
|
||
a=$(printf "a%.0s" $(seq 1 1200))
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"$a\"}]}}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question two\"}}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"short reply\"}]}}"
|
||
} > "$s/2026-01-01T00-00-00-000Z_smoke.jsonl"
|
||
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
|
||
echo "$out" | grep -q "Exported 1 session"
|
||
'
|
||
# The same guard from the other side: a session with no real assistant output
|
||
# (an abandoned prompt, whose bulk is injected skill text) must NOT be filed.
|
||
run "pi transcript exporter rejects an abandoned session" '
|
||
set -e
|
||
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
|
||
{
|
||
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke2\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
|
||
u=$(printf "u%.0s" $(seq 1 13000))
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"$u\"}}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"Ready. What would you like to work on?\"}]}}"
|
||
} > "$s/2026-01-01T00-00-00-000Z_smoke2.jsonl"
|
||
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
|
||
echo "$out" | grep -q "no sessions qualified"
|
||
'
|
||
# The remote-palace-without-inbox skip must ANNOUNCE itself, not vanish. This
|
||
# branch of entrypoint-user.sh runs at container start (not reachable from a
|
||
# `docker run` one-shot), so assert against the entrypoint that actually shipped
|
||
# in the image. Guards a silent regression back to the bare `:` no-op, which
|
||
# left a container contributing nothing to the palace with no artifact saying
|
||
# why — the log it would normally leave is written by the other branch.
|
||
run_expect "remote-palace-without-inbox skip is announced, not silent" \
|
||
"grep -o 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | head -1" \
|
||
"MemPalace catch-up skipped"
|
||
run "...and the skip notice names the variable that fixes it" \
|
||
"grep -A6 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | grep -q 'MEMPALACE_PI_SSH_TARGET'"
|
||
# A remote mine that FAILS must not report success. MCP answers a hard tool
|
||
# failure with HTTP 200 and the tool's own JSON escaped inside
|
||
# result.content[].text, so the feeder's old `'\"error\"' in body` check could
|
||
# never see it: on 2026-08-15 a mine that died with "source directory not found:
|
||
# '/data/feed/...'" logged "Done. Wing updated." and exited 0, and this
|
||
# container's transcripts were filed nowhere for a whole session. The feeder
|
||
# carries fixtures for that exact body; run them against the baked toolkit so a
|
||
# stale/reverted toolkit ref can't reintroduce a silent feed.
|
||
run "baked feeder detects a failed remote mine (no silent false success)" \
|
||
"mempalace-pi-session --self-test"
|
||
# v1.0.0 base additions — verify presence and basic functionality.
|
||
run "pandoc" "pandoc --version"
|
||
run "typst" "typst --version"
|
||
run "pandoc+typst PDF engine" "printf '# hi\n' | pandoc --pdf-engine=typst -o /tmp/_smoke.pdf - && test -s /tmp/_smoke.pdf; rm -f /tmp/_smoke.pdf"
|
||
run "graphviz (dot)" "dot -V"
|
||
run "imagemagick" "magick --version"
|
||
run "yq (mikefarah v4)" "yq --version | grep -qE 'mikefarah.*version v4'"
|
||
run "tldr (tealdeer)" "tldr --version"
|
||
run "socat" "socat -V"
|
||
run "studio-expose helper" "test -x /usr/local/bin/studio-expose"
|
||
run "image-baked pi-devbox-environment skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md"
|
||
run "image-baked credential-incident-response skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md"
|
||
run "global-AGENTS append snippet present" \
|
||
"test -f /usr/local/share/pi-devbox/pi-global-AGENTS.append.md"
|
||
run "pi-devbox block merged into pi-global-AGENTS.md" \
|
||
"grep -q 'pi-devbox:managed-block' /opt/pi-toolkit/pi-global-AGENTS.md"
|
||
run "mempalace session-start pointer merged into global AGENTS.md" \
|
||
"grep -q 'load the mempalace skill' /opt/pi-toolkit/pi-global-AGENTS.md"
|
||
# Vendored fallback skills (so a no-skillset container still resolves the
|
||
# AGENTS.md 'read the pi-extensions skill' pointer).
|
||
run "image-baked pi-extensions fallback skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md"
|
||
run "pi-extensions skill ships its helper" \
|
||
"test -f /usr/local/share/pi-devbox/skills/pi-extensions/evaluate-extension-usage.py"
|
||
run "image-baked mempalace fallback skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/mempalace/SKILL.md"
|
||
# Layered freshness: when the pinned pi-extensions clone carries the skill, the
|
||
# baked copy must be the fresh package copy (Option 1), not the stale snapshot.
|
||
run "pi-extensions skill refreshed from package when present" \
|
||
"if [ -f /opt/pi-extensions/skill/SKILL.md ]; then cmp -s /opt/pi-extensions/skill/SKILL.md /usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md; else true; fi"
|
||
# Runtime ownership handover (v1.8.5): the baked links are a FALLBACK, and
|
||
# skillset-OWNED skills must be repointed at the live clone when one is mounted.
|
||
# The list is data, so assert its content, not just its presence: mempalace in,
|
||
# pi-extensions deliberately out (its skillset copy is a lagging duplicate).
|
||
run "devbox-skill-reconcile helper present + executable" \
|
||
"test -x /usr/local/bin/devbox-skill-reconcile"
|
||
run "skillset-owned list ships and names mempalace" \
|
||
"grep -qx 'mempalace' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
|
||
run "skillset-owned list excludes pi-extensions (ownership)" \
|
||
"! grep -qx 'pi-extensions' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
|
||
|
||
# ── tmux 0-indexing (required for pi-studio variants) ─────────────────
|
||
echo ""
|
||
echo "── tmux config ──"
|
||
run_expect "/etc/tmux.conf has base-index 0" \
|
||
"cat /etc/tmux.conf" "set -g base-index 0"
|
||
run_expect "/etc/tmux.conf has pane-base-index 0" \
|
||
"cat /etc/tmux.conf" "set -g pane-base-index 0"
|
||
|
||
# ── Repo clones ───────────────────────────────────────────────────────
|
||
echo ""
|
||
echo "── Repo clones ──"
|
||
run "pi-toolkit clone" "test -d /opt/pi-toolkit && git -C /opt/pi-toolkit rev-parse --short HEAD"
|
||
run "pi-extensions clone" "test -d /opt/pi-extensions && git -C /opt/pi-extensions rev-parse --short HEAD"
|
||
run "pi-fork clone + node_modules" \
|
||
"test -f /opt/pi-fork/package.json && test -d /opt/pi-fork/node_modules"
|
||
# om is checked differently from pi-fork ON PURPOSE. It declares ZERO runtime
|
||
# dependencies: 8 devDependencies (omitted by --omit=dev) and 4 peerDependencies,
|
||
# which pi itself provides. npm 10 still materialised a node_modules for it, but
|
||
# that directory held exactly ONE file (.package-lock.json, 4 KB) and no nested
|
||
# package.json at all — 20 empty scope dirs. npm 11 stopped creating it, so the
|
||
# old `test -d node_modules` assertion went red on v1.9.0 while nothing about om
|
||
# had changed or broken. It was asserting an npm artefact, not a property of the
|
||
# shipped software. What actually has to hold is that the entry point pi loads
|
||
# exists, so assert THAT, straight out of the manifest pi reads
|
||
# (package.json -> pi.extensions), rather than a hardcoded path that could drift.
|
||
run "pi-observational-memory clone + declared pi entry point" \
|
||
"test -f /opt/pi-observational-memory/package.json && \
|
||
node -e 'const p=require(\"/opt/pi-observational-memory/package.json\"),f=require(\"fs\"),h=require(\"path\"); \
|
||
const l=(p.pi&&p.pi.extensions)||[]; \
|
||
if(!l.length){console.error(\"package.json declares no pi.extensions\");process.exit(1)} \
|
||
for(const e of l){const t=h.resolve(\"/opt/pi-observational-memory\",e); \
|
||
if(!f.existsSync(t)){console.error(\"declared entry missing: \"+t);process.exit(1)}} \
|
||
console.log(\"entries ok: \"+l.join(\",\"))'"
|
||
# ...and that the clone carries the AUTH FIX, not merely that it exists. om's
|
||
# pre-flight hasUsableAuth() check silently disabled `recall` for ~8 weeks once
|
||
# pi moved to request-time SigV4 signing and stopped exposing a static Bedrock
|
||
# key; upstream fixed it in ce9fc98, adopted in v1.8.4. PI_OBSMEM_REF tracks
|
||
# master, so an upstream revert or force-push would ship a dead `recall` with
|
||
# the clone assertion above still green — the exact gap flagged as open in the
|
||
# v1.8.5 changelog.
|
||
#
|
||
# Pin the markers to src/runtime.ts, the fix SITE, rather than grepping the
|
||
# repo: two of these three strings also appear under tests/, so a repo-wide
|
||
# grep stays green with runtime.ts itself reverted. That is a false green of the
|
||
# same family as the old skill-snapshot canary.
|
||
run "pi-observational-memory carries the ce9fc98 auth fix (recall stays alive)" '
|
||
f=/opt/pi-observational-memory/src/runtime.ts
|
||
test -f "$f" || { echo "fix site missing: $f" >&2; exit 1; }
|
||
for m in availability_recheck providerCredentialConfigured hasConfiguredAuth; do
|
||
grep -q "$m" "$f" || { echo "marker absent from runtime.ts: $m" >&2; exit 1; }
|
||
done
|
||
echo ok
|
||
'
|
||
# pi-atelier: deliberately NO node_modules assertion, unlike its siblings —
|
||
# it declares zero runtime dependencies (only peerDeps, satisfied by the baked
|
||
# pi) and has no build step, so Dockerfile.variant skips `npm install` for it.
|
||
# Assert what pi actually loads instead: the entry point named by its
|
||
# package.json `pi.extensions` key.
|
||
run "pi-atelier clone + entry point" \
|
||
"test -f /opt/pi-atelier/package.json && test -f /opt/pi-atelier/extensions/index.ts"
|
||
|
||
# ── pi <-> pi-atelier compatibility floor (executable, not a comment) ──
|
||
# pi-atelier < 0.7.1 wraps pi's PRIVATE TUI renderer in a way that recurses
|
||
# under pi >= 0.84: pi hangs at startup burning CPU, with no error. Upstream
|
||
# fixed it in 0.7.1/0.7.2, but atelier's peerDependencies still say
|
||
# `>=0.80.7`, so neither npm nor pi can warn about the real floor. Both
|
||
# versions are pinned in Dockerfile.variant; this makes a bad PAIRING fail the
|
||
# build instead of publishing an image whose TUI never starts.
|
||
run_expect "pi-atelier >= 0.7.1 floor for pi >= 0.84 (startup-hang guard)" \
|
||
'ge() { [ "$(printf "%s\n%s\n" "$1" "$2" | sort -V | head -n1)" = "$2" ]; }; AV=$(jq -r ".version // empty" /opt/pi-atelier/package.json 2>/dev/null); PV=$(pi --version 2>/dev/null | grep -oE "[0-9]+\.[0-9]+\.[0-9]+" | head -n1); if [ -z "$AV" ] || [ -z "$PV" ]; then echo "unreadable versions (atelier=$AV pi=$PV)"; elif ge "$PV" 0.84.0 && ! ge "$AV" 0.7.1; then echo "VIOLATION: pi $PV with pi-atelier $AV"; else echo "compatible: pi $PV + pi-atelier $AV"; fi' \
|
||
"compatible:"
|
||
|
||
# pi-studio is present only in the :latest-studio variant. Auto-detect by
|
||
# probing /opt/pi-studio so this one script covers both variants.
|
||
if docker run --rm --entrypoint="" "$IMAGE" sh -c 'test -d /opt/pi-studio' >/dev/null 2>&1; then
|
||
STUDIO_VARIANT=1
|
||
echo " ℹ️ pi-studio detected — running studio assertions"
|
||
run "pi-studio clone + node_modules" \
|
||
"test -f /opt/pi-studio/package.json && test -d /opt/pi-studio/node_modules"
|
||
run "pi-studio prebuilt client bundle" \
|
||
"test -f /opt/pi-studio/client/studio-client.js"
|
||
else
|
||
STUDIO_VARIANT=0
|
||
echo " ℹ️ pi-studio not present (non-studio variant) — skipping studio clone checks"
|
||
fi
|
||
|
||
# ── Build provenance (manifest + OCI labels) ─────────────────────────
|
||
echo ""
|
||
echo "── Build provenance ──"
|
||
run "/etc/pi-devbox/build-manifest.json present" \
|
||
"test -f /etc/pi-devbox/build-manifest.json"
|
||
# These next checks replace three that grepped the manifest for the FIELD NAME
|
||
# and never looked at the value:
|
||
#
|
||
# run_expect "manifest records pi_version" "cat …manifest.json" '"pi_version"'
|
||
#
|
||
# which passes on {"pi_version": ""} and on {"pi_version": null}. The tell was
|
||
# visible in its own passing output — `✅ manifest records pi_version (got
|
||
# "pi_version")` echoes the key back as the thing it claims to have found.
|
||
# Two failure modes were therefore invisible: a key that survives with an empty
|
||
# or garbage value, and a key that vanishes from the manifest while every
|
||
# remaining value still looks fine.
|
||
#
|
||
# Those two need SEPARATE assertions, and the reason is a trap worth keeping in
|
||
# writing: an "every component value is a valid SHA" loop passes VACUOUSLY on
|
||
# components:{} — jq's all() over an empty list is true — so the value check
|
||
# alone would go green on a manifest that lost every component. Mutation-tested
|
||
# 2026-08-25 across nine fabricated manifests (empty map, deleted key, "",
|
||
# null, "unknown", 12-hex truncation, 40 non-hex chars, legit null pi-studio).
|
||
run "manifest declares every required component key" '
|
||
req="pi-toolkit pi-extensions pi-fork pi-observational-memory pi-atelier mempalace-toolkit pi-studio"
|
||
for k in $req; do
|
||
jq -e --arg k "$k" "(.components|has(\$k))" /etc/pi-devbox/build-manifest.json >/dev/null \
|
||
|| { echo "manifest lost component key: $k" >&2; exit 1; }
|
||
done
|
||
'
|
||
# Subsumes the old `! grep -q \"unknown\"` check ("unknown" is not 40-hex), and
|
||
# also catches "", null and truncated SHAs, which that grep let through. null is
|
||
# legitimate for pi-studio alone: the non-studio variant has no such clone.
|
||
run "manifest component values are resolved 40-hex commits" '
|
||
jq -e "
|
||
.components
|
||
| to_entries
|
||
| all(if .key == \"pi-studio\" and .value == null then true
|
||
else (.value|type) == \"string\" and (.value|test(\"^[0-9a-f]{40}\$\")) end)
|
||
" /etc/pi-devbox/build-manifest.json >/dev/null
|
||
'
|
||
# pi_version against ground truth, same shape as the mempalace check below.
|
||
# Chains with the "pi version matches build arg" assertion earlier in this file:
|
||
# together they tie build arg -> installed binary -> recorded manifest, so a
|
||
# manifest written from a stale variable cannot pass by agreeing with itself.
|
||
run "manifest pi_version matches the installed pi" '
|
||
m=$(jq -r ".pi_version // empty" /etc/pi-devbox/build-manifest.json)
|
||
b=$(pi --version 2>/dev/null | head -n1 | tr -d "\r")
|
||
echo "manifest=[$m] installed=[$b]" >&2
|
||
[ -n "$m" ] && [ "$m" = "$b" ]
|
||
'
|
||
# Top-level provenance fields: assert the SHAPE of each value, and only when the
|
||
# field is populated. source_revision and build_date legitimately default to
|
||
# empty (Dockerfile.variant ARGs) on a plain local `docker build`, so demanding
|
||
# them would fail honest local smoke runs; a populated-but-malformed value is
|
||
# the actual defect. release_tag defaults to "dev", so empty means a broken write.
|
||
run "manifest top-level fields are well-formed, not merely present" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
t=$(jq -r ".release_tag // empty" $j)
|
||
r=$(jq -r ".source_revision // empty" $j)
|
||
d=$(jq -r ".build_date // empty" $j)
|
||
echo "release_tag=[$t] source_revision=[$r] build_date=[$d]" >&2
|
||
[ -n "$t" ] || { echo "release_tag empty (ARG default is dev)" >&2; exit 1; }
|
||
if [ -n "$r" ]; then
|
||
printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" || { echo "source_revision not a 40-hex commit" >&2; exit 1; }
|
||
fi
|
||
if [ -n "$d" ]; then
|
||
printf "%s" "$d" | grep -qE "^[0-9]{4}-[0-9]{2}-[0-9]{2}T" || { echo "build_date not ISO-8601" >&2; exit 1; }
|
||
fi
|
||
'
|
||
# mempalace CORE was absent from the manifest through v1.8.5: the toolkit SHA
|
||
# was recorded but the palace version behind the MCP tools was not, so a palace
|
||
# bug could not be correlated to an image version. Assert the field exists AND
|
||
# equals the installed binary — recording it from ARG MEMPALACE_VERSION instead
|
||
# would look identical here yet drift silently the first time an install
|
||
# resolved to something other than the pin, which is the whole reason this file
|
||
# is built from ground truth. `// empty` matters: jq -r prints the 4-char
|
||
# string "null" for a JSON null, which would satisfy a naive -n test.
|
||
run "manifest mempalace_version matches the installed core" '
|
||
m=$(jq -r ".mempalace_version // empty" /etc/pi-devbox/build-manifest.json)
|
||
b=$(mempalace --version 2>/dev/null | head -n1 | tr -d "\r"); b=${b##* }
|
||
echo "manifest=[$m] installed=[$b]" >&2
|
||
[ -n "$m" ] && [ "$m" = "$b" ]
|
||
'
|
||
# ... and, when CI supplies it, that the installed core is the version CI
|
||
# actually AUDITED (published + not yanked on PyPI, in resolve-versions). This
|
||
# does NOT duplicate the check above, which compares two properties of one
|
||
# image and so cannot notice that BOTH are the wrong version. The live failure
|
||
# mode it covers: the variant builds `FROM` a base tag chosen by base-decide's
|
||
# content hash, so a bug in that hashing (the reason scripts/check-base-hash.sh
|
||
# exists) could reuse a cached base built from an OLDER MEMPALACE_VERSION pin —
|
||
# internally consistent, silently stale, invisible to every other assertion.
|
||
if [ -n "${EXPECTED_MEMPALACE_VERSION:-}" ]; then
|
||
run "installed mempalace matches CI's audited pin (${EXPECTED_MEMPALACE_VERSION})" "
|
||
b=\$(mempalace --version 2>/dev/null | head -n1 | tr -d '\r'); b=\${b##* }
|
||
echo \"installed=[\$b] audited_pin=[${EXPECTED_MEMPALACE_VERSION}]\" >&2
|
||
[ \"\$b\" = \"${EXPECTED_MEMPALACE_VERSION}\" ]
|
||
"
|
||
fi
|
||
# Every component must be a resolved commit (or null for pi-studio in the
|
||
# non-studio variant) — now enforced by the 40-hex value check above, which
|
||
# strictly subsumes the old whole-file grep for '"unknown"'. Only rev() ever
|
||
# emits "unknown" and rev() feeds components only, so nothing is lost.
|
||
# pi-devbox-version wraps the manifest into a human-first command; verify the
|
||
# binary is present, executable, and that all three output modes work.
|
||
run "pi-devbox-version binary present + executable" \
|
||
"test -x /usr/local/bin/pi-devbox-version"
|
||
run_expect "pi-devbox-version human output shows release tag" \
|
||
"pi-devbox-version" "pi-devbox "
|
||
# --json is a verbatim `cat` of the manifest, so "round-trips" is assertable
|
||
# literally. The old form grepped the output for the string "release_tag" — the
|
||
# key name again — which would pass on a truncated or re-serialised dump.
|
||
run "pi-devbox-version --json round-trips the manifest byte-for-byte" '
|
||
a=$(cat /etc/pi-devbox/build-manifest.json)
|
||
b=$(pi-devbox-version --json)
|
||
[ "$a" = "$b" ] || { echo "--json output differs from the manifest on disk" >&2; exit 1; }
|
||
'
|
||
run_expect "pi-devbox-version --quiet is a compact one-liner" \
|
||
"pi-devbox-version --quiet | wc -l" "1"
|
||
# ── Vendored skill snapshot provenance ─────────────────────────────────
|
||
# The vendored mempalace skill is the one baked artefact with no /opt clone
|
||
# behind it (private upstream — see VENDORED.md), so until now the manifest
|
||
# could not say which skillset commit it came from. Two fields now travel with
|
||
# it: the CLAIMED ref (ARG default in Dockerfile.variant) and the MEASURED
|
||
# sha256 of the shipped bytes. Assert both are well-formed, and — separately —
|
||
# that the measurement still describes the file in the image.
|
||
#
|
||
# Kept as two assertions for the same reason the component checks are: one
|
||
# proves the fields are not empty/garbage, the other proves they are not merely
|
||
# self-consistent. A single combined check could pass on a manifest whose hash
|
||
# was computed from a file that was later overwritten (the pi-extensions skill
|
||
# copy at Dockerfile.variant:165 does exactly that kind of overwrite, one stage
|
||
# earlier), which is the failure this second one exists to catch.
|
||
run "manifest records the vendored skill snapshot provenance" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
r=$(jq -r ".skillset_snapshot_ref // empty" $j)
|
||
s=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
|
||
echo "ref=[$r] tree_sha256=[$s]" >&2
|
||
printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" \
|
||
|| { echo "skillset_snapshot_ref is not a 40-hex commit" >&2; exit 1; }
|
||
printf "%s" "$s" | grep -qxE "[0-9a-f]{64}" \
|
||
|| { echo "skillset_snapshot_tree_sha256 is not a 64-hex digest" >&2; exit 1; }
|
||
'
|
||
# Recomputes over the whole DIRECTORY with the same tree_sha256() pipeline
|
||
# Dockerfile.variant used to measure it, not a plain `sha256sum SKILL.md` —
|
||
# a file-only compare here would pass even if the manifest recorded a
|
||
# fingerprint over a directory that has since grown a second file (this is
|
||
# not hypothetical: pi-extensions already ships two files for its skill).
|
||
run "manifest skill fingerprint matches the baked snapshot" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
d=/usr/local/share/pi-devbox/skills/mempalace
|
||
m=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
|
||
a=$( (cd "$d" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum) | sha256sum | cut -d" " -f1)
|
||
echo "manifest=[$m] actual=[$a]" >&2
|
||
[ -n "$m" ] && [ "$m" = "$a" ]
|
||
'
|
||
|
||
# ── Which pi-extensions skill copy shipped ──────────────────────────────
|
||
# Closes the silent-fallback hole. The refresh in Dockerfile.variant is guarded
|
||
# by `[ -f /opt/pi-extensions/skill/SKILL.md ]`, so a build whose clone predates
|
||
# the co-located skill keeps the vendored floor and still succeeds GREEN, with
|
||
# nothing recording that a snapshot shipped instead of the package copy. Measured
|
||
# 2026-09-10: the floor had been stale since 2026-07-30, so that path would have
|
||
# shipped a six-week-old skill in silence. The floor is fresh now and gated by the
|
||
# skill-floor lint job, but "the fallback is currently harmless" is a fact with a
|
||
# shelf life, whereas "the image says which copy it got" keeps working.
|
||
#
|
||
# vendored-floor FAILS here rather than merely warning: these images track main,
|
||
# where the package has co-located skill/ since fa04d20, so a fallback means the
|
||
# clone did not resolve as intended and that is a defect to investigate. A fork
|
||
# deliberately pointing at a mirror without skill/ is the one case that should
|
||
# edit this assertion — which is the honest place for that decision to surface.
|
||
run "manifest names which pi-extensions skill copy shipped" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
s=$(jq -r ".pi_extensions_skill_source // empty" $j)
|
||
h=$(jq -r ".pi_extensions_skill_tree_sha256 // empty" $j)
|
||
echo "source=[$s] tree_sha256=[$h]" >&2
|
||
printf "%s" "$h" | grep -qxE "[0-9a-f]{64}" || {
|
||
echo "pi_extensions_skill_tree_sha256 is not a 64-hex digest" >&2; exit 1; }
|
||
case "$s" in
|
||
package) ;;
|
||
vendored-floor)
|
||
echo "FALLBACK: clone had no skill/ at this ref, so the image ships the committed floor" >&2; exit 1 ;;
|
||
divergent)
|
||
echo "MIXED: served directory is part package and part floor" >&2; exit 1 ;;
|
||
*)
|
||
echo "pi_extensions_skill_source absent or unrecognised" >&2; exit 1 ;;
|
||
esac
|
||
'
|
||
|
||
# Same shape as the mempalace fingerprint check above, and for the same reason: a
|
||
# recorded hash that is never recomputed is a claim, not a measurement.
|
||
run "recorded pi-extensions skill hash matches the served bytes" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
d=/usr/local/share/pi-devbox/skills/pi-extensions
|
||
m=$(jq -r ".pi_extensions_skill_tree_sha256 // empty" $j)
|
||
a=$( (cd "$d" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum) | sha256sum | cut -d" " -f1)
|
||
echo "manifest=[$m] actual=[$a]" >&2
|
||
[ -n "$m" ] && [ "$m" = "$a" ]
|
||
'
|
||
# OCI labels live in the image config, not the container fs — inspect them
|
||
# from the host docker rather than via `docker run`.
|
||
LBL=$(docker inspect --format '{{ index .Config.Labels "se.jordbo.pi-devbox.pi-extensions-ref" }}' "$IMAGE" 2>/dev/null || true)
|
||
if [ -n "$LBL" ] && [ "$LBL" != "<no value>" ]; then
|
||
printf " ✅ OCI label se.jordbo.pi-devbox.pi-extensions-ref=%s\n" "$LBL"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ OCI label se.jordbo.pi-devbox.pi-extensions-ref missing or empty\n"; FAIL=$((FAIL+1))
|
||
fi
|
||
|
||
# ── Runtime deployment (needs entrypoint to run) ──────────────────────
|
||
echo ""
|
||
echo "── Runtime deployment ──"
|
||
# Spin up a long-running container WITHOUT overriding the entrypoint, so
|
||
# the baked entrypoint chain (entrypoint.sh → entrypoint-user.sh) runs and
|
||
# deploys pi-toolkit + pi-extensions to ~/.pi/agent/. Override CMD to
|
||
# tail -f /dev/null so the container stays alive while we docker-exec.
|
||
CID=$(docker run -d --rm "$IMAGE" tail -f /dev/null)
|
||
cleanup() { docker rm -f "$CID" >/dev/null 2>&1 || true; }
|
||
trap cleanup EXIT
|
||
|
||
# Wait for entrypoint-user.sh to finish deploying pi-toolkit + extensions.
|
||
# Gate on BOTH the keybindings symlink (deployed by pi-toolkit) AND the
|
||
# mempalace.ts bridge (deployed last by entrypoint-user.sh) AND ≥4 *.ts
|
||
# extensions present. Parallel build load can otherwise sample the *.ts
|
||
# count mid-deploy and produce a flake. See opencode-devbox c6f9d11
|
||
# (2026-06-08) — same fix transplanted.
|
||
for i in $(seq 1 45); do
|
||
if docker exec "$CID" sh -c '
|
||
test -L /home/developer/.pi/agent/keybindings.json && \
|
||
test -L /home/developer/.pi/agent/extensions/mempalace.ts && \
|
||
test -L /home/developer/.agents/skills/pi-devbox-environment && \
|
||
test -L /home/developer/.agents/skills/pi-extensions && \
|
||
test -L /home/developer/.agents/skills/mempalace && \
|
||
count=$(ls -1 /home/developer/.pi/agent/extensions/*.ts 2>/dev/null | wc -l) && \
|
||
[ "$count" -ge 4 ]
|
||
' >/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
|
||
exec_test() {
|
||
local label="$1"; local cmd="$2"
|
||
if docker exec -u developer "$CID" sh -c "$cmd" >/dev/null 2>&1; then
|
||
printf " ✅ %s\n" "$label"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ %s\n" "$label"; FAIL=$((FAIL+1))
|
||
fi
|
||
}
|
||
|
||
exec_test "keybindings.json (pi-toolkit)" 'test -L $HOME/.pi/agent/keybindings.json && echo ok'
|
||
exec_test "extensions ≥ 4 (pi-extensions)" 'count=$(ls -1 $HOME/.pi/agent/extensions/*.ts 2>/dev/null | wc -l); [ $count -ge 4 ] && echo "$count extensions"'
|
||
exec_test "mempalace.ts bridge" 'test -L $HOME/.pi/agent/extensions/mempalace.ts && echo ok'
|
||
exec_test "settings.json bootstrapped" 'test -f $HOME/.pi/agent/settings.json && echo ok'
|
||
exec_test "pi-devbox-environment skill linked" 'test -L $HOME/.agents/skills/pi-devbox-environment && test -f $HOME/.agents/skills/pi-devbox-environment/SKILL.md && echo ok'
|
||
exec_test "pi-extensions skill linked (fallback)" 'test -L $HOME/.agents/skills/pi-extensions && test -f $HOME/.agents/skills/pi-extensions/SKILL.md && echo ok'
|
||
exec_test "mempalace skill linked (fallback)" 'test -L $HOME/.agents/skills/mempalace && test -f $HOME/.agents/skills/mempalace/SKILL.md && echo ok'
|
||
# The vendored mempalace snapshot is refreshed MANUALLY per release (see
|
||
# rootfs/usr/local/share/pi-devbox/skills/VENDORED.md). Through v1.8.4 it also
|
||
# silently SHADOWED the live skillset copy, so staleness was invisible — and the
|
||
# canary that was supposed to catch it could not: it grepped "Shared palace:
|
||
# multiple harnesses", a phrase present in BOTH the stale and the fresh copy.
|
||
# A snapshot canary must pin the NEWEST section, so update this string whenever
|
||
# the snapshot is refreshed — that is the point of it.
|
||
#
|
||
# v1.8.7: this fired for real, and on the release that changed the snapshot. The
|
||
# pinned phrase was "Attribute what you file yourself", the heading of the
|
||
# instruction telling agents to hand-stamp added_by — which that same release
|
||
# WITHDREW (RFC 001 §7.3.2 ranks agent-side stamping worst-possible; the bridge
|
||
# now does it). So the canary correctly reported "snapshot changed, expectation
|
||
# did not", and blocked publication of an otherwise-green build (81 passed, 1
|
||
# failed, twice). Two lessons kept in the assertion itself:
|
||
# * it is now BIDIRECTIONAL — the new phrase must be present AND the withdrawn
|
||
# one absent, so a re-vendored stale snapshot fails just as loudly as a
|
||
# forgotten bump. A one-way canary only catches half the drift.
|
||
# * a phrase canary can only ever detect "older than what I remembered to pin",
|
||
# never "older than skillset main".
|
||
#
|
||
# That structural limit is now addressed, but NOT by the "CI job diffing this
|
||
# file against the skillset repo" this comment used to point at (that pointer
|
||
# also dangled: it referenced an Unreleased changelog note that had become the
|
||
# v1.8.7 heading). A CI diff cannot be done without granting CI a credential
|
||
# for the PRIVATE skillset repo, and it would guard a file that on this fleet
|
||
# NO host reads — all four compose stacks mount a workspace containing the
|
||
# skillset, so devbox-skill-reconcile repoints this link at the live clone and
|
||
# the baked copy is a CI/no-mount fallback only. Instead the snapshot now
|
||
# carries its provenance (skillset_snapshot_ref + a measured
|
||
# skillset_snapshot_sha256 in build-manifest.json, written by
|
||
# scripts/vendor-mempalace-skill.sh), which moves the check to where the
|
||
# skillset actually IS: `scripts/vendor-mempalace-skill.sh --check` for a
|
||
# maintainer, and `pi-devbox-version` for an agent inside any container.
|
||
# This assertion is kept because it is orthogonal and free: it pins content,
|
||
# not provenance, so it still catches a re-vendored snapshot whose ref was
|
||
# bumped correctly but whose bytes came from the wrong place.
|
||
#
|
||
# v1.8.13: RE-PINNED on refresh a12fe5e -> e9e09d9, which is the whole point of
|
||
# the mechanism — the previous pair ("Provenance is stamped for you" present /
|
||
# "Attribute what you file yourself" absent) still passed against the NEW
|
||
# snapshot, so leaving it would have produced a canary that is green on both the
|
||
# old and the new bytes, i.e. blind to precisely the refresh it exists to
|
||
# witness. Same false-green family as the pre-v1.8.5 canary this comment warns
|
||
# about. The replacement pair was chosen by MEASURING direction against both
|
||
# files rather than by reading the diff: "Diaries self-heal; plain drawers do
|
||
# not" is new=1/old=0, "Agent diaries live in" is new=0/old=1 — so each string
|
||
# discriminates on its own and the pair still fails loudly in BOTH directions
|
||
# (forgotten bump AND re-vendored stale snapshot). Upstream content behind this
|
||
# refresh: the bare project-name wing convention and the <harness>@<device>
|
||
# added_by rule.
|
||
exec_test "mempalace skill snapshot is current" 'f=$HOME/.agents/skills/mempalace/SKILL.md; grep -q "Diaries self-heal; plain drawers do not" "$f" && ! grep -q "Agent diaries live in" "$f" && echo ok'
|
||
# Link TARGETS, not just link existence: with no skillset mounted (as here) the
|
||
# baked tree must be what resolves, for all four vendored skills.
|
||
exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
|
||
'for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
|
||
case "$(readlink -f $HOME/.agents/skills/$s)" in
|
||
/usr/local/share/pi-devbox/skills/$s) ;;
|
||
*) echo "$s resolves to $(readlink -f $HOME/.agents/skills/$s)" >&2; exit 1 ;;
|
||
esac
|
||
done; echo ok'
|
||
# ... and that the tool REPORTS that resolution, which is the half that was
|
||
# missing: a stale baked snapshot and a current live clone were
|
||
# indistinguishable from inside the container. CI mounts no skillset, so every
|
||
# vendored skill must report "baked" here — which also makes this a real test of
|
||
# the fallback path rather than of the environment it happens to run in.
|
||
exec_test "pi-devbox-version reports skill sources (all baked, no skillset here)" \
|
||
'out=$(pi-devbox-version)
|
||
echo "$out" | grep -q "skills:" || { echo "no skills section" >&2; exit 1; }
|
||
for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
|
||
echo "$out" | grep -qE "^ $s +baked( \([^)]*\))?$" \
|
||
|| { echo "$s not reported as baked" >&2; exit 1; }
|
||
done; echo ok'
|
||
# The optional " (...)" above is what pi-extensions now appends to say WHICH copy
|
||
# shipped — "baked (package copy)", or a loud FALLBACK/MIXED annotation. Without
|
||
# allowing it, adding that annotation turned this assertion red on v1.9.0 even
|
||
# though the state it reported was the correct one. The suffix is deliberately
|
||
# matched loosely rather than pinned to "(package copy)", because WHICH copy
|
||
# shipped is already asserted authoritatively above, against the manifest field
|
||
# and its measured tree hash, and duplicating that here in a regex would just
|
||
# create a second place to update whenever the wording changes.
|
||
# The boot banner must NOT carry the section: entrypoint-user.sh prints the
|
||
# version FIRST, before the baked links exist and long before the skillset
|
||
# deploy + reconcile run last, so anything it said about skill sources would be
|
||
# a pre-reconcile state that is about to change.
|
||
# A bare negative (`! grep -q "skills:"`) passes if the tool crashes or
|
||
# prints nothing at all — it cannot tell "correctly omitted the section"
|
||
# apart from "the binary is broken". Anchor it positively: the command must
|
||
# still succeed and still print its normal release-tag line.
|
||
exec_test "pi-devbox-version --no-skills omits the skills section" \
|
||
'out=$(pi-devbox-version --no-skills) && echo "$out" | grep -q "^pi-devbox " && ! echo "$out" | grep -q "skills:"'
|
||
exec_test "entrypoint prints the version banner with --no-skills" \
|
||
'grep -q "pi-devbox-version --no-skills" /usr/local/bin/entrypoint-user.sh'
|
||
# The handover path itself. CI never mounts a skillset, so without this the
|
||
# v1.8.5 fix would ship untested: fabricate a skillset + a skills dir holding
|
||
# baked-style links, run the reconciler, and assert all three outcomes —
|
||
# owned skill repointed, unowned skill left baked, user override untouched.
|
||
exec_test "reconciler: owned skill handed to live clone, others untouched" \
|
||
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/ss/skills/pi-extensions $t/skills
|
||
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo LIVE > $t/ss/skills/pi-extensions/SKILL.md
|
||
ln -s /usr/local/share/pi-devbox/skills/mempalace $t/skills/mempalace
|
||
ln -s /usr/local/share/pi-devbox/skills/pi-extensions $t/skills/pi-extensions
|
||
mkdir -p $t/skills/mine; echo MINE > $t/skills/mine/SKILL.md
|
||
devbox-skill-reconcile $t/ss $t/skills >/dev/null
|
||
devbox-skill-reconcile $t/ss $t/skills >/dev/null # idempotent
|
||
[ "$(readlink $t/skills/mempalace)" = "$t/ss/skills/mempalace" ] || { echo "owned skill NOT repointed" >&2; exit 1; }
|
||
[ "$(readlink $t/skills/pi-extensions)" = /usr/local/share/pi-devbox/skills/pi-extensions ] || { echo "unowned skill was repointed" >&2; exit 1; }
|
||
[ "$(cat $t/skills/mine/SKILL.md)" = MINE ] || { echo "user override clobbered" >&2; exit 1; }
|
||
rm -rf $t; echo ok'
|
||
# The case above cannot fail if the reconciler stops checking WHERE a link
|
||
# points — a mutation test showed all three of its assertions still passing with
|
||
# that guard deleted, which is the same false-green shape as the old snapshot
|
||
# canary. This one discriminates: an OWNED name (so it is considered) whose link
|
||
# is a user override pointing outside the baked tree (so it must be left alone).
|
||
exec_test "reconciler: user override on an owned name is left alone" \
|
||
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/skills $t/mine-skill
|
||
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo USERLINK > $t/mine-skill/SKILL.md
|
||
ln -sfn $t/mine-skill $t/skills/mempalace
|
||
devbox-skill-reconcile $t/ss $t/skills >/dev/null
|
||
[ "$(cat $t/skills/mempalace/SKILL.md)" = USERLINK ] || { echo "user symlink override clobbered" >&2; exit 1; }
|
||
rm -rf $t; echo ok'
|
||
# mempalace-census gained a /usr/local/bin symlink in v1.8.3; its three siblings
|
||
# had one since they were added, so this asserts the set stays complete.
|
||
exec_test "mempalace-census on PATH" 'command -v mempalace-census >/dev/null && mempalace-census --help >/dev/null && echo ok'
|
||
|
||
# pi-fork + pi-observational-memory are registered by entrypoint-user.sh via
|
||
# `pi install /opt/<pkg>`, which runs slightly after the keybindings marker.
|
||
#
|
||
# Assert against the `packages` ARRAY, never a whole-file grep: the settings
|
||
# template ships a top-level "pi-fork" CONFIG block, so `grep -q pi-fork
|
||
# settings.json` passes even when `pi install /opt/pi-fork` never ran. That
|
||
# false green is exactly why the missing `fork` tool shipped unnoticed from
|
||
# v1.0.0 through v1.6.3.
|
||
pkg_registered_cmd() {
|
||
printf "jq -e --arg n %s '(.packages // []) | any((type == \"string\") and (. == \"npm:\" + \$n or endswith(\"/\" + \$n)))' \$HOME/.pi/agent/settings.json" "$1"
|
||
}
|
||
|
||
for i in $(seq 1 15); do
|
||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-observational-memory)" \
|
||
>/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
exec_test "pi-fork registered in packages[] (fork tool)" \
|
||
"$(pkg_registered_cmd pi-fork)"
|
||
exec_test "pi-observational-memory registered in packages[] (recall tool)" \
|
||
"$(pkg_registered_cmd pi-observational-memory)"
|
||
|
||
# pi-studio registration (studio variant only) — registered by the same
|
||
# entrypoint-user.sh local-path install loop as fork/obsmem.
|
||
if [ "${STUDIO_VARIANT:-0}" = "1" ]; then
|
||
for i in $(seq 1 15); do
|
||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-studio)" \
|
||
>/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
exec_test "pi-studio registered in packages[] (/studio command + studio_* tools)" \
|
||
"$(pkg_registered_cmd pi-studio)"
|
||
fi
|
||
|
||
# pi-atelier registration. It is LAST in the entrypoint's install loop, so a
|
||
# pass here also means that loop ran to completion rather than dying midway.
|
||
for i in $(seq 1 15); do
|
||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-atelier)" \
|
||
>/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
exec_test "pi-atelier registered in packages[] (TUI sidebar)" \
|
||
"$(pkg_registered_cmd pi-atelier)"
|
||
# ...and registered from the vendored /opt copy, NOT as `npm:pi-atelier`: an
|
||
# npm: entry resolves through ~/.pi/npm-global on the config VOLUME, which
|
||
# outlives image upgrades and would silently keep an old, unaudited atelier —
|
||
# exactly the shape that pairs a stale 0.6.x with a new pi and hangs at startup.
|
||
exec_test "pi-atelier registered from /opt, not npm: (volume-shadowing guard)" \
|
||
'jq -e "((.packages // []) | any((type == \"string\") and endswith(\"/pi-atelier\"))) and (((.packages // []) | any(. == \"npm:pi-atelier\")) | not)" $HOME/.pi/agent/settings.json'
|
||
|
||
# agent-browser: the third package hit by ~/.pi/npm-global volume shadowing
|
||
# (after pi itself and pi-atelier). This build-time check is deliberately WEAK
|
||
# and says so: a `docker run` container has an EMPTY config volume, so it can
|
||
# only prove the image ships a sane copy and nothing in the image itself
|
||
# shadows it. The check that actually bites lives in
|
||
# recreate-sanity-check.sh, which runs where the volume is real — that is
|
||
# where a 7-week-old 0.27.0 was caught shadowing 0.35.2 on 2026-09-06.
|
||
# EXECUTION is ASSERTED here, not printed. Until 2026-09-07 the version was
|
||
# captured inside an echo with 2>/dev/null, so a binary that could not run at all
|
||
# still PASSED and simply printed version=[] -- the same failure class as the bare
|
||
# `node --version` two hundred lines up: a value displayed rather than compared.
|
||
#
|
||
# Why this exit code matters more than most: smoke runs `platforms: linux/amd64`
|
||
# on an x86 runner, i.e. NATIVE amd64, so this is the fleet's only recurring
|
||
# amd64 runtime proof for the linux-x64 ELF. No devbox can supply one -- every
|
||
# machine in the pi fleet is an Apple Silicon Mac (mbp-m1-2020; tor-ms22 = Mac
|
||
# Studio Mac13,1 M1 Max, verified 2026-08-17 by system_profiler; emb-7kj4vr4g =
|
||
# Apple Silicon, 4 routes 2026-09-07). Asking a device for that proof is asking
|
||
# for the impossible; CI already had it and was discarding it.
|
||
#
|
||
# KEEP PROSE OUT OF THE QUOTED BODY BELOW. On 2026-09-07 this explanation lived
|
||
# INSIDE the single-quoted argument and contained an apostrophe ("the fleet's").
|
||
# Inside '...' bash treats a backslash literally, so \' does not escape -- it
|
||
# CLOSES the string. The body silently truncated, the remaining lines were parsed
|
||
# by the RUNNER's shell instead of the container's, and `agent-browser --version`
|
||
# ran on a host that has no agent-browser: "line 770: command not found", release
|
||
# v1.8.14's smoke job failed after the base had already built. shellcheck caught
|
||
# it as SC2289 the same day and the red lint job went unread for 24h.
|
||
exec_test "agent-browser resolves under /usr (volume-shadowing guard, build-time half)" '
|
||
p=$(command -v agent-browser) || { echo "agent-browser not on PATH" >&2; exit 1; }
|
||
r=$(readlink -f "$p")
|
||
v=$(agent-browser --version) || { echo "agent-browser did not EXECUTE" >&2; exit 1; }
|
||
test -n "$v" || { echo "agent-browser --version produced no output" >&2; exit 1; }
|
||
echo "resolved=[$r] version=[$(printf %s "$v" | head -n1)]" >&2
|
||
case "$r" in /usr/*) ;; *) exit 1 ;; esac
|
||
test ! -d "$HOME/.pi/npm-global/lib/node_modules/agent-browser" || exit 1
|
||
echo ok
|
||
'
|
||
|
||
# pi-fork capability floor. `extensions: []` makes a fork child run with
|
||
# --no-extensions, which is the only MECHANICAL guarantee that a fork cannot
|
||
# file drawers or diary entries under the parent's identity — the mempalace
|
||
# bridge is an extension, so removing extensions removes the write path.
|
||
# Asserted because it is a security-shaped default that a settings merge or a
|
||
# hand-edit could silently drop, and its absence is invisible until a fork
|
||
# writes to the shared palace as you (measured twice: 2026-09-01, 2026-09-06).
|
||
# Deliberately compares to [] and not "is falsy": null means "load normal
|
||
# extensions", i.e. exactly the unguarded state this asserts against.
|
||
exec_test "pi-fork extensions floor is [] (forks cannot write to the palace)" \
|
||
'jq -e ".[\"pi-fork\"].extensions == []" $HOME/.pi/agent/settings.json'
|
||
|
||
# ── /tmp/sshcm directory created by entrypoint ────────────────────────
|
||
exec_test "/tmp/sshcm dir mode 700 (ssh ControlMaster)" \
|
||
'test -d /tmp/sshcm && [ "$(stat -c %a /tmp/sshcm)" = "700" ] && echo ok'
|
||
|
||
# ── Image size ────────────────────────────────────────────────────────
|
||
echo ""
|
||
echo "── Image size ──"
|
||
# Sum all layers via `docker history`. Docker's `image inspect --format='{{.Size}}'`
|
||
# returns ONLY the variant-unique layer when the base is content-addressed and
|
||
# shared (the case in this repo's two-phase build), which understates the
|
||
# user-facing image size by 2+ GB. Summing layer sizes from history is the
|
||
# metric Hub displays to users and the one we actually want to gate on.
|
||
SIZE_MB=$(docker history --format '{{.Size}}' "$IMAGE" | python3 -c '
|
||
import sys, re
|
||
total=0.0
|
||
for line in sys.stdin:
|
||
s=line.strip()
|
||
if s in ("0B", ""): continue
|
||
m=re.match(r"^([0-9.]+)(B|kB|MB|GB)$", s)
|
||
if not m: continue
|
||
v=float(m.group(1)); u=m.group(2)
|
||
mult={"B":1/1048576,"kB":1/1024,"MB":1,"GB":1024}[u]
|
||
total+=v*mult
|
||
print(int(total))
|
||
')
|
||
if [ -z "$SIZE_MB" ] || [ "$SIZE_MB" = "0" ]; then
|
||
printf " ⚠️ image size: could not parse — skipping check\n"
|
||
elif [ "$SIZE_MB" -le "$SIZE_THRESHOLD_MB" ]; then
|
||
printf " ✅ size: %d MB (threshold %d MB)\n" "$SIZE_MB" "$SIZE_THRESHOLD_MB"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ size: %d MB exceeds threshold %d MB\n" "$SIZE_MB" "$SIZE_THRESHOLD_MB"; FAIL=$((FAIL+1))
|
||
# A bare "too big" verdict cost a full CI-log dig plus a local npm bisect to
|
||
# attribute the v1.9.0 overshoot (+431 MB, which turned out to be npm 11
|
||
# installing 26 @esbuild platform binaries per pi-coding-agent copy). The
|
||
# container already knows where its bytes are, so make it say so: the biggest
|
||
# layers, and the biggest directories under the paths that historically grow.
|
||
# Same principle as the run() helper above — a red assertion should carry its
|
||
# own diagnostic rather than send the next reader spelunking.
|
||
echo " ── largest layers (docker history) ──"
|
||
docker history --format '{{.Size}}\t{{.CreatedBy}}' "$IMAGE" 2>/dev/null \
|
||
| grep -vE '^0B' | head -12 | sed 's/^/ /' | cut -c1-160
|
||
echo " ── largest directories in the image ──"
|
||
docker run --rm --entrypoint sh "$IMAGE" -c \
|
||
'du -sm /usr/lib/node_modules/* /opt/* /usr/local/share/ms-playwright 2>/dev/null | sort -rn | head -12' \
|
||
2>/dev/null | sed 's/^/ /' || echo " (could not inspect directories)"
|
||
echo " ── @esbuild platform dirs (npm 11 regression sentinel) ──"
|
||
docker run --rm --entrypoint sh "$IMAGE" -c \
|
||
'find /usr/lib/node_modules /opt -type d -regex ".*/@esbuild/[^/]+" -printf "%f\n" 2>/dev/null | sort | uniq -c | sort -rn | head' \
|
||
2>/dev/null | sed 's/^/ /' || true
|
||
fi
|
||
|
||
# ── Summary ───────────────────────────────────────────────────────────
|
||
echo ""
|
||
echo "=== Results: ${PASS} passed, ${FAIL} failed ==="
|
||
[ "$FAIL" -eq 0 ]
|