Joakim Persson
ff6fd1492a
Closes the half deliberately left open by cac5e00's skill-floor gate, and the
more important half: "the floor is currently fresh" is a fact with a shelf
life, whereas "the image says which copy it got" keeps working.
The refresh in Dockerfile.variant is guarded by
`[ -f /opt/pi-extensions/skill/SKILL.md ]`, so a build whose clone predates the
co-located skill keeps the vendored floor and still succeeds GREEN, with nothing
in the manifest, labels or logs separating that from a normal build. Afterwards
the two are indistinguishable by inspection -- same path, same filenames, same
permissions -- which is exactly how the floor went unnoticed from 2026-07-30 to
2026-09-10.
build-manifest.json gains pi_extensions_skill_source and
pi_extensions_skill_tree_sha256, MEASURED rather than passed as build-args, per
the ground-truth rule the surrounding block already follows -- and necessarily
so, since the outcome depends on the clone's contents and no ARG could express
it. Three values, because two would force a lie: package (served bytes equal
the clone's skill/), vendored-floor (clone had no skill/ at this ref), and
divergent (both exist but differ -- e.g. the clone ships SKILL.md but not
evaluate-extension-usage.py, so the served directory is a genuine MIX). No OCI
label mirrors these deliberately: LABEL cannot take a RUN-computed value, and a
label fed from an ARG would be the claim-not-measurement being removed here.
Two smoke assertions make the record a gate: the source must be named and be
`package` -- vendored-floor FAILS rather than warns, since these images track
main where the package has shipped skill/ since fa04d20, so a fallback means
the clone did not resolve as intended -- and the tree hash is recomputed over
the served directory, because a recorded hash never recompared is a claim.
pi-devbox-version annotates the line too: "baked (package copy)" normally, or a
yellow "(FALLBACK: vendored floor)". Its existing section reports which copy is
READ at runtime; this is the one fact decided at BUILD time and unrecoverable
later. Old images degrade cleanly -- field absent, jq // empty yields nothing,
line prints plain "baked" as before (verified against this v1.8.14 manifest).
Tested by running the exact logic against this container's real layout, with
the expected value written down before each: package (served == clone),
vendored-floor (clone path absent), divergent (clone lacking the .py while the
served dir has it), and null (empty served dir) -- all four as predicted. The
five pi-devbox-version render branches likewise, including the absent-field
case. Emitted JSON validated with jq for both the populated and null forms.
Gates green: lint-shell.sh (15 files), hadolint 2.15.1, actionlint 1.7.12,
check-base-hash.sh, check-skill-floor.sh, vendor-mempalace-skill.sh --check.
891 lines
53 KiB
Bash
Executable File
891 lines
53 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
# smoke-test.sh — sanity checks for the pi-devbox image
|
||
#
|
||
# Usage: ./scripts/smoke-test.sh <image>
|
||
#
|
||
# Verifies:
|
||
# - pi binary present and (if EXPECTED_PI_VERSION set) matches CI's resolved version
|
||
# - node MAJOR matches Dockerfile.base's ARG NODE_VERSION (if EXPECTED_NODE_MAJOR set)
|
||
# - mempalace core matches the audited pin (if EXPECTED_MEMPALACE_VERSION set)
|
||
# - new v1.0.0 base additions (pandoc, graphviz, imagemagick, yq, tealdeer)
|
||
# - typst PDF engine for pandoc (v1.4.0) — `pandoc --pdf-engine=typst`
|
||
# - non-modal editors nano + micro (alongside nvim)
|
||
# - terminfo for modern emulators: xterm-kitty, xterm-ghostty, wezterm,
|
||
# alacritty, foot (kitty-terminfo + ncurses-term + compiled ghostty alias)
|
||
# - tmux 0-indexing baked in /etc/tmux.conf (required for pi-studio variants)
|
||
# - pi-toolkit cloned at /opt/pi-toolkit
|
||
# - pi-extensions cloned at /opt/pi-extensions
|
||
# - pi-atelier vendored at /opt/pi-atelier, registered from /opt (not npm:),
|
||
# and >= the version floor pi's TUI requires (see the floor test)
|
||
# - pi-fork + pi-observational-memory cloned with node_modules baked
|
||
# - entrypoint deploys pi-toolkit keybindings symlink
|
||
# - entrypoint deploys ≥4 extensions
|
||
# - mempalace bridge symlink present
|
||
# - settings.json bootstrapped
|
||
# - pi-fork + pi-observational-memory registered in settings.json packages[]
|
||
# via `pi install`
|
||
# - pi-devbox-version command present + wraps the build manifest correctly
|
||
# (human, --json, --quiet)
|
||
# - (studio variant only, auto-detected) pi-studio cloned + prebuilt
|
||
# client bundle present + registered via `pi install`
|
||
# - image size within threshold
|
||
|
||
set -euo pipefail
|
||
|
||
IMAGE="${1:?usage: $0 <image>}"
|
||
PASS=0; FAIL=0
|
||
# pi-devbox v1.0.0 (decoupled from opencode-devbox) added pandoc, graphviz,
|
||
# imagemagick, yq, tealdeer, a baked /etc/tmux.conf, and the non-modal
|
||
# editors nano + micro (~15 MB combined). v1.6.0 baked in agent-browser +
|
||
# Playwright Chromium (~291 MB net after dropping the unused headless-shell
|
||
# build), which lifted the baseline. CI amd64 actuals observed on run 512
|
||
# (v1.6.1): 3411 MB non-studio, 3574 MB studio. Threshold below carries
|
||
# ~225 MB margin above the studio number to absorb minor arch/build-cache
|
||
# differences and small future growth without false reds, while still
|
||
# catching an unexpected +GB regression.
|
||
SIZE_THRESHOLD_MB=3800
|
||
|
||
# On failure, surface the last few lines the command produced. This used to
|
||
# discard output entirely (`>/dev/null 2>&1`), which made a red ❌ carry zero
|
||
# diagnostic weight: explaining the single v1.8.0 stage-default failure took a
|
||
# full CI-log dig plus a registry-config inspection, when the container had
|
||
# already printed the answer and thrown it away. Assertions that want a
|
||
# diagnostic just echo it to stderr — it stays hidden while they pass.
|
||
run() {
|
||
local label="$1"; local cmd="$2"
|
||
local out
|
||
if out=$(docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" 2>&1); then
|
||
printf " ✅ %s\n" "$label"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ %s\n" "$label"; FAIL=$((FAIL+1))
|
||
# `if`, not `&&` — a trailing false under `set -e` would abort the script.
|
||
if [ -n "$out" ]; then
|
||
printf " └─ %s\n" "$(printf '%s' "$out" | tail -3 | tr '\n' ' ' | cut -c1-300)"
|
||
fi
|
||
fi
|
||
}
|
||
|
||
# Stricter version of `run` that asserts an expected substring in stdout.
|
||
# Catches the "image bytes silently identical to previous release" class of
|
||
# regression — Docker layer cache hit on `npm install -g <pkg>` because the
|
||
# bare command string is identical across builds, even when `latest` would
|
||
# resolve differently. Discovered 2026-05-23 — every pi-devbox release
|
||
# v0.74.0..v0.75.5 had been shipping the same image bytes.
|
||
run_expect() {
|
||
local label="$1"; local cmd="$2"; local expect="$3"
|
||
local out
|
||
out=$(docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" 2>&1) || true
|
||
if echo "$out" | grep -Fq "$expect"; then
|
||
printf " ✅ %s (got %s)\n" "$label" "$expect"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ %s — expected substring %q, got: %s\n" "$label" "$expect" "$out"; FAIL=$((FAIL+1))
|
||
fi
|
||
}
|
||
|
||
echo "=== pi-devbox smoke test: $IMAGE ==="
|
||
echo ""
|
||
|
||
# ── Binaries ─────────────────────────────────────────────────────────
|
||
echo "── Binaries ──"
|
||
if [ -n "${EXPECTED_PI_VERSION:-}" ]; then
|
||
run_expect "pi version matches build arg" "pi --version" "$EXPECTED_PI_VERSION"
|
||
else
|
||
run "pi" "pi --version"
|
||
fi
|
||
# Until 2026-09-07 this was a bare `run "node" "node --version"`, which asserts
|
||
# only that the binary exists and exits 0 — the printed version was never
|
||
# compared to anything. A node major bump would therefore have passed this suite
|
||
# SILENTLY, while a reader skimming it would reasonably assume node regressions
|
||
# were covered. EXPECTED_NODE_MAJOR closes that: CI derives it from
|
||
# Dockerfile.base's ARG NODE_VERSION (the single source of truth), so this also
|
||
# catches a stale cached layer whose node does not match the declared ARG.
|
||
if [ -n "${EXPECTED_NODE_MAJOR:-}" ]; then
|
||
run_expect "node major matches Dockerfile ARG" "node --version" "v${EXPECTED_NODE_MAJOR}."
|
||
else
|
||
run "node" "node --version"
|
||
fi
|
||
run "git" "git --version"
|
||
# NOTE: the shellcheck binary is a GATE DEPENDENCY, not a convenience.
|
||
# scripts/lint-shell.sh is the release gate (the lint-gate job resolve-versions
|
||
# depends on) and it exits 2 when the binary is missing, by design — "a gate that
|
||
# cannot run must not pass". Measured on v1.8.14: it was absent from the image, so
|
||
# that gate could not be run by a developer in ANY container, only in CI.
|
||
# Asserted here so its absence fails a build instead of being discovered by a hook
|
||
# that then refuses every push (hooks/pre-push).
|
||
#
|
||
# This comment must not BEGIN with the tool's name: a line starting with
|
||
# `# shellcheck` is parsed as a DIRECTIVE, not a comment (SC1073/SC1072). The
|
||
# gate added in this same change caught that here, before the push.
|
||
run "shellcheck (lint gate dependency)" "shellcheck --version | grep -qE '^version: [0-9]'"
|
||
run "aws" "aws --version"
|
||
run "uv" "uv --version"
|
||
run "nvim" "nvim --version"
|
||
run "nano" "nano --version"
|
||
run "micro" "micro --version"
|
||
run "kitty-terminfo" "infocmp -x xterm-kitty >/dev/null 2>&1"
|
||
run "terminfo: modern emulators (ncurses-term)" 'for t in wezterm alacritty foot ghostty st-256color; do infocmp -x "$t" >/dev/null 2>&1 || exit 1; done'
|
||
run "terminfo: xterm-ghostty alias (tic)" "infocmp -x xterm-ghostty >/dev/null 2>&1"
|
||
run "nvim true-colour default (sysinit.vim)" "nvim --headless -c 'lua os.exit(vim.o.termguicolors and 0 or 1)'"
|
||
run "mempalace-mcp" "mempalace-mcp --help"
|
||
run "mempalace-pi-session on PATH" "mempalace-pi-session --help"
|
||
# The staging dir must sit next to the palace, not in a disposable cache: the
|
||
# palace keys per-source dedup on the STAGED path, so a stage that can be wiped
|
||
# while the palace survives lets `mempalace sync` prune every drawer mined from
|
||
# it. Assert the resolved default, not an env var — the guarantee is "stage
|
||
# shares the palace's lifetime", which an ENV pin would quietly break.
|
||
# NOTE: --sessions-dir gets an EMPTY temp dir, never /tmp. The stage banner is
|
||
# printed before any export, so nothing needs to be found — and pointing a
|
||
# default-staged run at a populated dir would export whatever transcripts it
|
||
# finds into the real stage, which is how a synthetic test session ends up
|
||
# staged for mining as if it were a real conversation.
|
||
#
|
||
# Asserted $HOME-RELATIVE, not against a literal /home/developer. `run` invokes
|
||
# `docker run --entrypoint=""`, and neither Dockerfile sets USER or ENV HOME
|
||
# (HOME is set by entrypoint-user.sh, which --entrypoint="" deliberately skips),
|
||
# so these assertions execute as root with HOME=/root. The original literal
|
||
# /home/developer form could therefore never match and failed the v1.8.0
|
||
# release — a test bug, not a product one: the stage resolution was correct all
|
||
# along, it just follows $HOME. The invariant under test ("the stage sits beside
|
||
# the palace, sharing its lifetime") is user-independent, so pinning the user
|
||
# was never part of it. A cache-dir default still fails the pattern below, which
|
||
# is the regression this guards.
|
||
#
|
||
# It went unnoticed for three days because this workflow only triggers on
|
||
# `push: tags: v*` — the assertion was added on a main push, so v1.8.0 was its
|
||
# first execution ever. Use the `smoke_only` workflow_dispatch input to run
|
||
# smoke against HEAD without cutting a tag.
|
||
run "pi stage defaults next to the palace (not a cache dir)" '
|
||
out=$(mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||
stage=$(echo "$out" | grep -oE "stage=[^ ]+" | head -1)
|
||
echo "resolved ${stage:-<no stage= line>} with HOME=$HOME" >&2
|
||
case "$stage" in
|
||
"stage=$HOME/.mempalace/pi-stage/"*) exit 0 ;;
|
||
*) exit 1 ;;
|
||
esac
|
||
'
|
||
# Companion to the above: the deployment-specific case the literal assertion was
|
||
# reaching for, done properly by supplying the HOME the container actually runs
|
||
# with instead of assuming it.
|
||
run "pi stage is palace-adjacent for the developer user" '
|
||
out=$(HOME=/home/developer mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||
echo "$out" | grep -oE "stage=[^ ]+" | head -1 >&2
|
||
echo "$out" | grep -q "stage=/home/developer/.mempalace/pi-stage/"
|
||
'
|
||
run "pi stage follows MEMPALACE_PALACE_PATH" '
|
||
out=$(MEMPALACE_PALACE_PATH=/tmp/alt/.mempalace/palace \
|
||
mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||
echo "$out" | grep -q "stage=/tmp/alt/.mempalace/pi-stage/"
|
||
'
|
||
# The feeder's --agent default is WHO a drawer is attributed to. mempalace core
|
||
# records neither the machine nor the harness on a write, and one shared bearer
|
||
# token means the server cannot tell clients apart, so toolkit c64ffa1 changed
|
||
# this default from $USER to pi@$MEMPALACE_PI_DEVICE — the one string that makes
|
||
# a write attributable to both. Nothing ever PRINTED the resolved value (the
|
||
# banner shows mode= and stage= only), so an image built from a pre-c64ffa1
|
||
# toolkit ref would ship unattributed writes with every check still green.
|
||
#
|
||
# `--help` assigns AGENT (script top) before it parses args, then exits 0 with
|
||
# no side effects — so `bash -x` observes the REAL resolution, env interpolation
|
||
# and fallback included, rather than grepping the source for a literal line that
|
||
# any reformat would break. Two-sided on purpose: device set => pi@<device>;
|
||
# device UNSET => must not be pi@anything. The second half is what fails against
|
||
# the old unconditional $USER default, which ignored the device entirely.
|
||
#
|
||
# Probes the PATH entry (a symlink into the /opt clone) rather than that clone
|
||
# path directly: this is the invocation the systemd/launchd timers and
|
||
# entrypoint-user.sh actually use, so it is the default that reaches the palace.
|
||
run "feeder resolves --agent to pi@<device> (drawer attribution)" '
|
||
f=$(command -v mempalace-pi-session) || { echo "feeder not on PATH" >&2; exit 1; }
|
||
with=$(MEMPALACE_PI_DEVICE=smoke-device bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
|
||
without=$(env -u MEMPALACE_PI_DEVICE bash -x $f --help 2>&1 | sed -n "s/^+* *AGENT=//p" | tail -n1)
|
||
echo "resolved with-device=[$with] without-device=[$without]" >&2
|
||
[ "$with" = "pi@smoke-device" ] || exit 1
|
||
case "$without" in pi@*) exit 1 ;; esac
|
||
echo ok
|
||
'
|
||
# Regression guard for the pi transcript exporter. If pi ever changes its
|
||
# session JSONL shape, the exporter stops recognising sessions and the palace
|
||
# silently gets nothing (or, worse, raw JSON chunked as prose). Feed it a
|
||
# synthetic session and assert it is actually exported. Uses --dry-run so no
|
||
# palace is touched, and a temp stage so nothing real is written.
|
||
run "pi transcript exporter recognises a pi session" '
|
||
set -e
|
||
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
|
||
{
|
||
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question one\"}}"
|
||
a=$(printf "a%.0s" $(seq 1 1200))
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"$a\"}]}}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question two\"}}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"short reply\"}]}}"
|
||
} > "$s/2026-01-01T00-00-00-000Z_smoke.jsonl"
|
||
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
|
||
echo "$out" | grep -q "Exported 1 session"
|
||
'
|
||
# The same guard from the other side: a session with no real assistant output
|
||
# (an abandoned prompt, whose bulk is injected skill text) must NOT be filed.
|
||
run "pi transcript exporter rejects an abandoned session" '
|
||
set -e
|
||
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
|
||
{
|
||
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke2\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
|
||
u=$(printf "u%.0s" $(seq 1 13000))
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"$u\"}}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"Ready. What would you like to work on?\"}]}}"
|
||
} > "$s/2026-01-01T00-00-00-000Z_smoke2.jsonl"
|
||
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
|
||
echo "$out" | grep -q "no sessions qualified"
|
||
'
|
||
# The remote-palace-without-inbox skip must ANNOUNCE itself, not vanish. This
|
||
# branch of entrypoint-user.sh runs at container start (not reachable from a
|
||
# `docker run` one-shot), so assert against the entrypoint that actually shipped
|
||
# in the image. Guards a silent regression back to the bare `:` no-op, which
|
||
# left a container contributing nothing to the palace with no artifact saying
|
||
# why — the log it would normally leave is written by the other branch.
|
||
run_expect "remote-palace-without-inbox skip is announced, not silent" \
|
||
"grep -o 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | head -1" \
|
||
"MemPalace catch-up skipped"
|
||
run "...and the skip notice names the variable that fixes it" \
|
||
"grep -A6 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | grep -q 'MEMPALACE_PI_SSH_TARGET'"
|
||
# A remote mine that FAILS must not report success. MCP answers a hard tool
|
||
# failure with HTTP 200 and the tool's own JSON escaped inside
|
||
# result.content[].text, so the feeder's old `'\"error\"' in body` check could
|
||
# never see it: on 2026-08-15 a mine that died with "source directory not found:
|
||
# '/data/feed/...'" logged "Done. Wing updated." and exited 0, and this
|
||
# container's transcripts were filed nowhere for a whole session. The feeder
|
||
# carries fixtures for that exact body; run them against the baked toolkit so a
|
||
# stale/reverted toolkit ref can't reintroduce a silent feed.
|
||
run "baked feeder detects a failed remote mine (no silent false success)" \
|
||
"mempalace-pi-session --self-test"
|
||
# v1.0.0 base additions — verify presence and basic functionality.
|
||
run "pandoc" "pandoc --version"
|
||
run "typst" "typst --version"
|
||
run "pandoc+typst PDF engine" "printf '# hi\n' | pandoc --pdf-engine=typst -o /tmp/_smoke.pdf - && test -s /tmp/_smoke.pdf; rm -f /tmp/_smoke.pdf"
|
||
run "graphviz (dot)" "dot -V"
|
||
run "imagemagick" "magick --version"
|
||
run "yq (mikefarah v4)" "yq --version | grep -qE 'mikefarah.*version v4'"
|
||
run "tldr (tealdeer)" "tldr --version"
|
||
run "socat" "socat -V"
|
||
run "studio-expose helper" "test -x /usr/local/bin/studio-expose"
|
||
run "image-baked pi-devbox-environment skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md"
|
||
run "image-baked credential-incident-response skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md"
|
||
run "global-AGENTS append snippet present" \
|
||
"test -f /usr/local/share/pi-devbox/pi-global-AGENTS.append.md"
|
||
run "pi-devbox block merged into pi-global-AGENTS.md" \
|
||
"grep -q 'pi-devbox:managed-block' /opt/pi-toolkit/pi-global-AGENTS.md"
|
||
run "mempalace session-start pointer merged into global AGENTS.md" \
|
||
"grep -q 'load the mempalace skill' /opt/pi-toolkit/pi-global-AGENTS.md"
|
||
# Vendored fallback skills (so a no-skillset container still resolves the
|
||
# AGENTS.md 'read the pi-extensions skill' pointer).
|
||
run "image-baked pi-extensions fallback skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md"
|
||
run "pi-extensions skill ships its helper" \
|
||
"test -f /usr/local/share/pi-devbox/skills/pi-extensions/evaluate-extension-usage.py"
|
||
run "image-baked mempalace fallback skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/mempalace/SKILL.md"
|
||
# Layered freshness: when the pinned pi-extensions clone carries the skill, the
|
||
# baked copy must be the fresh package copy (Option 1), not the stale snapshot.
|
||
run "pi-extensions skill refreshed from package when present" \
|
||
"if [ -f /opt/pi-extensions/skill/SKILL.md ]; then cmp -s /opt/pi-extensions/skill/SKILL.md /usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md; else true; fi"
|
||
# Runtime ownership handover (v1.8.5): the baked links are a FALLBACK, and
|
||
# skillset-OWNED skills must be repointed at the live clone when one is mounted.
|
||
# The list is data, so assert its content, not just its presence: mempalace in,
|
||
# pi-extensions deliberately out (its skillset copy is a lagging duplicate).
|
||
run "devbox-skill-reconcile helper present + executable" \
|
||
"test -x /usr/local/bin/devbox-skill-reconcile"
|
||
run "skillset-owned list ships and names mempalace" \
|
||
"grep -qx 'mempalace' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
|
||
run "skillset-owned list excludes pi-extensions (ownership)" \
|
||
"! grep -qx 'pi-extensions' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
|
||
|
||
# ── tmux 0-indexing (required for pi-studio variants) ─────────────────
|
||
echo ""
|
||
echo "── tmux config ──"
|
||
run_expect "/etc/tmux.conf has base-index 0" \
|
||
"cat /etc/tmux.conf" "set -g base-index 0"
|
||
run_expect "/etc/tmux.conf has pane-base-index 0" \
|
||
"cat /etc/tmux.conf" "set -g pane-base-index 0"
|
||
|
||
# ── Repo clones ───────────────────────────────────────────────────────
|
||
echo ""
|
||
echo "── Repo clones ──"
|
||
run "pi-toolkit clone" "test -d /opt/pi-toolkit && git -C /opt/pi-toolkit rev-parse --short HEAD"
|
||
run "pi-extensions clone" "test -d /opt/pi-extensions && git -C /opt/pi-extensions rev-parse --short HEAD"
|
||
run "pi-fork clone + node_modules" \
|
||
"test -f /opt/pi-fork/package.json && test -d /opt/pi-fork/node_modules"
|
||
run "pi-observational-memory clone + node_modules" \
|
||
"test -f /opt/pi-observational-memory/package.json && test -d /opt/pi-observational-memory/node_modules"
|
||
# ...and that the clone carries the AUTH FIX, not merely that it exists. om's
|
||
# pre-flight hasUsableAuth() check silently disabled `recall` for ~8 weeks once
|
||
# pi moved to request-time SigV4 signing and stopped exposing a static Bedrock
|
||
# key; upstream fixed it in ce9fc98, adopted in v1.8.4. PI_OBSMEM_REF tracks
|
||
# master, so an upstream revert or force-push would ship a dead `recall` with
|
||
# the clone assertion above still green — the exact gap flagged as open in the
|
||
# v1.8.5 changelog.
|
||
#
|
||
# Pin the markers to src/runtime.ts, the fix SITE, rather than grepping the
|
||
# repo: two of these three strings also appear under tests/, so a repo-wide
|
||
# grep stays green with runtime.ts itself reverted. That is a false green of the
|
||
# same family as the old skill-snapshot canary.
|
||
run "pi-observational-memory carries the ce9fc98 auth fix (recall stays alive)" '
|
||
f=/opt/pi-observational-memory/src/runtime.ts
|
||
test -f "$f" || { echo "fix site missing: $f" >&2; exit 1; }
|
||
for m in availability_recheck providerCredentialConfigured hasConfiguredAuth; do
|
||
grep -q "$m" "$f" || { echo "marker absent from runtime.ts: $m" >&2; exit 1; }
|
||
done
|
||
echo ok
|
||
'
|
||
# pi-atelier: deliberately NO node_modules assertion, unlike its siblings —
|
||
# it declares zero runtime dependencies (only peerDeps, satisfied by the baked
|
||
# pi) and has no build step, so Dockerfile.variant skips `npm install` for it.
|
||
# Assert what pi actually loads instead: the entry point named by its
|
||
# package.json `pi.extensions` key.
|
||
run "pi-atelier clone + entry point" \
|
||
"test -f /opt/pi-atelier/package.json && test -f /opt/pi-atelier/extensions/index.ts"
|
||
|
||
# ── pi <-> pi-atelier compatibility floor (executable, not a comment) ──
|
||
# pi-atelier < 0.7.1 wraps pi's PRIVATE TUI renderer in a way that recurses
|
||
# under pi >= 0.84: pi hangs at startup burning CPU, with no error. Upstream
|
||
# fixed it in 0.7.1/0.7.2, but atelier's peerDependencies still say
|
||
# `>=0.80.7`, so neither npm nor pi can warn about the real floor. Both
|
||
# versions are pinned in Dockerfile.variant; this makes a bad PAIRING fail the
|
||
# build instead of publishing an image whose TUI never starts.
|
||
run_expect "pi-atelier >= 0.7.1 floor for pi >= 0.84 (startup-hang guard)" \
|
||
'ge() { [ "$(printf "%s\n%s\n" "$1" "$2" | sort -V | head -n1)" = "$2" ]; }; AV=$(jq -r ".version // empty" /opt/pi-atelier/package.json 2>/dev/null); PV=$(pi --version 2>/dev/null | grep -oE "[0-9]+\.[0-9]+\.[0-9]+" | head -n1); if [ -z "$AV" ] || [ -z "$PV" ]; then echo "unreadable versions (atelier=$AV pi=$PV)"; elif ge "$PV" 0.84.0 && ! ge "$AV" 0.7.1; then echo "VIOLATION: pi $PV with pi-atelier $AV"; else echo "compatible: pi $PV + pi-atelier $AV"; fi' \
|
||
"compatible:"
|
||
|
||
# pi-studio is present only in the :latest-studio variant. Auto-detect by
|
||
# probing /opt/pi-studio so this one script covers both variants.
|
||
if docker run --rm --entrypoint="" "$IMAGE" sh -c 'test -d /opt/pi-studio' >/dev/null 2>&1; then
|
||
STUDIO_VARIANT=1
|
||
echo " ℹ️ pi-studio detected — running studio assertions"
|
||
run "pi-studio clone + node_modules" \
|
||
"test -f /opt/pi-studio/package.json && test -d /opt/pi-studio/node_modules"
|
||
run "pi-studio prebuilt client bundle" \
|
||
"test -f /opt/pi-studio/client/studio-client.js"
|
||
else
|
||
STUDIO_VARIANT=0
|
||
echo " ℹ️ pi-studio not present (non-studio variant) — skipping studio clone checks"
|
||
fi
|
||
|
||
# ── Build provenance (manifest + OCI labels) ─────────────────────────
|
||
echo ""
|
||
echo "── Build provenance ──"
|
||
run "/etc/pi-devbox/build-manifest.json present" \
|
||
"test -f /etc/pi-devbox/build-manifest.json"
|
||
# These next checks replace three that grepped the manifest for the FIELD NAME
|
||
# and never looked at the value:
|
||
#
|
||
# run_expect "manifest records pi_version" "cat …manifest.json" '"pi_version"'
|
||
#
|
||
# which passes on {"pi_version": ""} and on {"pi_version": null}. The tell was
|
||
# visible in its own passing output — `✅ manifest records pi_version (got
|
||
# "pi_version")` echoes the key back as the thing it claims to have found.
|
||
# Two failure modes were therefore invisible: a key that survives with an empty
|
||
# or garbage value, and a key that vanishes from the manifest while every
|
||
# remaining value still looks fine.
|
||
#
|
||
# Those two need SEPARATE assertions, and the reason is a trap worth keeping in
|
||
# writing: an "every component value is a valid SHA" loop passes VACUOUSLY on
|
||
# components:{} — jq's all() over an empty list is true — so the value check
|
||
# alone would go green on a manifest that lost every component. Mutation-tested
|
||
# 2026-08-25 across nine fabricated manifests (empty map, deleted key, "",
|
||
# null, "unknown", 12-hex truncation, 40 non-hex chars, legit null pi-studio).
|
||
run "manifest declares every required component key" '
|
||
req="pi-toolkit pi-extensions pi-fork pi-observational-memory pi-atelier mempalace-toolkit pi-studio"
|
||
for k in $req; do
|
||
jq -e --arg k "$k" "(.components|has(\$k))" /etc/pi-devbox/build-manifest.json >/dev/null \
|
||
|| { echo "manifest lost component key: $k" >&2; exit 1; }
|
||
done
|
||
'
|
||
# Subsumes the old `! grep -q \"unknown\"` check ("unknown" is not 40-hex), and
|
||
# also catches "", null and truncated SHAs, which that grep let through. null is
|
||
# legitimate for pi-studio alone: the non-studio variant has no such clone.
|
||
run "manifest component values are resolved 40-hex commits" '
|
||
jq -e "
|
||
.components
|
||
| to_entries
|
||
| all(if .key == \"pi-studio\" and .value == null then true
|
||
else (.value|type) == \"string\" and (.value|test(\"^[0-9a-f]{40}\$\")) end)
|
||
" /etc/pi-devbox/build-manifest.json >/dev/null
|
||
'
|
||
# pi_version against ground truth, same shape as the mempalace check below.
|
||
# Chains with the "pi version matches build arg" assertion earlier in this file:
|
||
# together they tie build arg -> installed binary -> recorded manifest, so a
|
||
# manifest written from a stale variable cannot pass by agreeing with itself.
|
||
run "manifest pi_version matches the installed pi" '
|
||
m=$(jq -r ".pi_version // empty" /etc/pi-devbox/build-manifest.json)
|
||
b=$(pi --version 2>/dev/null | head -n1 | tr -d "\r")
|
||
echo "manifest=[$m] installed=[$b]" >&2
|
||
[ -n "$m" ] && [ "$m" = "$b" ]
|
||
'
|
||
# Top-level provenance fields: assert the SHAPE of each value, and only when the
|
||
# field is populated. source_revision and build_date legitimately default to
|
||
# empty (Dockerfile.variant ARGs) on a plain local `docker build`, so demanding
|
||
# them would fail honest local smoke runs; a populated-but-malformed value is
|
||
# the actual defect. release_tag defaults to "dev", so empty means a broken write.
|
||
run "manifest top-level fields are well-formed, not merely present" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
t=$(jq -r ".release_tag // empty" $j)
|
||
r=$(jq -r ".source_revision // empty" $j)
|
||
d=$(jq -r ".build_date // empty" $j)
|
||
echo "release_tag=[$t] source_revision=[$r] build_date=[$d]" >&2
|
||
[ -n "$t" ] || { echo "release_tag empty (ARG default is dev)" >&2; exit 1; }
|
||
if [ -n "$r" ]; then
|
||
printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" || { echo "source_revision not a 40-hex commit" >&2; exit 1; }
|
||
fi
|
||
if [ -n "$d" ]; then
|
||
printf "%s" "$d" | grep -qE "^[0-9]{4}-[0-9]{2}-[0-9]{2}T" || { echo "build_date not ISO-8601" >&2; exit 1; }
|
||
fi
|
||
'
|
||
# mempalace CORE was absent from the manifest through v1.8.5: the toolkit SHA
|
||
# was recorded but the palace version behind the MCP tools was not, so a palace
|
||
# bug could not be correlated to an image version. Assert the field exists AND
|
||
# equals the installed binary — recording it from ARG MEMPALACE_VERSION instead
|
||
# would look identical here yet drift silently the first time an install
|
||
# resolved to something other than the pin, which is the whole reason this file
|
||
# is built from ground truth. `// empty` matters: jq -r prints the 4-char
|
||
# string "null" for a JSON null, which would satisfy a naive -n test.
|
||
run "manifest mempalace_version matches the installed core" '
|
||
m=$(jq -r ".mempalace_version // empty" /etc/pi-devbox/build-manifest.json)
|
||
b=$(mempalace --version 2>/dev/null | head -n1 | tr -d "\r"); b=${b##* }
|
||
echo "manifest=[$m] installed=[$b]" >&2
|
||
[ -n "$m" ] && [ "$m" = "$b" ]
|
||
'
|
||
# ... and, when CI supplies it, that the installed core is the version CI
|
||
# actually AUDITED (published + not yanked on PyPI, in resolve-versions). This
|
||
# does NOT duplicate the check above, which compares two properties of one
|
||
# image and so cannot notice that BOTH are the wrong version. The live failure
|
||
# mode it covers: the variant builds `FROM` a base tag chosen by base-decide's
|
||
# content hash, so a bug in that hashing (the reason scripts/check-base-hash.sh
|
||
# exists) could reuse a cached base built from an OLDER MEMPALACE_VERSION pin —
|
||
# internally consistent, silently stale, invisible to every other assertion.
|
||
if [ -n "${EXPECTED_MEMPALACE_VERSION:-}" ]; then
|
||
run "installed mempalace matches CI's audited pin (${EXPECTED_MEMPALACE_VERSION})" "
|
||
b=\$(mempalace --version 2>/dev/null | head -n1 | tr -d '\r'); b=\${b##* }
|
||
echo \"installed=[\$b] audited_pin=[${EXPECTED_MEMPALACE_VERSION}]\" >&2
|
||
[ \"\$b\" = \"${EXPECTED_MEMPALACE_VERSION}\" ]
|
||
"
|
||
fi
|
||
# Every component must be a resolved commit (or null for pi-studio in the
|
||
# non-studio variant) — now enforced by the 40-hex value check above, which
|
||
# strictly subsumes the old whole-file grep for '"unknown"'. Only rev() ever
|
||
# emits "unknown" and rev() feeds components only, so nothing is lost.
|
||
# pi-devbox-version wraps the manifest into a human-first command; verify the
|
||
# binary is present, executable, and that all three output modes work.
|
||
run "pi-devbox-version binary present + executable" \
|
||
"test -x /usr/local/bin/pi-devbox-version"
|
||
run_expect "pi-devbox-version human output shows release tag" \
|
||
"pi-devbox-version" "pi-devbox "
|
||
# --json is a verbatim `cat` of the manifest, so "round-trips" is assertable
|
||
# literally. The old form grepped the output for the string "release_tag" — the
|
||
# key name again — which would pass on a truncated or re-serialised dump.
|
||
run "pi-devbox-version --json round-trips the manifest byte-for-byte" '
|
||
a=$(cat /etc/pi-devbox/build-manifest.json)
|
||
b=$(pi-devbox-version --json)
|
||
[ "$a" = "$b" ] || { echo "--json output differs from the manifest on disk" >&2; exit 1; }
|
||
'
|
||
run_expect "pi-devbox-version --quiet is a compact one-liner" \
|
||
"pi-devbox-version --quiet | wc -l" "1"
|
||
# ── Vendored skill snapshot provenance ─────────────────────────────────
|
||
# The vendored mempalace skill is the one baked artefact with no /opt clone
|
||
# behind it (private upstream — see VENDORED.md), so until now the manifest
|
||
# could not say which skillset commit it came from. Two fields now travel with
|
||
# it: the CLAIMED ref (ARG default in Dockerfile.variant) and the MEASURED
|
||
# sha256 of the shipped bytes. Assert both are well-formed, and — separately —
|
||
# that the measurement still describes the file in the image.
|
||
#
|
||
# Kept as two assertions for the same reason the component checks are: one
|
||
# proves the fields are not empty/garbage, the other proves they are not merely
|
||
# self-consistent. A single combined check could pass on a manifest whose hash
|
||
# was computed from a file that was later overwritten (the pi-extensions skill
|
||
# copy at Dockerfile.variant:165 does exactly that kind of overwrite, one stage
|
||
# earlier), which is the failure this second one exists to catch.
|
||
run "manifest records the vendored skill snapshot provenance" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
r=$(jq -r ".skillset_snapshot_ref // empty" $j)
|
||
s=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
|
||
echo "ref=[$r] tree_sha256=[$s]" >&2
|
||
printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" \
|
||
|| { echo "skillset_snapshot_ref is not a 40-hex commit" >&2; exit 1; }
|
||
printf "%s" "$s" | grep -qxE "[0-9a-f]{64}" \
|
||
|| { echo "skillset_snapshot_tree_sha256 is not a 64-hex digest" >&2; exit 1; }
|
||
'
|
||
# Recomputes over the whole DIRECTORY with the same tree_sha256() pipeline
|
||
# Dockerfile.variant used to measure it, not a plain `sha256sum SKILL.md` —
|
||
# a file-only compare here would pass even if the manifest recorded a
|
||
# fingerprint over a directory that has since grown a second file (this is
|
||
# not hypothetical: pi-extensions already ships two files for its skill).
|
||
run "manifest skill fingerprint matches the baked snapshot" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
d=/usr/local/share/pi-devbox/skills/mempalace
|
||
m=$(jq -r ".skillset_snapshot_tree_sha256 // empty" $j)
|
||
a=$( (cd "$d" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum) | sha256sum | cut -d" " -f1)
|
||
echo "manifest=[$m] actual=[$a]" >&2
|
||
[ -n "$m" ] && [ "$m" = "$a" ]
|
||
'
|
||
|
||
# ── Which pi-extensions skill copy shipped ──────────────────────────────
|
||
# Closes the silent-fallback hole. The refresh in Dockerfile.variant is guarded
|
||
# by `[ -f /opt/pi-extensions/skill/SKILL.md ]`, so a build whose clone predates
|
||
# the co-located skill keeps the vendored floor and still succeeds GREEN, with
|
||
# nothing recording that a snapshot shipped instead of the package copy. Measured
|
||
# 2026-09-10: the floor had been stale since 2026-07-30, so that path would have
|
||
# shipped a six-week-old skill in silence. The floor is fresh now and gated by the
|
||
# skill-floor lint job, but "the fallback is currently harmless" is a fact with a
|
||
# shelf life, whereas "the image says which copy it got" keeps working.
|
||
#
|
||
# vendored-floor FAILS here rather than merely warning: these images track main,
|
||
# where the package has co-located skill/ since fa04d20, so a fallback means the
|
||
# clone did not resolve as intended and that is a defect to investigate. A fork
|
||
# deliberately pointing at a mirror without skill/ is the one case that should
|
||
# edit this assertion — which is the honest place for that decision to surface.
|
||
run "manifest names which pi-extensions skill copy shipped" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
s=$(jq -r ".pi_extensions_skill_source // empty" $j)
|
||
h=$(jq -r ".pi_extensions_skill_tree_sha256 // empty" $j)
|
||
echo "source=[$s] tree_sha256=[$h]" >&2
|
||
printf "%s" "$h" | grep -qxE "[0-9a-f]{64}" || {
|
||
echo "pi_extensions_skill_tree_sha256 is not a 64-hex digest" >&2; exit 1; }
|
||
case "$s" in
|
||
package) ;;
|
||
vendored-floor)
|
||
echo "FALLBACK: clone had no skill/ at this ref, so the image ships the committed floor" >&2; exit 1 ;;
|
||
divergent)
|
||
echo "MIXED: served directory is part package and part floor" >&2; exit 1 ;;
|
||
*)
|
||
echo "pi_extensions_skill_source absent or unrecognised" >&2; exit 1 ;;
|
||
esac
|
||
'
|
||
|
||
# Same shape as the mempalace fingerprint check above, and for the same reason: a
|
||
# recorded hash that is never recomputed is a claim, not a measurement.
|
||
run "recorded pi-extensions skill hash matches the served bytes" '
|
||
j=/etc/pi-devbox/build-manifest.json
|
||
d=/usr/local/share/pi-devbox/skills/pi-extensions
|
||
m=$(jq -r ".pi_extensions_skill_tree_sha256 // empty" $j)
|
||
a=$( (cd "$d" && find . -type f -print | LC_ALL=C sort | xargs -r sha256sum) | sha256sum | cut -d" " -f1)
|
||
echo "manifest=[$m] actual=[$a]" >&2
|
||
[ -n "$m" ] && [ "$m" = "$a" ]
|
||
'
|
||
# OCI labels live in the image config, not the container fs — inspect them
|
||
# from the host docker rather than via `docker run`.
|
||
LBL=$(docker inspect --format '{{ index .Config.Labels "se.jordbo.pi-devbox.pi-extensions-ref" }}' "$IMAGE" 2>/dev/null || true)
|
||
if [ -n "$LBL" ] && [ "$LBL" != "<no value>" ]; then
|
||
printf " ✅ OCI label se.jordbo.pi-devbox.pi-extensions-ref=%s\n" "$LBL"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ OCI label se.jordbo.pi-devbox.pi-extensions-ref missing or empty\n"; FAIL=$((FAIL+1))
|
||
fi
|
||
|
||
# ── Runtime deployment (needs entrypoint to run) ──────────────────────
|
||
echo ""
|
||
echo "── Runtime deployment ──"
|
||
# Spin up a long-running container WITHOUT overriding the entrypoint, so
|
||
# the baked entrypoint chain (entrypoint.sh → entrypoint-user.sh) runs and
|
||
# deploys pi-toolkit + pi-extensions to ~/.pi/agent/. Override CMD to
|
||
# tail -f /dev/null so the container stays alive while we docker-exec.
|
||
CID=$(docker run -d --rm "$IMAGE" tail -f /dev/null)
|
||
cleanup() { docker rm -f "$CID" >/dev/null 2>&1 || true; }
|
||
trap cleanup EXIT
|
||
|
||
# Wait for entrypoint-user.sh to finish deploying pi-toolkit + extensions.
|
||
# Gate on BOTH the keybindings symlink (deployed by pi-toolkit) AND the
|
||
# mempalace.ts bridge (deployed last by entrypoint-user.sh) AND ≥4 *.ts
|
||
# extensions present. Parallel build load can otherwise sample the *.ts
|
||
# count mid-deploy and produce a flake. See opencode-devbox c6f9d11
|
||
# (2026-06-08) — same fix transplanted.
|
||
for i in $(seq 1 45); do
|
||
if docker exec "$CID" sh -c '
|
||
test -L /home/developer/.pi/agent/keybindings.json && \
|
||
test -L /home/developer/.pi/agent/extensions/mempalace.ts && \
|
||
test -L /home/developer/.agents/skills/pi-devbox-environment && \
|
||
test -L /home/developer/.agents/skills/pi-extensions && \
|
||
test -L /home/developer/.agents/skills/mempalace && \
|
||
count=$(ls -1 /home/developer/.pi/agent/extensions/*.ts 2>/dev/null | wc -l) && \
|
||
[ "$count" -ge 4 ]
|
||
' >/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
|
||
exec_test() {
|
||
local label="$1"; local cmd="$2"
|
||
if docker exec -u developer "$CID" sh -c "$cmd" >/dev/null 2>&1; then
|
||
printf " ✅ %s\n" "$label"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ %s\n" "$label"; FAIL=$((FAIL+1))
|
||
fi
|
||
}
|
||
|
||
exec_test "keybindings.json (pi-toolkit)" 'test -L $HOME/.pi/agent/keybindings.json && echo ok'
|
||
exec_test "extensions ≥ 4 (pi-extensions)" 'count=$(ls -1 $HOME/.pi/agent/extensions/*.ts 2>/dev/null | wc -l); [ $count -ge 4 ] && echo "$count extensions"'
|
||
exec_test "mempalace.ts bridge" 'test -L $HOME/.pi/agent/extensions/mempalace.ts && echo ok'
|
||
exec_test "settings.json bootstrapped" 'test -f $HOME/.pi/agent/settings.json && echo ok'
|
||
exec_test "pi-devbox-environment skill linked" 'test -L $HOME/.agents/skills/pi-devbox-environment && test -f $HOME/.agents/skills/pi-devbox-environment/SKILL.md && echo ok'
|
||
exec_test "pi-extensions skill linked (fallback)" 'test -L $HOME/.agents/skills/pi-extensions && test -f $HOME/.agents/skills/pi-extensions/SKILL.md && echo ok'
|
||
exec_test "mempalace skill linked (fallback)" 'test -L $HOME/.agents/skills/mempalace && test -f $HOME/.agents/skills/mempalace/SKILL.md && echo ok'
|
||
# The vendored mempalace snapshot is refreshed MANUALLY per release (see
|
||
# rootfs/usr/local/share/pi-devbox/skills/VENDORED.md). Through v1.8.4 it also
|
||
# silently SHADOWED the live skillset copy, so staleness was invisible — and the
|
||
# canary that was supposed to catch it could not: it grepped "Shared palace:
|
||
# multiple harnesses", a phrase present in BOTH the stale and the fresh copy.
|
||
# A snapshot canary must pin the NEWEST section, so update this string whenever
|
||
# the snapshot is refreshed — that is the point of it.
|
||
#
|
||
# v1.8.7: this fired for real, and on the release that changed the snapshot. The
|
||
# pinned phrase was "Attribute what you file yourself", the heading of the
|
||
# instruction telling agents to hand-stamp added_by — which that same release
|
||
# WITHDREW (RFC 001 §7.3.2 ranks agent-side stamping worst-possible; the bridge
|
||
# now does it). So the canary correctly reported "snapshot changed, expectation
|
||
# did not", and blocked publication of an otherwise-green build (81 passed, 1
|
||
# failed, twice). Two lessons kept in the assertion itself:
|
||
# * it is now BIDIRECTIONAL — the new phrase must be present AND the withdrawn
|
||
# one absent, so a re-vendored stale snapshot fails just as loudly as a
|
||
# forgotten bump. A one-way canary only catches half the drift.
|
||
# * a phrase canary can only ever detect "older than what I remembered to pin",
|
||
# never "older than skillset main".
|
||
#
|
||
# That structural limit is now addressed, but NOT by the "CI job diffing this
|
||
# file against the skillset repo" this comment used to point at (that pointer
|
||
# also dangled: it referenced an Unreleased changelog note that had become the
|
||
# v1.8.7 heading). A CI diff cannot be done without granting CI a credential
|
||
# for the PRIVATE skillset repo, and it would guard a file that on this fleet
|
||
# NO host reads — all four compose stacks mount a workspace containing the
|
||
# skillset, so devbox-skill-reconcile repoints this link at the live clone and
|
||
# the baked copy is a CI/no-mount fallback only. Instead the snapshot now
|
||
# carries its provenance (skillset_snapshot_ref + a measured
|
||
# skillset_snapshot_sha256 in build-manifest.json, written by
|
||
# scripts/vendor-mempalace-skill.sh), which moves the check to where the
|
||
# skillset actually IS: `scripts/vendor-mempalace-skill.sh --check` for a
|
||
# maintainer, and `pi-devbox-version` for an agent inside any container.
|
||
# This assertion is kept because it is orthogonal and free: it pins content,
|
||
# not provenance, so it still catches a re-vendored snapshot whose ref was
|
||
# bumped correctly but whose bytes came from the wrong place.
|
||
#
|
||
# v1.8.13: RE-PINNED on refresh a12fe5e -> e9e09d9, which is the whole point of
|
||
# the mechanism — the previous pair ("Provenance is stamped for you" present /
|
||
# "Attribute what you file yourself" absent) still passed against the NEW
|
||
# snapshot, so leaving it would have produced a canary that is green on both the
|
||
# old and the new bytes, i.e. blind to precisely the refresh it exists to
|
||
# witness. Same false-green family as the pre-v1.8.5 canary this comment warns
|
||
# about. The replacement pair was chosen by MEASURING direction against both
|
||
# files rather than by reading the diff: "Diaries self-heal; plain drawers do
|
||
# not" is new=1/old=0, "Agent diaries live in" is new=0/old=1 — so each string
|
||
# discriminates on its own and the pair still fails loudly in BOTH directions
|
||
# (forgotten bump AND re-vendored stale snapshot). Upstream content behind this
|
||
# refresh: the bare project-name wing convention and the <harness>@<device>
|
||
# added_by rule.
|
||
exec_test "mempalace skill snapshot is current" 'f=$HOME/.agents/skills/mempalace/SKILL.md; grep -q "Diaries self-heal; plain drawers do not" "$f" && ! grep -q "Agent diaries live in" "$f" && echo ok'
|
||
# Link TARGETS, not just link existence: with no skillset mounted (as here) the
|
||
# baked tree must be what resolves, for all four vendored skills.
|
||
exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
|
||
'for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
|
||
case "$(readlink -f $HOME/.agents/skills/$s)" in
|
||
/usr/local/share/pi-devbox/skills/$s) ;;
|
||
*) echo "$s resolves to $(readlink -f $HOME/.agents/skills/$s)" >&2; exit 1 ;;
|
||
esac
|
||
done; echo ok'
|
||
# ... and that the tool REPORTS that resolution, which is the half that was
|
||
# missing: a stale baked snapshot and a current live clone were
|
||
# indistinguishable from inside the container. CI mounts no skillset, so every
|
||
# vendored skill must report "baked" here — which also makes this a real test of
|
||
# the fallback path rather than of the environment it happens to run in.
|
||
exec_test "pi-devbox-version reports skill sources (all baked, no skillset here)" \
|
||
'out=$(pi-devbox-version)
|
||
echo "$out" | grep -q "skills:" || { echo "no skills section" >&2; exit 1; }
|
||
for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
|
||
echo "$out" | grep -qE "^ $s +baked$" \
|
||
|| { echo "$s not reported as baked" >&2; exit 1; }
|
||
done; echo ok'
|
||
# The boot banner must NOT carry the section: entrypoint-user.sh prints the
|
||
# version FIRST, before the baked links exist and long before the skillset
|
||
# deploy + reconcile run last, so anything it said about skill sources would be
|
||
# a pre-reconcile state that is about to change.
|
||
# A bare negative (`! grep -q "skills:"`) passes if the tool crashes or
|
||
# prints nothing at all — it cannot tell "correctly omitted the section"
|
||
# apart from "the binary is broken". Anchor it positively: the command must
|
||
# still succeed and still print its normal release-tag line.
|
||
exec_test "pi-devbox-version --no-skills omits the skills section" \
|
||
'out=$(pi-devbox-version --no-skills) && echo "$out" | grep -q "^pi-devbox " && ! echo "$out" | grep -q "skills:"'
|
||
exec_test "entrypoint prints the version banner with --no-skills" \
|
||
'grep -q "pi-devbox-version --no-skills" /usr/local/bin/entrypoint-user.sh'
|
||
# The handover path itself. CI never mounts a skillset, so without this the
|
||
# v1.8.5 fix would ship untested: fabricate a skillset + a skills dir holding
|
||
# baked-style links, run the reconciler, and assert all three outcomes —
|
||
# owned skill repointed, unowned skill left baked, user override untouched.
|
||
exec_test "reconciler: owned skill handed to live clone, others untouched" \
|
||
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/ss/skills/pi-extensions $t/skills
|
||
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo LIVE > $t/ss/skills/pi-extensions/SKILL.md
|
||
ln -s /usr/local/share/pi-devbox/skills/mempalace $t/skills/mempalace
|
||
ln -s /usr/local/share/pi-devbox/skills/pi-extensions $t/skills/pi-extensions
|
||
mkdir -p $t/skills/mine; echo MINE > $t/skills/mine/SKILL.md
|
||
devbox-skill-reconcile $t/ss $t/skills >/dev/null
|
||
devbox-skill-reconcile $t/ss $t/skills >/dev/null # idempotent
|
||
[ "$(readlink $t/skills/mempalace)" = "$t/ss/skills/mempalace" ] || { echo "owned skill NOT repointed" >&2; exit 1; }
|
||
[ "$(readlink $t/skills/pi-extensions)" = /usr/local/share/pi-devbox/skills/pi-extensions ] || { echo "unowned skill was repointed" >&2; exit 1; }
|
||
[ "$(cat $t/skills/mine/SKILL.md)" = MINE ] || { echo "user override clobbered" >&2; exit 1; }
|
||
rm -rf $t; echo ok'
|
||
# The case above cannot fail if the reconciler stops checking WHERE a link
|
||
# points — a mutation test showed all three of its assertions still passing with
|
||
# that guard deleted, which is the same false-green shape as the old snapshot
|
||
# canary. This one discriminates: an OWNED name (so it is considered) whose link
|
||
# is a user override pointing outside the baked tree (so it must be left alone).
|
||
exec_test "reconciler: user override on an owned name is left alone" \
|
||
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/skills $t/mine-skill
|
||
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo USERLINK > $t/mine-skill/SKILL.md
|
||
ln -sfn $t/mine-skill $t/skills/mempalace
|
||
devbox-skill-reconcile $t/ss $t/skills >/dev/null
|
||
[ "$(cat $t/skills/mempalace/SKILL.md)" = USERLINK ] || { echo "user symlink override clobbered" >&2; exit 1; }
|
||
rm -rf $t; echo ok'
|
||
# mempalace-census gained a /usr/local/bin symlink in v1.8.3; its three siblings
|
||
# had one since they were added, so this asserts the set stays complete.
|
||
exec_test "mempalace-census on PATH" 'command -v mempalace-census >/dev/null && mempalace-census --help >/dev/null && echo ok'
|
||
|
||
# pi-fork + pi-observational-memory are registered by entrypoint-user.sh via
|
||
# `pi install /opt/<pkg>`, which runs slightly after the keybindings marker.
|
||
#
|
||
# Assert against the `packages` ARRAY, never a whole-file grep: the settings
|
||
# template ships a top-level "pi-fork" CONFIG block, so `grep -q pi-fork
|
||
# settings.json` passes even when `pi install /opt/pi-fork` never ran. That
|
||
# false green is exactly why the missing `fork` tool shipped unnoticed from
|
||
# v1.0.0 through v1.6.3.
|
||
pkg_registered_cmd() {
|
||
printf "jq -e --arg n %s '(.packages // []) | any((type == \"string\") and (. == \"npm:\" + \$n or endswith(\"/\" + \$n)))' \$HOME/.pi/agent/settings.json" "$1"
|
||
}
|
||
|
||
for i in $(seq 1 15); do
|
||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-observational-memory)" \
|
||
>/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
exec_test "pi-fork registered in packages[] (fork tool)" \
|
||
"$(pkg_registered_cmd pi-fork)"
|
||
exec_test "pi-observational-memory registered in packages[] (recall tool)" \
|
||
"$(pkg_registered_cmd pi-observational-memory)"
|
||
|
||
# pi-studio registration (studio variant only) — registered by the same
|
||
# entrypoint-user.sh local-path install loop as fork/obsmem.
|
||
if [ "${STUDIO_VARIANT:-0}" = "1" ]; then
|
||
for i in $(seq 1 15); do
|
||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-studio)" \
|
||
>/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
exec_test "pi-studio registered in packages[] (/studio command + studio_* tools)" \
|
||
"$(pkg_registered_cmd pi-studio)"
|
||
fi
|
||
|
||
# pi-atelier registration. It is LAST in the entrypoint's install loop, so a
|
||
# pass here also means that loop ran to completion rather than dying midway.
|
||
for i in $(seq 1 15); do
|
||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-atelier)" \
|
||
>/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
exec_test "pi-atelier registered in packages[] (TUI sidebar)" \
|
||
"$(pkg_registered_cmd pi-atelier)"
|
||
# ...and registered from the vendored /opt copy, NOT as `npm:pi-atelier`: an
|
||
# npm: entry resolves through ~/.pi/npm-global on the config VOLUME, which
|
||
# outlives image upgrades and would silently keep an old, unaudited atelier —
|
||
# exactly the shape that pairs a stale 0.6.x with a new pi and hangs at startup.
|
||
exec_test "pi-atelier registered from /opt, not npm: (volume-shadowing guard)" \
|
||
'jq -e "((.packages // []) | any((type == \"string\") and endswith(\"/pi-atelier\"))) and (((.packages // []) | any(. == \"npm:pi-atelier\")) | not)" $HOME/.pi/agent/settings.json'
|
||
|
||
# agent-browser: the third package hit by ~/.pi/npm-global volume shadowing
|
||
# (after pi itself and pi-atelier). This build-time check is deliberately WEAK
|
||
# and says so: a `docker run` container has an EMPTY config volume, so it can
|
||
# only prove the image ships a sane copy and nothing in the image itself
|
||
# shadows it. The check that actually bites lives in
|
||
# recreate-sanity-check.sh, which runs where the volume is real — that is
|
||
# where a 7-week-old 0.27.0 was caught shadowing 0.35.2 on 2026-09-06.
|
||
# EXECUTION is ASSERTED here, not printed. Until 2026-09-07 the version was
|
||
# captured inside an echo with 2>/dev/null, so a binary that could not run at all
|
||
# still PASSED and simply printed version=[] -- the same failure class as the bare
|
||
# `node --version` two hundred lines up: a value displayed rather than compared.
|
||
#
|
||
# Why this exit code matters more than most: smoke runs `platforms: linux/amd64`
|
||
# on an x86 runner, i.e. NATIVE amd64, so this is the fleet's only recurring
|
||
# amd64 runtime proof for the linux-x64 ELF. No devbox can supply one -- every
|
||
# machine in the pi fleet is an Apple Silicon Mac (mbp-m1-2020; tor-ms22 = Mac
|
||
# Studio Mac13,1 M1 Max, verified 2026-08-17 by system_profiler; emb-7kj4vr4g =
|
||
# Apple Silicon, 4 routes 2026-09-07). Asking a device for that proof is asking
|
||
# for the impossible; CI already had it and was discarding it.
|
||
#
|
||
# KEEP PROSE OUT OF THE QUOTED BODY BELOW. On 2026-09-07 this explanation lived
|
||
# INSIDE the single-quoted argument and contained an apostrophe ("the fleet's").
|
||
# Inside '...' bash treats a backslash literally, so \' does not escape -- it
|
||
# CLOSES the string. The body silently truncated, the remaining lines were parsed
|
||
# by the RUNNER's shell instead of the container's, and `agent-browser --version`
|
||
# ran on a host that has no agent-browser: "line 770: command not found", release
|
||
# v1.8.14's smoke job failed after the base had already built. shellcheck caught
|
||
# it as SC2289 the same day and the red lint job went unread for 24h.
|
||
exec_test "agent-browser resolves under /usr (volume-shadowing guard, build-time half)" '
|
||
p=$(command -v agent-browser) || { echo "agent-browser not on PATH" >&2; exit 1; }
|
||
r=$(readlink -f "$p")
|
||
v=$(agent-browser --version) || { echo "agent-browser did not EXECUTE" >&2; exit 1; }
|
||
test -n "$v" || { echo "agent-browser --version produced no output" >&2; exit 1; }
|
||
echo "resolved=[$r] version=[$(printf %s "$v" | head -n1)]" >&2
|
||
case "$r" in /usr/*) ;; *) exit 1 ;; esac
|
||
test ! -d "$HOME/.pi/npm-global/lib/node_modules/agent-browser" || exit 1
|
||
echo ok
|
||
'
|
||
|
||
# pi-fork capability floor. `extensions: []` makes a fork child run with
|
||
# --no-extensions, which is the only MECHANICAL guarantee that a fork cannot
|
||
# file drawers or diary entries under the parent's identity — the mempalace
|
||
# bridge is an extension, so removing extensions removes the write path.
|
||
# Asserted because it is a security-shaped default that a settings merge or a
|
||
# hand-edit could silently drop, and its absence is invisible until a fork
|
||
# writes to the shared palace as you (measured twice: 2026-09-01, 2026-09-06).
|
||
# Deliberately compares to [] and not "is falsy": null means "load normal
|
||
# extensions", i.e. exactly the unguarded state this asserts against.
|
||
exec_test "pi-fork extensions floor is [] (forks cannot write to the palace)" \
|
||
'jq -e ".[\"pi-fork\"].extensions == []" $HOME/.pi/agent/settings.json'
|
||
|
||
# ── /tmp/sshcm directory created by entrypoint ────────────────────────
|
||
exec_test "/tmp/sshcm dir mode 700 (ssh ControlMaster)" \
|
||
'test -d /tmp/sshcm && [ "$(stat -c %a /tmp/sshcm)" = "700" ] && echo ok'
|
||
|
||
# ── Image size ────────────────────────────────────────────────────────
|
||
echo ""
|
||
echo "── Image size ──"
|
||
# Sum all layers via `docker history`. Docker's `image inspect --format='{{.Size}}'`
|
||
# returns ONLY the variant-unique layer when the base is content-addressed and
|
||
# shared (the case in this repo's two-phase build), which understates the
|
||
# user-facing image size by 2+ GB. Summing layer sizes from history is the
|
||
# metric Hub displays to users and the one we actually want to gate on.
|
||
SIZE_MB=$(docker history --format '{{.Size}}' "$IMAGE" | python3 -c '
|
||
import sys, re
|
||
total=0.0
|
||
for line in sys.stdin:
|
||
s=line.strip()
|
||
if s in ("0B", ""): continue
|
||
m=re.match(r"^([0-9.]+)(B|kB|MB|GB)$", s)
|
||
if not m: continue
|
||
v=float(m.group(1)); u=m.group(2)
|
||
mult={"B":1/1048576,"kB":1/1024,"MB":1,"GB":1024}[u]
|
||
total+=v*mult
|
||
print(int(total))
|
||
')
|
||
if [ -z "$SIZE_MB" ] || [ "$SIZE_MB" = "0" ]; then
|
||
printf " ⚠️ image size: could not parse — skipping check\n"
|
||
elif [ "$SIZE_MB" -le "$SIZE_THRESHOLD_MB" ]; then
|
||
printf " ✅ size: %d MB (threshold %d MB)\n" "$SIZE_MB" "$SIZE_THRESHOLD_MB"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ size: %d MB exceeds threshold %d MB\n" "$SIZE_MB" "$SIZE_THRESHOLD_MB"; FAIL=$((FAIL+1))
|
||
fi
|
||
|
||
# ── Summary ───────────────────────────────────────────────────────────
|
||
echo ""
|
||
echo "=== Results: ${PASS} passed, ${FAIL} failed ==="
|
||
[ "$FAIL" -eq 0 ]
|