joakimp
b5810654f6
Baked skill links won over the live skillset clone for all three vendored skills, so a pushed edit to skills/mempalace/SKILL.md was invisible in every container until the next image build -- measured on two hosts (live md5 129bcc4752 vs baked 5236024fef). Cause was ordering, not intent: the baked links are created early with a create-only-when-absent guard to close a smoke readiness race, and the skillset deploy runs last and treats them as foreign. The comment claimed the opposite of the behaviour. The fix is not "skillset always wins". Ownership is per-skill: pi-extensions is owned by its package repo and copied over the snapshot at build time, so the skillset's lagging duplicate must keep losing; pi-devbox-environment is authored here. Only mempalace is skillset-owned. devbox-skill-reconcile therefore runs after the deploy and repoints only the names in skills/skillset-owned.txt, replacing a link solely when it points into the baked tree, so a real directory or a link pointing elsewhere is never disturbed. Precedence is now user override -> live clone (owned names) -> baked snapshot, with the early links intact as the fallback so the readiness race stays closed. Reviewing that turned up a latent boot-abort in the pre-existing baked-link block: `[ ! -e "$link" ]` is TRUE for a dangling symlink, so once a link can point into /workspace/skillset, a vanished mount makes plain `ln -s` fail with "File exists" -- and under `set -euo pipefail` that aborts container start before `exec "$@"`. Reachable on `docker restart` or a host reboot, not on a recreate, since ~/.agents is not a volume on any host. Now `ln -sfn`, which heals the link back to the baked fallback. Smoke additions cover what let this ship: the stale-snapshot canary grepped a phrase present in BOTH the stale and fresh copies, so it passed throughout; it now pins the newest section. Link targets are asserted, not just `test -L`; the owned-list content is asserted both ways; and the reconciler's replace path -- which no CI container exercises, since none mounts a skillset -- is covered by fabricating one. A mutation test showed the obvious three assertions still pass with the "is this link ours?" guard deleted, so a discriminating case was added: an owned name whose link is a user override outside the baked tree. Also refreshes the mempalace snapshot to skillset 670f7f1 (without it the fix helps only hosts that mount skillset) and corrects README, which documented the old, wrong precedence in three places. Verified with 12 fixture cases plus 2 mutants: ownership respected against the real trees, user overrides preserved, relative/trailing-slash/CRLF/space/glob inputs handled, dangling link healed, read-only skills dir exits 0, idempotent.
523 lines
30 KiB
Bash
Executable File
523 lines
30 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
# smoke-test.sh — sanity checks for the pi-devbox image
|
||
#
|
||
# Usage: ./scripts/smoke-test.sh <image>
|
||
#
|
||
# Verifies:
|
||
# - pi binary present and (if EXPECTED_PI_VERSION set) matches CI's resolved version
|
||
# - new v1.0.0 base additions (pandoc, graphviz, imagemagick, yq, tealdeer)
|
||
# - typst PDF engine for pandoc (Unreleased) — `pandoc --pdf-engine=typst`
|
||
# - non-modal editors nano + micro (alongside nvim)
|
||
# - terminfo for modern emulators: xterm-kitty, xterm-ghostty, wezterm,
|
||
# alacritty, foot (kitty-terminfo + ncurses-term + compiled ghostty alias)
|
||
# - tmux 0-indexing baked in /etc/tmux.conf (required for pi-studio variants)
|
||
# - pi-toolkit cloned at /opt/pi-toolkit
|
||
# - pi-extensions cloned at /opt/pi-extensions
|
||
# - pi-atelier vendored at /opt/pi-atelier, registered from /opt (not npm:),
|
||
# and >= the version floor pi's TUI requires (see the floor test)
|
||
# - pi-fork + pi-observational-memory cloned with node_modules baked
|
||
# - entrypoint deploys pi-toolkit keybindings symlink
|
||
# - entrypoint deploys ≥4 extensions
|
||
# - mempalace bridge symlink present
|
||
# - settings.json bootstrapped
|
||
# - pi-fork + pi-observational-memory registered in settings.json packages[]
|
||
# via `pi install`
|
||
# - pi-devbox-version command present + wraps the build manifest correctly
|
||
# (human, --json, --quiet)
|
||
# - (studio variant only, auto-detected) pi-studio cloned + prebuilt
|
||
# client bundle present + registered via `pi install`
|
||
# - image size within threshold
|
||
|
||
set -euo pipefail
|
||
|
||
IMAGE="${1:?usage: $0 <image>}"
|
||
PASS=0; FAIL=0
|
||
# pi-devbox v1.0.0 (decoupled from opencode-devbox) added pandoc, graphviz,
|
||
# imagemagick, yq, tealdeer, a baked /etc/tmux.conf, and the non-modal
|
||
# editors nano + micro (~15 MB combined). v1.6.0 baked in agent-browser +
|
||
# Playwright Chromium (~291 MB net after dropping the unused headless-shell
|
||
# build), which lifted the baseline. CI amd64 actuals observed on run 512
|
||
# (v1.6.1): 3411 MB non-studio, 3574 MB studio. Threshold below carries
|
||
# ~225 MB margin above the studio number to absorb minor arch/build-cache
|
||
# differences and small future growth without false reds, while still
|
||
# catching an unexpected +GB regression.
|
||
SIZE_THRESHOLD_MB=3800
|
||
|
||
# On failure, surface the last few lines the command produced. This used to
|
||
# discard output entirely (`>/dev/null 2>&1`), which made a red ❌ carry zero
|
||
# diagnostic weight: explaining the single v1.8.0 stage-default failure took a
|
||
# full CI-log dig plus a registry-config inspection, when the container had
|
||
# already printed the answer and thrown it away. Assertions that want a
|
||
# diagnostic just echo it to stderr — it stays hidden while they pass.
|
||
run() {
|
||
local label="$1"; local cmd="$2"
|
||
local out
|
||
if out=$(docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" 2>&1); then
|
||
printf " ✅ %s\n" "$label"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ %s\n" "$label"; FAIL=$((FAIL+1))
|
||
# `if`, not `&&` — a trailing false under `set -e` would abort the script.
|
||
if [ -n "$out" ]; then
|
||
printf " └─ %s\n" "$(printf '%s' "$out" | tail -3 | tr '\n' ' ' | cut -c1-300)"
|
||
fi
|
||
fi
|
||
}
|
||
|
||
# Stricter version of `run` that asserts an expected substring in stdout.
|
||
# Catches the "image bytes silently identical to previous release" class of
|
||
# regression — Docker layer cache hit on `npm install -g <pkg>` because the
|
||
# bare command string is identical across builds, even when `latest` would
|
||
# resolve differently. Discovered 2026-05-23 — every pi-devbox release
|
||
# v0.74.0..v0.75.5 had been shipping the same image bytes.
|
||
run_expect() {
|
||
local label="$1"; local cmd="$2"; local expect="$3"
|
||
local out
|
||
out=$(docker run --rm --entrypoint="" "$IMAGE" sh -c "$cmd" 2>&1) || true
|
||
if echo "$out" | grep -Fq "$expect"; then
|
||
printf " ✅ %s (got %s)\n" "$label" "$expect"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ %s — expected substring %q, got: %s\n" "$label" "$expect" "$out"; FAIL=$((FAIL+1))
|
||
fi
|
||
}
|
||
|
||
echo "=== pi-devbox smoke test: $IMAGE ==="
|
||
echo ""
|
||
|
||
# ── Binaries ─────────────────────────────────────────────────────────
|
||
echo "── Binaries ──"
|
||
if [ -n "${EXPECTED_PI_VERSION:-}" ]; then
|
||
run_expect "pi version matches build arg" "pi --version" "$EXPECTED_PI_VERSION"
|
||
else
|
||
run "pi" "pi --version"
|
||
fi
|
||
run "node" "node --version"
|
||
run "git" "git --version"
|
||
run "aws" "aws --version"
|
||
run "uv" "uv --version"
|
||
run "nvim" "nvim --version"
|
||
run "nano" "nano --version"
|
||
run "micro" "micro --version"
|
||
run "kitty-terminfo" "infocmp -x xterm-kitty >/dev/null 2>&1"
|
||
run "terminfo: modern emulators (ncurses-term)" 'for t in wezterm alacritty foot ghostty st-256color; do infocmp -x "$t" >/dev/null 2>&1 || exit 1; done'
|
||
run "terminfo: xterm-ghostty alias (tic)" "infocmp -x xterm-ghostty >/dev/null 2>&1"
|
||
run "nvim true-colour default (sysinit.vim)" "nvim --headless -c 'lua os.exit(vim.o.termguicolors and 0 or 1)'"
|
||
run "mempalace-mcp" "mempalace-mcp --help"
|
||
run "mempalace-pi-session on PATH" "mempalace-pi-session --help"
|
||
# The staging dir must sit next to the palace, not in a disposable cache: the
|
||
# palace keys per-source dedup on the STAGED path, so a stage that can be wiped
|
||
# while the palace survives lets `mempalace sync` prune every drawer mined from
|
||
# it. Assert the resolved default, not an env var — the guarantee is "stage
|
||
# shares the palace's lifetime", which an ENV pin would quietly break.
|
||
# NOTE: --sessions-dir gets an EMPTY temp dir, never /tmp. The stage banner is
|
||
# printed before any export, so nothing needs to be found — and pointing a
|
||
# default-staged run at a populated dir would export whatever transcripts it
|
||
# finds into the real stage, which is how a synthetic test session ends up
|
||
# staged for mining as if it were a real conversation.
|
||
#
|
||
# Asserted $HOME-RELATIVE, not against a literal /home/developer. `run` invokes
|
||
# `docker run --entrypoint=""`, and neither Dockerfile sets USER or ENV HOME
|
||
# (HOME is set by entrypoint-user.sh, which --entrypoint="" deliberately skips),
|
||
# so these assertions execute as root with HOME=/root. The original literal
|
||
# /home/developer form could therefore never match and failed the v1.8.0
|
||
# release — a test bug, not a product one: the stage resolution was correct all
|
||
# along, it just follows $HOME. The invariant under test ("the stage sits beside
|
||
# the palace, sharing its lifetime") is user-independent, so pinning the user
|
||
# was never part of it. A cache-dir default still fails the pattern below, which
|
||
# is the regression this guards.
|
||
#
|
||
# It went unnoticed for three days because this workflow only triggers on
|
||
# `push: tags: v*` — the assertion was added on a main push, so v1.8.0 was its
|
||
# first execution ever. Use the `smoke_only` workflow_dispatch input to run
|
||
# smoke against HEAD without cutting a tag.
|
||
run "pi stage defaults next to the palace (not a cache dir)" '
|
||
out=$(mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||
stage=$(echo "$out" | grep -oE "stage=[^ ]+" | head -1)
|
||
echo "resolved ${stage:-<no stage= line>} with HOME=$HOME" >&2
|
||
case "$stage" in
|
||
"stage=$HOME/.mempalace/pi-stage/"*) exit 0 ;;
|
||
*) exit 1 ;;
|
||
esac
|
||
'
|
||
# Companion to the above: the deployment-specific case the literal assertion was
|
||
# reaching for, done properly by supplying the HOME the container actually runs
|
||
# with instead of assuming it.
|
||
run "pi stage is palace-adjacent for the developer user" '
|
||
out=$(HOME=/home/developer mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||
echo "$out" | grep -oE "stage=[^ ]+" | head -1 >&2
|
||
echo "$out" | grep -q "stage=/home/developer/.mempalace/pi-stage/"
|
||
'
|
||
run "pi stage follows MEMPALACE_PALACE_PATH" '
|
||
out=$(MEMPALACE_PALACE_PATH=/tmp/alt/.mempalace/palace \
|
||
mempalace-pi-session --dry-run --reason smoke --sessions-dir "$(mktemp -d)" 2>&1) || true
|
||
echo "$out" | grep -q "stage=/tmp/alt/.mempalace/pi-stage/"
|
||
'
|
||
# Regression guard for the pi transcript exporter. If pi ever changes its
|
||
# session JSONL shape, the exporter stops recognising sessions and the palace
|
||
# silently gets nothing (or, worse, raw JSON chunked as prose). Feed it a
|
||
# synthetic session and assert it is actually exported. Uses --dry-run so no
|
||
# palace is touched, and a temp stage so nothing real is written.
|
||
run "pi transcript exporter recognises a pi session" '
|
||
set -e
|
||
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
|
||
{
|
||
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question one\"}}"
|
||
a=$(printf "a%.0s" $(seq 1 1200))
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"$a\"}]}}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"question two\"}}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"short reply\"}]}}"
|
||
} > "$s/2026-01-01T00-00-00-000Z_smoke.jsonl"
|
||
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
|
||
echo "$out" | grep -q "Exported 1 session"
|
||
'
|
||
# The same guard from the other side: a session with no real assistant output
|
||
# (an abandoned prompt, whose bulk is injected skill text) must NOT be filed.
|
||
run "pi transcript exporter rejects an abandoned session" '
|
||
set -e
|
||
d=$(mktemp -d); s="$d/sessions/--workspace--"; mkdir -p "$s"
|
||
{
|
||
printf "%s\n" "{\"type\":\"session\",\"version\":1,\"id\":\"smoke2\",\"cwd\":\"/workspace\",\"timestamp\":\"2026-01-01T00:00:00Z\"}"
|
||
u=$(printf "u%.0s" $(seq 1 13000))
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"user\",\"content\":\"$u\"}}"
|
||
printf "%s\n" "{\"type\":\"message\",\"message\":{\"role\":\"assistant\",\"content\":[{\"type\":\"text\",\"text\":\"Ready. What would you like to work on?\"}]}}"
|
||
} > "$s/2026-01-01T00-00-00-000Z_smoke2.jsonl"
|
||
out=$(mempalace-pi-session --dry-run --sessions-dir "$d/sessions" --stage "$d/stage" 2>&1)
|
||
echo "$out" | grep -q "no sessions qualified"
|
||
'
|
||
# The remote-palace-without-inbox skip must ANNOUNCE itself, not vanish. This
|
||
# branch of entrypoint-user.sh runs at container start (not reachable from a
|
||
# `docker run` one-shot), so assert against the entrypoint that actually shipped
|
||
# in the image. Guards a silent regression back to the bare `:` no-op, which
|
||
# left a container contributing nothing to the palace with no artifact saying
|
||
# why — the log it would normally leave is written by the other branch.
|
||
run_expect "remote-palace-without-inbox skip is announced, not silent" \
|
||
"grep -o 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | head -1" \
|
||
"MemPalace catch-up skipped"
|
||
run "...and the skip notice names the variable that fixes it" \
|
||
"grep -A6 'MemPalace catch-up skipped' /usr/local/bin/entrypoint-user.sh | grep -q 'MEMPALACE_PI_SSH_TARGET'"
|
||
# A remote mine that FAILS must not report success. MCP answers a hard tool
|
||
# failure with HTTP 200 and the tool's own JSON escaped inside
|
||
# result.content[].text, so the feeder's old `'\"error\"' in body` check could
|
||
# never see it: on 2026-08-15 a mine that died with "source directory not found:
|
||
# '/data/feed/...'" logged "Done. Wing updated." and exited 0, and this
|
||
# container's transcripts were filed nowhere for a whole session. The feeder
|
||
# carries fixtures for that exact body; run them against the baked toolkit so a
|
||
# stale/reverted toolkit ref can't reintroduce a silent feed.
|
||
run "baked feeder detects a failed remote mine (no silent false success)" \
|
||
"mempalace-pi-session --self-test"
|
||
# v1.0.0 base additions — verify presence and basic functionality.
|
||
run "pandoc" "pandoc --version"
|
||
run "typst" "typst --version"
|
||
run "pandoc+typst PDF engine" "printf '# hi\n' | pandoc --pdf-engine=typst -o /tmp/_smoke.pdf - && test -s /tmp/_smoke.pdf; rm -f /tmp/_smoke.pdf"
|
||
run "graphviz (dot)" "dot -V"
|
||
run "imagemagick" "magick --version"
|
||
run "yq (mikefarah v4)" "yq --version | grep -qE 'mikefarah.*version v4'"
|
||
run "tldr (tealdeer)" "tldr --version"
|
||
run "socat" "socat -V"
|
||
run "studio-expose helper" "test -x /usr/local/bin/studio-expose"
|
||
run "image-baked pi-devbox-environment skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md"
|
||
run "global-AGENTS append snippet present" \
|
||
"test -f /usr/local/share/pi-devbox/pi-global-AGENTS.append.md"
|
||
run "pi-devbox block merged into pi-global-AGENTS.md" \
|
||
"grep -q 'pi-devbox:managed-block' /opt/pi-toolkit/pi-global-AGENTS.md"
|
||
run "mempalace session-start pointer merged into global AGENTS.md" \
|
||
"grep -q 'load the mempalace skill' /opt/pi-toolkit/pi-global-AGENTS.md"
|
||
# Vendored fallback skills (so a no-skillset container still resolves the
|
||
# AGENTS.md 'read the pi-extensions skill' pointer).
|
||
run "image-baked pi-extensions fallback skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md"
|
||
run "pi-extensions skill ships its helper" \
|
||
"test -f /usr/local/share/pi-devbox/skills/pi-extensions/evaluate-extension-usage.py"
|
||
run "image-baked mempalace fallback skill" \
|
||
"test -f /usr/local/share/pi-devbox/skills/mempalace/SKILL.md"
|
||
# Layered freshness: when the pinned pi-extensions clone carries the skill, the
|
||
# baked copy must be the fresh package copy (Option 1), not the stale snapshot.
|
||
run "pi-extensions skill refreshed from package when present" \
|
||
"if [ -f /opt/pi-extensions/skill/SKILL.md ]; then cmp -s /opt/pi-extensions/skill/SKILL.md /usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md; else true; fi"
|
||
# Runtime ownership handover (v1.8.5): the baked links are a FALLBACK, and
|
||
# skillset-OWNED skills must be repointed at the live clone when one is mounted.
|
||
# The list is data, so assert its content, not just its presence: mempalace in,
|
||
# pi-extensions deliberately out (its skillset copy is a lagging duplicate).
|
||
run "devbox-skill-reconcile helper present + executable" \
|
||
"test -x /usr/local/bin/devbox-skill-reconcile"
|
||
run "skillset-owned list ships and names mempalace" \
|
||
"grep -qx 'mempalace' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
|
||
run "skillset-owned list excludes pi-extensions (ownership)" \
|
||
"! grep -qx 'pi-extensions' /usr/local/share/pi-devbox/skills/skillset-owned.txt"
|
||
|
||
# ── tmux 0-indexing (required for pi-studio variants) ─────────────────
|
||
echo ""
|
||
echo "── tmux config ──"
|
||
run_expect "/etc/tmux.conf has base-index 0" \
|
||
"cat /etc/tmux.conf" "set -g base-index 0"
|
||
run_expect "/etc/tmux.conf has pane-base-index 0" \
|
||
"cat /etc/tmux.conf" "set -g pane-base-index 0"
|
||
|
||
# ── Repo clones ───────────────────────────────────────────────────────
|
||
echo ""
|
||
echo "── Repo clones ──"
|
||
run "pi-toolkit clone" "test -d /opt/pi-toolkit && git -C /opt/pi-toolkit rev-parse --short HEAD"
|
||
run "pi-extensions clone" "test -d /opt/pi-extensions && git -C /opt/pi-extensions rev-parse --short HEAD"
|
||
run "pi-fork clone + node_modules" \
|
||
"test -f /opt/pi-fork/package.json && test -d /opt/pi-fork/node_modules"
|
||
run "pi-observational-memory clone + node_modules" \
|
||
"test -f /opt/pi-observational-memory/package.json && test -d /opt/pi-observational-memory/node_modules"
|
||
# pi-atelier: deliberately NO node_modules assertion, unlike its siblings —
|
||
# it declares zero runtime dependencies (only peerDeps, satisfied by the baked
|
||
# pi) and has no build step, so Dockerfile.variant skips `npm install` for it.
|
||
# Assert what pi actually loads instead: the entry point named by its
|
||
# package.json `pi.extensions` key.
|
||
run "pi-atelier clone + entry point" \
|
||
"test -f /opt/pi-atelier/package.json && test -f /opt/pi-atelier/extensions/index.ts"
|
||
|
||
# ── pi <-> pi-atelier compatibility floor (executable, not a comment) ──
|
||
# pi-atelier < 0.7.1 wraps pi's PRIVATE TUI renderer in a way that recurses
|
||
# under pi >= 0.84: pi hangs at startup burning CPU, with no error. Upstream
|
||
# fixed it in 0.7.1/0.7.2, but atelier's peerDependencies still say
|
||
# `>=0.80.7`, so neither npm nor pi can warn about the real floor. Both
|
||
# versions are pinned in Dockerfile.variant; this makes a bad PAIRING fail the
|
||
# build instead of publishing an image whose TUI never starts.
|
||
run_expect "pi-atelier >= 0.7.1 floor for pi >= 0.84 (startup-hang guard)" \
|
||
'ge() { [ "$(printf "%s\n%s\n" "$1" "$2" | sort -V | head -n1)" = "$2" ]; }; AV=$(jq -r ".version // empty" /opt/pi-atelier/package.json 2>/dev/null); PV=$(pi --version 2>/dev/null | grep -oE "[0-9]+\.[0-9]+\.[0-9]+" | head -n1); if [ -z "$AV" ] || [ -z "$PV" ]; then echo "unreadable versions (atelier=$AV pi=$PV)"; elif ge "$PV" 0.84.0 && ! ge "$AV" 0.7.1; then echo "VIOLATION: pi $PV with pi-atelier $AV"; else echo "compatible: pi $PV + pi-atelier $AV"; fi' \
|
||
"compatible:"
|
||
|
||
# pi-studio is present only in the :latest-studio variant. Auto-detect by
|
||
# probing /opt/pi-studio so this one script covers both variants.
|
||
if docker run --rm --entrypoint="" "$IMAGE" sh -c 'test -d /opt/pi-studio' >/dev/null 2>&1; then
|
||
STUDIO_VARIANT=1
|
||
echo " ℹ️ pi-studio detected — running studio assertions"
|
||
run "pi-studio clone + node_modules" \
|
||
"test -f /opt/pi-studio/package.json && test -d /opt/pi-studio/node_modules"
|
||
run "pi-studio prebuilt client bundle" \
|
||
"test -f /opt/pi-studio/client/studio-client.js"
|
||
else
|
||
STUDIO_VARIANT=0
|
||
echo " ℹ️ pi-studio not present (non-studio variant) — skipping studio clone checks"
|
||
fi
|
||
|
||
# ── Build provenance (manifest + OCI labels) ─────────────────────────
|
||
echo ""
|
||
echo "── Build provenance ──"
|
||
run "/etc/pi-devbox/build-manifest.json present" \
|
||
"test -f /etc/pi-devbox/build-manifest.json"
|
||
run_expect "manifest records pi-extensions component" \
|
||
"cat /etc/pi-devbox/build-manifest.json" '"pi-extensions"'
|
||
run_expect "manifest records pi-atelier" \
|
||
"cat /etc/pi-devbox/build-manifest.json" '"pi-atelier"'
|
||
run_expect "manifest records pi_version" \
|
||
"cat /etc/pi-devbox/build-manifest.json" '"pi_version"'
|
||
# Every component must be a resolved commit (or null for pi-studio in the
|
||
# non-studio variant) — 'unknown' means a clone silently failed to resolve.
|
||
run "manifest has no unresolved ('unknown') components" \
|
||
"! grep -q '\"unknown\"' /etc/pi-devbox/build-manifest.json"
|
||
# pi-devbox-version wraps the manifest into a human-first command (this
|
||
# PR); verify the binary is present, executable, and both output modes work.
|
||
run "pi-devbox-version binary present + executable" \
|
||
"test -x /usr/local/bin/pi-devbox-version"
|
||
run_expect "pi-devbox-version human output shows release tag" \
|
||
"pi-devbox-version" "pi-devbox "
|
||
run_expect "pi-devbox-version --json round-trips the manifest" \
|
||
"pi-devbox-version --json" '"release_tag"'
|
||
run_expect "pi-devbox-version --quiet is a compact one-liner" \
|
||
"pi-devbox-version --quiet | wc -l" "1"
|
||
# OCI labels live in the image config, not the container fs — inspect them
|
||
# from the host docker rather than via `docker run`.
|
||
LBL=$(docker inspect --format '{{ index .Config.Labels "se.jordbo.pi-devbox.pi-extensions-ref" }}' "$IMAGE" 2>/dev/null || true)
|
||
if [ -n "$LBL" ] && [ "$LBL" != "<no value>" ]; then
|
||
printf " ✅ OCI label se.jordbo.pi-devbox.pi-extensions-ref=%s\n" "$LBL"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ OCI label se.jordbo.pi-devbox.pi-extensions-ref missing or empty\n"; FAIL=$((FAIL+1))
|
||
fi
|
||
|
||
# ── Runtime deployment (needs entrypoint to run) ──────────────────────
|
||
echo ""
|
||
echo "── Runtime deployment ──"
|
||
# Spin up a long-running container WITHOUT overriding the entrypoint, so
|
||
# the baked entrypoint chain (entrypoint.sh → entrypoint-user.sh) runs and
|
||
# deploys pi-toolkit + pi-extensions to ~/.pi/agent/. Override CMD to
|
||
# tail -f /dev/null so the container stays alive while we docker-exec.
|
||
CID=$(docker run -d --rm "$IMAGE" tail -f /dev/null)
|
||
cleanup() { docker rm -f "$CID" >/dev/null 2>&1 || true; }
|
||
trap cleanup EXIT
|
||
|
||
# Wait for entrypoint-user.sh to finish deploying pi-toolkit + extensions.
|
||
# Gate on BOTH the keybindings symlink (deployed by pi-toolkit) AND the
|
||
# mempalace.ts bridge (deployed last by entrypoint-user.sh) AND ≥4 *.ts
|
||
# extensions present. Parallel build load can otherwise sample the *.ts
|
||
# count mid-deploy and produce a flake. See opencode-devbox c6f9d11
|
||
# (2026-06-08) — same fix transplanted.
|
||
for i in $(seq 1 45); do
|
||
if docker exec "$CID" sh -c '
|
||
test -L /home/developer/.pi/agent/keybindings.json && \
|
||
test -L /home/developer/.pi/agent/extensions/mempalace.ts && \
|
||
test -L /home/developer/.agents/skills/pi-devbox-environment && \
|
||
test -L /home/developer/.agents/skills/pi-extensions && \
|
||
test -L /home/developer/.agents/skills/mempalace && \
|
||
count=$(ls -1 /home/developer/.pi/agent/extensions/*.ts 2>/dev/null | wc -l) && \
|
||
[ "$count" -ge 4 ]
|
||
' >/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
|
||
exec_test() {
|
||
local label="$1"; local cmd="$2"
|
||
if docker exec -u developer "$CID" sh -c "$cmd" >/dev/null 2>&1; then
|
||
printf " ✅ %s\n" "$label"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ %s\n" "$label"; FAIL=$((FAIL+1))
|
||
fi
|
||
}
|
||
|
||
exec_test "keybindings.json (pi-toolkit)" 'test -L $HOME/.pi/agent/keybindings.json && echo ok'
|
||
exec_test "extensions ≥ 4 (pi-extensions)" 'count=$(ls -1 $HOME/.pi/agent/extensions/*.ts 2>/dev/null | wc -l); [ $count -ge 4 ] && echo "$count extensions"'
|
||
exec_test "mempalace.ts bridge" 'test -L $HOME/.pi/agent/extensions/mempalace.ts && echo ok'
|
||
exec_test "settings.json bootstrapped" 'test -f $HOME/.pi/agent/settings.json && echo ok'
|
||
exec_test "pi-devbox-environment skill linked" 'test -L $HOME/.agents/skills/pi-devbox-environment && test -f $HOME/.agents/skills/pi-devbox-environment/SKILL.md && echo ok'
|
||
exec_test "pi-extensions skill linked (fallback)" 'test -L $HOME/.agents/skills/pi-extensions && test -f $HOME/.agents/skills/pi-extensions/SKILL.md && echo ok'
|
||
exec_test "mempalace skill linked (fallback)" 'test -L $HOME/.agents/skills/mempalace && test -f $HOME/.agents/skills/mempalace/SKILL.md && echo ok'
|
||
# The vendored mempalace snapshot is refreshed MANUALLY per release (see
|
||
# rootfs/usr/local/share/pi-devbox/skills/VENDORED.md). Through v1.8.4 it also
|
||
# silently SHADOWED the live skillset copy, so staleness was invisible — and the
|
||
# canary that was supposed to catch it could not: it grepped "Shared palace:
|
||
# multiple harnesses", a phrase present in BOTH the stale and the fresh copy.
|
||
# A snapshot canary must pin the NEWEST section, so update this string whenever
|
||
# the snapshot is refreshed — that is the point of it.
|
||
exec_test "mempalace skill snapshot is current" 'grep -q "Attribute what you file yourself" $HOME/.agents/skills/mempalace/SKILL.md && echo ok'
|
||
# Link TARGETS, not just link existence: with no skillset mounted (as here) the
|
||
# baked tree must be what resolves, for all three vendored skills.
|
||
exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
|
||
'for s in mempalace pi-extensions pi-devbox-environment; do
|
||
case "$(readlink -f $HOME/.agents/skills/$s)" in
|
||
/usr/local/share/pi-devbox/skills/$s) ;;
|
||
*) echo "$s resolves to $(readlink -f $HOME/.agents/skills/$s)" >&2; exit 1 ;;
|
||
esac
|
||
done; echo ok'
|
||
# The handover path itself. CI never mounts a skillset, so without this the
|
||
# v1.8.5 fix would ship untested: fabricate a skillset + a skills dir holding
|
||
# baked-style links, run the reconciler, and assert all three outcomes —
|
||
# owned skill repointed, unowned skill left baked, user override untouched.
|
||
exec_test "reconciler: owned skill handed to live clone, others untouched" \
|
||
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/ss/skills/pi-extensions $t/skills
|
||
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo LIVE > $t/ss/skills/pi-extensions/SKILL.md
|
||
ln -s /usr/local/share/pi-devbox/skills/mempalace $t/skills/mempalace
|
||
ln -s /usr/local/share/pi-devbox/skills/pi-extensions $t/skills/pi-extensions
|
||
mkdir -p $t/skills/mine; echo MINE > $t/skills/mine/SKILL.md
|
||
devbox-skill-reconcile $t/ss $t/skills >/dev/null
|
||
devbox-skill-reconcile $t/ss $t/skills >/dev/null # idempotent
|
||
[ "$(readlink $t/skills/mempalace)" = "$t/ss/skills/mempalace" ] || { echo "owned skill NOT repointed" >&2; exit 1; }
|
||
[ "$(readlink $t/skills/pi-extensions)" = /usr/local/share/pi-devbox/skills/pi-extensions ] || { echo "unowned skill was repointed" >&2; exit 1; }
|
||
[ "$(cat $t/skills/mine/SKILL.md)" = MINE ] || { echo "user override clobbered" >&2; exit 1; }
|
||
rm -rf $t; echo ok'
|
||
# The case above cannot fail if the reconciler stops checking WHERE a link
|
||
# points — a mutation test showed all three of its assertions still passing with
|
||
# that guard deleted, which is the same false-green shape as the old snapshot
|
||
# canary. This one discriminates: an OWNED name (so it is considered) whose link
|
||
# is a user override pointing outside the baked tree (so it must be left alone).
|
||
exec_test "reconciler: user override on an owned name is left alone" \
|
||
'set -e; t=$(mktemp -d); mkdir -p $t/ss/skills/mempalace $t/skills $t/mine-skill
|
||
echo LIVE > $t/ss/skills/mempalace/SKILL.md; echo USERLINK > $t/mine-skill/SKILL.md
|
||
ln -sfn $t/mine-skill $t/skills/mempalace
|
||
devbox-skill-reconcile $t/ss $t/skills >/dev/null
|
||
[ "$(cat $t/skills/mempalace/SKILL.md)" = USERLINK ] || { echo "user symlink override clobbered" >&2; exit 1; }
|
||
rm -rf $t; echo ok'
|
||
# mempalace-census gained a /usr/local/bin symlink in v1.8.3; its three siblings
|
||
# had one since they were added, so this asserts the set stays complete.
|
||
exec_test "mempalace-census on PATH" 'command -v mempalace-census >/dev/null && mempalace-census --help >/dev/null && echo ok'
|
||
|
||
# pi-fork + pi-observational-memory are registered by entrypoint-user.sh via
|
||
# `pi install /opt/<pkg>`, which runs slightly after the keybindings marker.
|
||
#
|
||
# Assert against the `packages` ARRAY, never a whole-file grep: the settings
|
||
# template ships a top-level "pi-fork" CONFIG block, so `grep -q pi-fork
|
||
# settings.json` passes even when `pi install /opt/pi-fork` never ran. That
|
||
# false green is exactly why the missing `fork` tool shipped unnoticed from
|
||
# v1.0.0 through v1.6.3.
|
||
pkg_registered_cmd() {
|
||
printf "jq -e --arg n %s '(.packages // []) | any((type == \"string\") and (. == \"npm:\" + \$n or endswith(\"/\" + \$n)))' \$HOME/.pi/agent/settings.json" "$1"
|
||
}
|
||
|
||
for i in $(seq 1 15); do
|
||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-observational-memory)" \
|
||
>/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
exec_test "pi-fork registered in packages[] (fork tool)" \
|
||
"$(pkg_registered_cmd pi-fork)"
|
||
exec_test "pi-observational-memory registered in packages[] (recall tool)" \
|
||
"$(pkg_registered_cmd pi-observational-memory)"
|
||
|
||
# pi-studio registration (studio variant only) — registered by the same
|
||
# entrypoint-user.sh local-path install loop as fork/obsmem.
|
||
if [ "${STUDIO_VARIANT:-0}" = "1" ]; then
|
||
for i in $(seq 1 15); do
|
||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-studio)" \
|
||
>/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
exec_test "pi-studio registered in packages[] (/studio command + studio_* tools)" \
|
||
"$(pkg_registered_cmd pi-studio)"
|
||
fi
|
||
|
||
# pi-atelier registration. It is LAST in the entrypoint's install loop, so a
|
||
# pass here also means that loop ran to completion rather than dying midway.
|
||
for i in $(seq 1 15); do
|
||
if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-atelier)" \
|
||
>/dev/null 2>&1; then
|
||
break
|
||
fi
|
||
sleep 1
|
||
done
|
||
exec_test "pi-atelier registered in packages[] (TUI sidebar)" \
|
||
"$(pkg_registered_cmd pi-atelier)"
|
||
# ...and registered from the vendored /opt copy, NOT as `npm:pi-atelier`: an
|
||
# npm: entry resolves through ~/.pi/npm-global on the config VOLUME, which
|
||
# outlives image upgrades and would silently keep an old, unaudited atelier —
|
||
# exactly the shape that pairs a stale 0.6.x with a new pi and hangs at startup.
|
||
exec_test "pi-atelier registered from /opt, not npm: (volume-shadowing guard)" \
|
||
'jq -e "((.packages // []) | any((type == \"string\") and endswith(\"/pi-atelier\"))) and (((.packages // []) | any(. == \"npm:pi-atelier\")) | not)" $HOME/.pi/agent/settings.json'
|
||
|
||
# ── /tmp/sshcm directory created by entrypoint ────────────────────────
|
||
exec_test "/tmp/sshcm dir mode 700 (ssh ControlMaster)" \
|
||
'test -d /tmp/sshcm && [ "$(stat -c %a /tmp/sshcm)" = "700" ] && echo ok'
|
||
|
||
# ── Image size ────────────────────────────────────────────────────────
|
||
echo ""
|
||
echo "── Image size ──"
|
||
# Sum all layers via `docker history`. Docker's `image inspect --format='{{.Size}}'`
|
||
# returns ONLY the variant-unique layer when the base is content-addressed and
|
||
# shared (the case in this repo's two-phase build), which understates the
|
||
# user-facing image size by 2+ GB. Summing layer sizes from history is the
|
||
# metric Hub displays to users and the one we actually want to gate on.
|
||
SIZE_MB=$(docker history --format '{{.Size}}' "$IMAGE" | python3 -c '
|
||
import sys, re
|
||
total=0.0
|
||
for line in sys.stdin:
|
||
s=line.strip()
|
||
if s in ("0B", ""): continue
|
||
m=re.match(r"^([0-9.]+)(B|kB|MB|GB)$", s)
|
||
if not m: continue
|
||
v=float(m.group(1)); u=m.group(2)
|
||
mult={"B":1/1048576,"kB":1/1024,"MB":1,"GB":1024}[u]
|
||
total+=v*mult
|
||
print(int(total))
|
||
')
|
||
if [ -z "$SIZE_MB" ] || [ "$SIZE_MB" = "0" ]; then
|
||
printf " ⚠️ image size: could not parse — skipping check\n"
|
||
elif [ "$SIZE_MB" -le "$SIZE_THRESHOLD_MB" ]; then
|
||
printf " ✅ size: %d MB (threshold %d MB)\n" "$SIZE_MB" "$SIZE_THRESHOLD_MB"; PASS=$((PASS+1))
|
||
else
|
||
printf " ❌ size: %d MB exceeds threshold %d MB\n" "$SIZE_MB" "$SIZE_THRESHOLD_MB"; FAIL=$((FAIL+1))
|
||
fi
|
||
|
||
# ── Summary ───────────────────────────────────────────────────────────
|
||
echo ""
|
||
echo "=== Results: ${PASS} passed, ${FAIL} failed ==="
|
||
[ "$FAIL" -eq 0 ]
|