joakimp/pi-devbox

Fork 0

Files

T

joakimp e8ddeaf89f

Lint / hadolint (push) Successful in 9s

Details

Lint / actionlint (push) Successful in 1m39s

Details

skills: a gate that could pass without checking, and a mailbox that never empties

Fixes the three blockers and seven should-fixes from pi@emb-7kj4vr4g's review
(logstream correlation skills-provenance-review, full text in
drawer_pi-devbox_reviews_e43e766641c9ec85217bc6ce). Every finding was
reproduced by execution here before being fixed; two were refined by that
reproduction rather than taken as given.

BLOCKER 1 — the provenance gate could print OK and exit 0 without verifying.
`git show <ref>:<path> | sha256sum` hashes EMPTY STDIN when the ref does not
resolve, so at_ref was never empty and the UNKNOWN branch was dead code.
Measured: a bogus ref reported MISMATCH — accusing the snapshot of lying when
the real cause was an incomplete clone, and the operator's natural remedy for
MISMATCH is to re-run the refresh, which rewrites provenance to silence the
complaint; and with a 0-byte snapshot against a 0-byte upstream file it printed
"OK: exactly skillset@aaaaaaa" with exit 0 for a ref that does not exist. The
script already had the sha_empty idiom and had applied it to blob_sha but not
to at_ref. Existence is now PROVEN with git cat-file -e before anything is
hashed, at two levels (ref resolves / path exists at it) because those deserve
different messages. Same defect class as the canary it replaces: a check that
can succeed without checking. A second, unflagged instance of the same pipeline
shape in blob_sha was found and fixed too.

Exit codes split, because the old contract failed the sanctioned case: 0
truthful (including stale, with a NOTICE), 1 a lying record only, 2 cannot
determine. AGENTS.md step 2 promised "the message distinguishes the two" and
was the thing this branch was breaking; rewritten to state all three.

BLOCKER 3 — VENDORED.md contradicted itself in the release whose stated
invariant is non-contradiction: its hand-maintained provenance line named
skillset 670f7f1, seven commits behind the ARG and itself the commit that told
agents to hand-stamp added_by — the withdrawn instruction this work exists to
stop shipping — while its cp recipe contradicted the "not cp" rule 20 lines
above. Line removed (nothing forced it to move when the ARGs did); 670f7f1 kept
only as a labelled cautionary example. The pi-extensions half was verified
redundant (CI require_sha resolves PI_EXTENSIONS_REF) before removal.

SHOULD-FIXES: `<root> --check`, the spelling VENDORED.md documented, silently
ran a REFRESH because only $1 was parsed (both tools now parse all args and
reject unknown ones); refresh at a detached/older HEAD silently rewound ref and
bytes (now refused unless the recorded ref is an ancestor, --force to override);
upstream_dirty was computed and never used in check mode; --no-skills --json
printed human text and broke jq; --help was a hardcoded sed range this branch
had already made stale; the fingerprint hashed SKILL.md alone so a live skill
differing only in a sibling file reported "identical", and pi-extensions already
ships two files, so it is now a per-skill TREE hash with the manifest field
renamed skillset_snapshot_tree_sha256; the --no-skills smoke assertion was
negative-only and passed on a crashed binary. mktemp+mv left files 0600 — CI was
unaffected since the index records 100644, so the blast radius was local builds
only, narrower than the review inferred.

Snapshot resynced c04cd15 -> 5fd0d5c so the no-clone fallback carries the
CORRECTED coordination protocol rather than the withdrawn one; --check is now OK
with no staleness notice, and the bidirectional canary re-verified against the
new bytes. Local validation is bash -n only (shellcheck, hadolint and actionlint
are all absent in this container) — CI remains the shellcheck gate.

2026-08-26 14:38:07 +02:00

17 KiB

Raw Permalink Blame History

AGENTS.md — pi-devbox

Self-contained Docker image for the pi coding-agent. Decoupled from opencode-devbox at v1.0.0 (2026-06-09); previously pi-devbox was a thin re-brand of opencode-devbox's pi-only variant.

Repository layout

Dockerfile.base — multi-arch base layer with system packages, GitHub-binary tools (fzf, eza, zoxide, neovim, bat, gosu, gitleaks, git-lfs, uv, gitea-mcp, tealdeer), AWS CLI v2, mempalace + toolkit, Node.js, Python toolchain, locales, ssh ControlMaster defaults, and /etc/tmux.conf with 0-indexed sessions.
Dockerfile.variant — FROM base-<hash>, adds pi + companions (pi-toolkit, pi-extensions, pi-fork, pi-observational-memory) and, when INSTALL_STUDIO=true, vendors pi-studio to /opt/pi-studio (-studio variant). Also appends the pi-devbox managed block from pi-global-AGENTS.append.md onto pi-toolkit's pi-global-AGENTS.md (the single global instruction slot pi loads) so containers proactively load the baked pi-devbox-environment skill. Idempotent via a marker grep. After the pinned clones it also refreshes the vendored pi-extensions fallback skill by copying /opt/pi-extensions/skill/ over the committed rootfs/ snapshot (Option 1 over Option 2 — see skills/VENDORED.md).
entrypoint.sh — UID/GID alignment as root, then drops to developer.
entrypoint-user.sh — per-container start: prints the pi-devbox-version banner first (which build/commit is running, from the manifest below), then SSH ControlMaster socket dir, LAN-access setup, MemPalace init, pi-toolkit + pi-extensions deploy, mempalace-bridge symlink, fork/recall + pi-studio pi-install, optional studio-expose bridge (when STUDIO_EXPOSE=1), image-baked skills symlink-in, skillset deploy.
rootfs/ — files baked into the image (bash aliases, inputrc, setup-lan-access.sh, studio-expose helper, pi-devbox-version — wraps /etc/pi-devbox/build-manifest.json into a human-readable summary + live drift check, see README “Build provenance”). Also usr/local/share/pi-devbox/skills/<name>/SKILL.md — image-baked agent skills (the repo-authored pi-devbox-environment, plus vendored fallback copies of pi-extensions and mempalace — see skills/VENDORED.md) symlinked into ~/.agents/skills/ by the entrypoint, available with or without a mounted skillset — plus usr/local/share/pi-devbox/pi-global-AGENTS.append.md (the global-AGENTS pointer concatenated in Dockerfile.variant).
scripts/smoke-test.sh — sanity checks run by CI before pushing to Hub.
.gitea/workflows/docker-publish.yml — two-phase CI (base-decide → build-base → smoke → build-variant → promote-base-latest → update-description). The -studio variant adds independent smoke-studio + build-variant-studio jobs that gate only the -studio tags (never the core :latest release).

Versioning scheme

Tags follow semver. v1.0.0 is the first decoupled release; future minor bumps add variants (-studio, -studio-tex) or significant base additions (e.g. v1.2.0 image-baked agent skills); patch bumps follow pi npm version updates and small fixes.
Docker Hub tags: joakimp/pi-devbox:vX.Y.Z + joakimp/pi-devbox:latest
- (since v1.1.0) joakimp/pi-devbox:vX.Y.Z-studio + joakimp/pi-devbox:latest-studio. Internal tags: joakimp/pi-devbox:base-<hash> (content-addressed) + joakimp/pi-devbox:base-latest (alias of most recent base).

Release-day checklist

Confirm pi --version resolves from npm to the expected version (curl -sf 'https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/latest' | jq -r .version). Check release notes at https://github.com/earendil-works/pi/releases for the upstream changelog to include in CHANGELOG.md.
Refresh the vendored mempalace skill snapshot if the skillset moved: scripts/vendor-mempalace-skill.sh --check (reads a real skillset clone, writes nothing). Three exit codes, not two — a stale-but-truthful record is not a release blocker, so don't treat any non-zero exit as "must refresh" without reading which one it was:
- 0 — the record is truthful. This includes stale-but-truthful (upstream has moved past the recorded ref, or the local clone has uncommitted changes) — a NOTICE is printed, but nothing is lying. Skipping the refresh in this case is the legitimate, sanctioned outcome — every enrolled host reads its own live skillset clone, so the baked copy is only a no-mount fallback. What is not legitimate is skipping it silently: the drift is visible here, in pi-devbox-version, and in the manifest, so decide rather than forget.
- 1 — a confirmed problem: the vendored bytes provably do NOT match the file at the recorded ref (a lying record), or the recorded ref doesn't even resolve to that path in this clone. Refresh.
- 2 — cannot determine (the recorded ref itself isn't resolvable in this clone — commonly a shallow checkout missing history). Fetch full history and re-check before deciding; don't refresh blind. Refresh with scripts/vendor-mempalace-skill.sh, which rewrites the file and the ARG together so they cannot drift apart, and refuses (exit 1) rather than silently rewinding provenance if the skillset clone's HEAD is behind the already-recorded ref (detached HEAD, older checkout) — pass --force only if that rewind is genuinely intended. Two consequences to accept deliberately on an actual refresh: the snapshot is hashed into base_tag, so it costs a base rebuild (~67 min); and if the section the phrase canary names has changed, re-pin it in scripts/smoke-test.sh.
Update CHANGELOG.md Unreleased → vX.Y.Z section.
Verify docker compose up works locally with the current latest image if you're upgrading users from a previous version. Then run the post-recreate sanity check inside the running container to confirm persisted volumes survived and the pi runtime wiring re-deployed (not just that the container booted): docker compose exec devbox bash scripts/recreate-sanity-check.sh --expected-version X.Y.Z (or just pi-devbox-sanity --expected-version X.Y.Z if cli_utils/bin is on PATH). This is the runtime peer of the build-time smoke-test.sh gate.
Push tag: git tag vX.Y.Z && git push origin vX.Y.Z.
Watch CI: smoke job builds amd64 only and asserts size + extensions + pi version + new-base-tooling presence. Variant build is multi-arch (amd64 + arm64) only after smoke passes. A tag push fires only docker-publish.yml — lint.yml is scoped to branches: ['**'], which excludes tag refs on purpose (the tagged tree was already linted when the commit hit main, and a fast lint run sorting above the slow publish run made releases look finished before anything shipped). Verified on v1.8.4: refs/tags/v1.8.4 produced run 571 (publish) and nothing else. Still filter discovery on head_sha and the workflow path — see Gitea API access below — because that guard costs nothing and a future workflow added on v* would silently reintroduce the ambiguity.
Verify the Hub tags appear (latest + vX.Y.Z, the -studio pair, plus base-latest if the base was rebuilt this run).
Revoke any short-lived Gitea PAT used during the release at gitea.jordbo.se/user/settings/applications. N/A if you used the GITEA_ACCESS_TOKEN env var instead (see Gitea API access below) — its lifecycle is managed host-side, nothing to revoke.

Verifying this repo's reality from inside a container

Most work on this repo happens inside a pi-devbox container, inspecting a host or a peer over SSH. That setup manufactures convincing false negatives, so when you are about to report that something is absent, unreachable, or not running, suspect your own command first. Recurring instances:

docker is not on the host's non-interactive SSH PATH. ssh mac 'docker ps' says command not found on a host that plainly runs Docker; use /usr/local/bin/docker (or command -v docker first). Every step in the Release-day checklist that inspects a running container hits this.
Don't | head -N a search whose answer you don't already know. The host's ~/.ssh/config is ~500 lines; a head -20 "proved" a peer absent that was defined at line 454.
The deployment compose file is not this repo's. docker-compose.yml here is a template pinning :latest; a real host runs its own per-machine file (find it with docker inspect <container> --format '{{ index .Config.Labels "com.docker.compose.project.config_files" }}'). Recreating from the repo copy can silently move a host off :latest-studio onto :latest.
A live SSH ControlMaster hides remote auth changes — after editing a peer's authorized_keys, prove access with -o ControlPath=none -o ControlMaster=no, or the breakage surfaces in a later session instead.

Depth and further mechanisms: the repo-authored pi-devbox-environment skill (rootfs/usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md) §2 and §3 — that file is the one an agent actually loads mid-session, whereas this AGENTS.md is only auto-read when the cwd is this repo.

Gitea API access (env token)

GITEA_ACCESS_TOKEN + GITEA_HOST are passed into the container from the host .env via docker-compose.yml (${GITEA_ACCESS_TOKEN:-} / ${GITEA_HOST:-}), primarily to enable the gitea-mcp server. They are not baked into the image. When configured, they are also available for any direct Gitea API interaction from inside the container — inspecting CI runs, checking published tags, listing commits — e.g. curl -H "Authorization: token $GITEA_ACCESS_TOKEN" "$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs?limit=20". Prefer this over a short-lived PAT file when the env token is present (the ci-release-watcher skill auto-detects it). Public-repo GET listings work unauthenticated too, so the token matters mainly for private repos or rate-limit headroom; its lifecycle is host-managed, so there is nothing to revoke after use. Never echo the token value (including into logs).

Gotcha — a tag push fires EVERY workflow whose triggers match the tag ref. lint.yml uses a bare push: trigger, so a release tag yields both a lint run and the publish run. The listing is newest-first and lint sorts above the publish run, so "take the first run whose path contains refs/tags/<tag>" picks the wrong one reliably, not occasionally. Real listing for v1.6.4:

id=531  #104  lint.yml@refs/tags/v1.6.4          <- wrong; sorts first
id=530  #103  docker-publish.yml@refs/tags/v1.6.4 <- the release build
id=529  #102  lint.yml@refs/heads/main            <- same commit, linted on push

Lint goes green in minutes while the image is still building, so watching it makes a release look finished when nothing has been published yet.

Gotcha — the jobs endpoint takes the internal id, NOT the run_number the UI shows as #104. The two diverge widely, and GET .../actions/runs/<run_number>/jobs does not error — it silently returns a different run's jobs. Always read id from the run listing:

# Which runs did this tag/commit trigger? Filter on head_sha; never trust
# ordering or run numbering. limit=20, not 5 — with two runs per push the
# publish run falls off a 5-item window fast.
curl -sS -H "Authorization: token $GITEA_ACCESS_TOKEN" \
  "$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs?limit=20" \
  | jq --arg sha "$(git rev-list -n1 vX.Y.Z)" \
      '.workflow_runs[] | select(.head_sha==$sha) | {id, run_number, path, status, conclusion}'
# pick the id whose .path starts with docker-publish.yml, then:
curl -sS -H "Authorization: token $GITEA_ACCESS_TOKEN" \
  "$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs/<id>/jobs" \
  | jq '.jobs[] | {name, status, conclusion}'

Watcher config for this repo (ci-release-watcher skill, hub-only shape — pi-devbox has no downstream host to deploy to):

EXPECT_WORKFLOW=docker-publish.yml — the skill's preflight_run() aborts at startup if the run id belongs to lint instead.
EXPECTED_FRESH_TAGS='vX.Y.Z latest vX.Y.Z-studio latest-studio'
EXPECTED_EXISTS_TAGS='base-latest' — existence only: it is content-addressed and legitimately keeps its old timestamp when the base is a cache hit.
CRITICAL_JOBS='build-variant build-variant-studio' — job names are matched exactly (critical.issubset(succeeded)), so the studio variant must be listed explicitly; the skill's default omits it. Leave promote-base-latest out: it legitimately skips on a base cache hit, which would misclassify a good run. update-description is the cosmetic post-publish job.

Cache-hit footgun (must-know)

PI_VERSION defaults to latest in Dockerfile.variant but CI must resolve it to a concrete version string before passing as a build-arg. Otherwise the build-arg string is byte-identical across releases → identical layer hash → registry buildcache silently reuses the old layer. resolve-versions job in the workflow handles this.

Discovered in pi-devbox 2026-05-23 (every release v0.74.0..v0.75.5 shipped the same image bytes); preventatively fixed for PI_VERSION + PI_FORK_REF + PI_OBSMEM_REF.

Smoke-test gate

scripts/smoke-test.sh runs amd64-only against a freshly-built variant image. Verifies binaries, repo clones, runtime deployment (waits for keybindings + mempalace bridge + ≥4 extensions before sampling — fixes the parallel-build-load race documented in opencode-devbox c6f9d11 2026-06-08), and image size threshold (3500 MB; revisit after a few releases as actuals settle).

If smoke fails on size threshold but build is otherwise fine: bump SIZE_THRESHOLD_MB in scripts/smoke-test.sh in a follow-up commit and re-run. The threshold exists to catch runaway growth (an accidental texlive bake-in, a forgotten chrome dependency), not to block ordinary upstream bumps.

Build pipeline notes

Two-phase: base + variant. Base is rebuilt only when Dockerfile.base, rootfs/, or entrypoint*.sh change (CI computes a content hash and probes Hub for an existing base-<hash> tag).
base-latest alias is promoted from base-<hash> via crane copy (manifest copy, no rebuild) only when the base actually changed.
docker buildx build --push retry: 3 attempts with backoff for transient Hub blips. Deterministic failures fail all 3 and the job fails as expected.
Registry buildcache disabled: buildkit's cache-export hits HTTP 400 on Hub CDN since ~2026-05-23. Image push works fine; we pay the full base build on Dockerfile.base change, but base tags are content- addressed so unchanged bases short-circuit at the probe step.

Decoupling history (briefly)

Pre-v1.0.0 pi-devbox was FROM joakimp/pi-devbox:base-pi-only, where base-pi-only was a tag built by opencode-devbox CI (with INSTALL_OPENCODE=false in their variant Dockerfile) and pushed under the pi-devbox repo as an internal building-block tag. This setup required rebuilding opencode-devbox before pi-devbox could be tagged and meant pi-devbox docs needed cross-referencing into opencode-devbox.

v1.0.0 brings pi install logic into this repo, drops the cross-repo dependency, and the base-pi-only* tags from opencode-devbox become deprecated artifacts (to be removed in opencode-devbox v2.0.0).

What we DON'T install (and why)

No texlive (~600 MB–1 GB). PDF export from pandoc / pi-studio works out of the box via typst (~30 MB static binary), which the base ships as the pandoc PDF engine (pandoc --pdf-engine=typst) — small enough to live in base rather than a dedicated :latest-studio-tex variant. We don't bake in a full TeX Live: it's heavy and typst covers the common Markdown→PDF case. Users needing LaTeX-exact output can install the higher-fidelity fallback on demand: sudo apt-get install texlive-xetex texlive-latex-recommended (then pandoc --pdf-engine=xelatex).
pi-studio ships in the :latest-studio variant (since v1.1.0), vendored to /opt/pi-studio and registered at container start via pi install /opt/pi-studio (see Dockerfile.variant INSTALL_STUDIO). The default :latest image stays studio-free. Note: pi-studio binds 127.0.0.1 inside the container, so browser access needs host networking or the bundled studio-expose bridge (socat; auto-starts when STUDIO_EXPOSE=1) — see README "Using pi-studio".
No Julia/R/GHCi/Clojure runtimes. Use uv run --with X for Python REPLs; apt install other-language runtimes ad-hoc per container if needed.

Backward compatibility

The host ~/.mempalace bind-mount path is unchanged.
Volume names (devbox-pi-config, devbox-ssh-local, devbox-shell-history, devbox-zoxide, devbox-nvim-data, devbox-uv; optional devbox-palace, devbox-chroma-cache) are unchanged.
~/.pi/agent/ layout inside the container is unchanged; existing named volumes work without recreation.
The :latest and vX.Y.Z Hub tags continue to point at a "base + pi" image. Same tag, same shape, just built differently.

17 KiB Raw Permalink Blame History Unescape Escape