Joakim Persson
fbc1f86612
Three false negatives in one session, all self-inflicted, all convincing because the command "succeeded": a `| head -20` proved an SSH peer absent that sits at line 454 of a ~500-line config; `ssh mac 'docker ps'` proved the host had no Docker, when the non-interactive PATH simply lacks /usr/local/bin; and `grep 'ssh '` proved no ControlMaster was running, when those processes rename themselves to `ssh: <path> [mux]`. Same root cause each time, so it goes in the skill rather than in a commit message: a positive result carries its own evidence, absence has to be earned. The skill (rootfs/, symlinked into ~/.agents/skills) is BAKED, so this is an image change and is logged in CHANGELOG Unreleased accordingly. Its §3 also now records that a live ControlMaster socket makes later commands authenticate not at all -- after editing a peer's authorized_keys, "it still works" proves nothing; prove it with -o ControlPath=none, or the breakage waits for a future session that has no memory of the edit. AGENTS.md: corrected a stale CI claim while placing the pointer. It said a tag push produces two runs including lint; lint.yml has since been scoped to branches: ['**'], which excludes tag refs, and refs/tags/v1.8.4 duly produced run 571 (publish) and nothing else. Kept the head_sha + workflow path filter advice, which is cheap and guards against a future v*-triggered workflow. Added a short section on verifying this repo from inside a container, including that docker-compose.yml here is a TEMPLATE pinning :latest while a real host runs its own per-machine file -- recreating from the repo copy can silently move a host off :latest-studio. Placement note: AGENTS.md is only auto-read when the cwd is this repo, so the durable rule lives in the skill, which loads by description match in any pi-devbox session.
273 lines
15 KiB
Markdown
273 lines
15 KiB
Markdown
# AGENTS.md — pi-devbox
|
||
|
||
Self-contained Docker image for the **pi coding-agent**. Decoupled from
|
||
opencode-devbox at v1.0.0 (2026-06-09); previously pi-devbox was a thin
|
||
re-brand of opencode-devbox's `pi-only` variant.
|
||
|
||
## Repository layout
|
||
|
||
- `Dockerfile.base` — multi-arch base layer with system packages,
|
||
GitHub-binary tools (fzf, eza, zoxide, neovim, bat, gosu, gitleaks,
|
||
git-lfs, uv, gitea-mcp, tealdeer), AWS CLI v2, mempalace + toolkit,
|
||
Node.js, Python toolchain, locales, ssh ControlMaster defaults, and
|
||
`/etc/tmux.conf` with 0-indexed sessions.
|
||
- `Dockerfile.variant` — `FROM base-<hash>`, adds pi + companions
|
||
(`pi-toolkit`, `pi-extensions`, `pi-fork`, `pi-observational-memory`)
|
||
and, when `INSTALL_STUDIO=true`, vendors `pi-studio` to `/opt/pi-studio`
|
||
(`-studio` variant). Also appends the pi-devbox managed block from
|
||
`pi-global-AGENTS.append.md` onto pi-toolkit's `pi-global-AGENTS.md` (the
|
||
single global instruction slot pi loads) so containers proactively load the
|
||
baked `pi-devbox-environment` skill. Idempotent via a marker grep. After the
|
||
pinned clones it also refreshes the vendored `pi-extensions` fallback skill
|
||
by copying `/opt/pi-extensions/skill/` over the committed `rootfs/` snapshot
|
||
(Option 1 over Option 2 — see `skills/VENDORED.md`).
|
||
- `entrypoint.sh` — UID/GID alignment as root, then drops to `developer`.
|
||
- `entrypoint-user.sh` — per-container start: prints the `pi-devbox-version`
|
||
banner first (which build/commit is running, from the manifest below),
|
||
then SSH ControlMaster socket dir, LAN-access setup, MemPalace init,
|
||
pi-toolkit + pi-extensions deploy, mempalace-bridge symlink, fork/recall +
|
||
pi-studio pi-install, optional `studio-expose` bridge (when
|
||
`STUDIO_EXPOSE=1`), image-baked skills symlink-in, skillset deploy.
|
||
- `rootfs/` — files baked into the image (bash aliases, inputrc,
|
||
setup-lan-access.sh, `studio-expose` helper, `pi-devbox-version` — wraps
|
||
`/etc/pi-devbox/build-manifest.json` into a human-readable summary + live
|
||
drift check, see README “Build provenance”). Also
|
||
`usr/local/share/pi-devbox/skills/<name>/SKILL.md` — image-baked agent
|
||
skills (the repo-authored `pi-devbox-environment`, plus vendored fallback
|
||
copies of `pi-extensions` and `mempalace` — see `skills/VENDORED.md`)
|
||
symlinked into `~/.agents/skills/` by the entrypoint, available with or
|
||
without a mounted skillset — plus
|
||
`usr/local/share/pi-devbox/pi-global-AGENTS.append.md` (the global-AGENTS
|
||
pointer concatenated in `Dockerfile.variant`).
|
||
- `scripts/smoke-test.sh` — sanity checks run by CI before pushing to Hub.
|
||
- `.gitea/workflows/docker-publish.yml` — two-phase CI (base-decide →
|
||
build-base → smoke → build-variant → promote-base-latest →
|
||
update-description). The `-studio` variant adds independent
|
||
`smoke-studio` + `build-variant-studio` jobs that gate only the
|
||
`-studio` tags (never the core `:latest` release).
|
||
|
||
## Versioning scheme
|
||
|
||
- Tags follow semver. **v1.0.0** is the first decoupled release; future
|
||
minor bumps add variants (`-studio`, `-studio-tex`) or significant base
|
||
additions (e.g. v1.2.0 image-baked agent skills); patch bumps follow
|
||
pi npm version updates and small fixes.
|
||
- Docker Hub tags: `joakimp/pi-devbox:vX.Y.Z` + `joakimp/pi-devbox:latest`
|
||
+ (since v1.1.0) `joakimp/pi-devbox:vX.Y.Z-studio` +
|
||
`joakimp/pi-devbox:latest-studio`.
|
||
Internal tags: `joakimp/pi-devbox:base-<hash>` (content-addressed) +
|
||
`joakimp/pi-devbox:base-latest` (alias of most recent base).
|
||
|
||
## Release-day checklist
|
||
|
||
1. Confirm `pi --version` resolves from npm to the expected version
|
||
(`curl -sf 'https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/latest' | jq -r .version`).
|
||
Check release notes at https://github.com/earendil-works/pi/releases for
|
||
the upstream changelog to include in `CHANGELOG.md`.
|
||
2. Update `CHANGELOG.md` Unreleased → vX.Y.Z section.
|
||
3. Verify `docker compose up` works locally with the current `latest` image
|
||
if you're upgrading users from a previous version. Then run the
|
||
**post-recreate sanity check** inside the running container to confirm
|
||
persisted volumes survived and the pi runtime wiring re-deployed (not just
|
||
that the container booted):
|
||
`docker compose exec devbox bash scripts/recreate-sanity-check.sh --expected-version X.Y.Z`
|
||
(or just `pi-devbox-sanity --expected-version X.Y.Z` if `cli_utils/bin` is
|
||
on PATH). This is the runtime peer of the build-time `smoke-test.sh` gate.
|
||
4. Push tag: `git tag vX.Y.Z && git push origin vX.Y.Z`.
|
||
5. Watch CI: smoke job builds amd64 only and asserts size + extensions +
|
||
pi version + new-base-tooling presence. Variant build is multi-arch
|
||
(amd64 + arm64) only after smoke passes. A tag push fires **only**
|
||
`docker-publish.yml` — `lint.yml` is scoped to `branches: ['**']`, which
|
||
excludes tag refs on purpose (the tagged tree was already linted when the
|
||
commit hit `main`, and a fast lint run sorting above the slow publish run
|
||
made releases look finished before anything shipped). Verified on v1.8.4:
|
||
`refs/tags/v1.8.4` produced run 571 (publish) and nothing else. Still filter
|
||
discovery on `head_sha` **and** the workflow `path` — see *Gitea API access*
|
||
below — because that guard costs nothing and a future workflow added on `v*`
|
||
would silently reintroduce the ambiguity.
|
||
6. Verify the Hub tags appear (latest + vX.Y.Z, the `-studio` pair, plus
|
||
base-latest if the base was rebuilt this run).
|
||
7. **Revoke any short-lived Gitea PAT** used during the release at
|
||
`gitea.jordbo.se/user/settings/applications`. N/A if you used the
|
||
`GITEA_ACCESS_TOKEN` env var instead (see *Gitea API access* below) —
|
||
its lifecycle is managed host-side, nothing to revoke.
|
||
|
||
## Verifying this repo's reality from inside a container
|
||
|
||
Most work on this repo happens **inside** a pi-devbox container, inspecting a
|
||
host or a peer over SSH. That setup manufactures convincing false negatives, so
|
||
when you are about to report that something is **absent, unreachable, or not
|
||
running**, suspect your own command first. Recurring instances:
|
||
|
||
- **`docker` is not on the host's non-interactive SSH `PATH`.** `ssh mac 'docker
|
||
ps'` says *command not found* on a host that plainly runs Docker; use
|
||
`/usr/local/bin/docker` (or `command -v docker` first). Every step in the
|
||
*Release-day checklist* that inspects a running container hits this.
|
||
- **Don't `| head -N` a search whose answer you don't already know.** The host's
|
||
`~/.ssh/config` is ~500 lines; a `head -20` "proved" a peer absent that was
|
||
defined at line 454.
|
||
- **The deployment compose file is not this repo's.** `docker-compose.yml` here
|
||
is a template pinning `:latest`; a real host runs its own per-machine file
|
||
(find it with `docker inspect <container> --format '{{ index .Config.Labels
|
||
"com.docker.compose.project.config_files" }}'`). Recreating from the repo copy
|
||
can silently move a host off `:latest-studio` onto `:latest`.
|
||
- **A live SSH ControlMaster hides remote auth changes** — after editing a
|
||
peer's `authorized_keys`, prove access with `-o ControlPath=none -o
|
||
ControlMaster=no`, or the breakage surfaces in a later session instead.
|
||
|
||
Depth and further mechanisms: the repo-authored `pi-devbox-environment` skill
|
||
(`rootfs/usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md`) §2
|
||
and §3 — that file is the one an agent actually loads mid-session, whereas this
|
||
`AGENTS.md` is only auto-read when the cwd *is* this repo.
|
||
|
||
## Gitea API access (env token)
|
||
|
||
`GITEA_ACCESS_TOKEN` + `GITEA_HOST` are passed into the container from the
|
||
host `.env` via `docker-compose.yml` (`${GITEA_ACCESS_TOKEN:-}` /
|
||
`${GITEA_HOST:-}`), primarily to enable the `gitea-mcp` server. They are
|
||
**not** baked into the image. When configured, they are also available for
|
||
**any** direct Gitea API interaction from inside the container — inspecting
|
||
CI runs, checking published tags, listing commits — e.g.
|
||
`curl -H "Authorization: token $GITEA_ACCESS_TOKEN" "$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs?limit=20"`.
|
||
Prefer this over a short-lived PAT file when the env token is present (the
|
||
`ci-release-watcher` skill auto-detects it). Public-repo GET listings work
|
||
unauthenticated too, so the token matters mainly for private repos or
|
||
rate-limit headroom; its lifecycle is host-managed, so there is nothing to
|
||
revoke after use. Never echo the token value (including into logs).
|
||
|
||
**Gotcha — a tag push fires EVERY workflow whose triggers match the tag ref.**
|
||
`lint.yml` uses a bare `push:` trigger, so a release tag yields *both* a lint run
|
||
and the publish run. The listing is newest-first and lint sorts **above** the
|
||
publish run, so "take the first run whose `path` contains `refs/tags/<tag>`"
|
||
picks the wrong one **reliably, not occasionally**. Real listing for v1.6.4:
|
||
|
||
```
|
||
id=531 #104 lint.yml@refs/tags/v1.6.4 <- wrong; sorts first
|
||
id=530 #103 docker-publish.yml@refs/tags/v1.6.4 <- the release build
|
||
id=529 #102 lint.yml@refs/heads/main <- same commit, linted on push
|
||
```
|
||
|
||
Lint goes green in minutes while the image is still building, so watching it
|
||
makes a release look finished when nothing has been published yet.
|
||
|
||
**Gotcha — the jobs endpoint takes the internal `id`, NOT the `run_number` the
|
||
UI shows as `#104`.** The two diverge widely, and `GET
|
||
.../actions/runs/<run_number>/jobs` does **not** error — it silently returns a
|
||
*different* run's jobs. Always read `id` from the run listing:
|
||
|
||
```bash
|
||
# Which runs did this tag/commit trigger? Filter on head_sha; never trust
|
||
# ordering or run numbering. limit=20, not 5 — with two runs per push the
|
||
# publish run falls off a 5-item window fast.
|
||
curl -sS -H "Authorization: token $GITEA_ACCESS_TOKEN" \
|
||
"$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs?limit=20" \
|
||
| jq --arg sha "$(git rev-list -n1 vX.Y.Z)" \
|
||
'.workflow_runs[] | select(.head_sha==$sha) | {id, run_number, path, status, conclusion}'
|
||
# pick the id whose .path starts with docker-publish.yml, then:
|
||
curl -sS -H "Authorization: token $GITEA_ACCESS_TOKEN" \
|
||
"$GITEA_HOST/api/v1/repos/joakimp/pi-devbox/actions/runs/<id>/jobs" \
|
||
| jq '.jobs[] | {name, status, conclusion}'
|
||
```
|
||
|
||
**Watcher config for this repo** (`ci-release-watcher` skill, hub-only shape —
|
||
pi-devbox has no downstream host to deploy to):
|
||
|
||
- `EXPECT_WORKFLOW=docker-publish.yml` — the skill's `preflight_run()` aborts at
|
||
startup if the run id belongs to lint instead.
|
||
- `EXPECTED_FRESH_TAGS='vX.Y.Z latest vX.Y.Z-studio latest-studio'`
|
||
- `EXPECTED_EXISTS_TAGS='base-latest'` — existence only: it is content-addressed
|
||
and legitimately keeps its old timestamp when the base is a cache hit.
|
||
- `CRITICAL_JOBS='build-variant build-variant-studio'` — job names are matched
|
||
**exactly** (`critical.issubset(succeeded)`), so the studio variant must be
|
||
listed explicitly; the skill's default omits it. Leave `promote-base-latest`
|
||
out: it legitimately skips on a base cache hit, which would misclassify a good
|
||
run. `update-description` is the cosmetic post-publish job.
|
||
|
||
## Cache-hit footgun (must-know)
|
||
|
||
`PI_VERSION` defaults to `latest` in `Dockerfile.variant` but **CI must
|
||
resolve it to a concrete version string** before passing as a build-arg.
|
||
Otherwise the build-arg string is byte-identical across releases →
|
||
identical layer hash → registry buildcache silently reuses the old
|
||
layer. `resolve-versions` job in the workflow handles this.
|
||
|
||
Discovered in pi-devbox 2026-05-23 (every release v0.74.0..v0.75.5
|
||
shipped the same image bytes); preventatively fixed for `PI_VERSION` +
|
||
`PI_FORK_REF` + `PI_OBSMEM_REF`.
|
||
|
||
## Smoke-test gate
|
||
|
||
`scripts/smoke-test.sh` runs amd64-only against a freshly-built variant
|
||
image. Verifies binaries, repo clones, runtime deployment (waits for
|
||
keybindings + mempalace bridge + ≥4 extensions before sampling — fixes
|
||
the parallel-build-load race documented in opencode-devbox c6f9d11
|
||
2026-06-08), and image size threshold (3500 MB; revisit after a few
|
||
releases as actuals settle).
|
||
|
||
If smoke fails on size threshold but build is otherwise fine: bump
|
||
`SIZE_THRESHOLD_MB` in scripts/smoke-test.sh in a follow-up commit and
|
||
re-run. The threshold exists to catch *runaway* growth (an accidental
|
||
texlive bake-in, a forgotten chrome dependency), not to block ordinary
|
||
upstream bumps.
|
||
|
||
## Build pipeline notes
|
||
|
||
- **Two-phase**: base + variant. Base is rebuilt only when
|
||
`Dockerfile.base`, `rootfs/`, or `entrypoint*.sh` change (CI computes
|
||
a content hash and probes Hub for an existing `base-<hash>` tag).
|
||
- **`base-latest` alias** is promoted from `base-<hash>` via `crane copy`
|
||
(manifest copy, no rebuild) only when the base actually changed.
|
||
- **`docker buildx build --push` retry**: 3 attempts with backoff for
|
||
transient Hub blips. Deterministic failures fail all 3 and the job
|
||
fails as expected.
|
||
- **Registry buildcache disabled**: buildkit's cache-export hits HTTP 400
|
||
on Hub CDN since ~2026-05-23. Image push works fine; we pay the full
|
||
base build on Dockerfile.base change, but base tags are content-
|
||
addressed so unchanged bases short-circuit at the probe step.
|
||
|
||
## Decoupling history (briefly)
|
||
|
||
Pre-v1.0.0 pi-devbox was `FROM joakimp/pi-devbox:base-pi-only`, where
|
||
`base-pi-only` was a tag built by **opencode-devbox CI** (with
|
||
`INSTALL_OPENCODE=false` in their variant Dockerfile) and pushed under
|
||
the pi-devbox repo as an internal building-block tag. This setup
|
||
required rebuilding opencode-devbox before pi-devbox could be tagged
|
||
and meant pi-devbox docs needed cross-referencing into opencode-devbox.
|
||
|
||
v1.0.0 brings pi install logic into this repo, drops the cross-repo
|
||
dependency, and the `base-pi-only*` tags from opencode-devbox become
|
||
deprecated artifacts (to be removed in opencode-devbox v2.0.0).
|
||
|
||
## What we DON'T install (and why)
|
||
|
||
- **No texlive** (~600 MB–1 GB). PDF export from pandoc / pi-studio works
|
||
out of the box via **`typst`** (~30 MB static binary), which the base ships
|
||
as the pandoc PDF engine (`pandoc --pdf-engine=typst`) — small enough to live
|
||
in base rather than a dedicated `:latest-studio-tex` variant. We don't bake in
|
||
a full TeX Live: it's heavy and typst covers the common Markdown→PDF case.
|
||
Users needing LaTeX-exact output can install the higher-fidelity fallback on
|
||
demand: `sudo apt-get install texlive-xetex texlive-latex-recommended` (then
|
||
`pandoc --pdf-engine=xelatex`).
|
||
- **pi-studio** ships in the `:latest-studio` variant (since v1.1.0),
|
||
vendored to `/opt/pi-studio` and registered at container start via
|
||
`pi install /opt/pi-studio` (see Dockerfile.variant `INSTALL_STUDIO`).
|
||
The default `:latest` image stays studio-free. Note: pi-studio binds
|
||
`127.0.0.1` inside the container, so browser access needs host
|
||
networking or the bundled `studio-expose` bridge (socat; auto-starts
|
||
when `STUDIO_EXPOSE=1`) — see README "Using pi-studio".
|
||
- **No Julia/R/GHCi/Clojure runtimes**. Use `uv run --with X` for
|
||
Python REPLs; `apt install` other-language runtimes ad-hoc per
|
||
container if needed.
|
||
|
||
## Backward compatibility
|
||
|
||
- The host `~/.mempalace` bind-mount path is unchanged.
|
||
- Volume names (`devbox-pi-config`, `devbox-ssh-local`,
|
||
`devbox-shell-history`, `devbox-zoxide`, `devbox-nvim-data`,
|
||
`devbox-uv`; optional `devbox-palace`, `devbox-chroma-cache`) are
|
||
unchanged.
|
||
- `~/.pi/agent/` layout inside the container is unchanged; existing
|
||
named volumes work without recreation.
|
||
- The `:latest` and `vX.Y.Z` Hub tags continue to point at a "base + pi"
|
||
image. Same tag, same shape, just built differently.
|