From 43cd6e22f2abe44593af2f1f4405b3001601db94 Mon Sep 17 00:00:00 2001 From: Joakim Persson Date: Fri, 7 Aug 2026 21:34:41 +0200 Subject: [PATCH] v1.7.0: bundle pi-atelier at a pinned tag; pin pi to an audited 0.84.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two changes that belong together, because the first is what makes the second dangerous to get wrong. pi-atelier (TUI sidebar + status rail) is now vendored to /opt/pi-atelier at PI_ATELIER_REF=v0.8.0 and registered by entrypoint-user.sh — the pi-fork / pi-observational-memory / pi-studio pattern, deliberately NOT `pi install npm:pi-atelier`, which writes into ~/.pi/npm-global on the config volume where it shadows the image and pins nothing. Unlike its siblings it gets no `npm install`: atelier declares zero runtime deps (peerDeps only, satisfied by the baked pi) and has no build step, so pi loads its TypeScript straight from the checkout via package.json `pi.extensions`. pi is no longer resolved to npm `latest` at build time. The pin lives in Dockerfile.variant and CI reads it from there, so a local `docker build` and a CI release ship the same versions by construction. The pin is a CHECKPOINT, NOT A FREEZE: bumping stays a one-line change; what stops is *unreviewed* adoption of whatever shipped that morning, in the same build that then gets tagged and published. CI fails when a pin is not concrete or not actually published on npm, and warns — never adopts — when npm latest moves ahead, naming what to re-check. Why this pairing needed care: pi-atelier 0.6.0/0.7.0 wrap pi's PRIVATE TUI renderer, and under pi 0.84 that wrapper recurses — pi hangs at startup burning CPU with no error. Upstream fixed the recursion in 0.7.1 and restored the non-overlapping split in 0.7.2; 0.8.0 is additive on top. atelier's own peerDependencies still say >=0.80.7, which does not express that floor, so nothing in npm metadata could have warned us. The floor is therefore encoded as an executable rule — pi >= 0.84 => pi-atelier >= 0.7.1 — asserted in both smoke-test.sh (build time) and recreate-sanity-check.sh (after a real recreate), verified against a 4x4 version matrix. Existing volumes needed migration, not just vendoring: a hand-installed `npm:pi-atelier` entry is counted as already-registered by the entrypoint guard, so every existing volume would have kept its unpinned npm copy — and a 0.6.x copy next to pi 0.84 is exactly the startup hang. The entrypoint now drops that one exact string (settings.json.bak.atelier. backup, distinct prefix so it cannot clobber the template merge's backup in the same second) and lets the pinned /opt copy register. Tested against a real settings.json: only that entry removed, other packages and all keys intact, idempotent, and unparseable JSON leaves the file untouched. DEVBOX_ATELIER=0 opts out entirely — in the entrypoint rather than via `pi uninstall`, because this component's failure mode is "pi will not start", which cannot be repaired from inside pi. 0.84.1 was audited for this release, not merely adopted: theme/TUI changes are additive, the session format is unchanged (CURRENT_SESSION_VERSION = 3 in both 0.83.0 and 0.84.1 with an identical migrateV1ToV2/migrateV2ToV3 ladder, so existing transcripts are neither migrated nor at risk and pi-session-repair stays valid), and the Node engine floor is unmoved at >=22.19.0. CI resolves the atelier tag to its PEELED commit SHA — atelier uses annotated tags, so the unpeeled ref is a tag object, not a commit; pi-studio's lightweight tags never exposed that distinction. Also: docs for overriding the read-only ~/.ssh/config from the container — container-only keys in ~/.ssh-local, hardened authorized_keys, the fact that `from=` must allow the HOST's addresses because container egress is NAT'd through it, and the macOS-only-keyword trap (`UseKeychain` is fatal to Linux OpenSSH and takes out dssh/pi --ssh while the host keeps working). Corrects two claims in "Naming LAN peers": ssh-lan.conf is not ProxyJump-only, and first-time creation does need one restart because the Include is emitted only when the file already exists at start. --- .env.example | 8 + .gitea/workflows/docker-publish.yml | 79 ++++++++- CHANGELOG.md | 145 +++++++++++++++- DOCKER_HUB.md | 3 +- Dockerfile.variant | 68 ++++++-- README.md | 251 ++++++++++++++++++++++++++-- entrypoint-user.sh | 62 ++++++- scripts/recreate-sanity-check.sh | 50 ++++++ scripts/smoke-test.sh | 40 +++++ 9 files changed, 667 insertions(+), 39 deletions(-) diff --git a/.env.example b/.env.example index d5492e1..bee2d69 100644 --- a/.env.example +++ b/.env.example @@ -43,6 +43,14 @@ SSH_KEY_PATH=~/.ssh # the host, so bare `dssh user@` works on whatever LAN you're roaming on. # DEVBOX_LAN_AUTOJUMP_PRIVATE=0 +# ── pi-atelier (TUI sidebar) ───────────────────────────────────────── +# The image vendors pi-atelier at a pinned, audited tag and registers it on +# container start. Set to 0 to opt out: the entrypoint then removes it from +# pi's `packages[]` instead of registering it. This lives here rather than +# being a `pi uninstall` because a broken TUI extension's failure mode is +# "pi will not start", which you cannot fix from inside pi. +# DEVBOX_ATELIER=1 + # ── Git Configuration ──────────────────────────────────────────────── GIT_USER_NAME= GIT_USER_EMAIL= diff --git a/.gitea/workflows/docker-publish.yml b/.gitea/workflows/docker-publish.yml index f6a88f6..bee31cf 100644 --- a/.gitea/workflows/docker-publish.yml +++ b/.gitea/workflows/docker-publish.yml @@ -136,8 +136,16 @@ jobs: extensions_ref: ${{ steps.resolve.outputs.extensions_ref }} studio_ref: ${{ steps.resolve.outputs.studio_ref }} studio_tag: ${{ steps.resolve.outputs.studio_tag }} + atelier_ref: ${{ steps.resolve.outputs.atelier_ref }} + atelier_tag: ${{ steps.resolve.outputs.atelier_tag }} mempalace_toolkit_ref: ${{ steps.resolve.outputs.mempalace_toolkit_ref }} steps: + # Needed since v1.7.0: the pi version and the pi-atelier tag are now + # PINNED IN Dockerfile.variant and read from it here, so this job has to + # see the repo. Keeping the pins in the Dockerfile (rather than duplicated + # in this workflow) means a local `docker build` and CI ship the same + # versions by construction, and a bump is one reviewable line. + - uses: actions/checkout@v4 - name: Resolve pi version + companion refs id: resolve shell: bash @@ -157,13 +165,35 @@ jobs: fi } - # pi version from npm (catthehacker/ubuntu:act-latest's npm is not - # reliably on PATH in act_runner job containers, so query directly). - PI_VERSION=$(curl -sf "https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/latest" | jq -r '.version' 2>/dev/null || true) - if ! printf '%s' "${PI_VERSION:-}" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+'; then - echo "::error::Could not resolve pi version from npm (got '${PI_VERSION:-}')." + # ── pi version: from the PIN, not from npm `latest` ─────────── + # Until v1.7.0 this followed npm `latest`, which meant every release + # silently adopted whatever pi had shipped that morning — unaudited — + # in the same build that then got tagged and published. A pi minor + # can move the TUI/renderer internals that pi-atelier wraps (0.84 vs + # atelier 0.6.0: startup hang, sustained CPU) or the session `.jsonl` + # format that pi-session-repair parses. The pin makes adoption an + # explicit, reviewable act; the drift warning below makes it a + # prompt rather than a surprise. + PI_VERSION=$(sed -n 's/^ARG PI_VERSION=\([^[:space:]]*\).*/\1/p' Dockerfile.variant | head -n1) + if ! printf '%s' "${PI_VERSION:-}" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+$'; then + echo "::error::ARG PI_VERSION in Dockerfile.variant is not a concrete version (got '${PI_VERSION:-}'). CI refuses to build from a floating pi version — see the pin policy comment above that ARG." exit 1 fi + # The pin must actually exist on npm: catches a typo, an unpublished + # version, or one yanked after we audited it — at resolve time, with + # a clear message, instead of as an `npm install` failure mid-build. + PI_PUBLISHED=$(curl -sf "https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/${PI_VERSION}" | jq -r '.version // empty' 2>/dev/null || true) + if [ "${PI_PUBLISHED:-}" != "${PI_VERSION}" ]; then + echo "::error::Pinned pi version ${PI_VERSION} is not published on npm (registry returned '${PI_PUBLISHED:-}'). Fix ARG PI_VERSION in Dockerfile.variant." + exit 1 + fi + # Informational only — a newer pi must never be adopted implicitly. + # `|| true`: a transient registry failure must not fail a release + # whose version is already pinned and verified above. + PI_NPM_LATEST=$(curl -sf "https://registry.npmjs.org/@earendil-works%2Fpi-coding-agent/latest" | jq -r '.version // empty' 2>/dev/null || true) + if [ -n "${PI_NPM_LATEST:-}" ] && [ "${PI_NPM_LATEST}" != "${PI_VERSION}" ]; then + echo "::warning::pi ${PI_NPM_LATEST} is published; this build ships the audited pin ${PI_VERSION}. To adopt it: read the upstream CHANGELOG for every version in between (TUI/theme API, session .jsonl format, extension loader, Node engine), re-check pi-atelier's floor, then bump ARG PI_VERSION in Dockerfile.variant and note the audit in CHANGELOG.md." + fi echo "pi_version=${PI_VERSION}" >> "$GITHUB_OUTPUT" # pi-fork / pi-observational-memory (GitHub) → commit SHAs. @@ -176,6 +206,26 @@ jobs: echo "fork_ref=${FORK_REF}" >> "$GITHUB_OUTPUT" echo "obsmem_ref=${OBSMEM_REF}" >> "$GITHUB_OUTPUT" + # pi-atelier → the PINNED TAG's commit SHA. Unlike fork/obsmem + # (which track a branch head) atelier wraps pi's private TUI + # renderer, so its version is pinned in Dockerfile.variant and read + # from there; we only resolve tag → SHA, for reproducibility and to + # defeat the cache-hit footgun. Never floats to a branch. + ATELIER_TAG=$(sed -n 's/^ARG PI_ATELIER_REF=\([^[:space:]]*\).*/\1/p' Dockerfile.variant | head -n1) + if ! printf '%s' "${ATELIER_TAG:-}" | grep -qE '^v?[0-9]+\.[0-9]+\.[0-9]+$'; then + echo "::error::ARG PI_ATELIER_REF in Dockerfile.variant is not a semver tag (got '${ATELIER_TAG:-}'). pi-atelier must stay pinned to a tag — see the floor note above that ARG." + exit 1 + fi + ATELIER_LS=$(git ls-remote --tags "https://github.com/michaelmjhhhh/pi-atelier.git" || true) + # Peeled ^{} line first (annotated tags), then the direct ref. + ATELIER_REF=$(printf '%s\n' "$ATELIER_LS" | awk -v t="refs/tags/${ATELIER_TAG}^{}" '$2==t{print $1}') + if [ -z "$ATELIER_REF" ]; then + ATELIER_REF=$(printf '%s\n' "$ATELIER_LS" | awk -v t="refs/tags/${ATELIER_TAG}" '$2==t{print $1}') + fi + require_sha PI_ATELIER_REF "$ATELIER_REF" + echo "atelier_ref=${ATELIER_REF}" >> "$GITHUB_OUTPUT" + echo "atelier_tag=${ATELIER_TAG}" >> "$GITHUB_OUTPUT" + # pi-toolkit / pi-extensions (Gitea) → commit SHAs. Gitea API # requires auth even for public-repo commit listing. TOOLKIT_REF=$(curl -sf -H "$AUTH_HEADER" \ @@ -230,7 +280,8 @@ jobs: echo "studio_ref=${STUDIO_REF}" >> "$GITHUB_OUTPUT" echo "studio_tag=${STUDIO_TAG}" >> "$GITHUB_OUTPUT" - echo "Resolved PI_VERSION=${PI_VERSION}" + echo "Resolved PI_VERSION=${PI_VERSION} (pinned in Dockerfile.variant; npm latest is ${PI_NPM_LATEST:-unknown})" + echo "Resolved PI_ATELIER_REF=${ATELIER_REF} (pi-atelier ${ATELIER_TAG}, pinned)" echo "Resolved PI_FORK_REF=${FORK_REF}, PI_OBSMEM_REF=${OBSMEM_REF}" echo "Resolved PI_TOOLKIT_REF=${TOOLKIT_REF}, PI_EXTENSIONS_REF=${EXTENSIONS_REF}" echo "Resolved PI_STUDIO_REF=${STUDIO_REF} (pi-studio ${STUDIO_TAG})" @@ -357,6 +408,8 @@ jobs: PI_TOOLKIT_REF=${{ needs.resolve-versions.outputs.toolkit_ref }} PI_EXTENSIONS_REF=${{ needs.resolve-versions.outputs.extensions_ref }} MEMPALACE_TOOLKIT_REF=${{ needs.resolve-versions.outputs.mempalace_toolkit_ref }} + PI_ATELIER_REF=${{ needs.resolve-versions.outputs.atelier_ref }} + PI_ATELIER_VERSION=${{ needs.resolve-versions.outputs.atelier_tag }} RELEASE_TAG=smoke SOURCE_REVISION=${{ github.sha }} - name: Smoke test (amd64) @@ -417,6 +470,8 @@ jobs: PI_STUDIO_REF=${{ needs.resolve-versions.outputs.studio_ref }} PI_STUDIO_VERSION=${{ needs.resolve-versions.outputs.studio_tag }} MEMPALACE_TOOLKIT_REF=${{ needs.resolve-versions.outputs.mempalace_toolkit_ref }} + PI_ATELIER_REF=${{ needs.resolve-versions.outputs.atelier_ref }} + PI_ATELIER_VERSION=${{ needs.resolve-versions.outputs.atelier_tag }} RELEASE_TAG=smoke-studio SOURCE_REVISION=${{ github.sha }} - name: Smoke test studio (amd64) @@ -471,6 +526,8 @@ jobs: TOOLKIT_REF: ${{ needs.resolve-versions.outputs.toolkit_ref }} EXTENSIONS_REF: ${{ needs.resolve-versions.outputs.extensions_ref }} MEMPALACE_TOOLKIT_REF: ${{ needs.resolve-versions.outputs.mempalace_toolkit_ref }} + ATELIER_REF: ${{ needs.resolve-versions.outputs.atelier_ref }} + ATELIER_TAG: ${{ needs.resolve-versions.outputs.atelier_tag }} run: | set -euo pipefail TAG_FLAGS=() @@ -490,8 +547,10 @@ jobs: --build-arg "PI_TOOLKIT_REF=${TOOLKIT_REF}" \ --build-arg "PI_EXTENSIONS_REF=${EXTENSIONS_REF}" \ --build-arg "MEMPALACE_TOOLKIT_REF=${MEMPALACE_TOOLKIT_REF}" \ + --build-arg "PI_ATELIER_REF=${ATELIER_REF}" \ + --build-arg "PI_ATELIER_VERSION=${ATELIER_TAG}" \ --build-arg "IMAGE_TITLE=pi-devbox" \ - --build-arg "IMAGE_DESCRIPTION=pi-devbox ${RELEASE_TAG} — core variant: pi coding agent CLI ${PI_VERSION}, pi-toolkit, extensions (fork + observational-memory), MemPalace. No browser UI — see the -studio tags for that." \ + --build-arg "IMAGE_DESCRIPTION=pi-devbox ${RELEASE_TAG} — core variant: pi coding agent CLI ${PI_VERSION}, pi-toolkit, extensions (fork + observational-memory + atelier ${ATELIER_TAG} TUI sidebar), MemPalace. No browser UI — see the -studio tags for that." \ --build-arg "RELEASE_TAG=${RELEASE_TAG}" \ --build-arg "BUILD_DATE=${BUILD_DATE}" \ --build-arg "SOURCE_REVISION=${GITHUB_SHA:-}" \ @@ -561,6 +620,8 @@ jobs: STUDIO_REF: ${{ needs.resolve-versions.outputs.studio_ref }} STUDIO_TAG: ${{ needs.resolve-versions.outputs.studio_tag }} MEMPALACE_TOOLKIT_REF: ${{ needs.resolve-versions.outputs.mempalace_toolkit_ref }} + ATELIER_REF: ${{ needs.resolve-versions.outputs.atelier_ref }} + ATELIER_TAG: ${{ needs.resolve-versions.outputs.atelier_tag }} run: | set -euo pipefail TAG_FLAGS=() @@ -582,7 +643,9 @@ jobs: --build-arg "MEMPALACE_TOOLKIT_REF=${MEMPALACE_TOOLKIT_REF}" \ --build-arg "INSTALL_STUDIO=true" \ --build-arg "IMAGE_TITLE=pi-devbox (studio)" \ - --build-arg "IMAGE_DESCRIPTION=pi-devbox ${RELEASE_TAG} — studio variant: everything in the core variant (pi ${PI_VERSION}, pi-toolkit, fork + observational-memory, MemPalace) plus the pi-studio browser UI ${STUDIO_TAG}." \ + --build-arg "PI_ATELIER_REF=${ATELIER_REF}" \ + --build-arg "PI_ATELIER_VERSION=${ATELIER_TAG}" \ + --build-arg "IMAGE_DESCRIPTION=pi-devbox ${RELEASE_TAG} — studio variant: everything in the core variant (pi ${PI_VERSION}, pi-toolkit, fork + observational-memory + atelier ${ATELIER_TAG}, MemPalace) plus the pi-studio browser UI ${STUDIO_TAG}." \ --build-arg "PI_STUDIO_REF=${STUDIO_REF}" \ --build-arg "PI_STUDIO_VERSION=${STUDIO_TAG}" \ --build-arg "RELEASE_TAG=${RELEASE_TAG}" \ diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c7fc72..bd89fb0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,13 +11,101 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`). --- -## Unreleased +## v1.7.0 — 2026-08-07 -Staged, not yet tagged — more changes are queued for the next release before CI -is kicked off. +Minor release. Headline: **pi-atelier is now part of the image** — the TUI +sidebar/status rail every container previously had to hand-install — and **pi is +pinned to an audited version instead of tracking npm `latest`**. + +*Why minor and not patch:* the policy above reserves patch for "pi version bumps, +smaller fixes" and minor for "new variants, significant base additions". Bundling +a new companion package into every image is the same shape as v1.1.0, which went +minor for bundling pi-studio; v1.4.0 likewise went minor for adding typst. This +release also adds a new build-arg pair, a new opt-out env var, and a settings +migration, so patch would understate it. + +### Added + +- **pi-atelier vendored at `/opt/pi-atelier`, pinned to `v0.8.0`** — the TUI + sidebar (ordered panels, split-pane, themes) is now part of the image instead + of something each user hand-installs. Vendored + registered at container start + by `entrypoint-user.sh`, the same pattern as pi-fork/pi-observational-memory/ + pi-studio, and deliberately **not** `pi install npm:pi-atelier`: an npm + install writes into `~/.pi/npm-global` on the config volume, which shadows the + image and pins nothing — the footgun that once hid a missing `fork` tool for + six weeks. Unlike its siblings it gets **no `npm install`**: pi-atelier + declares zero runtime dependencies (only peerDeps, satisfied by the baked pi) + and has no build step, so pi loads its TypeScript straight from the checkout + (`pi.extensions` → `extensions/index.ts`). +- **A version FLOOR, encoded as an executable test.** pi-atelier 0.6.0/0.7.0 + wrap pi's private TUI renderer in a way that recurses under pi 0.84: pi hangs + at startup with sustained CPU and no error message. Upstream fixed the + recursion in 0.7.1 and restored the non-overlapping split in 0.7.2 + ("avoiding the recursive render path that caused startup hangs and sustained + CPU usage"); 0.8.0 is additive on top of that. atelier's own + `peerDependencies` still say `>=0.80.7`, which does **not** express the floor, + so nothing in npm metadata could have warned us. `smoke-test.sh` and + `recreate-sanity-check.sh` now assert the pairing rule **pi ≥ 0.84 ⇒ + pi-atelier ≥ 0.7.1** — verified against a 4×4 version matrix — so a bad + combination fails the build instead of publishing an image whose TUI never + starts. CI resolves the pinned tag to its **peeled commit SHA**; atelier uses + annotated tags, so the unpeeled ref is a tag object, not a commit (pi-studio's + lightweight tags never exposed that distinction). +- **`DEVBOX_ATELIER=0`** opts out: the entrypoint removes pi-atelier from pi's + `packages[]` instead of registering it. The switch lives in the entrypoint + rather than being "just run `pi uninstall`" because this component's failure + mode is *pi will not start*, which cannot be repaired from inside pi. +- **Migration for hand-installed copies.** A pre-existing `npm:pi-atelier` entry + is dropped from `packages[]` (with a `settings.json.bak.atelier.` backup) + so the pinned `/opt` copy takes over. This is not cosmetic: the registration + guard counts `npm:` as already-registered, so without this step every + existing volume would have kept its unpinned npm copy — and a 0.6.x copy + alongside pi 0.84 is exactly the startup hang above. Only that one exact + string is removed; jq-parse failures or a missing file leave settings + untouched, and the backup prefix is distinct from the template merge's so two + rewrites in the same second cannot overwrite each other's backup. ### Changed +- **pi-toolkit's `pi-atelier.json` modernised to atelier's current schema** + (pi-toolkit `0e1369e`, cross-repo — it reaches the image through the pinned + `PI_TOOLKIT_REF` clone). The seeded config had been written against the pre-0.7 + vocabulary: `segments` → `segmentLayout` with explicit per-segment visibility, + `ornament: "none"` → `{"id":"brand","visible":false}`, `showExtensionStatuses` + → `{"id":"statuses","visible":true}`, plus the sidebar toggles that did not + exist when it was written (`showSidebarAgent`, `showSidebarTodos`, and + `showSidebarOnStartup`, new in atelier 0.8.0). Upstream still reads the old + keys, but only as non-authoritative legacy inputs, so the file worked while + silently missing every sidebar control added since. Verified by loading the old + and new file through pi-atelier 0.8.0's own `loadConfig()`: zero warnings from + each and an identical *effective* config, so it is a pure schema + modernisation — every deliberate choice (compact density, 60/85 context + thresholds, notifications off) is preserved. `sidebarPanelLayout` is left unset + on purpose so the panel set tracks upstream as atelier adds panels. +- **pi is now PINNED, not `latest`: `PI_VERSION=0.84.1`** (`Dockerfile.variant`). + CI's `resolve-versions` job used to resolve `@earendil-works/pi-coding-agent` + to npm `latest`, which meant every release silently adopted whatever pi had + shipped that morning — unaudited — in the same build that then got tagged and + published. A pi minor can move the private TUI/renderer internals pi-atelier + wraps (0.84 vs atelier 0.6.0: startup hang) or the session `.jsonl` format + `pi-session-repair` parses. **The pin is a checkpoint, not a freeze** — + bumping stays a routine one-line change; what stops is *unreviewed* adoption. + 0.84.1 was audited for this release: theme/TUI additions are additive, the + session format is unchanged (`CURRENT_SESSION_VERSION = 3` in both 0.83.0 and + 0.84.1, identical `migrateV1ToV2`/`migrateV2ToV3` ladder, so existing + transcripts are neither migrated nor at risk), and the Node engine floor is + unmoved at `>=22.19.0`. + - The pins live in the **Dockerfiles** and CI reads them from there (a + `checkout` was added to `resolve-versions`), so a local `docker build` and a + CI release ship the same versions by construction instead of by convention. + - CI **fails** the build when the pin is not a concrete version, and when the + pinned version is not actually published on npm — catching a typo, an + unpublished version, or one yanked after we audited it, at resolve time with + a clear message rather than as an `npm install` error mid-build. + - CI **warns** (`::warning::`, never adopts) when npm `latest` is ahead of the + pin, naming the newer version and what to re-check. That warning is the + prompt to audit and bump — not something to silence. + - **mempalace pin `3.5.0` → `3.6.0`** (`Dockerfile.base` `MEMPALACE_VERSION`), in lockstep with opencode-devbox v2.9.0 as the pin's own comment requires. 3.6.0 (2026-07-17) is PyPI latest and is additive/reliability only — secure @@ -35,6 +123,57 @@ is kicked off. ### Documentation +- **New README section: "Using pi-atelier (TUI sidebar)"** — what the status rail + and sidebar give you, the `alt+a` / `/atelier` entry points, session-scoped + `/atelier sidebar on|off` versus persistent Save, and `DEVBOX_ATELIER=0` to opt + out. Plus the config story: why `~/.pi/agent/pi-atelier.json` is **copied, not + symlinked** (atelier saves via write-temp-then-`rename(2)`, and `rename` + replaces a symlink rather than following it, so a symlink would silently detach + on the first save), why `install.sh` therefore only seeds it when absent, which + keys are current versus legacy-compatibility, and the 92-column auto-hide / + 64-column main-pane floor so a narrow terminal degrades gracefully. +- **Documents how to authenticate the container to a LAN peer with its own key** + (README: *Giving the container its own key for a peer*) — the gap the existing + *Naming LAN peers* section left open. That section explained `ProxyJump` + *routing* while asserting `HostName`/`User`/`IdentityFile` are "inherited from + the matching block in your real `~/.ssh/config`", which is precisely what fails + in a container: host keys are normally passphrase-protected and unlocked by the + macOS Keychain or an `ssh-agent`, neither of which exists here, so the key can + never be decrypted — `Permission denied (publickey)` while the identical + `ssh peer` works fine in a host terminal — and `~/.ssh` is read-only, so no + usable key can be added there either. The new walkthrough (throwaway example + keys) covers a passphraseless keypair in the `devbox-ssh-local` volume so it + survives `--force-recreate`; a hardened `authorized_keys` line (`restrict`, + `from=`, optional `permitopen`); the non-obvious detail that `from=` must allow + the **host's** addresses, plural, because container egress is NAT'd through the + host and a roaming laptop presents a different one per network (a `from=` + mismatch is indistinguishable from a wrong key in the error message); the + `IdentityFile` override in the host-owned `ssh-lan.conf`; and verification with + `-o ControlPath=none` so a warm ControlMaster cannot fake a pass. States + explicitly that no private key is in the published image — the volume is + created at runtime on the operator's own machine. +- **Corrects two claims in *Naming LAN peers***: (1) `ssh-lan.conf` is not + `ProxyJump`-only — it is `Include`d before `~/.ssh/config`, so by + first-value-wins *any* option set there wins, which is what makes the + `IdentityFile` override above possible; (2) "newly added peers work + immediately, no container or session restart needed" holds only for *edits* to + an existing file. Creating it for the first time **does** need one restart, + because `setup-lan-access.sh` emits the + `Include ~/.config/devbox-shell/ssh-lan.conf` line only + `if [ -r "$SSH_LAN_CONF" ]` at container start — until then ssh never reads it, + which presents exactly as "my override is being ignored". +- **Adds *macOS-only keywords in a shared `~/.ssh/config`***. The same file is + read by macOS ssh and by the container's Linux OpenSSH, where macOS-only + keywords are fatal rather than ignored: one `UseKeychain yes` in a `Host *` + block yields `Bad configuration option: usekeychain` / + `terminating, 1 bad configuration options` and takes down `dssh`/`dscp`, + `pi --ssh`, `scp` and every helper that shells out to ssh — while the host + keeps working, so it presents as a container regression rather than a host + config error. Fix is `IgnoreUnknown UseKeychain` *ahead of* the keyword (macOS + still honours it, Linux skips it), plus keeping such a `Host *` block below + OrbStack's `Include ~/.orbstack/ssh/config`, which documents in its own comment + that it must come first. + - Documents **per-variant image description labels** (committed and pushed after the v1.6.4 tag without a changelog entry). Both published variants used to inherit `Dockerfile.base`'s `description="pi-devbox — base image diff --git a/DOCKER_HUB.md b/DOCKER_HUB.md index b68db79..65db460 100644 --- a/DOCKER_HUB.md +++ b/DOCKER_HUB.md @@ -46,7 +46,8 @@ Full setup guide — authentication for each provider (Anthropic, OpenAI, Gemini ### pi and companions -- **pi `{{PI_VERSION}}`** ([`@earendil-works/pi-coding-agent`](https://www.npmjs.com/package/@earendil-works/pi-coding-agent)) — installed at `/usr/bin/pi` +- **pi `{{PI_VERSION}}`** ([`@earendil-works/pi-coding-agent`](https://www.npmjs.com/package/@earendil-works/pi-coding-agent)) — installed at `/usr/bin/pi`, pinned to an audited version (not npm `latest`) +- **pi-atelier** — TUI sidebar (ordered panels, split-pane, themes), vendored at `/opt/pi-atelier` and pinned to an audited tag; the exact tag is in the image labels (`se.jordbo.pi-devbox.pi-atelier-version`) and `/etc/pi-devbox/build-manifest.json` - **[pi-toolkit](https://gitea.jordbo.se/joakimp/pi-toolkit)** — keybindings (mosh/tmux-friendly Shift+Enter, Ctrl+J, Alt+J newline bindings), AWS env loader, settings template - **[pi-extensions](https://gitea.jordbo.se/joakimp/pi-extensions)** — 7 user-facing extensions: `ext-toggle`, `mcp-loader`, `todo`, `ssh-controlmaster`, `notify`, `git-checkpoint`, `confirm-destructive` - **`fork`** ([pi-fork](https://github.com/elpapi42/pi-fork)) and **`recall`** ([pi-observational-memory](https://github.com/elpapi42/pi-observational-memory)) tools diff --git a/Dockerfile.variant b/Dockerfile.variant index f82e86f..a228004 100644 --- a/Dockerfile.variant +++ b/Dockerfile.variant @@ -29,16 +29,34 @@ ARG USER_NAME=developer # runs each repo's install.sh on container start so symlinks land under # ~/.pi/agent/ on the named volume. # -# PI_VERSION should be passed explicitly by CI as a concrete version -# (resolved from `npm view @earendil-works/pi-coding-agent version`). -# The default `latest` is for local dev convenience only — it has a -# known cache-hit footgun in registry-cached CI builds: the resulting -# build-arg string is byte-identical across builds, the layer-hash is -# identical, and the registry buildcache silently reuses the layer -# from whatever pi version was current when the cache was first -# populated. CI MUST pass a resolved concrete version. See pi-devbox -# v0.75.5b 2026-05-23 for the discovery + canonical fix. -ARG PI_VERSION=latest +# ── pi version pin: an AUDITED CHECKPOINT, not a freeze ────────────── +# PI_VERSION is pinned to a version whose upstream CHANGELOG has been read +# against this image's integration surface: the theme/TUI API that pi-atelier +# couples to, the session `.jsonl` format that `pi-session-repair` parses, the +# extension/package loader, and the Node engine floor. CI reads THIS LINE as +# the single source of truth (see the `resolve-versions` job) and no longer +# follows npm `latest` — following it meant every release silently adopted +# whatever pi shipped that morning, unaudited, in the very build that then got +# tagged and published. +# +# BUMPING IS ROUTINE AND EXPECTED — the pin exists to force a look, not to +# hold a version forever: +# 1. Read the upstream CHANGELOG for every version between old and new. +# 2. Re-check the companions that couple to pi's private TUI/renderer +# internals — pi-atelier above all (see PI_ATELIER_REF below for the +# 0.6.0-under-pi-0.84 startup-hang precedent). +# 3. Bump this line, record the audit in CHANGELOG.md, then tag. +# CI fails the build if this pin is not a published npm version, and warns — +# without adopting it — when npm `latest` has moved ahead. That warning is the +# prompt to do step 1; it is not something to silence. +# +# A concrete version here ALSO defeats the registry-buildcache cache-hit +# footgun that `latest` carried: a byte-identical build-arg string produced an +# identical layer hash, so the cache reused the layer from whatever pi was +# current when it was first populated (shipped the same bytes for pi-devbox +# v0.74.0..v0.75.5; discovered + fixed in v0.75.5b, 2026-05-23). The `latest` +# branch below is kept only for a deliberate local `docker build` override. +ARG PI_VERSION=0.84.1 ARG PI_TOOLKIT_REF=main ARG PI_EXTENSIONS_REF=main # Repo URLs default to the canonical gitea origin but are overridable so a @@ -54,6 +72,29 @@ ARG PI_FORK_REPO=https://github.com/elpapi42/pi-fork.git ARG PI_FORK_REF=master ARG PI_OBSMEM_REPO=https://github.com/elpapi42/pi-observational-memory.git ARG PI_OBSMEM_REF=master +# pi-atelier (TUI sidebar: ordered panels, split-pane, themes) is PINNED TO A +# TAG, which CI resolves to that tag's commit SHA — same treatment as +# pi-studio, for reproducibility plus cache-busting. +# +# This floor is hard-earned. pi-atelier 0.6.0/0.7.0 wrapped pi's PRIVATE TUI +# renderer, and under pi 0.84 that wrapper recursed: pi hung at startup with +# sustained CPU. Upstream fixed the recursion in 0.7.1 and restored the +# non-overlapping split in 0.7.2 — "avoiding the recursive render path that +# caused startup hangs and sustained CPU usage". Its own peerDependencies +# still say `>=0.80.7`, which does NOT encode that floor, so nothing would +# have warned us: NEVER pair pi-atelier < 0.7.1 with pi >= 0.84. Bump this +# pin and PI_VERSION together, checking atelier's CHANGELOG for the pi +# version it claims to track. +# +# No `npm install` step, unlike pi-fork/pi-observational-memory/pi-studio: +# pi-atelier declares ZERO runtime dependencies (only peerDeps, satisfied by +# the baked pi) and has no build step — pi loads its TypeScript directly from +# the /opt checkout. Adding an install here would be a no-op that only costs +# build time. +ARG PI_ATELIER_REPO=https://github.com/michaelmjhhhh/pi-atelier.git +ARG PI_ATELIER_REF=v0.8.0 +# Human-readable tag PI_ATELIER_REF was resolved from; recorded as a label. +ARG PI_ATELIER_VERSION=v0.8.0 RUN set -e && \ # git_fetch_ref: clone-equivalent helper that accepts EITHER a branch name @@ -87,12 +128,14 @@ RUN set -e && \ git_fetch_ref "${PI_EXTENSIONS_REPO}" "${PI_EXTENSIONS_REF}" /opt/pi-extensions && \ git_fetch_ref "${PI_FORK_REPO}" "${PI_FORK_REF}" /opt/pi-fork && \ git_fetch_ref "${PI_OBSMEM_REPO}" "${PI_OBSMEM_REF}" /opt/pi-observational-memory && \ + git_fetch_ref "${PI_ATELIER_REPO}" "${PI_ATELIER_REF}" /opt/pi-atelier && \ (cd /opt/pi-fork && npm install --omit=dev --no-audit --no-fund) && \ (cd /opt/pi-observational-memory && npm install --omit=dev --no-audit --no-fund) && \ echo "pi-toolkit at $(cd /opt/pi-toolkit && git rev-parse --short HEAD)" && \ echo "pi-extensions at $(cd /opt/pi-extensions && git rev-parse --short HEAD)" && \ echo "pi-fork at $(cd /opt/pi-fork && git rev-parse --short HEAD)" && \ - echo "pi-observational-memory at $(cd /opt/pi-observational-memory && git rev-parse --short HEAD)" + echo "pi-observational-memory at $(cd /opt/pi-observational-memory && git rev-parse --short HEAD)" && \ + echo "pi-atelier at $(cd /opt/pi-atelier && git rev-parse --short HEAD) (${PI_ATELIER_VERSION})" # ── Image-baked skill refresh: pi-extensions (Option 1 over Option 2) ── # rootfs ships a VENDORED snapshot of the pi-extensions skill at @@ -239,6 +282,8 @@ LABEL org.opencontainers.image.version="${RELEASE_TAG}" \ se.jordbo.pi-devbox.pi-extensions-ref="${PI_EXTENSIONS_REF}" \ se.jordbo.pi-devbox.pi-fork-ref="${PI_FORK_REF}" \ se.jordbo.pi-devbox.pi-obsmem-ref="${PI_OBSMEM_REF}" \ + se.jordbo.pi-devbox.pi-atelier-ref="${PI_ATELIER_REF}" \ + se.jordbo.pi-devbox.pi-atelier-version="${PI_ATELIER_VERSION}" \ se.jordbo.pi-devbox.mempalace-toolkit-ref="${MEMPALACE_TOOLKIT_REF}" \ se.jordbo.pi-devbox.pi-studio-ref="${PI_STUDIO_REF}" \ se.jordbo.pi-devbox.pi-studio-version="${PI_STUDIO_VERSION}" @@ -265,6 +310,7 @@ RUN set -e; \ echo " \"pi-extensions\": \"$(rev /opt/pi-extensions)\","; \ echo " \"pi-fork\": \"$(rev /opt/pi-fork)\","; \ echo " \"pi-observational-memory\": \"$(rev /opt/pi-observational-memory)\","; \ + echo " \"pi-atelier\": \"$(rev /opt/pi-atelier)\","; \ echo " \"mempalace-toolkit\": \"$(rev /opt/mempalace-toolkit)\","; \ echo " \"pi-studio\": ${STUDIO_REV}"; \ echo " }"; \ diff --git a/README.md b/README.md index b204b7f..766a4bb 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,8 @@ on the host. mempalace integration, etc.) - `pi-fork` — the `fork` tool for spawning sub-agents - `pi-observational-memory` — the `recall` tool for session compaction +- `pi-atelier` — TUI sidebar: ordered panels, split-pane, themes. Pinned to an + audited tag; see [Version pins](#version-pins-pi-pi-atelier-mempalace) ### MemPalace (AI memory) @@ -338,6 +340,59 @@ DOT syntax errors instead of crashing. Then in Studio: open the PNG (or a `.md` that embeds it) and hit **refresh-from-disk** after each edit. Note: SVG is **not** in Studio's local-image-link allowlist — use PNG. +## Using pi-atelier (TUI sidebar) + +`pi-atelier` is bundled in **both** variants (vendored at `/opt/pi-atelier`, +pinned — see [Version pins](#version-pins-pi-pi-atelier-mempalace)). It adds two +things to pi's terminal UI: + +- a **status rail** — activity, token/cost metrics, context usage, model, git + state, extension statuses, and a menu; +- a **sidebar** — ordered panels (agent, activity, alerts, TODOs, context, + workspace, usage, tools) in a split pane beside the transcript. + +Nothing needs installing; the entrypoint registers it on container start, and it +binds on the next pi start (or `/reload`). + +| Action | How | +|---|---| +| Open the atelier menu | `alt+a`, or `/atelier` | +| Toggle the sidebar for this session | `/atelier sidebar on` / `off` | +| Change settings persistently | atelier menu → **Settings**, then **Save** | +| Turn the whole thing off | `DEVBOX_ATELIER=0` in `.env` | + +If your terminal or keymap swallows `alt+a`, use `/atelier` and pick a different +`shortcut` in the config file below. + +### Config + +Config lives at `~/.pi/agent/pi-atelier.json` on the `devbox-pi-config` volume, +seeded from pi-toolkit with container-appropriate defaults: compact density, +context warnings at 60/85 % (earlier than upstream's 70/90), sidebar tool names +on, and desktop completion notifications **off** (a container has nowhere useful +to pop a toast). + +It is **copied, not symlinked** — atelier rewrites this exact path when you hit +**Save**, using write-temp-then-`rename(2)`, and `rename` replaces a symlink with +a regular file instead of following it. A symlink would silently detach on your +first save. Consequently pi-toolkit's `install.sh` only seeds the file when it is +absent: once you have saved your own preferences, image upgrades leave them +alone, and `install.sh` prints a diff hint instead of clobbering. + +The seeded file uses atelier's **current** schema — `segmentLayout` with explicit +per-segment visibility, plus `showSidebarAgent` / `showSidebarTodos` / +`showSidebarOnStartup`. Older configs written against the pre-0.7 vocabulary +(`segments`, `ornament`, `showExtensionStatuses`) still load, but only through +upstream's legacy-compatibility shims — so if you are carrying one on an old +volume, expect it to keep working while missing every sidebar control added +since. `sidebarPanelLayout` is deliberately left unset so the panel set follows +upstream's product default as atelier adds panels; set it only if you want to +pin the order yourself. + +The sidebar auto-hides below 92 terminal columns and keeps the main pane at +least 64 columns wide, so a narrow terminal degrades to the plain TUI rather +than a squeezed one. + ## docker-compose.yml — basic shape ```yaml @@ -554,6 +609,35 @@ User-level overrides in `~/.ssh/config` win because Debian's `/etc/ssh/ssh_config` includes `/etc/ssh/ssh_config.d/*.conf` before the `Host *` block. +### macOS-only keywords in a shared `~/.ssh/config` + +The same `~/.ssh/config` is read by macOS ssh *and* by the Linux OpenSSH inside +the container (the sidecar `Include`s it). macOS-only keywords are **fatal** +there, not ignored — a single `UseKeychain yes` in a `Host *` block takes down +every ssh call in the container: + +``` +/home/developer/.ssh/config: line 2: Bad configuration option: usekeychain +/home/developer/.ssh/config: terminating, 1 bad configuration options +``` + +That breaks `dssh`/`dscp`, `pi --ssh`, `scp`, and anything that shells out to +ssh (including CI/deploy helpers), while the host keeps working perfectly — so +it presents as a container regression rather than a host config error. Guard the +keyword on the host, *before* it is used: + +```diff + Host * ++ IgnoreUnknown UseKeychain + UseKeychain yes + AddKeysToAgent yes +``` + +`IgnoreUnknown` is understood by both implementations: macOS still honours +`UseKeychain`, Linux skips it. Also keep such a `Host *` block **below** any +`Include` that must come first — OrbStack's own `Include ~/.orbstack/ssh/config` +says so in a comment, and a `Host *` block above it silently violates that. + ### Per-host `ControlPath` on a read-only `~/.ssh` `~/.ssh` is usually bind-mounted read-only, so a user `~/.ssh/config` that @@ -573,7 +657,10 @@ this without editing the read-only config: jump via the host, add `ProxyJump host` overrides in the host-owned `~/.config/devbox-shell/ssh-lan.conf` (see [Naming LAN peers](#naming-lan-peers)) rather than the read-only - `~/.ssh/config`. + `~/.ssh/config`. If the peer also rejects the host's key — the usual case, + since host keys are normally passphrase-protected and the container has no + Keychain or agent — see + [Giving the container its own key for a peer](#giving-the-container-its-own-key-for-a-peer). ## tmux and 0-indexed sessions @@ -646,6 +733,7 @@ repoint each one at a mirror, another host, or a local `file://` path | `MEMPALACE_TOOLKIT_REPO` | `https://gitea.jordbo.se/joakimp/mempalace-toolkit.git` | base | | `PI_FORK_REPO` | `https://github.com/elpapi42/pi-fork.git` | variant | | `PI_OBSMEM_REPO` | `https://github.com/elpapi42/pi-observational-memory.git` | variant | +| `PI_ATELIER_REPO` | `https://github.com/michaelmjhhhh/pi-atelier.git` | variant | | `PI_STUDIO_REPO` | `https://github.com/omaclaren/pi-studio.git` | variant | Each has a matching `*_REF` arg (branch name or commit SHA). Example — build @@ -750,14 +838,90 @@ Host pve pve-2 alpserv-2 lagret ProxyJump host ``` -`HostName` / `User` / `IdentityFile` are inherited from the matching block in -your real `~/.ssh/config` (first-value-wins, so only `ProxyJump` is taken from -here). This file is `Include`d *before* `~/.ssh/config` and read fresh on every -connection — newly added peers work immediately, no container or session -restart needed — and the peer names stay out of the published image (they're a -fact about your specific LAN, not the image). Alternatively, set -`DEVBOX_LAN_AUTOJUMP_PRIVATE=1` to ProxyJump *any* RFC1918 address through the -host without naming peers (see `.env.example`). +Any option can be set here, not just `ProxyJump`: the file is `Include`d +*before* `~/.ssh/config` and ssh takes the **first** value it sees for each +option, so whatever you put here wins while everything you omit is inherited +from the matching block in your real `~/.ssh/config`. Peer names stay out of the +published image (they are a fact about your LAN, not the image). Alternatively, +set `DEVBOX_LAN_AUTOJUMP_PRIVATE=1` to ProxyJump *any* RFC1918 address through +the host without naming peers (see `.env.example`). + +Once the file exists it is re-read on every connection, so *edits* take effect +immediately — no container or session restart. **Creating it for the first time +does need one restart**, because `setup-lan-access.sh` only emits the +`Include ~/.config/devbox-shell/ssh-lan.conf` line when the file is already +readable at container start (`if [ -r "$SSH_LAN_CONF" ]`). Until then ssh never +looks at it — which reads exactly like "my override is being ignored". + +#### Giving the container its own key for a peer + +`ProxyJump` fixes *routing*; it does not fix *authentication*, and inheriting +the host's `IdentityFile` usually fails inside the container: + +- Host keys are commonly passphrase-protected, and that passphrase is unlocked + by the macOS Keychain or a running `ssh-agent`. The container has neither, so + the key can never be decrypted — `Permission denied (publickey)` even though + the identical `ssh peer` works in a host terminal. +- `~/.ssh` is mounted read-only, so you can neither drop a container-usable key + in there nor edit `~/.ssh/config` from inside. + +The answer is a **container-only keypair** in `~/.ssh-local/` — the named volume +`devbox-ssh-local`, so it survives `docker compose up -d --force-recreate` — +plus an `IdentityFile` override in the host-owned `ssh-lan.conf`. Note that +nothing is baked into the *published image*: that volume is created on your +machine at runtime, so no private key ever ships to Docker Hub, and a fresh pull +elsewhere generates its own. (Every key below is a throwaway example.) + +**1. In the container** — generate a passphraseless key (there is no agent to +unlock a protected one): + +```bash +ssh-keygen -t ed25519 -N '' -C "devbox-$(hostname)" \ + -f ~/.ssh-local/mypeer_devbox_ed25519 +cat ~/.ssh-local/mypeer_devbox_ed25519.pub +# ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEXAMPLE0000EXAMPLE0000EXAMPLE0000ex devbox-0d11ec7731c7 +``` + +**2. On the peer** — authorize it narrowly rather than bare: + +``` +from="192.168.1.0/24,192.168.4.0/24,10.8.0.7",restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEXAMPLE0000EXAMPLE0000EXAMPLE0000ex devbox-mymachine +``` + +`restrict` disables pty, agent/X11 and port forwarding; append +`port-forwarding` and `permitopen="127.0.0.1:"` after it if you need one +specific tunnel. `from=` must list the **host's** addresses, not the +container's: container egress is NAT'd through the host, so the peer sees the +host's LAN address (confirm with `echo $SSH_CLIENT` on first login). List every +network the host roams — e.g. both home WLAN subnets plus its VPN address — +because a `from=` mismatch is indistinguishable from a wrong key in the error +message. + +**3. On the host** — point the peer at that key in +`~/.config/devbox-shell/ssh-lan.conf`: + +``` +Host mypeer mypeer.home.arpa + HostName 192.168.1.142 + User myuser + IdentityFile ~/.ssh-local/mypeer_devbox_ed25519 + IdentitiesOnly yes + # ProxyJump host # only if the container cannot reach the peer directly +``` + +That path exists only inside containers, which is why it belongs here rather +than in the shared `~/.ssh/config`. + +**4. First time only** — restart the container so the `Include` is emitted (see +above), then verify with the master socket bypassed, so a warm connection cannot +fake a pass: + +```bash +ssh -F ~/.ssh-local/config -o ControlPath=none mypeer 'echo $SSH_CLIENT' +``` + +Use one key per machine (`devbox-mbp`, `devbox-studio`, …) so a single +`authorized_keys` line can be revoked without locking out the others. ### Smoke-testing a local build @@ -794,9 +958,72 @@ pi-devbox follows semver-ish: - **Minor** — new variants, significant base additions. - **Patch** — pi version bumps, smaller fixes. -The `pi --version` inside the image is asserted by smoke tests to -match the release tag's pi component, so version drift between the -image and the tag is caught at CI time. +The `pi --version` inside the image is asserted by smoke tests to match the +version CI resolved (since v1.7.0, the pin below), so drift between what was +intended and what actually got baked is caught at CI time rather than on a +user's pull. + +### Version pins: pi, pi-atelier, mempalace + +Three components are pinned to an exact version **in the repo** instead of being +resolved to `latest` at build time: + +| Component | Pin | Where | +|---|---|---| +| pi | `0.84.1` | `ARG PI_VERSION` — `Dockerfile.variant` | +| pi-atelier | `v0.8.0` | `ARG PI_ATELIER_REF` — `Dockerfile.variant` | +| mempalace | `3.6.0` | `ARG MEMPALACE_VERSION` — `Dockerfile.base` | + +The objective is **not** to freeze versions. Bumping is routine — usually one +line plus a changelog note. The objective is that adopting a new upstream +version is a deliberate, reviewable act, not a side effect of whatever happened +to be published the morning CI ran. Each of these has already drawn blood: + +- **pi** — a minor release can move the private TUI/renderer internals that + pi-atelier wraps, or the session `.jsonl` format `pi-session-repair` parses. +- **pi-atelier** — 0.6.0/0.7.0 hang pi 0.84 **at startup**, burning CPU with no + error (fixed in 0.7.1/0.7.2). Its `peerDependencies` still say `>=0.80.7`, so + nothing in the npm metadata expresses the real floor. +- **mempalace** — an unpinned install once swept in the broken `diary_write` MCP + tool schema of 3.3.x/3.4.0, which is why that pin's comment requires a + tool-schema review before every bump. + +CI enforces this rather than trusting it: + +- `resolve-versions` reads the pins **out of the Dockerfiles** — single source of + truth, so a local `docker build` and a CI release ship the same versions — and + fails the build if a pin is not concrete, not a semver tag, or not actually + published on npm. +- When npm has a newer pi than the pin, CI emits a `::warning::` naming it. That + warning is the prompt to audit and bump; it never adopts the version. +- `smoke-test.sh` asserts the image's `pi --version` equals the pin, and + separately asserts the pairing rule **pi ≥ 0.84 ⇒ pi-atelier ≥ 0.7.1**, so a + bad combination fails the build instead of publishing a TUI that never starts. + +To bump pi: read the upstream CHANGELOG for every intervening version (TUI/theme +API, session format, extension loader, Node engine floor), re-check pi-atelier's +CHANGELOG for the pi version it claims to track, then edit the one `ARG` line and +record what you checked in `CHANGELOG.md`. + +#### If you previously hand-installed pi-atelier + +A hand-installed `pi install npm:pi-atelier` lands in `~/.pi/npm-global`, which +is on the `devbox-pi-config` **volume** — so it outlives image upgrades and stays +at whatever version you installed, unpinned and unaudited. Since the image now +vendors an audited pi-atelier at `/opt/pi-atelier`, the entrypoint removes a +lingering `npm:pi-atelier` entry from `packages[]` (after backing +`settings.json` up to `settings.json.bak.atelier.`) and registers the +pinned `/opt` copy instead. Nothing else in your settings is touched, and the +npm-global copy itself is left on disk — only the registration changes. + +This matters more than it sounds: leaving a 0.6.x npm copy registered alongside +pi 0.84 is precisely the combination that hangs at startup. + +To opt out of pi-atelier entirely, set `DEVBOX_ATELIER=0` in `.env`. The +entrypoint then removes any pi-atelier entry from `packages[]` on start. That +switch lives in the entrypoint — not in a pi command — deliberately: this +component's failure mode is "pi will not start", which you cannot repair with +`pi uninstall`. ## Acknowledgements diff --git a/entrypoint-user.sh b/entrypoint-user.sh index af8fd3b..bfa53cc 100755 --- a/entrypoint-user.sh +++ b/entrypoint-user.sh @@ -169,9 +169,9 @@ if command -v pi &>/dev/null; then "$HOME/.pi/agent/extensions/mempalace.ts" fi - # pi-fork (fork tool) + pi-observational-memory (recall tool) + (in the - # :latest-studio variant only) pi-studio (/studio command + studio_* - # tools + theme). These are pi packages (not symlink-style extensions): + # pi-fork (fork tool) + pi-observational-memory (recall tool) + pi-atelier + # (TUI sidebar panels/split-pane) + (in the :latest-studio variant only) + # pi-studio (/studio command + studio_* tools + theme). These are pi packages (not symlink-style extensions): # they're cloned to /opt with node_modules baked at BUILD time, then # registered here via `pi install `. A local-path install is # instant + in-place (pi loads the extension directly from /opt) + @@ -206,9 +206,63 @@ if command -v pi &>/dev/null; then fi } - for _pkg in /opt/pi-fork /opt/pi-observational-memory /opt/pi-studio; do + # ── pi-atelier: retire a stale `npm:pi-atelier`, plus an opt-out ────── + # The image now vendors pi-atelier at a pinned, audited tag (PI_ATELIER_REF + # in Dockerfile.variant). A leftover `npm:pi-atelier` entry from a + # hand-install resolves through ~/.pi/npm-global, which lives on the + # devbox-pi-config VOLUME — so it survives image upgrades and keeps whatever + # version was installed by hand, unpinned and unaudited. That is not + # academic: pi-atelier < 0.7.1 makes pi >= 0.84 hang at startup with + # sustained CPU, so leaving it in place turns a pi bump into a TUI that will + # not start. And `_pi_pkg_registered` deliberately counts `npm:` as + # registered (it respects a user's own npm install), so the loop below would + # never replace it. + # + # We only DELETE the exact `npm:pi-atelier` string; the loop then registers + # /opt/pi-atelier in pi's own canonical serialization, so this code never has + # to guess the stored relative-path form. Idempotent — after the rewrite + # there is no npm entry left to match. + # + # DEVBOX_ATELIER=0 goes further and removes pi-atelier from `packages` + # altogether. That escape hatch lives HERE, in the entrypoint, precisely + # because this component's known failure mode is "pi will not start" — which + # you cannot repair with `pi uninstall`. + _pi_atelier_drop() { + # $1 = jq predicate over one `packages` entry, selecting what to REMOVE. + # Returns 0 only when the file was actually rewritten (caller logs), 1 for + # "nothing to do" — including missing jq or unparseable JSON, which must + # never clobber user settings. Backs up first, same convention as the + # template merge above. + _ad_settings="$HOME/.pi/agent/settings.json" + [ -f "$_ad_settings" ] || return 1 + command -v jq >/dev/null 2>&1 || return 1 + _ad_new=$(jq "(.packages // []) |= map(select(($1) | not))" "$_ad_settings" 2>/dev/null) || return 1 + [ -n "$_ad_new" ] || return 1 + if printf '%s' "$_ad_new" | jq -e --slurpfile cur "$_ad_settings" '. == $cur[0]' >/dev/null 2>&1; then + return 1 + fi + # `.bak.atelier.` rather than the merge's plain `.bak.` prefix: both can + # fire in the same startup, and a bare seconds-resolution timestamp would + # make the second cp overwrite the first one's backup. + cp "$_ad_settings" "${_ad_settings}.bak.atelier.$(date +%Y%m%d-%H%M%S)" + printf '%s\n' "$_ad_new" > "$_ad_settings" + return 0 + } + if [ "${DEVBOX_ATELIER:-1}" = "0" ]; then + if _pi_atelier_drop '(. == "npm:pi-atelier") or ((type == "string") and endswith("/pi-atelier"))'; then + echo "pi-atelier: unregistered per DEVBOX_ATELIER=0 (settings backup saved)" + fi + elif [ -d /opt/pi-atelier ]; then + if _pi_atelier_drop '. == "npm:pi-atelier"'; then + echo "pi-atelier: dropped stale npm: registration — the pinned /opt copy takes over (settings backup saved)" + fi + fi + + for _pkg in /opt/pi-fork /opt/pi-observational-memory /opt/pi-studio /opt/pi-atelier; do [ -d "$_pkg" ] || continue _name=$(basename "$_pkg") + # DEVBOX_ATELIER=0 → leave pi-atelier unregistered (handled just above). + if [ "$_name" = "pi-atelier" ] && [ "${DEVBOX_ATELIER:-1}" = "0" ]; then continue; fi if ! _pi_pkg_registered "$_name"; then pi install "$_pkg" >/dev/null 2>&1 || \ echo "WARN: pi install $_name failed (continuing)" diff --git a/scripts/recreate-sanity-check.sh b/scripts/recreate-sanity-check.sh index 9eacb11..d86f25e 100755 --- a/scripts/recreate-sanity-check.sh +++ b/scripts/recreate-sanity-check.sh @@ -218,6 +218,15 @@ _pkg_registered() { fi } +# True when a literal `npm:pi-atelier` entry is still present — the +# volume-resident registration the entrypoint migrates away from. +_npm_atelier_present() { + _s="$HOME/.pi/agent/settings.json" + [ -f "$_s" ] || return 1 + command -v jq >/dev/null 2>&1 || return 1 + jq -e '(.packages // []) | any(. == "npm:pi-atelier")' "$_s" >/dev/null 2>&1 +} + if [ -f "$HOME/.pi/agent/settings.json" ]; then for pkg in pi-fork pi-observational-memory; do if _pkg_registered "$pkg"; then @@ -234,6 +243,47 @@ if [ -f "$HOME/.pi/agent/settings.json" ]; then fail "pi-studio NOT in settings.json packages[] (studio variant)" fi fi + + # pi-atelier — vendored from v1.7.0 on. Absent on older images, and + # deliberately unregistered when DEVBOX_ATELIER=0; neither is a failure. + if [ -d /opt/pi-atelier ]; then + if [ "${DEVBOX_ATELIER:-1}" = "0" ]; then + if _pkg_registered pi-atelier; then + fail "pi-atelier still in packages[] despite DEVBOX_ATELIER=0" + else + pass "pi-atelier unregistered (DEVBOX_ATELIER=0, as requested)" + fi + elif _pkg_registered pi-atelier; then + pass "pi-atelier registered in settings.json packages[]" + else + fail "pi-atelier NOT in settings.json packages[] (sidebar will not load)" + fi + if _npm_atelier_present; then + fail "stale npm:pi-atelier still in packages[] — it resolves through the ~/.pi/npm-global VOLUME and shadows the pinned /opt copy (entrypoint migration did not run)" + fi + fi +fi + +# ── pi <-> pi-atelier compatibility floor ───────────────────────────── +# atelier < 0.7.1 wraps pi's private TUI renderer in a way that recurses under +# pi >= 0.84: pi hangs at startup burning CPU, with no error message. atelier's +# own peerDependencies (>=0.80.7) do not encode this. Assert it here too, not +# just in the build-time smoke test: this script runs after a real +# `--force-recreate` on a live box, where a volume-resident old copy is exactly +# what could bite. +if [ -d /opt/pi-atelier ] && command -v jq >/dev/null 2>&1; then + _ge() { [ "$(printf '%s\n%s\n' "$1" "$2" | sort -V | head -n1)" = "$2" ]; } + _av=$(jq -r '.version // empty' /opt/pi-atelier/package.json 2>/dev/null || true) + _pv=$(pi --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -n1 || true) + if [ -n "$_av" ] && [ -n "$_pv" ]; then + if _ge "$_pv" 0.84.0 && ! _ge "$_av" 0.7.1; then + fail "pi $_pv with pi-atelier $_av — atelier < 0.7.1 hangs pi >= 0.84 at startup (bump PI_ATELIER_REF in Dockerfile.variant)" + else + pass "pi $_pv + pi-atelier $_av (compatibility floor OK)" + fi + else + warn "could not compare pi/pi-atelier versions (pi='$_pv' atelier='$_av')" + fi fi echo diff --git a/scripts/smoke-test.sh b/scripts/smoke-test.sh index 6b1016d..115924c 100755 --- a/scripts/smoke-test.sh +++ b/scripts/smoke-test.sh @@ -13,6 +13,8 @@ # - tmux 0-indexing baked in /etc/tmux.conf (required for pi-studio variants) # - pi-toolkit cloned at /opt/pi-toolkit # - pi-extensions cloned at /opt/pi-extensions +# - pi-atelier vendored at /opt/pi-atelier, registered from /opt (not npm:), +# and >= the version floor pi's TUI requires (see the floor test) # - pi-fork + pi-observational-memory cloned with node_modules baked # - entrypoint deploys pi-toolkit keybindings symlink # - entrypoint deploys ≥4 extensions @@ -137,6 +139,24 @@ run "pi-fork clone + node_modules" \ "test -f /opt/pi-fork/package.json && test -d /opt/pi-fork/node_modules" run "pi-observational-memory clone + node_modules" \ "test -f /opt/pi-observational-memory/package.json && test -d /opt/pi-observational-memory/node_modules" +# pi-atelier: deliberately NO node_modules assertion, unlike its siblings — +# it declares zero runtime dependencies (only peerDeps, satisfied by the baked +# pi) and has no build step, so Dockerfile.variant skips `npm install` for it. +# Assert what pi actually loads instead: the entry point named by its +# package.json `pi.extensions` key. +run "pi-atelier clone + entry point" \ + "test -f /opt/pi-atelier/package.json && test -f /opt/pi-atelier/extensions/index.ts" + +# ── pi <-> pi-atelier compatibility floor (executable, not a comment) ── +# pi-atelier < 0.7.1 wraps pi's PRIVATE TUI renderer in a way that recurses +# under pi >= 0.84: pi hangs at startup burning CPU, with no error. Upstream +# fixed it in 0.7.1/0.7.2, but atelier's peerDependencies still say +# `>=0.80.7`, so neither npm nor pi can warn about the real floor. Both +# versions are pinned in Dockerfile.variant; this makes a bad PAIRING fail the +# build instead of publishing an image whose TUI never starts. +run_expect "pi-atelier >= 0.7.1 floor for pi >= 0.84 (startup-hang guard)" \ + 'ge() { [ "$(printf "%s\n%s\n" "$1" "$2" | sort -V | head -n1)" = "$2" ]; }; AV=$(jq -r ".version // empty" /opt/pi-atelier/package.json 2>/dev/null); PV=$(pi --version 2>/dev/null | grep -oE "[0-9]+\.[0-9]+\.[0-9]+" | head -n1); if [ -z "$AV" ] || [ -z "$PV" ]; then echo "unreadable versions (atelier=$AV pi=$PV)"; elif ge "$PV" 0.84.0 && ! ge "$AV" 0.7.1; then echo "VIOLATION: pi $PV with pi-atelier $AV"; else echo "compatible: pi $PV + pi-atelier $AV"; fi' \ + "compatible:" # pi-studio is present only in the :latest-studio variant. Auto-detect by # probing /opt/pi-studio so this one script covers both variants. @@ -159,6 +179,8 @@ run "/etc/pi-devbox/build-manifest.json present" \ "test -f /etc/pi-devbox/build-manifest.json" run_expect "manifest records pi-extensions component" \ "cat /etc/pi-devbox/build-manifest.json" '"pi-extensions"' +run_expect "manifest records pi-atelier" \ + "cat /etc/pi-devbox/build-manifest.json" '"pi-atelier"' run_expect "manifest records pi_version" \ "cat /etc/pi-devbox/build-manifest.json" '"pi_version"' # Every component must be a resolved commit (or null for pi-studio in the @@ -271,6 +293,24 @@ if [ "${STUDIO_VARIANT:-0}" = "1" ]; then "$(pkg_registered_cmd pi-studio)" fi +# pi-atelier registration. It is LAST in the entrypoint's install loop, so a +# pass here also means that loop ran to completion rather than dying midway. +for i in $(seq 1 15); do + if docker exec -u developer "$CID" sh -c "$(pkg_registered_cmd pi-atelier)" \ + >/dev/null 2>&1; then + break + fi + sleep 1 +done +exec_test "pi-atelier registered in packages[] (TUI sidebar)" \ + "$(pkg_registered_cmd pi-atelier)" +# ...and registered from the vendored /opt copy, NOT as `npm:pi-atelier`: an +# npm: entry resolves through ~/.pi/npm-global on the config VOLUME, which +# outlives image upgrades and would silently keep an old, unaudited atelier — +# exactly the shape that pairs a stale 0.6.x with a new pi and hangs at startup. +exec_test "pi-atelier registered from /opt, not npm: (volume-shadowing guard)" \ + 'jq -e "((.packages // []) | any((type == \"string\") and endswith(\"/pi-atelier\"))) and (((.packages // []) | any(. == \"npm:pi-atelier\")) | not)" $HOME/.pi/agent/settings.json' + # ── /tmp/sshcm directory created by entrypoint ──────────────────────── exec_test "/tmp/sshcm dir mode 700 (ssh ControlMaster)" \ 'test -d /tmp/sshcm && [ "$(stat -c %a /tmp/sshcm)" = "700" ] && echo ok'