main
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joakim Persson
|
ff6fd1492a |
feat(manifest): record WHICH pi-extensions skill copy shipped
Closes the half deliberately left open by cac5e00's skill-floor gate, and the
more important half: "the floor is currently fresh" is a fact with a shelf
life, whereas "the image says which copy it got" keeps working.
The refresh in Dockerfile.variant is guarded by
`[ -f /opt/pi-extensions/skill/SKILL.md ]`, so a build whose clone predates the
co-located skill keeps the vendored floor and still succeeds GREEN, with nothing
in the manifest, labels or logs separating that from a normal build. Afterwards
the two are indistinguishable by inspection -- same path, same filenames, same
permissions -- which is exactly how the floor went unnoticed from 2026-07-30 to
2026-09-10.
build-manifest.json gains pi_extensions_skill_source and
pi_extensions_skill_tree_sha256, MEASURED rather than passed as build-args, per
the ground-truth rule the surrounding block already follows -- and necessarily
so, since the outcome depends on the clone's contents and no ARG could express
it. Three values, because two would force a lie: package (served bytes equal
the clone's skill/), vendored-floor (clone had no skill/ at this ref), and
divergent (both exist but differ -- e.g. the clone ships SKILL.md but not
evaluate-extension-usage.py, so the served directory is a genuine MIX). No OCI
label mirrors these deliberately: LABEL cannot take a RUN-computed value, and a
label fed from an ARG would be the claim-not-measurement being removed here.
Two smoke assertions make the record a gate: the source must be named and be
`package` -- vendored-floor FAILS rather than warns, since these images track
main where the package has shipped skill/ since
|
||
 joakimp
|
e8ddeaf89f |
skills: a gate that could pass without checking, and a mailbox that never empties
Fixes the three blockers and seven should-fixes from pi@emb-7kj4vr4g's review (logstream correlation skills-provenance-review, full text in drawer_pi-devbox_reviews_e43e766641c9ec85217bc6ce). Every finding was reproduced by execution here before being fixed; two were refined by that reproduction rather than taken as given. BLOCKER 1 — the provenance gate could print OK and exit 0 without verifying. `git show <ref>:<path> | sha256sum` hashes EMPTY STDIN when the ref does not resolve, so at_ref was never empty and the UNKNOWN branch was dead code. Measured: a bogus ref reported MISMATCH — accusing the snapshot of lying when the real cause was an incomplete clone, and the operator's natural remedy for MISMATCH is to re-run the refresh, which rewrites provenance to silence the complaint; and with a 0-byte snapshot against a 0-byte upstream file it printed "OK: exactly skillset@aaaaaaa" with exit 0 for a ref that does not exist. The script already had the sha_empty idiom and had applied it to blob_sha but not to at_ref. Existence is now PROVEN with git cat-file -e before anything is hashed, at two levels (ref resolves / path exists at it) because those deserve different messages. Same defect class as the canary it replaces: a check that can succeed without checking. A second, unflagged instance of the same pipeline shape in blob_sha was found and fixed too. Exit codes split, because the old contract failed the sanctioned case: 0 truthful (including stale, with a NOTICE), 1 a lying record only, 2 cannot determine. AGENTS.md step 2 promised "the message distinguishes the two" and was the thing this branch was breaking; rewritten to state all three. BLOCKER 3 — VENDORED.md contradicted itself in the release whose stated invariant is non-contradiction: its hand-maintained provenance line named skillset 670f7f1, seven commits behind the ARG and itself the commit that told agents to hand-stamp added_by — the withdrawn instruction this work exists to stop shipping — while its cp recipe contradicted the "not cp" rule 20 lines above. Line removed (nothing forced it to move when the ARGs did); 670f7f1 kept only as a labelled cautionary example. The pi-extensions half was verified redundant (CI require_sha resolves PI_EXTENSIONS_REF) before removal. SHOULD-FIXES: `<root> --check`, the spelling VENDORED.md documented, silently ran a REFRESH because only $1 was parsed (both tools now parse all args and reject unknown ones); refresh at a detached/older HEAD silently rewound ref and bytes (now refused unless the recorded ref is an ancestor, --force to override); upstream_dirty was computed and never used in check mode; --no-skills --json printed human text and broke jq; --help was a hardcoded sed range this branch had already made stale; the fingerprint hashed SKILL.md alone so a live skill differing only in a sibling file reported "identical", and pi-extensions already ships two files, so it is now a per-skill TREE hash with the manifest field renamed skillset_snapshot_tree_sha256; the --no-skills smoke assertion was negative-only and passed on a crashed binary. mktemp+mv left files 0600 — CI was unaffected since the index records 100644, so the blast radius was local builds only, narrower than the review inferred. Snapshot resynced c04cd15 -> 5fd0d5c so the no-clone fallback carries the CORRECTED coordination protocol rather than the withdrawn one; --check is now OK with no staleness notice, and the bidirectional canary re-verified against the new bytes. Local validation is bash -n only (shellcheck, hadolint and actionlint are all absent in this container) — CI remains the shellcheck gate. |
||
|
joakimp
|
e070e0bcbf |
skills: record the vendored snapshot's provenance, and report which copy wins
Found while verifying v1.8.7 from inside a fresh container: the baked mempalace
snapshot is read by no host on this fleet. devbox-skill-reconcile repoints
~/.agents/skills/mempalace at the mounted live clone (the v1.8.5 fix working as
designed), and all four compose stacks mount a workspace containing the
skillset. So the phrase canary that blocked v1.8.7's first tag polices a file
nobody opens, while the drift that could actually mislead an agent — a git pull
nobody ran in /workspace/skillset — was invisible from inside the container and
is invisible to CI by construction.
Record provenance instead of policing it, and move the check to where the
skillset actually is:
- Dockerfile.variant: ARG SKILLSET_SNAPSHOT_REF (the claim) + a sha256 of the
shipped bytes measured in the manifest layer (the fact), as manifest siblings
rather than components{} members, plus an OCI label. An ARG default, not a
CI-resolved output: no credential for the private skillset, no change at any
of the four variant build call sites, and a local docker build records what CI
does. Variant-only, so no base rebuild — check-base-hash.sh scans
Dockerfile.base alone, verified by running it.
- pi-devbox-version: a skills: section naming baked vs live <repo> @ <sha> per
vendored skill, and for mempalace whether the live copy is identical to the
baked fingerprint, at the same commit with uncommitted edits, or divergent.
entrypoint-user.sh passes the new --no-skills, because the banner prints
before the links exist and long before the reconcile runs.
- scripts/vendor-mempalace-skill.sh: refresh the file and rewrite the ref
together (a cp without an ARG bump makes the manifest lie, which is worse than
anonymity); --check verifies the claim against a real clone.
- 5 new smoke assertions (78 -> 83), mutation-tested through the real sh -c
path: 6 fabricated manifests, where a well-formed hash of the wrong file
proves the two manifest assertions are not redundant; the all-baked reporting
test verified to FAIL against a live-skillset environment.
Reviewed mid-flight by pi@emb-7kj4vr4g over the logstream (correlation
skillset-vendor-drift), which retracted its own earlier recommendation of a
build-time byte-compare against skillset HEAD and supplied the better framing:
the invariant is NON-CONTRADICTION, not currency. Byte parity on a fallback
would have cost a resync commit plus a ~67-min base rebuild for each of the four
skillset commits pushed in one evening. Its warning also found a real bug here:
the script now CONSTRUCTS the snapshot from `git show HEAD:<path>` instead of
copying the working tree, because a clean `git diff` says nothing about an
untracked file — the one input the first draft would have recorded a false ref
for. Tested: untracked, unstaged and staged-but-uncommitted all refuse, atomically.
Also fixes three stale in-repo markers of the same class the canary belongs to
(true when written, silently false at release): two dangling "Unreleased"
pointers and a typst line still marked Unreleased five releases after v1.4.0.
|
||
|
joakimp
|
93f986e90e |
v1.8.6: adopt pi 0.84.3 + mempalace 3.8.0, close the v1.8.5 doc/observability gaps
Lint / hadolint (push) Successful in 9s
Publish Docker Image / resolve-versions (push) Successful in 15s
Publish Docker Image / base-decide (push) Successful in 8s
Lint / actionlint (push) Successful in 1m9s
Publish Docker Image / build-base (push) Successful in 41m23s
Publish Docker Image / smoke (push) Successful in 4m50s
Publish Docker Image / smoke-studio (push) Successful in 5m5s
Publish Docker Image / build-variant (push) Successful in 15m46s
Publish Docker Image / update-description (push) Successful in 7s
Publish Docker Image / promote-base-latest (push) Successful in 15s
Publish Docker Image / build-variant-studio (push) Successful in 19m59s
Three coupled pieces of work, all of which ride on the base rebuild that the
mempalace bump forces anyway.
DRIFT ADOPTED
- pi 0.84.2 -> 0.84.3. Its release notes carry a "Breaking Changes" line
(GoogleThinkingLevel -> GoogleApiThinkingLevel). Audited before adopting:
zero references across all four vendored companions (pi-fork,
pi-observational-memory, pi-atelier, pi-studio), so it is inert for us. The
reason to adopt is two skill-discovery fixes that land directly on v1.8.5's
vendored-skill work: nested Markdown skills inside grouping directories were
not discovered, and root README.md/AGENTS.md in skill dirs were reported as
broken skills.
- mempalace core 3.7.1 -> 3.8.0. Additive/reliability only. Its sync fix
(#2320/#2322) stops sync --apply deleting drawers whose source_file was
unreachable *at that moment* -- which does NOT relax the standing landmine
against sync on the shared palace, because that landmine is about paths
permanently absent from whichever host runs the sync. Different failure
shape; the caution stands.
DOCS -- three defects, one of them public
- DOCKER_HUB.md advertised "neovim (LazyVim defaults)". Nothing in the image
installs LazyVim; the only nvim config is a 19-line sysinit.vim. CI PATCHes
this file into the Docker Hub description on every release, so this was a
false claim published to the world. Removed.
- agent-browser + Playwright + Chromium is the single largest addition in the
image (~625 MB) and had zero mentions in README, DOCKER_HUB or THIRD_PARTY --
it was documented only to agents, in the AGENTS.md managed block. Now
documented to humans, including the Chromium licence dimension.
- typst and socat appeared in README prose but not in the "What's inside"
inventory. Added.
OBSERVABILITY -- the three gaps v1.8.5 listed as still open
- build-manifest.json now records mempalace core, read from the live binary
(ground truth, not the build ARG). Placed as a sibling of pi_version rather
than inside components{}, because pi-devbox-version renders that map through
[0:12] and would truncate a version string.
- smoke asserts the pi-observational-memory clone actually CONTAINS the ce9fc98
auth fix, pinned to src/runtime.ts. Deliberately not a repo-wide grep: two of
the three markers also live under tests/, so the repo-wide form stays green
with the fix site reverted. That is the third false-green of this exact family
in this repo (canary phrase in both snapshots; reconciler fixture using a
non-owned name; now this) -- pin containment checks to the fix site.
- smoke asserts the feeder's pi@<device> agent default behaviourally. The
earlier audit concluded this needed a --print-config added upstream; it does
not. AGENT is assigned before arg parsing, so `bash -x mempalace-pi-session
--help` observes the real resolution with no toolkit change. Two-sided:
device set => pi@<device>, unset => must not be pi@*.
- pi-devbox-version now prints a palace: line with the same live-vs-baked drift
detection pi already had. This matters more than it looks: mempalace is the
one component that is both client (here) and server (synlig), so skew between
them is a real failure mode. Degrades quietly on pre-v1.8.6 images.
Deferred deliberately: a native arm64 act_runner on tor-ms22 (the current
runner is on synlig, x86_64, so every arm64 layer ships QEMU-emulated).
Analysis and caveats filed to the palace rather than actioned here.
|
||
|
joakimp
|
b5810654f6 |
skills: let the skillset own the skills it owns, and stop a dangling link from killing boot
Baked skill links won over the live skillset clone for all three vendored skills, so a pushed edit to skills/mempalace/SKILL.md was invisible in every container until the next image build -- measured on two hosts (live md5 129bcc4752 vs baked 5236024fef). Cause was ordering, not intent: the baked links are created early with a create-only-when-absent guard to close a smoke readiness race, and the skillset deploy runs last and treats them as foreign. The comment claimed the opposite of the behaviour. The fix is not "skillset always wins". Ownership is per-skill: pi-extensions is owned by its package repo and copied over the snapshot at build time, so the skillset's lagging duplicate must keep losing; pi-devbox-environment is authored here. Only mempalace is skillset-owned. devbox-skill-reconcile therefore runs after the deploy and repoints only the names in skills/skillset-owned.txt, replacing a link solely when it points into the baked tree, so a real directory or a link pointing elsewhere is never disturbed. Precedence is now user override -> live clone (owned names) -> baked snapshot, with the early links intact as the fallback so the readiness race stays closed. Reviewing that turned up a latent boot-abort in the pre-existing baked-link block: `[ ! -e "$link" ]` is TRUE for a dangling symlink, so once a link can point into /workspace/skillset, a vanished mount makes plain `ln -s` fail with "File exists" -- and under `set -euo pipefail` that aborts container start before `exec "$@"`. Reachable on `docker restart` or a host reboot, not on a recreate, since ~/.agents is not a volume on any host. Now `ln -sfn`, which heals the link back to the baked fallback. Smoke additions cover what let this ship: the stale-snapshot canary grepped a phrase present in BOTH the stale and fresh copies, so it passed throughout; it now pins the newest section. Link targets are asserted, not just `test -L`; the owned-list content is asserted both ways; and the reconciler's replace path -- which no CI container exercises, since none mounts a skillset -- is covered by fabricating one. A mutation test showed the obvious three assertions still pass with the "is this link ours?" guard deleted, so a discriminating case was added: an owned name whose link is a user override outside the baked tree. Also refreshes the mempalace snapshot to skillset 670f7f1 (without it the fix helps only hosts that mount skillset) and corrects README, which documented the old, wrong precedence in three places. Verified with 12 fixture cases plus 2 mutants: ownership respected against the real trees, user overrides preserved, relative/trailing-slash/CRLF/space/glob inputs handled, dangling link healed, read-only skills dir exits 0, idempotent. |
||
|
joakimp
|
fb49828826 |
base: add pi-devbox-version command + startup banner
Wraps /etc/pi-devbox/build-manifest.json (already written at docker-build time in Dockerfile.variant) into a human-readable summary instead of requiring users to know the manifest path and pipe it through jq themselves. - rootfs/usr/local/bin/pi-devbox-version: human (default) / --json / --quiet output modes. Also flags live drift — compares the baked pi_version against the actually-running `pi --version` and warns on mismatch rather than trusting the manifest blindly (same ground-truth philosophy as the manifest generation itself). Exits 1 with a short stderr notice on images built before the manifest existed, instead of failing silently. - entrypoint-user.sh: calls it as the very first line. CMD is `bash -l` with tty:true in compose, so this banner lands directly above the first prompt on container start — no separate motd/bashrc hook needed (deliberately not wired into .bash_aliases, which would reprint on every docker exec -it). - Dockerfile.base: COPY + chmod, same pattern as dot-watch/studio-expose. - scripts/smoke-test.sh: 4 new checks (binary present+executable, human output has release tag, --json round-trips the manifest, --quiet is a single line). - README.md / AGENTS.md / CHANGELOG.md updated. |
||
 pi
|
ab5ff8ec56 |
feat: bundle dot-watch helper for live graphviz .dot -> PNG re-render in Studio
pi-studio renders Mermaid natively but has no DOT renderer. Its markdown preview displays local PNG/JPG/GIF/WEBP images, so dot-watch closes the loop for Graphviz: edit .dot -> auto-render <name>.png -> Studio refresh-from-disk shows the update. Uses mtime polling (no inotify dep). - rootfs/usr/local/bin/dot-watch: the helper (executable) - Dockerfile.base: COPY + chmod, following the studio-expose pattern - README.md: 'Graphviz diagrams in Studio' subsection - CHANGELOG.md: Unreleased entry graphviz was already in the base image; no new package. |
||
|
pi
|
7d8ee4cea1 |
feat(studio): bundle studio-expose bridge + socat (opt-in STUDIO_EXPOSE)
pi-studio binds the container's 127.0.0.1, which a published Docker port can't reach. Add a robust, portable bridge rather than a doc-only one-liner: - Dockerfile.base: add socat (~1 MB, generally useful TCP relay). - rootfs/usr/local/bin/studio-expose: socat TCP relay listening on the container's egress IPv4 (not 0.0.0.0 — that would EADDRINUSE against Studio's loopback listener) forwarding to 127.0.0.1:PORT on the SAME port, so Studio's printed token URL works verbatim. Robust egress-IP detection (hostname -I, loopback-filtered; ip route get fallback), --help, port validation, foreground. - entrypoint-user.sh: opt-in STUDIO_EXPOSE=1 auto-starts the bridge in the background (studio variant only). Default OFF — Studio stays loopback-only (its secure default) unless explicitly opted in. - README: 'Using pi-studio' now documents host-networking (A) and the studio-expose/STUDIO_EXPOSE bridge (B) with a security note; ssh -L for remote, mosh caveat retained. - smoke-test: assert socat + studio-expose present (base-level). - CHANGELOG/AGENTS updated. No tag — stopping for review. |