skills: record the vendored snapshot's provenance, and report which copy wins
Found while verifying v1.8.7 from inside a fresh container: the baked mempalace
snapshot is read by no host on this fleet. devbox-skill-reconcile repoints
~/.agents/skills/mempalace at the mounted live clone (the v1.8.5 fix working as
designed), and all four compose stacks mount a workspace containing the
skillset. So the phrase canary that blocked v1.8.7's first tag polices a file
nobody opens, while the drift that could actually mislead an agent — a git pull
nobody ran in /workspace/skillset — was invisible from inside the container and
is invisible to CI by construction.
Record provenance instead of policing it, and move the check to where the
skillset actually is:
- Dockerfile.variant: ARG SKILLSET_SNAPSHOT_REF (the claim) + a sha256 of the
shipped bytes measured in the manifest layer (the fact), as manifest siblings
rather than components{} members, plus an OCI label. An ARG default, not a
CI-resolved output: no credential for the private skillset, no change at any
of the four variant build call sites, and a local docker build records what CI
does. Variant-only, so no base rebuild — check-base-hash.sh scans
Dockerfile.base alone, verified by running it.
- pi-devbox-version: a skills: section naming baked vs live <repo> @ <sha> per
vendored skill, and for mempalace whether the live copy is identical to the
baked fingerprint, at the same commit with uncommitted edits, or divergent.
entrypoint-user.sh passes the new --no-skills, because the banner prints
before the links exist and long before the reconcile runs.
- scripts/vendor-mempalace-skill.sh: refresh the file and rewrite the ref
together (a cp without an ARG bump makes the manifest lie, which is worse than
anonymity); --check verifies the claim against a real clone.
- 5 new smoke assertions (78 -> 83), mutation-tested through the real sh -c
path: 6 fabricated manifests, where a well-formed hash of the wrong file
proves the two manifest assertions are not redundant; the all-baked reporting
test verified to FAIL against a live-skillset environment.
Reviewed mid-flight by pi@emb-7kj4vr4g over the logstream (correlation
skillset-vendor-drift), which retracted its own earlier recommendation of a
build-time byte-compare against skillset HEAD and supplied the better framing:
the invariant is NON-CONTRADICTION, not currency. Byte parity on a fallback
would have cost a resync commit plus a ~67-min base rebuild for each of the four
skillset commits pushed in one evening. Its warning also found a real bug here:
the script now CONSTRUCTS the snapshot from `git show HEAD:<path>` instead of
copying the working tree, because a clean `git diff` says nothing about an
untracked file — the one input the first draft would have recorded a false ref
for. Tested: untracked, unstaged and staged-but-uncommitted all refuse, atomically.
Also fixes three stale in-repo markers of the same class the canary belongs to
(true when written, silently false at release): two dangling "Unreleased"
pointers and a typst line still marked Unreleased five releases after v1.4.0.
This commit is contained in:
+179
-1
@@ -11,6 +11,184 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`).
|
||||
|
||||
---
|
||||
|
||||
## Unreleased
|
||||
|
||||
The vendored `mempalace` skill snapshot stops being anonymous, and the
|
||||
container starts saying which copy of each skill it is actually reading.
|
||||
|
||||
**Also carried, previously undocumented:** `dbb7879` resynced the vendored
|
||||
`mempalace` snapshot to skillset `c04cd15` ("the withdrawal only holds where the
|
||||
bridge is live"), landed after the v1.8.7 tag and so absent from that image.
|
||||
⚠️ **A base rebuild is forced** (~67 min): both that resync and the
|
||||
`pi-devbox-version` / `entrypoint-user.sh` changes below touch inputs to
|
||||
`base_tag` (`rootfs/` and `entrypoint*.sh`). The provenance recording itself
|
||||
adds nothing to that cost — it lives entirely in `Dockerfile.variant`.
|
||||
|
||||
Both come from one finding, made while verifying v1.8.7 from inside a freshly
|
||||
recreated container: **the baked `mempalace` snapshot is read by no host on this
|
||||
fleet.** `~/.agents/skills/mempalace` is a symlink to `/workspace/skillset/skills/mempalace`
|
||||
— `entrypoint-user.sh` links the baked skill only `if [ ! -e ]`, and
|
||||
`devbox-skill-reconcile` then repoints the skillset-owned ones at the live clone
|
||||
(that is the v1.8.5 fix working as designed). All four compose stacks in
|
||||
`docker-compose-repo` mount a workspace containing the skillset, so the vendored
|
||||
copy is a CI/no-mount **fallback** and nothing else. Which means the
|
||||
`mempalace skill snapshot is current` canary — the assertion that blocked
|
||||
v1.8.7's first tag — polices a file that no agent on this fleet ever opens,
|
||||
while the drift that *could* actually mislead an agent (a `git pull` nobody ran
|
||||
in `/workspace/skillset`) was invisible from inside the container and is
|
||||
invisible to CI by construction.
|
||||
|
||||
**The rejected fix is worth recording, because it was the obvious one.** The
|
||||
old comment in `scripts/smoke-test.sh` said the real answer was "a CI job
|
||||
diffing this file against the skillset repo". It isn't:
|
||||
|
||||
| Objection | Detail |
|
||||
|---|---|
|
||||
| needs a credential CI does not have | the skillset is **private** (`ssh://git@gitea.jordbo.se:2222/joakimp/skillset.git`); every build-time clone in this image uses anonymous HTTPS, and `resolve-versions`' `gitea_sha()` is explicitly documented as public-repo-only — its 401/403 path exists to survive a *stale token against a public repo*, so a private 403 would return empty and `require_sha` would hard-abort the release |
|
||||
| makes another repo's branch able to fail this build | the same pi-devbox commit would go green today and red tomorrow, and a release could be blocked by an edit in an unrelated repo — precisely the shape of the run 589 failure, but automated and permanent |
|
||||
| pure churn, and it is measurable | pi@emb-7kj4vr4g pushed **four** skillset commits in one evening (`d9dbbbd`, `b740d51`, `3324bd0`, `c04cd15`); a byte-parity gate would have demanded a pi-devbox resync commit **and a ~67-minute base rebuild for each one**, to keep current a copy almost nobody resolves |
|
||||
| guards the wrong artefact | see above: on this fleet, nobody reads it |
|
||||
|
||||
**The invariant is not currency, it is non-contradiction** — the framing comes
|
||||
from pi@emb-7kj4vr4g's review (logstream `project/pi-devbox`, correlation
|
||||
`skillset-vendor-drift`, which also **retracted** its own earlier build-time
|
||||
byte-compare recommendation). A stale-but-self-consistent fallback is harmless;
|
||||
a stale fallback carrying a **withdrawn instruction** is a live footgun, and
|
||||
this project has already paid for that one — through v1.8.4 the baked snapshot
|
||||
*shadowed* the live clone, which is how superseded attribution guidance kept
|
||||
reaching agents. That is precisely what the bidirectional canary asserts, and
|
||||
why it stays.
|
||||
|
||||
So provenance is **recorded** rather than policed, and the check moves to where
|
||||
the skillset actually is — a maintainer's clone, or any running container.
|
||||
|
||||
### Added
|
||||
|
||||
- **`build-manifest.json` now records the vendored snapshot's provenance:
|
||||
`skillset_snapshot_ref` (which skillset commit the bytes are claimed to come
|
||||
from) and `skillset_snapshot_sha256` (the bytes that actually shipped).** The
|
||||
ref is a plain `ARG` **default in `Dockerfile.variant`**, deliberately not a
|
||||
CI-resolved output, which buys three things at once: it needs no credential
|
||||
for a private repo; it keeps a local `docker build` and CI identical by
|
||||
construction (the same reasoning that put `MEMPALACE_VERSION` in
|
||||
`Dockerfile.base` rather than duplicating it in the workflow); and it requires
|
||||
**no change at any of the four `Dockerfile.variant` call sites** (`smoke`,
|
||||
`smoke-studio`, `build-variant`, `build-variant-studio`), whose `--build-arg`
|
||||
lists are hand-duplicated and therefore easy to under-apply to only two.
|
||||
Also emitted as OCI label `se.jordbo.pi-devbox.skillset-snapshot-ref`, so it
|
||||
is readable off the registry without pulling the image.
|
||||
|
||||
Two design points, each arrived at from the file's own rules:
|
||||
|
||||
- **The ref is a claim; the hash is measured.** `Dockerfile.variant` writes
|
||||
the manifest from ground truth (`rev()` on each `/opt` clone, the live
|
||||
`pi --version`), so the snapshot hash is computed with `sha256sum` in that
|
||||
same layer rather than passed in. A build where the two disagree is exactly
|
||||
what the new smoke assertions catch.
|
||||
- **They are siblings, not members of `components{}`.** That map means "HEAD
|
||||
of a clone present in this image" and the skillset is not cloned here —
|
||||
calling it a component would be a lie a future reader would act on. It is
|
||||
also load-bearing mechanically: `pi-devbox-version` renders every
|
||||
`components{}` value with `.value[0:12]`, which would truncate a 64-hex
|
||||
digest into something that looks like a short commit. Same reasoning as
|
||||
`mempalace_version`'s existing comment.
|
||||
|
||||
⚠️ **Costs no base rebuild.** `base_tag` hashes `Dockerfile.base` + `rootfs/`
|
||||
+ `entrypoint*.sh` + the mempalace-toolkit SHA; `Dockerfile.variant` is in
|
||||
none of it. `scripts/check-base-hash.sh` scans `Dockerfile.base` **only**
|
||||
(`DF="Dockerfile.base"`, single hardcoded path), so a new `*_REF` ARG in the
|
||||
variant is invisible to that guard — correctly, since it changes nothing
|
||||
about the base's contents.
|
||||
|
||||
- **`pi-devbox-version` gained a `skills:` section** reporting, per vendored
|
||||
skill, whether the live copy is `baked` or a `live <repo> @ <sha>` clone —
|
||||
and for `mempalace`, whether that live copy matches the baked fingerprint:
|
||||
`(identical to baked snapshot)`, `(baked snapshot <ref> + uncommitted edits)`
|
||||
when the clone is at the recorded commit but the bytes differ, or
|
||||
`(baked snapshot <ref> — live copy differs)`. Same live-vs-baked shape as the
|
||||
existing `pi:`/`palace:` drift annotations. **This is the check CI cannot do
|
||||
and a container can, for free**, since every host that matters already has the
|
||||
skillset mounted. The list iterates the baked tree rather than a hardcoded
|
||||
name list, so vendoring a fourth skill needs no edit here.
|
||||
|
||||
`entrypoint-user.sh` calls it with the new **`--no-skills`** flag: the banner
|
||||
is printed FIRST, before the baked links exist and long before the skillset
|
||||
deploy and reconcile run last, so anything it said about skill sources would
|
||||
describe a state that is about to change. Wrong-but-plausible is worse than
|
||||
absent. (This is the one part of the change that touches `rootfs/` and
|
||||
`entrypoint-user.sh`, so it does cost a base rebuild — already sunk, since
|
||||
`dbb7879` refreshed the vendored snapshot.)
|
||||
|
||||
- **`scripts/vendor-mempalace-skill.sh`** — refreshes the snapshot and rewrites
|
||||
the recorded ref *together*, because a `cp` without a matching ARG bump
|
||||
produces a manifest that confidently lies, which is worse than the anonymous
|
||||
snapshot it replaced. Refuses to record a ref when the upstream file has
|
||||
uncommitted modifications (no commit describes those bytes, so recording one
|
||||
would be a fabrication) — checked on that one file, not the whole tree, so
|
||||
unrelated work in progress in the skillset does not block a vendoring.
|
||||
`--check` answers "is the committed snapshot really `skillset@<recorded
|
||||
ref>`?" and separately reports staleness against the clone's HEAD.
|
||||
|
||||
Counterfactual-tested rather than reasoned about, against throwaway clones:
|
||||
a tampered snapshot reports `MISMATCH` **and** `STALE` (rc 1); a ref rolled
|
||||
back to the previous skillset commit reports `MISMATCH` with content
|
||||
unchanged (rc 1) and a subsequent refresh fixes only the ref, leaving the
|
||||
bytes alone; unstaged and staged-but-uncommitted upstream edits are refused
|
||||
with distinct messages and the snapshot left byte-identical, i.e. the refusal
|
||||
is atomic.
|
||||
|
||||
**Hardened after review** by pi@emb-7kj4vr4g, whose warning was that a resync
|
||||
script must "write the ref it ACTUALLY copied from, or the provenance field
|
||||
inherits the same class of bug the canary just had". The first draft copied
|
||||
the working tree and guarded it with `git diff` — which says nothing about an
|
||||
**untracked** file, and can be clean on a detached or behind checkout while
|
||||
`HEAD` names something else. The snapshot is now *constructed* from
|
||||
`git show HEAD:<path>`, so the recorded pair cannot be a lie by construction,
|
||||
and the untracked case is refused explicitly (tested: it was the one input the
|
||||
first draft would have silently recorded a false ref for). Both new scripts are
|
||||
`bash -n` clean and `shellcheck -S error` clean — the gate v1.8.7 added.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Three stale in-repo markers, all the same failure class.** Two "Unreleased"
|
||||
pointers — `scripts/smoke-test.sh` pointed the reader at "the Unreleased
|
||||
changelog note", and the v1.8.6 correction at "the Unreleased entry above";
|
||||
that section became the `## v1.8.7` heading at release time and neither
|
||||
back-reference was updated. The third: `scripts/smoke-test.sh`'s own coverage
|
||||
list still advertised "typst PDF engine for pandoc **(Unreleased)**", five
|
||||
releases after typst shipped in v1.4.0. Same class as the canary they sit
|
||||
next to: true when written, silently false at release, with nothing checking
|
||||
them. The smoke comment now describes the mechanism that actually shipped
|
||||
(and why the CI-diff idea it advertised was rejected); the changelog one names
|
||||
v1.8.7; the typst line names v1.4.0.
|
||||
|
||||
### Not fixed, deliberately
|
||||
|
||||
- **CI still cannot tell you the vendored snapshot is behind `skillset` main.**
|
||||
That needs a read-only deploy key for a private repo threaded into
|
||||
`resolve-versions`, to warn about a file no host on this fleet reads. Revisit
|
||||
when a no-skillset container becomes a real deployment (shipping the image
|
||||
outside the fleet, or a CI-only agent) — at which point the honest gate is a
|
||||
**warning**, matching the existing `PI_VERSION`/`MEMPALACE_VERSION` policy
|
||||
(concreteness → error, newer-release-exists → warning), never a build
|
||||
failure.
|
||||
- **The phrase canary stays.** It is orthogonal and free: it pins *content*
|
||||
where the new fields pin *provenance*, so it still catches a re-vendored
|
||||
snapshot whose ref was bumped correctly but whose bytes came from the wrong
|
||||
place — and, per the review above, asserting the **absence of withdrawn
|
||||
guidance** is the half of it that earns its keep. Its comment now states the
|
||||
limit instead of promising a fix.
|
||||
- **v1.8.7's published image has no recorded ref**, and that is expected: the
|
||||
field arrives here. Worth knowing when reading one, since the tag move
|
||||
`ebd0de0` → `f645e66` means the published v1.8.7 carries a pre-`dbb7879`
|
||||
snapshot, i.e. its baked mempalace skill lacks c04cd15's "confirm the bridge
|
||||
actually stamps" caveat. Harmless — on v1.8.7 the bridge *is* live, so that
|
||||
caveat self-retires, and every enrolled host reads the live clone anyway.
|
||||
`pi-devbox-version` degrades quietly on such an image: no fingerprint, no
|
||||
annotation, verified against the real v1.8.7 manifest.
|
||||
|
||||
---
|
||||
|
||||
## v1.8.7 — 2026-08-25
|
||||
|
||||
Patch release, and the fastest turnaround in the series (~9 h after v1.8.6) for
|
||||
@@ -457,7 +635,7 @@ and did not require a toolkit-side change.
|
||||
**CORRECTION (2026-08-25, post-tag):** this bullet is wrong and was never
|
||||
true of the tagged tree. `pi-devbox-version` *does* print a `palace:` line
|
||||
in human mode, with live-vs-baked drift detection, degrading quietly on
|
||||
pre-v1.8.6 manifests. Nothing is open here. See the Unreleased entry above.
|
||||
pre-v1.8.6 manifests. Nothing is open here. See the v1.8.7 entry above.
|
||||
|
||||
**Resolved during this release, not left open:** the feeder `--agent`
|
||||
default behavioural hook initially looked like it might need a
|
||||
|
||||
Reference in New Issue
Block a user