skills: record the vendored snapshot's provenance, and report which copy wins
Found while verifying v1.8.7 from inside a fresh container: the baked mempalace
snapshot is read by no host on this fleet. devbox-skill-reconcile repoints
~/.agents/skills/mempalace at the mounted live clone (the v1.8.5 fix working as
designed), and all four compose stacks mount a workspace containing the
skillset. So the phrase canary that blocked v1.8.7's first tag polices a file
nobody opens, while the drift that could actually mislead an agent — a git pull
nobody ran in /workspace/skillset — was invisible from inside the container and
is invisible to CI by construction.
Record provenance instead of policing it, and move the check to where the
skillset actually is:
- Dockerfile.variant: ARG SKILLSET_SNAPSHOT_REF (the claim) + a sha256 of the
shipped bytes measured in the manifest layer (the fact), as manifest siblings
rather than components{} members, plus an OCI label. An ARG default, not a
CI-resolved output: no credential for the private skillset, no change at any
of the four variant build call sites, and a local docker build records what CI
does. Variant-only, so no base rebuild — check-base-hash.sh scans
Dockerfile.base alone, verified by running it.
- pi-devbox-version: a skills: section naming baked vs live <repo> @ <sha> per
vendored skill, and for mempalace whether the live copy is identical to the
baked fingerprint, at the same commit with uncommitted edits, or divergent.
entrypoint-user.sh passes the new --no-skills, because the banner prints
before the links exist and long before the reconcile runs.
- scripts/vendor-mempalace-skill.sh: refresh the file and rewrite the ref
together (a cp without an ARG bump makes the manifest lie, which is worse than
anonymity); --check verifies the claim against a real clone.
- 5 new smoke assertions (78 -> 83), mutation-tested through the real sh -c
path: 6 fabricated manifests, where a well-formed hash of the wrong file
proves the two manifest assertions are not redundant; the all-baked reporting
test verified to FAIL against a live-skillset environment.
Reviewed mid-flight by pi@emb-7kj4vr4g over the logstream (correlation
skillset-vendor-drift), which retracted its own earlier recommendation of a
build-time byte-compare against skillset HEAD and supplied the better framing:
the invariant is NON-CONTRADICTION, not currency. Byte parity on a fallback
would have cost a resync commit plus a ~67-min base rebuild for each of the four
skillset commits pushed in one evening. Its warning also found a real bug here:
the script now CONSTRUCTS the snapshot from `git show HEAD:<path>` instead of
copying the working tree, because a clean `git diff` says nothing about an
untracked file — the one input the first draft would have recorded a false ref
for. Tested: untracked, unstaged and staged-but-uncommitted all refuse, atomically.
Also fixes three stale in-repo markers of the same class the canary belongs to
(true when written, silently false at release): two dangling "Unreleased"
pointers and a typst line still marked Unreleased five releases after v1.4.0.
This commit is contained in:
+71
-3
@@ -7,7 +7,7 @@
|
||||
# - pi binary present and (if EXPECTED_PI_VERSION set) matches CI's resolved version
|
||||
# - mempalace core matches the audited pin (if EXPECTED_MEMPALACE_VERSION set)
|
||||
# - new v1.0.0 base additions (pandoc, graphviz, imagemagick, yq, tealdeer)
|
||||
# - typst PDF engine for pandoc (Unreleased) — `pandoc --pdf-engine=typst`
|
||||
# - typst PDF engine for pandoc (v1.4.0) — `pandoc --pdf-engine=typst`
|
||||
# - non-modal editors nano + micro (alongside nvim)
|
||||
# - terminfo for modern emulators: xterm-kitty, xterm-ghostty, wezterm,
|
||||
# alacritty, foot (kitty-terminfo + ncurses-term + compiled ghostty alias)
|
||||
@@ -463,6 +463,38 @@ run "pi-devbox-version --json round-trips the manifest byte-for-byte" '
|
||||
'
|
||||
run_expect "pi-devbox-version --quiet is a compact one-liner" \
|
||||
"pi-devbox-version --quiet | wc -l" "1"
|
||||
# ── Vendored skill snapshot provenance ─────────────────────────────────
|
||||
# The vendored mempalace skill is the one baked artefact with no /opt clone
|
||||
# behind it (private upstream — see VENDORED.md), so until now the manifest
|
||||
# could not say which skillset commit it came from. Two fields now travel with
|
||||
# it: the CLAIMED ref (ARG default in Dockerfile.variant) and the MEASURED
|
||||
# sha256 of the shipped bytes. Assert both are well-formed, and — separately —
|
||||
# that the measurement still describes the file in the image.
|
||||
#
|
||||
# Kept as two assertions for the same reason the component checks are: one
|
||||
# proves the fields are not empty/garbage, the other proves they are not merely
|
||||
# self-consistent. A single combined check could pass on a manifest whose hash
|
||||
# was computed from a file that was later overwritten (the pi-extensions skill
|
||||
# copy at Dockerfile.variant:165 does exactly that kind of overwrite, one stage
|
||||
# earlier), which is the failure this second one exists to catch.
|
||||
run "manifest records the vendored skill snapshot provenance" '
|
||||
j=/etc/pi-devbox/build-manifest.json
|
||||
r=$(jq -r ".skillset_snapshot_ref // empty" $j)
|
||||
s=$(jq -r ".skillset_snapshot_sha256 // empty" $j)
|
||||
echo "ref=[$r] sha256=[$s]" >&2
|
||||
printf "%s" "$r" | grep -qxE "[0-9a-f]{40}" \
|
||||
|| { echo "skillset_snapshot_ref is not a 40-hex commit" >&2; exit 1; }
|
||||
printf "%s" "$s" | grep -qxE "[0-9a-f]{64}" \
|
||||
|| { echo "skillset_snapshot_sha256 is not a 64-hex digest" >&2; exit 1; }
|
||||
'
|
||||
run "manifest skill fingerprint matches the baked snapshot" '
|
||||
j=/etc/pi-devbox/build-manifest.json
|
||||
f=/usr/local/share/pi-devbox/skills/mempalace/SKILL.md
|
||||
m=$(jq -r ".skillset_snapshot_sha256 // empty" $j)
|
||||
a=$(sha256sum "$f" | cut -d" " -f1)
|
||||
echo "manifest=[$m] actual=[$a]" >&2
|
||||
[ -n "$m" ] && [ "$m" = "$a" ]
|
||||
'
|
||||
# OCI labels live in the image config, not the container fs — inspect them
|
||||
# from the host docker rather than via `docker run`.
|
||||
LBL=$(docker inspect --format '{{ index .Config.Labels "se.jordbo.pi-devbox.pi-extensions-ref" }}' "$IMAGE" 2>/dev/null || true)
|
||||
@@ -539,8 +571,24 @@ exec_test "mempalace skill linked (fallback)" 'test -L $HOME/.agents/skills
|
||||
# one absent, so a re-vendored stale snapshot fails just as loudly as a
|
||||
# forgotten bump. A one-way canary only catches half the drift.
|
||||
# * a phrase canary can only ever detect "older than what I remembered to pin",
|
||||
# never "older than skillset main". The real fix is a CI job diffing this
|
||||
# file against the skillset repo — see the Unreleased changelog note.
|
||||
# never "older than skillset main".
|
||||
#
|
||||
# That structural limit is now addressed, but NOT by the "CI job diffing this
|
||||
# file against the skillset repo" this comment used to point at (that pointer
|
||||
# also dangled: it referenced an Unreleased changelog note that had become the
|
||||
# v1.8.7 heading). A CI diff cannot be done without granting CI a credential
|
||||
# for the PRIVATE skillset repo, and it would guard a file that on this fleet
|
||||
# NO host reads — all four compose stacks mount a workspace containing the
|
||||
# skillset, so devbox-skill-reconcile repoints this link at the live clone and
|
||||
# the baked copy is a CI/no-mount fallback only. Instead the snapshot now
|
||||
# carries its provenance (skillset_snapshot_ref + a measured
|
||||
# skillset_snapshot_sha256 in build-manifest.json, written by
|
||||
# scripts/vendor-mempalace-skill.sh), which moves the check to where the
|
||||
# skillset actually IS: `scripts/vendor-mempalace-skill.sh --check` for a
|
||||
# maintainer, and `pi-devbox-version` for an agent inside any container.
|
||||
# This assertion is kept because it is orthogonal and free: it pins content,
|
||||
# not provenance, so it still catches a re-vendored snapshot whose ref was
|
||||
# bumped correctly but whose bytes came from the wrong place.
|
||||
exec_test "mempalace skill snapshot is current" 'f=$HOME/.agents/skills/mempalace/SKILL.md; grep -q "Provenance is stamped for you" "$f" && ! grep -q "Attribute what you file yourself" "$f" && echo ok'
|
||||
# Link TARGETS, not just link existence: with no skillset mounted (as here) the
|
||||
# baked tree must be what resolves, for all three vendored skills.
|
||||
@@ -551,6 +599,26 @@ exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
|
||||
*) echo "$s resolves to $(readlink -f $HOME/.agents/skills/$s)" >&2; exit 1 ;;
|
||||
esac
|
||||
done; echo ok'
|
||||
# ... and that the tool REPORTS that resolution, which is the half that was
|
||||
# missing: a stale baked snapshot and a current live clone were
|
||||
# indistinguishable from inside the container. CI mounts no skillset, so every
|
||||
# vendored skill must report "baked" here — which also makes this a real test of
|
||||
# the fallback path rather than of the environment it happens to run in.
|
||||
exec_test "pi-devbox-version reports skill sources (all baked, no skillset here)" \
|
||||
'out=$(pi-devbox-version)
|
||||
echo "$out" | grep -q "skills:" || { echo "no skills section" >&2; exit 1; }
|
||||
for s in mempalace pi-extensions pi-devbox-environment; do
|
||||
echo "$out" | grep -qE "^ $s +baked$" \
|
||||
|| { echo "$s not reported as baked" >&2; exit 1; }
|
||||
done; echo ok'
|
||||
# The boot banner must NOT carry the section: entrypoint-user.sh prints the
|
||||
# version FIRST, before the baked links exist and long before the skillset
|
||||
# deploy + reconcile run last, so anything it said about skill sources would be
|
||||
# a pre-reconcile state that is about to change.
|
||||
exec_test "pi-devbox-version --no-skills omits the skills section" \
|
||||
'! pi-devbox-version --no-skills | grep -q "skills:"'
|
||||
exec_test "entrypoint prints the version banner with --no-skills" \
|
||||
'grep -q "pi-devbox-version --no-skills" /usr/local/bin/entrypoint-user.sh'
|
||||
# The handover path itself. CI never mounts a skillset, so without this the
|
||||
# v1.8.5 fix would ship untested: fabricate a skillset + a skills dir holding
|
||||
# baked-style links, run the reconciler, and assert all three outcomes —
|
||||
|
||||
Executable
+159
@@ -0,0 +1,159 @@
|
||||
#!/usr/bin/env bash
|
||||
# vendor-mempalace-skill.sh — refresh the vendored mempalace skill snapshot
|
||||
# AND its recorded provenance, together, so the two cannot drift apart.
|
||||
#
|
||||
# WHY THIS EXISTS
|
||||
# ---------------
|
||||
# rootfs/usr/local/share/pi-devbox/skills/mempalace/SKILL.md is a snapshot of a
|
||||
# file owned by the PRIVATE skillset repo (see VENDORED.md). Because the image
|
||||
# cannot clone that repo, refreshing the snapshot was a manual `cp` — and the
|
||||
# result was anonymous: nothing recorded WHICH skillset commit the bytes came
|
||||
# from. The only staleness check available was a hand-maintained phrase canary
|
||||
# in scripts/smoke-test.sh, which by construction detects "older than the phrase
|
||||
# I remembered to pin", never "older than skillset main".
|
||||
#
|
||||
# Two facts now travel with the snapshot: the skillset commit it was taken from
|
||||
# (ARG SKILLSET_SNAPSHOT_REF in Dockerfile.variant) and the sha256 of the bytes
|
||||
# themselves (measured at build time into build-manifest.json). This script is
|
||||
# the only thing that should ever write the first one, because a `cp` without a
|
||||
# matching ARG bump produces a manifest that CONFIDENTLY LIES — worse than the
|
||||
# anonymous snapshot it replaced.
|
||||
#
|
||||
# USAGE
|
||||
# scripts/vendor-mempalace-skill.sh [skillset-root] refresh + record
|
||||
# scripts/vendor-mempalace-skill.sh --check [root] verify, write nothing
|
||||
#
|
||||
# skillset-root defaults to /workspace/skillset, then $HOME/skillset.
|
||||
#
|
||||
# --check answers "is the committed snapshot really skillset@<recorded ref>?"
|
||||
# — the question CI cannot answer without a credential for the private repo,
|
||||
# and which anyone with the skillset checked out can answer for free. Exit 0
|
||||
# when the snapshot is honest and current, 1 when it is not.
|
||||
#
|
||||
# EXIT STATUS
|
||||
# 0 refreshed (or already up to date; --check: snapshot verified)
|
||||
# 1 refused: dirty upstream file, missing repo, or --check mismatch
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
DOCKERFILE="Dockerfile.variant"
|
||||
VENDORED="rootfs/usr/local/share/pi-devbox/skills/mempalace/SKILL.md"
|
||||
ARG_NAME="SKILLSET_SNAPSHOT_REF"
|
||||
REL_PATH="skills/mempalace/SKILL.md"
|
||||
|
||||
MODE="refresh"
|
||||
if [ "${1:-}" = "--check" ]; then
|
||||
MODE="check"
|
||||
shift
|
||||
fi
|
||||
|
||||
ROOT="${1:-}"
|
||||
if [ -z "$ROOT" ]; then
|
||||
for candidate in /workspace/skillset "$HOME/skillset"; do
|
||||
if [ -d "$candidate/.git" ]; then
|
||||
ROOT="$candidate"
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
die() { printf '%s: %s\n' "$(basename "$0")" "$1" >&2; exit 1; }
|
||||
|
||||
[ -n "$ROOT" ] || die "no skillset clone found (pass one: $(basename "$0") /path/to/skillset)"
|
||||
[ -d "$ROOT/.git" ] || die "not a git clone: $ROOT"
|
||||
[ -f "$ROOT/$REL_PATH" ] || die "no $REL_PATH in $ROOT"
|
||||
[ -f "$VENDORED" ] || die "vendored snapshot missing: $VENDORED"
|
||||
|
||||
head_sha=$(git -C "$ROOT" rev-parse HEAD 2>/dev/null) || die "cannot read HEAD of $ROOT"
|
||||
recorded=$(grep -oE "^ARG ${ARG_NAME}=[0-9a-f]{40}$" "$DOCKERFILE" | cut -d= -f2 || true)
|
||||
[ -n "$recorded" ] || die "no 'ARG ${ARG_NAME}=<40-hex>' line in $DOCKERFILE"
|
||||
|
||||
sha_of() { sha256sum "$1" | cut -d' ' -f1; }
|
||||
sha_empty=$(printf '' | sha256sum | cut -d' ' -f1)
|
||||
vendored_sha=$(sha_of "$VENDORED")
|
||||
upstream_sha=$(sha_of "$ROOT/$REL_PATH")
|
||||
|
||||
# The recorded ref must PROVABLY describe the recorded bytes. Reviewed by
|
||||
# pi@emb-7kj4vr4g (logstream correlation skillset-vendor-drift, 2026-08-26):
|
||||
# "make sure the resync script writes the ref it ACTUALLY copied from, or the
|
||||
# provenance field inherits the same class of bug the canary just had." So this
|
||||
# does not copy the working tree and then hope a `git diff` was enough — a
|
||||
# clean-looking `git diff` says nothing about an UNTRACKED file, and a detached
|
||||
# or behind checkout can be clean while HEAD names something else. Instead the
|
||||
# snapshot is CONSTRUCTED from the ref being recorded (below), and the working
|
||||
# tree is compared only so a local edit produces a refusal rather than a silent
|
||||
# surprise. Order matters here: everything this block reads is defined above it.
|
||||
blob_sha=$(git -C "$ROOT" show "HEAD:$REL_PATH" 2>/dev/null | sha256sum | cut -d' ' -f1 || true)
|
||||
upstream_dirty=""
|
||||
if [ -z "$blob_sha" ] || [ "$blob_sha" = "$sha_empty" ]; then
|
||||
upstream_dirty="not present at HEAD (untracked, or absent at this commit)"
|
||||
elif [ "$blob_sha" != "$upstream_sha" ]; then
|
||||
if ! git -C "$ROOT" diff --quiet -- "$REL_PATH" 2>/dev/null; then
|
||||
upstream_dirty="modified but not committed"
|
||||
elif ! git -C "$ROOT" diff --cached --quiet -- "$REL_PATH" 2>/dev/null; then
|
||||
upstream_dirty="staged but not committed"
|
||||
else
|
||||
upstream_dirty="different at HEAD than in the working tree"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$MODE" = "check" ]; then
|
||||
# Compare the committed snapshot against the file AT THE RECORDED REF, not
|
||||
# against the working tree: the question is whether the record is truthful,
|
||||
# which is independent of how current it is. Both are reported.
|
||||
at_ref=$(git -C "$ROOT" show "${recorded}:${REL_PATH}" 2>/dev/null | sha256sum | cut -d' ' -f1 || true)
|
||||
printf 'recorded ref: %s\n' "$recorded"
|
||||
printf 'vendored sha256: %s\n' "$vendored_sha"
|
||||
printf 'sha256 at ref: %s\n' "${at_ref:-<ref not present in this clone>}"
|
||||
printf 'skillset HEAD: %s (%s)\n' "$head_sha" "$(sha_of "$ROOT/$REL_PATH")"
|
||||
rc=0
|
||||
if [ -z "$at_ref" ]; then
|
||||
printf 'UNKNOWN: %s is not in this clone — fetch, or check against a complete one\n' "$recorded" >&2
|
||||
rc=1
|
||||
elif [ "$at_ref" != "$vendored_sha" ]; then
|
||||
printf 'MISMATCH: the vendored snapshot is NOT the file at the recorded ref\n' >&2
|
||||
rc=1
|
||||
else
|
||||
printf 'OK: the vendored snapshot is exactly skillset@%s:%s\n' "${recorded:0:7}" "$REL_PATH"
|
||||
fi
|
||||
if [ "$vendored_sha" != "$upstream_sha" ]; then
|
||||
printf 'STALE: the working tree of %s differs from the snapshot (HEAD %s)\n' \
|
||||
"$ROOT" "${head_sha:0:7}" >&2
|
||||
rc=1
|
||||
fi
|
||||
exit "$rc"
|
||||
fi
|
||||
|
||||
[ -z "$upstream_dirty" ] || die "$ROOT/$REL_PATH is $upstream_dirty — commit it first, or the recorded ref would not describe these bytes"
|
||||
|
||||
if [ "$vendored_sha" = "$upstream_sha" ] && [ "$recorded" = "$head_sha" ]; then
|
||||
printf 'already current: snapshot == skillset@%s\n' "${head_sha:0:7}"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Written FROM THE REF, not copied from the working tree, so the pair cannot be
|
||||
# a lie by construction. Via a temp file so a failed write cannot leave a
|
||||
# half-vendored snapshot behind.
|
||||
snap_tmp=$(mktemp)
|
||||
if ! git -C "$ROOT" show "HEAD:$REL_PATH" > "$snap_tmp" 2>/dev/null; then
|
||||
rm -f -- "$snap_tmp"
|
||||
die "cannot read HEAD:$REL_PATH from $ROOT"
|
||||
fi
|
||||
mv -- "$snap_tmp" "$VENDORED"
|
||||
[ "$(sha_of "$VENDORED")" = "$blob_sha" ] \
|
||||
|| die "internal: written snapshot does not match HEAD:$REL_PATH"
|
||||
# In-place, and only the exact pinned line: a broad sed on this Dockerfile could
|
||||
# rewrite one of the other *_REF ARGs.
|
||||
tmp=$(mktemp)
|
||||
sed "s|^ARG ${ARG_NAME}=.*$|ARG ${ARG_NAME}=${head_sha}|" "$DOCKERFILE" > "$tmp"
|
||||
mv -- "$tmp" "$DOCKERFILE"
|
||||
|
||||
new_recorded=$(grep -oE "^ARG ${ARG_NAME}=[0-9a-f]{40}$" "$DOCKERFILE" | cut -d= -f2 || true)
|
||||
[ "$new_recorded" = "$head_sha" ] || die "failed to rewrite ${ARG_NAME} in $DOCKERFILE"
|
||||
|
||||
printf 'snapshot: %s -> %s\n' "${vendored_sha:0:12}" "$(sha_of "$VENDORED" | cut -c1-12)"
|
||||
printf 'ref: %s -> %s\n' "${recorded:0:7}" "${head_sha:0:7}"
|
||||
printf '\nNOTE: %s is hashed into base_tag, so this costs a base rebuild\n' "$VENDORED"
|
||||
printf 'on the next tag (~67 min). Also re-pin the phrase canary in\n'
|
||||
printf 'scripts/smoke-test.sh if the section it names changed.\n'
|
||||
Reference in New Issue
Block a user