skills: let the skillset own the skills it owns, and stop a dangling link from killing boot
Baked skill links won over the live skillset clone for all three vendored skills, so a pushed edit to skills/mempalace/SKILL.md was invisible in every container until the next image build -- measured on two hosts (live md5 129bcc4752 vs baked 5236024fef). Cause was ordering, not intent: the baked links are created early with a create-only-when-absent guard to close a smoke readiness race, and the skillset deploy runs last and treats them as foreign. The comment claimed the opposite of the behaviour. The fix is not "skillset always wins". Ownership is per-skill: pi-extensions is owned by its package repo and copied over the snapshot at build time, so the skillset's lagging duplicate must keep losing; pi-devbox-environment is authored here. Only mempalace is skillset-owned. devbox-skill-reconcile therefore runs after the deploy and repoints only the names in skills/skillset-owned.txt, replacing a link solely when it points into the baked tree, so a real directory or a link pointing elsewhere is never disturbed. Precedence is now user override -> live clone (owned names) -> baked snapshot, with the early links intact as the fallback so the readiness race stays closed. Reviewing that turned up a latent boot-abort in the pre-existing baked-link block: `[ ! -e "$link" ]` is TRUE for a dangling symlink, so once a link can point into /workspace/skillset, a vanished mount makes plain `ln -s` fail with "File exists" -- and under `set -euo pipefail` that aborts container start before `exec "$@"`. Reachable on `docker restart` or a host reboot, not on a recreate, since ~/.agents is not a volume on any host. Now `ln -sfn`, which heals the link back to the baked fallback. Smoke additions cover what let this ship: the stale-snapshot canary grepped a phrase present in BOTH the stale and fresh copies, so it passed throughout; it now pins the newest section. Link targets are asserted, not just `test -L`; the owned-list content is asserted both ways; and the reconciler's replace path -- which no CI container exercises, since none mounts a skillset -- is covered by fabricating one. A mutation test showed the obvious three assertions still pass with the "is this link ours?" guard deleted, so a discriminating case was added: an owned name whose link is a user override outside the baked tree. Also refreshes the mempalace snapshot to skillset 670f7f1 (without it the fix helps only hosts that mount skillset) and corrects README, which documented the old, wrong precedence in three places. Verified with 12 fixture cases plus 2 mutants: ownership respected against the real trees, user overrides preserved, relative/trailing-slash/CRLF/space/glob inputs handled, dangling link healed, read-only skills dir exits 0, idempotent.
This commit is contained in:
Executable
+91
@@ -0,0 +1,91 @@
|
||||
#!/bin/sh
|
||||
# devbox-skill-reconcile — hand skillset-OWNED skills back to the live clone.
|
||||
#
|
||||
# WHY THIS EXISTS
|
||||
# ---------------
|
||||
# entrypoint-user.sh links the image-baked skills into ~/.agents/skills/ EARLY
|
||||
# (before pi-deploy), because the smoke readiness probe gates on markers that
|
||||
# only land later, and a link created after that gate produced a flaky
|
||||
# assertion. Those links are created with a `[ ! -e ]` guard — "only when
|
||||
# absent" — and the skillset deploy runs LAST, treating already-present links
|
||||
# as foreign and leaving them alone. Net effect through v1.8.4: the baked copy
|
||||
# always won, so an edit pushed to a skillset-owned skill was invisible in
|
||||
# every container until the next image build (measured on two hosts: live
|
||||
# skillset md5 129bcc4752 vs baked 5236024fef, the new section absent).
|
||||
#
|
||||
# The fix is NOT "the skillset always wins". Ownership is per-skill (see
|
||||
# rootfs/usr/local/share/pi-devbox/skills/VENDORED.md):
|
||||
#
|
||||
# pi-devbox-environment authored in pi-devbox → baked IS canonical
|
||||
# pi-extensions owned by the package repo, copied over the snapshot
|
||||
# at build time; skillset carries a DOWNSTREAM copy
|
||||
# that can lag → baked must keep winning
|
||||
# mempalace owned by the skillset repo; baked is a snapshot
|
||||
# fallback for containers with no skillset mounted
|
||||
# → the live clone must win when it is present
|
||||
#
|
||||
# So only skills listed in skills/skillset-owned.txt are handed over. Baked
|
||||
# links stay as the fallback (the early-link race fix is untouched), and a user
|
||||
# override always beats both: a real directory is never replaced, and neither is
|
||||
# a symlink that already points somewhere other than the baked tree.
|
||||
#
|
||||
# Usage: devbox-skill-reconcile <skillset-root> [skills-dir] [baked-src]
|
||||
# skillset-root the mounted skillset repo (contains skills/<name>/)
|
||||
# skills-dir default $HOME/.agents/skills
|
||||
# baked-src default /usr/local/share/pi-devbox/skills
|
||||
#
|
||||
# Idempotent, and silent unless it changes something. Exits 0 when there is
|
||||
# nothing to do (no skillset, no list) so the entrypoint never fails on it.
|
||||
set -eu
|
||||
|
||||
SKILLSET_ROOT="${1:-}"
|
||||
SKILLS_DIR="${2:-$HOME/.agents/skills}"
|
||||
BAKED_SRC="${3:-/usr/local/share/pi-devbox/skills}"
|
||||
BAKED_SRC="${BAKED_SRC%/}" # a trailing slash would make the prefix
|
||||
# match below ("$BAKED_SRC"/*) match nothing
|
||||
|
||||
[ -n "$SKILLSET_ROOT" ] || exit 0
|
||||
[ -d "$SKILLSET_ROOT/skills" ] || exit 0
|
||||
[ -d "$SKILLS_DIR" ] || exit 0
|
||||
|
||||
# Absolutise BOTH roots before they are used, because each has its own way of
|
||||
# failing silently when relative: a relative symlink TARGET is resolved against
|
||||
# the link's directory (~/.agents/skills), not $PWD, so it would dangle on
|
||||
# creation; and a relative BAKED_SRC would never prefix-match the absolute
|
||||
# target that `readlink` reports, so every skill would be skipped and the fix
|
||||
# would look like it had simply done nothing.
|
||||
SKILLSET_ROOT=$(CDPATH= cd -- "$SKILLSET_ROOT" 2>/dev/null && pwd) || exit 0
|
||||
BAKED_SRC=$(CDPATH= cd -- "$BAKED_SRC" 2>/dev/null && pwd) || exit 0
|
||||
OWNED_LIST="$BAKED_SRC/skillset-owned.txt"
|
||||
[ -f "$OWNED_LIST" ] || exit 0
|
||||
|
||||
while IFS= read -r _line || [ -n "$_line" ]; do
|
||||
# strip comments and surrounding whitespace; skip blanks
|
||||
_name=$(printf '%s\n' "$_line" | sed -e 's/#.*$//' -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
|
||||
[ -n "$_name" ] || continue
|
||||
# defensive: a list entry must be a plain skill name, never a path
|
||||
case "$_name" in */*|.*) continue ;; esac
|
||||
|
||||
_live="$SKILLSET_ROOT/skills/$_name"
|
||||
_link="$SKILLS_DIR/$_name"
|
||||
|
||||
# the skillset does not ship it → the baked fallback is all there is
|
||||
[ -d "$_live" ] || continue
|
||||
# a real directory is a user override → never touch
|
||||
[ -L "$_link" ] || continue
|
||||
|
||||
# only ever replace OUR OWN link. readlink is deliberate: `readlink -f`
|
||||
# would resolve a link that already points into the skillset clone and,
|
||||
# since both trees hold a same-named skill, could not tell them apart.
|
||||
_target=$(readlink "$_link" 2>/dev/null || true)
|
||||
case "$_target" in
|
||||
"$BAKED_SRC"/*|"$BAKED_SRC") ;; # baked link → ours to replace
|
||||
*) continue ;; # user/foreign target → leave alone
|
||||
esac
|
||||
|
||||
# -n so an existing symlink-to-directory is replaced rather than followed
|
||||
# (without it, ln would create $_link/$_name inside the baked tree).
|
||||
if ln -sfn "$_live" "$_link" 2>/dev/null; then
|
||||
printf 'skill %s: baked snapshot -> live skillset (%s)\n' "$_name" "$_live"
|
||||
fi
|
||||
done < "$OWNED_LIST"
|
||||
@@ -1,9 +1,10 @@
|
||||
# Vendored fallback skills
|
||||
|
||||
Most directories here are **image-baked skills** that `entrypoint-user.sh`
|
||||
symlinks into `~/.agents/skills/` on container start (only when a skill of the
|
||||
same name is not already present, so a mounted `skillset` repo or a user
|
||||
override always wins).
|
||||
symlinks into `~/.agents/skills/` on container start. They are the **fallback**
|
||||
layer: see *Runtime precedence* below for which copy actually wins when a
|
||||
`skillset` repo is mounted (through v1.8.4 the answer was "always the baked
|
||||
one", which was a bug).
|
||||
|
||||
| skill | owner | how it gets here |
|
||||
|-------|-------|------------------|
|
||||
@@ -38,6 +39,35 @@ its skill file needed baking.
|
||||
*different* skill, `opencode-mempalace-bridge`), so there is no public
|
||||
package source to copy from. This snapshot is refreshed manually per release.
|
||||
|
||||
## Runtime precedence (v1.8.5+)
|
||||
|
||||
The baked links are created **early** in `entrypoint-user.sh` (before pi-deploy,
|
||||
to close a smoke readiness race) with a create-only-when-absent guard, and the
|
||||
skillset deploy runs **last** and treats them as foreign links. Through v1.8.4
|
||||
that combination meant the baked snapshot always won: an edit pushed to
|
||||
`skillset/skills/mempalace/SKILL.md` was invisible in every container until the
|
||||
next image build (measured on two hosts — live `md5 129bcc4752` vs baked
|
||||
`5236024fef`, new section absent). Editing those skills *appeared* to work.
|
||||
|
||||
`devbox-skill-reconcile` now runs immediately after the skillset deploy and
|
||||
repoints the links for skills the **skillset owns**, listed one per line in
|
||||
`skillset-owned.txt`. Precedence, highest first:
|
||||
|
||||
1. **user override** — a real directory, or a symlink pointing outside the baked
|
||||
tree; never touched by anything
|
||||
2. **live skillset clone** — but only for names in `skillset-owned.txt`
|
||||
3. **baked snapshot** — everything else, and every skill when no skillset is
|
||||
mounted
|
||||
|
||||
Ownership is per-skill on purpose: `pi-extensions`' authoritative source is the
|
||||
package repo (copied over the snapshot at build), and `skillset` carries a
|
||||
downstream copy that can lag, so handing it to the clone would *regress* the
|
||||
skill. Only `mempalace` is skillset-owned today.
|
||||
|
||||
Verify with `readlink -f ~/.agents/skills/<skill>` — not by reading the
|
||||
entrypoint. Smoke covers both directions (baked resolution with no skillset
|
||||
mounted, plus a fabricated-skillset run of the reconciler).
|
||||
|
||||
## Refreshing the snapshots
|
||||
|
||||
cp <pi-extensions-pkg>/skill/SKILL.md pi-extensions/SKILL.md
|
||||
@@ -50,4 +80,9 @@ also carries a copy, but it is a downstream duplicate and can lag), and
|
||||
`mempalace` from `skillset`. Copying `pi-extensions` from `skillset` would
|
||||
regress the snapshot to whatever that repo last mirrored.
|
||||
|
||||
Snapshot provenance at last refresh: skillset `936fed8`, pi-extensions pkg `e73cb9f`.
|
||||
Snapshot provenance at last refresh: skillset `670f7f1`, pi-extensions pkg `e73cb9f`.
|
||||
|
||||
When you refresh the `mempalace` snapshot, also update the phrase asserted by
|
||||
the "mempalace skill snapshot is current" smoke test — it deliberately pins the
|
||||
**newest** section, because the previous canary grepped a phrase that survived
|
||||
the very edit that made the snapshot stale, and so passed on stale content.
|
||||
|
||||
@@ -293,6 +293,7 @@ Zechner's pi-coding-agent). Implications:
|
||||
When the palace is **central** (shared across machines), five more things apply:
|
||||
|
||||
- **Check which machine a conversation came from.** Transcripts are fed per device, so `source_path` reads `…/mempalace-feed/<device>/pi_<uuid>.jsonl` while the displayed `source_file` is only the basename. One search can legitimately return hits from several machines at once — look at the device segment before attributing a decision to *this* project.
|
||||
- **Attribute what you file yourself.** Drawers now carry `device` and `agent_kind` metadata (plus `device_source`/`agent_kind_source` recording *how* each was determined, so an inference is never mistaken for a fact). Mined content gets these for free — the inbox path gives the device, the filename shape gives the harness — and a timer on the palace host re-stamps hourly, because live re-mining replaces metadata rows and silently drops earlier stamps. But for anything **you** file by hand, the only signal is what you pass: set `added_by="<harness>@<device>"` (e.g. `pi@emb-7kj4vr4g`, from `$MEMPALACE_PI_DEVICE`) on `add_drawer`/`checkpoint`/`mine`. Skip it and your drawer joins the ~16k historic `/workspace` project mines that are permanently unattributable, because `/workspace` exists identically on every devbox. Note the palace preserves `source_file` in full (see `source_path`) but *displays* only the basename — so a device prefix there survives storage even though it looks stripped.
|
||||
- **Mined drawers carry the MINE date, not the session date.** When history is imported, or re-mined on the palace host, `filed_at`/`created_at` is the *import* time — so sorting by them does not give chronological order. Real session time is recoverable from the UUIDv7 in `pi_<uuid>.jsonl`: the first 12 hex digits are milliseconds since the epoch (and UUIDv7 sorts lexicographically in time order, so a plain filename sort is already chronological). Agent-authored drawers and diaries have no such backdoor — for those `filed_at` is the only chronology, which is why it must never be restamped.
|
||||
- **Beware the timezone mismatch when you combine those.** Palace `filed_at`/`created_at` are naive timestamps in the palace host's local time, while a UUIDv7 decodes to UTC. Comparing them directly introduces a silent offset (2 h for a CEST host). Normalise before drawing conclusions about ordering.
|
||||
- **`agent_name` is not device-scoped.** `mempalace_diary_read(agent_name="pi")` returns *every* machine's `pi` diary, interleaved. Read the entry before assuming it is your own history.
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
# Skills in this directory whose OWNER is the skillset repo.
|
||||
#
|
||||
# Read by devbox-skill-reconcile, which runs after the skillset deploy in
|
||||
# entrypoint-user.sh: for each name below, if the mounted skillset ships a
|
||||
# skill of that name, the baked link in ~/.agents/skills/ is repointed at the
|
||||
# live clone. The baked copy remains the fallback for containers started
|
||||
# WITHOUT a skillset mount, and a user override always wins over both.
|
||||
#
|
||||
# Add a name here ONLY if the skillset repo is the authoritative source (see
|
||||
# the ownership table in VENDORED.md). Do NOT add:
|
||||
# pi-devbox-environment — authored in this repo; baked IS canonical
|
||||
# pi-extensions — owned by the pi-extensions package repo and copied
|
||||
# over the snapshot at build time; the skillset copy
|
||||
# is a downstream duplicate that can lag, so letting
|
||||
# it win would regress the skill.
|
||||
mempalace
|
||||
Reference in New Issue
Block a user