From 057827a84b0f02f1f9683ebdd0a4eb2d26cb6130 Mon Sep 17 00:00:00 2001 From: "pi@mbp-m1-2020" Date: Sat, 24 Oct 2026 11:26:26 +0200 Subject: [PATCH] skills: trim two descriptions to a routing sentence, and assert their pointers Every skill's name and description sit in the system prompt of EVERY request -- that block is the routing index -- while bodies are fetched on demand through the read tool. Measured on this device: 24 discoverable skills cost 10,609 description chars (~2,652 tokens) per request, against 474,656 chars of bodies (~119k tokens) that cost nothing until used. Bodies are 45x the descriptions, which is what the spec's 1024-char description limit is protecting. credential-incident-response (1022, two characters under the limit) and pi-devbox-environment (923) were each carrying a table of contents of what their body covers. Trimmed to what a description is actually for -- answering "should I load this?" -- reclaiming 1,087 chars (~271 tokens) of permanent context. 4,154 -> 3,067 across the five baked skills. WHY ONLY THESE TWO. Both are named BY PATH from the always-loaded managed block, so neither depends on description-matching to be found. pi-extensions is left alone deliberately: check-skill-floor.sh compares the vendored copy to the package repo by DIRECTORY HASH, so trimming ours alone registers as drift and turns that gate red; and it is the one skill that also ships in the skillset repo, where it runs on hosts with no pi-devbox managed block and genuinely needs its description. pi-devbox-firstboot stays fuller than needed because it is the skill a device with nothing else has to find. NOTHING WAS DELETED, ONLY RELOCATED -- and that was checked rather than assumed: all 18 enumerated items were confirmed present in the bodies BEFORE cutting (frontmatter stripped, so the description's own line could not score its own claim), and `git diff --stat` confirms both bodies are byte-identical. THE TRIM CREATES A DEPENDENCY, SO IT IS NOW ASSERTED. smoke-test.sh checks both pointers by path, next to the firstboot one. The premise of the cut is "the block names them", which holds only while those lines exist -- reword the block, drop a pointer, and the skill silently stops being reachable for a device that did not already know it was there. Verified discriminating: a reworded block fails the grep. An instrument failure caught on the way, and the reason the coverage check was re-run: my first pass used `grep -E 'a\|b'`, where \| is a LITERAL pipe in ERE, so 8 of 18 items read as 0 hits. Every zero was a pattern containing \| and every non-zero lacked one -- that correlation, not the content, was the finding. With real alternation all 18 are present. Loader re-verified on both trimmed skills: rc=0, no skill diagnostics. --- CHANGELOG.md | 26 +++++++++++++++++++ .../credential-incident-response/SKILL.md | 8 +----- .../skills/pi-devbox-environment/SKILL.md | 9 +------ scripts/smoke-test.sh | 11 ++++++++ 4 files changed, 39 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 87ea52c..10062cc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -73,6 +73,32 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`). the vendored copies are deliberately in scope: the image ships those bytes, so the violation is ours to carry whoever typed it. +- **Two baked skills' `description` fields trimmed to a routing sentence, + reclaiming ~1,087 characters of permanently-loaded context.** Every skill's + name and description sit in the system prompt of *every* request — that block + is the routing index — while bodies are fetched on demand through the read + tool. Measured here: 24 discoverable skills cost 10,609 description chars + (~2,652 tokens) on every request, against 474,656 chars of bodies (~119k + tokens) that cost nothing until used. `credential-incident-response` (1022, + i.e. two characters under the spec limit) and `pi-devbox-environment` (923) + were each carrying a table of contents of what their body covers. + + Both are named **by path** from the always-loaded managed block, so neither + depends on description-matching to be found — which is what makes the cut + safe, and is why only these two were cut. `pi-extensions` was left alone for + two reasons: `check-skill-floor.sh` compares the vendored copy to the package + repo by directory hash, so trimming ours alone registers as drift; and it is + the one skill that also ships in the `skillset` repo, where it runs on hosts + with no pi-devbox managed block and so genuinely needs its description to be + found. All 18 enumerated items were verified present in the bodies first — + the cut relocates nothing, and both bodies are byte-identical. + + `smoke-test.sh` now asserts both pointers by path, alongside the firstboot + one. The trim's premise is "the block names them", and that premise holds only + while those lines exist: reword the block, drop a pointer, and the skill + silently stops being reachable for a device that did not already know it was + there. Verified discriminating with a reworded negative control. + ### Fixed - **CI: a smoke job's image could be deleted mid-test by another job's disk diff --git a/rootfs/usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md b/rootfs/usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md index e6a6858..b51e545 100644 --- a/rootfs/usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md +++ b/rootfs/usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md @@ -6,13 +6,7 @@ description: >- agent-authored note. Load this whenever a task involves a leaked/exposed secret, a token rotation, a "is this credential still live?" question, deciding whether to delete or scrub stored content, proving a corpus is clean, or - choosing scopes for a new API token. Covers the mandatory order of operations - (probe the issuer FIRST — severity before cleanliness), leak-free identity via - sha256[:8] fingerprints and when publishing one is safe, - why revocation beats deletion for anything already replicated, scopes derived - from measured consumers, the three places a secret hides in a Chroma palace, how to prove ABSENCE rather than assume it (instrument strength, - census vs class passes, the tokenisation trap where quoting decides detectability, why git filters never run on symlinks, self-tests that abort), - where this fleet's secrets live, and what rotation does NOT fix. + choosing scopes for a new API token. --- # Credential incident response diff --git a/rootfs/usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md b/rootfs/usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md index d3e1b71..e1779ff 100644 --- a/rootfs/usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md +++ b/rootfs/usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md @@ -6,14 +6,7 @@ description: >- shell prompt is prefixed `[devbox]`, or `~/.ssh-local/config` is present) and the task touches any of: reaching the Docker host or its LAN, SSH, DNS name resolution, what survives container recreate (persistence vs ephemerality), - running Python or other REPLs, tmux, or the pi-studio browser UI. Covers the - persistence model, the interactive-vs-tool-shell alias gotcha - (dssh/dscp/cat=bat exist only in interactive bash), host + LAN SSH - reachability and ControlMaster, split-horizon DNS mechanisms, the tmux - 0-index constraint, uv-first Python, and pi-studio reachability. This skill - teaches MECHANISMS only — concrete hostnames, usernames, internal domains, - nameservers, and even the host OS vary per deployment and MUST be discovered - at runtime, never assumed or hardcoded. + running Python or other REPLs, tmux, or the pi-studio browser UI. --- # pi-devbox environment diff --git a/scripts/smoke-test.sh b/scripts/smoke-test.sh index 3990129..792ea26 100755 --- a/scripts/smoke-test.sh +++ b/scripts/smoke-test.sh @@ -304,6 +304,17 @@ run "mempalace session-start pointer merged into global AGENTS.md" \ # rewording the paragraph must not be able to silently break the link. run "firstboot skill pointer merged into global AGENTS.md" \ "grep -q 'pi-devbox-firstboot/SKILL.md' /opt/pi-toolkit/pi-global-AGENTS.md" +# Same path assertion for the two skills whose descriptions were deliberately +# TRIMMED to a routing sentence. Their frontmatter no longer enumerates what +# they cover, on the explicit premise that this block names them by path and so +# they never depend on description-matching to be found. That premise is only +# true while these lines exist, which is exactly why it is asserted rather than +# trusted: reword the block, drop a pointer, and the skill silently stops being +# reachable for a device that did not already know it was there. +run "pi-devbox-environment pointer merged into global AGENTS.md" \ + "grep -q 'pi-devbox-environment/SKILL.md' /opt/pi-toolkit/pi-global-AGENTS.md" +run "credential-incident-response pointer merged into global AGENTS.md" \ + "grep -q 'credential-incident-response/SKILL.md' /opt/pi-toolkit/pi-global-AGENTS.md" # Vendored fallback skills (so a no-skillset container still resolves the # AGENTS.md 'read the pi-extensions skill' pointer). run "image-baked pi-extensions fallback skill" \