ci: gate baked skill frontmatter against the loader's own limits
Lint / skill-floor (push) Successful in 9s
Lint / hadolint (push) Successful in 10s
Lint / doc-drift (push) Successful in 18s
Lint / skill-frontmatter (push) Successful in 19s
Lint / actionlint (push) Successful in 32s

Found while adding pi-devbox-firstboot, whose description was drafted at 1056
characters against pi's MAX_DESCRIPTION_LENGTH of 1024. pi does not reject it:
dist/core/skills.js pushes a diagnostic and then loads the skill anyway --
"Still load the skill even with warnings, unless description is missing or
empty". So the skill works, the violation is real, and the only thing that
would ever report it is a diagnostic inside a live pi session. Nothing in CI
starts one, so the warning is invisible here by construction. A spec violation
that still works is the most durable kind of rot: nothing forces the fix.

Unlike check-skill-floor.sh, this was NOT free to add. Measured headroom on the
five baked skills: credential-incident-response 1022 (2 chars spare),
pi-extensions 1003 (21), pi-devbox-firstboot 952, pi-devbox-environment 923,
mempalace 254. Two of five are within one sentence of the limit, so "nobody
would write a description that long" is already false here.

It also catches the sibling failure that is a silent DROP rather than a
warning: a missing or empty description returns skill: null, so the file ships,
the symlink resolves, smoke-test's `test -f` passes, pi-devbox-version lists it
-- and no agent can ever see it. Name rules are checked too, including against
the directory, because the loader falls back to parentDirName and a mismatch
silently renames the skill while every path-based pointer (the managed AGENTS
block included) keeps naming the directory.

Truth table, each mutation verified to differ materially from the shipped tree:
shipped 0; over-long description 1; missing description 1 (reported as fatal,
not a mere violation); name != directory 1; no frontmatter 1; directory with no
SKILL.md 1; absent skills tree 2. Exit codes match lint-shell.sh,
check-skill-floor.sh and check-doc-drift.sh -- a gate that cannot run must not
pass.

python3-yaml rather than grep/awk because `description: >-` is a folded block
scalar whose value spans lines and joins with spaces, so a line-oriented length
check measures the wrong string. Same package the actionlint job already
installs, same parser check-doc-drift.sh already uses.

The vendored copies (pi-extensions, mempalace) are deliberately in scope: an
upstream edit over the limit turns this red on a file this repo does not author,
which is precedented by check-skill-floor.sh's coupling and correct because the
image ships those bytes. The fix in that case is to refresh the snapshot, not to
exempt the file.
This commit is contained in:
pi@mbp-m1-2020
2026-10-24 00:32:13 +02:00
parent 5d6ff537c4
commit 206fd1ec35
3 changed files with 257 additions and 0 deletions
+46
View File
@@ -249,3 +249,49 @@ jobs:
- name: Doc claims match the build files
run: bash scripts/check-doc-drift.sh
skill-frontmatter:
# Gate the image-baked skills' frontmatter against the spec pi's own loader
# measures. Its own job, for the same reason as skill-floor and doc-drift:
# "a shipped skill violates the spec" should be a distinct red name, not a
# line buried in a job about workflow syntax.
#
# The gap it closes, measured 2026-10-23 while adding pi-devbox-firstboot:
# that skill's description was drafted at 1056 chars against the loader's
# MAX_DESCRIPTION_LENGTH of 1024. pi does NOT reject it — dist/core/skills.js
# pushes a diagnostic and then loads the skill anyway ("Still load the skill
# even with warnings, unless description is missing or empty"). So the only
# thing that would ever report the violation is a diagnostic inside a live pi
# session, and nothing in CI starts one. A spec violation that still works is
# the most durable kind of rot: nothing forces the fix.
#
# Not free in the way skill-floor was free: two of the five baked skills are
# already within ~20 chars of the limit (credential-incident-response 1022,
# pi-extensions 1003), so this fires on a plausible next edit rather than on
# a hypothetical one.
#
# The sibling failure it also catches is a silent DROP rather than a warning:
# a missing/empty description returns skill: null, so the file ships, the
# symlink resolves, smoke-test's `test -f` passes, pi-devbox-version lists
# it — and no agent can ever see it.
#
# Exit codes 0 valid / 1 violation / 2 cannot-run, matching lint-shell.sh,
# check-skill-floor.sh and check-doc-drift.sh. A missing skills tree or a
# python3 without the yaml module is a red 2, not a green tick.
runs-on: ubuntu-latest
container:
image: catthehacker/ubuntu:act-latest
steps:
- uses: actions/checkout@v4
# python3-yaml because `description: >-` is a folded block scalar: the
# value spans lines and joins with spaces, so a line-oriented length
# check measures the wrong string. Same package the actionlint job
# installs, same parser check-doc-drift.sh already relies on.
- name: Install python3-yaml
run: |
apt-get update
apt-get install -y --no-install-recommends python3-yaml
- name: Baked skill frontmatter is valid
run: bash scripts/check-skill-frontmatter.sh