ci: gate baked skill frontmatter against the loader's own limits
Found while adding pi-devbox-firstboot, whose description was drafted at 1056 characters against pi's MAX_DESCRIPTION_LENGTH of 1024. pi does not reject it: dist/core/skills.js pushes a diagnostic and then loads the skill anyway -- "Still load the skill even with warnings, unless description is missing or empty". So the skill works, the violation is real, and the only thing that would ever report it is a diagnostic inside a live pi session. Nothing in CI starts one, so the warning is invisible here by construction. A spec violation that still works is the most durable kind of rot: nothing forces the fix. Unlike check-skill-floor.sh, this was NOT free to add. Measured headroom on the five baked skills: credential-incident-response 1022 (2 chars spare), pi-extensions 1003 (21), pi-devbox-firstboot 952, pi-devbox-environment 923, mempalace 254. Two of five are within one sentence of the limit, so "nobody would write a description that long" is already false here. It also catches the sibling failure that is a silent DROP rather than a warning: a missing or empty description returns skill: null, so the file ships, the symlink resolves, smoke-test's `test -f` passes, pi-devbox-version lists it -- and no agent can ever see it. Name rules are checked too, including against the directory, because the loader falls back to parentDirName and a mismatch silently renames the skill while every path-based pointer (the managed AGENTS block included) keeps naming the directory. Truth table, each mutation verified to differ materially from the shipped tree: shipped 0; over-long description 1; missing description 1 (reported as fatal, not a mere violation); name != directory 1; no frontmatter 1; directory with no SKILL.md 1; absent skills tree 2. Exit codes match lint-shell.sh, check-skill-floor.sh and check-doc-drift.sh -- a gate that cannot run must not pass. python3-yaml rather than grep/awk because `description: >-` is a folded block scalar whose value spans lines and joins with spaces, so a line-oriented length check measures the wrong string. Same package the actionlint job already installs, same parser check-doc-drift.sh already uses. The vendored copies (pi-extensions, mempalace) are deliberately in scope: an upstream edit over the limit turns this red on a file this repo does not author, which is precedented by check-skill-floor.sh's coupling and correct because the image ships those bytes. The fix in that case is to refresh the snapshot, not to exempt the file.
This commit is contained in:
@@ -51,6 +51,28 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`).
|
||||
the round only through that line, so rewording the paragraph must not be able
|
||||
to break the link silently.
|
||||
|
||||
- **`scripts/check-skill-frontmatter.sh` + a `skill-frontmatter` lint job:
|
||||
baked skill frontmatter is now gated against the limits pi's own loader
|
||||
measures.** Found while adding the skill above, whose `description` was
|
||||
drafted at 1056 characters against the loader's `MAX_DESCRIPTION_LENGTH` of
|
||||
1024. pi does **not** reject that — `dist/core/skills.js` pushes a diagnostic
|
||||
and then loads the skill anyway ("Still load the skill even with warnings,
|
||||
unless description is missing or empty"), so the only thing that would ever
|
||||
report the violation is a diagnostic inside a live pi session — and nothing in
|
||||
CI starts one. A spec violation that still works is the most durable kind of
|
||||
rot, because nothing forces the fix.
|
||||
|
||||
Unlike `check-skill-floor.sh`, this one was **not** free to add: two of the
|
||||
five baked skills are already within ~20 characters of the limit
|
||||
(`credential-incident-response` 1022, `pi-extensions` 1003), so it fires on a
|
||||
plausible next edit rather than a hypothetical one. It also catches the
|
||||
sibling failure that is a silent **drop** rather than a warning — missing or
|
||||
empty `description` returns `skill: null`, so the file ships, the symlink
|
||||
resolves, smoke-test's `test -f` passes, `pi-devbox-version` lists it, and no
|
||||
agent can ever see it. Exit codes 0 valid / 1 violation / 2 cannot-run, and
|
||||
the vendored copies are deliberately in scope: the image ships those bytes,
|
||||
so the violation is ours to carry whoever typed it.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **CI: a smoke job's image could be deleted mid-test by another job's disk
|
||||
|
||||
Reference in New Issue
Block a user