diff --git a/.gitea/workflows/lint.yml b/.gitea/workflows/lint.yml index f597ad5..688c319 100644 --- a/.gitea/workflows/lint.yml +++ b/.gitea/workflows/lint.yml @@ -172,3 +172,46 @@ jobs: - name: Vendored pi-extensions skill floor matches the package run: bash scripts/check-skill-floor.sh + + doc-drift: + # Gate hand-maintained doc claims against the build files they describe. + # Its own job for the same reason as skill-floor: "the docs lie" should be a + # distinct red name, not a line buried in a job about workflow syntax. + # + # The gap it closes, measured 2026-09-10 while preparing v1.9.0 — five + # claims had rotted, every one of them a fact written by hand in a file + # nothing verified: + # * README.md's "Version pins" table was wrong on ALL THREE rows (pi + # 0.84.4 vs 0.85.1, pi-atelier v0.10.0 vs v0.10.1, mempalace 3.8.0 vs + # 3.9.0) — and that table exists specifically to be the reviewable + # record of what the repo freezes on purpose, so a wrong row destroys + # the only thing it is for. + # * README.md listed already-shipped typst PDF export under "Planned for + # an upcoming minor release", marked "(shipped in Unreleased/base)". + # * DOCKER_HUB.md claimed "Node.js v22" while v1.9.0 ships Node 24. + # + # DOCKER_HUB.md is why this is a gate and not a habit. It is PUBLISHED — + # update-description POSTs it to Docker Hub as full_description on every tag + # — and it had gone eight releases (v1.8.6 -> v1.9.0) untouched. Nothing + # generates it and nothing checked it, so the only thing keeping it true was + # someone remembering. It is also read from the TAG, so a fix pushed to main + # after tagging never reaches the published page. + # + # Cheap and hermetic on purpose: every check compares a doc string against a + # value that exists in this repo, so no network, no token, no built image, + # and no sibling clone. Claims that genuinely need a running container (image + # sizes, the "N mempalace_* tools" count) are deliberately left out — a gate + # that cannot evaluate a claim honestly would have to guess, and a guessing + # gate is worse than none. Assert those in scripts/smoke-test.sh instead. + # + # Exit codes 0 in sync / 1 drift / 2 cannot-run, matching lint-shell.sh and + # check-skill-floor.sh. A renamed ARG makes the gate blind, so that is a red + # 2, not a green tick. + runs-on: ubuntu-latest + container: + image: catthehacker/ubuntu:act-latest + steps: + - uses: actions/checkout@v4 + + - name: Doc claims match the build files + run: bash scripts/check-doc-drift.sh diff --git a/AGENTS.md b/AGENTS.md index f355efa..cd676b1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -92,7 +92,44 @@ re-brand of opencode-devbox's `pi-only` variant. is hashed into `base_tag`, so it costs a base rebuild (~67 min); and if the section the phrase canary names has changed, re-pin it in `scripts/smoke-test.sh`. -3. Update `CHANGELOG.md` Unreleased → vX.Y.Z section. +3. **Update the docs this release makes stale — BEFORE you tag.** Rename + `CHANGELOG.md`'s `## Unreleased` to `## vX.Y.Z — YYYY-MM-DD` (em dash, as + every prior release heading uses), then run the gate: + + ```bash + bash scripts/check-doc-drift.sh # 0 in sync / 1 drift / 2 cannot run + ``` + + It compares README.md's version-pin table against the ARGs it names, and + DOCKER_HUB.md's Node claim against `ARG NODE_VERSION`, plus Hub's + 25 000-char limit, unsubstituted `{{PLACEHOLDERS}}`, and stale `Unreleased` + pointers in user-facing docs. + + **Why before and not after:** `docker-publish.yml` runs `actions/checkout@v4` + with no `ref:`, so every job reads `github.ref` — the **tag**. A doc fix + pushed to `main` after tagging does not reach the release, and for + `DOCKER_HUB.md` it does not reach the published Hub page either, because + `update-description` POSTs that file as Docker Hub's `full_description` from + the tag's tree. Getting it in afterwards means re-pointing the tag, which is + its own hazard (v1.8.14 went `601fc98` → `361babd` and broke deploy + verification until `git fetch --tags --force`). + + The gate is deliberately narrow — it only checks claims verifiable from files + in this repo. Still eyeball, because these are NOT gated: + - counts and sizes (`~1.1 GB`, "N `mempalace_*` tools", "7 extensions") — + they need a running image; assert them in `scripts/smoke-test.sh` instead + - feature prose that quietly became false, e.g. a "Planned for an upcoming + release" section describing something that already shipped + - `Dockerfile.base`'s `# BASE_REBUILD_DATE:` marker. Ungated on purpose: + `base_tag` hashes Dockerfile.base's content, comments included, so + demanding it be current would force a ~60 min base rebuild on a release + that touched no base files. **Fix it when the base is already rebuilding — + then it is free.** + + Measured cost of skipping this, 2026-09-10 (v1.9.0): five stale claims, one + of them published. README's pin table was wrong on all three rows, and + DOCKER_HUB.md — untouched for eight releases — still said Node v22 while the + image shipped Node 24. 4. Verify `docker compose up` works locally with the current `latest` image if you're upgrading users from a previous version. Then run the **post-recreate sanity check** inside the running container to confirm diff --git a/CHANGELOG.md b/CHANGELOG.md index bd94221..c56fea0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,89 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`). --- +## Unreleased + +**A gate for documentation drift, because five claims rotted in one release and +one of them was published.** Preparing v1.9.0 turned up a cluster of stale +facts, all the same shape — a value written once by hand, in a file nothing +verifies, about a number that lives somewhere else and moved: + +- `README.md`'s "Version pins" table was wrong on **all three rows**: pi + `0.84.4` vs `ARG PI_VERSION=0.85.1`, pi-atelier `v0.10.0` vs `v0.10.1`, + mempalace `3.8.0` vs `3.9.0`. That table is the worst possible place for this, + because it exists *specifically* to be the reviewable record of what the repo + freezes deliberately — so a wrong row destroys the only thing it is for. +- `README.md` listed already-shipped typst PDF export under "Planned for an + upcoming minor release", carrying the self-contradicting marker "(shipped in + Unreleased/base)". The **fourth** instance of the stale-`Unreleased`-pointer + class this changelog already documented three of. +- `DOCKER_HUB.md` claimed "Node.js v22" while v1.9.0 ships Node 24. + +The last one is why this became a gate rather than a resolution to be careful. +`DOCKER_HUB.md` is **published**: `update-description` POSTs it to Docker Hub as +`full_description` on every tag. It had gone **eight releases** (v1.8.6 → +v1.9.0) without a touch. Nothing generates it — CI only substitutes +`{{PI_VERSION}}` — and nothing checked it, so the sole mechanism keeping it true +was whoever remembered. Worse, it is read from the **tag**, so the stale page +published with v1.9.0 anyway and the fix could only ride the next release. + +**New: `scripts/check-doc-drift.sh` + a `doc-drift` job in `lint.yml`.** Seven +checks, all comparing a doc string to a value that exists in this repo, so it +needs no network, no token, no built image, and no sibling clone: + +- README's three pin-table rows vs the ARGs they name *by name* +- `DOCKER_HUB.md`'s Node claim vs `ARG NODE_VERSION` +- placeholders CI will not substitute — the publish step greps for leftovers of + `{{PI_VERSION}}` only, so any *second* token sails through and publishes + literally +- `DOCKER_HUB.md` under Docker Hub's 25 000-char `full_description` limit + (previously discoverable only as a non-200 *after* the full build) +- `Unreleased` appearing in a user-facing doc, which is always a pointer that + outlived what it pointed at + +Exit codes match `lint-shell.sh` and `check-skill-floor.sh`: `0` in sync, `1` +drift, `2` cannot run — a renamed ARG makes the gate blind, which is a red `2`, +never a green tick. Verified with **15 controls**: every check fails when its +claim is broken, the real v1.9.0 Node bug is caught, and two false-positive +controls pass — the first version of the placeholder check wrongly flagged +`README.md:900`'s `docker inspect --format '{{json .Config.Labels}}'`, a Go +template in a legitimate example, so the pattern is now anchored to the +UPPER_SNAKE convention CI actually substitutes. **The gate was wrong, not the +doc** — which is the whole reason a gate gets negative controls. + +Deliberately **not** gated, and the reasons matter more than the list: + +- Counts and sizes (`~1.1 GB`, "N `mempalace_*` tools", "7 extensions") need a + running image. A gate that cannot evaluate a claim honestly would have to + guess, and a guessing gate is worse than none — assert these in + `scripts/smoke-test.sh`, where a real image exists. +- `Dockerfile.base`'s `# BASE_REBUILD_DATE:` marker, itself stale (2026-07-13, + three base rebuilds ago). `base_tag` hashes Dockerfile.base's *content*, + comments included, so demanding it be current would force a ~60 min base + rebuild on a release that touched no base files at all. It is free to fix + while the base is *already* rebuilding, and expensive at any other moment. + That cost asymmetry is now written into the release checklist rather than + enforced. + +**Release checklist step 3 rewritten** (`AGENTS.md`) around the mechanism that +made this expensive: `docker-publish.yml` runs `actions/checkout@v4` with no +`ref:`, so every job reads `github.ref` — the tag. Docs must be correct *before* +tagging; afterwards the only routes are re-pointing the tag (its own hazard — +v1.8.14 went `601fc98` → `361babd` and broke deploy verification until +`git fetch --tags --force`) or waiting for the next release. The step now also +names what the gate cannot see, so "gate is green" is not mistaken for "docs are +true". The same reflex went into the `ci-release-watcher` skill, as the first +correctness rule — it is the only one that expires once the tag exists. + +Also fixed in passing: README's `pi-devbox-version` sample was v1.5.0-era and +structurally outdated (it predated the `palace:` line the surrounding prose +advertises, the `pi-atelier` component, and the whole `skills:` block). Replaced +with real observed output rather than hand-written text. `DOCKER_HUB.md`'s "7 +user-facing extensions" was **verified correct**; its "29 `mempalace_*` tools" +is stale (a live client shows 45) but left alone rather than corrected on a +guess, since that count cannot be attributed to the baked 3.9.0 server without +measuring it. + ## v1.9.0 — 2026-09-10 **`shellcheck` is now in the image, because the release gate it depends on could diff --git a/scripts/check-doc-drift.sh b/scripts/check-doc-drift.sh new file mode 100755 index 0000000..52bd57e --- /dev/null +++ b/scripts/check-doc-drift.sh @@ -0,0 +1,246 @@ +#!/usr/bin/env bash +# check-doc-drift.sh — fail when a hand-maintained doc claim contradicts the +# build files it describes. +# +# THE DEFECT CLASS THIS EXISTS TO CATCH, measured 2026-09-10 while preparing +# v1.9.0. Five separate claims had rotted, all of them the same shape: a fact +# written once by hand, in a file nothing verifies, about a value that lives +# somewhere else and moved. +# +# 1..3. README.md's "Version pins" table was wrong on EVERY row — pi `0.84.4` +# vs ARG PI_VERSION=0.85.1, pi-atelier `v0.10.0` vs v0.10.1, mempalace +# `3.8.0` vs 3.9.0. That table is the worst possible place for this: it +# exists precisely to be the reviewable record of what is deliberately +# frozen, so when it lies, the review it enables is worthless. +# 4. README.md carried a "Planned for an upcoming minor release" section +# listing typst PDF export, which had ALREADY SHIPPED, tagged with a +# self-contradicting "(shipped in Unreleased/base)" marker. The +# CHANGELOG had already documented three earlier instances of exactly +# this stale-"Unreleased"-pointer class (see its v1.8.7 notes). +# 5. DOCKER_HUB.md claimed "Node.js v22" while this release ships Node 24. +# This one is the reason the gate exists at all: DOCKER_HUB.md is +# PUBLISHED. `update-description` in docker-publish.yml POSTs it to Hub +# as full_description on every tag, so unlike README.md — which no +# workflow or gate reads — a stale claim here is what users see. +# +# WHY A GATE AND NOT "REMEMBER TO CHECK". DOCKER_HUB.md had gone eight releases +# (v1.8.6 → v1.9.0) without a touch. Nothing generates it and nothing verifies +# it; the only mechanism keeping it true was whoever remembered. That is the +# same failure mode check-skill-floor.sh was written for, and the same fix: +# convert "someone remembers" into "CI refuses". +# +# WHY THESE FIVE CHECKS AND NOT MORE. Every check here compares a doc string to +# a value that EXISTS IN THIS REPO, so it can never be wrong about the world and +# needs no network, no token, and no built image. Claims that require a running +# container to verify (image sizes, the "N mempalace_* tools" count) are +# deliberately NOT gated: a check that cannot be evaluated honestly at lint time +# would either be skipped or guessed, and a guessing gate is worse than none. +# If you want those, assert them in scripts/smoke-test.sh where a real image is +# available. +# +# DELIBERATELY NOT GATED: Dockerfile.base's `# BASE_REBUILD_DATE:` comment, which +# is also stale (2026-07-13, three base rebuilds ago). base_tag is a hash of +# Dockerfile.base's CONTENT plus rootfs/, comments included, so a gate that +# demanded that comment be current would force a ~60 min base rebuild on any +# release that touched no base files at all. Fix it when you are already +# rebuilding the base — then it is free. This is a real cost asymmetry, not +# laziness. +# +# EXIT CODES (same contract as lint-shell.sh and check-skill-floor.sh): +# 0 every checked claim matches +# 1 at least one claim has drifted +# 2 cannot run (a file or ARG this gate reads is missing/unparseable) +# A gate that cannot run must not pass, so a missing input is 2, never 0. + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$REPO_ROOT" + +README="README.md" +HUB="DOCKER_HUB.md" +DF_VARIANT="Dockerfile.variant" +DF_BASE="Dockerfile.base" + +# Docker Hub rejects a full_description longer than this. docker-publish.yml has +# no size check of its own; it only notices via a non-200 from the API, i.e. +# after paying the whole build. Catching it here makes it a 2-second failure. +HUB_MAX_CHARS=25000 + +WARN_ONLY=0 +FAILURES=0 + +usage() { + cat <<'EOF' +Usage: check-doc-drift.sh [--warn-only] [-h|--help] + +Compares hand-written claims in README.md and DOCKER_HUB.md against the build +files they describe (Dockerfile.base, Dockerfile.variant). + + --warn-only Report drift but exit 0 (advisory use, e.g. a local pre-push hook). + +Exit: 0 = in sync, 1 = drift, 2 = cannot run. +EOF +} + +while [ $# -gt 0 ]; do + case "$1" in + --warn-only) WARN_ONLY=1; shift ;; + -h|--help) usage; exit 0 ;; + *) echo "::error::unknown argument: $1" >&2; usage >&2; exit 2 ;; + esac +done + +for f in "$README" "$HUB" "$DF_VARIANT" "$DF_BASE"; do + if [ ! -f "$f" ]; then + echo "::error::$f not found (cwd $PWD). Cannot evaluate doc drift, so this is exit 2, not a pass." + exit 2 + fi +done + +# Read `ARG NAME=value` from a Dockerfile. Exit 2 when absent: if the ARG this +# gate is built around has been renamed, the gate is measuring nothing and must +# say so rather than silently comparing against an empty string. +read_arg() { + local file="$1" name="$2" value + value="$(sed -n "s/^ARG ${name}=\\(.*\\)\$/\\1/p" "$file" | head -1)" + if [ -z "$value" ]; then + echo "::error::ARG ${name} not found in ${file}. It was probably renamed;" >&2 + echo "::error::update check-doc-drift.sh to match, because this gate is now blind." >&2 + exit 2 + fi + printf '%s' "$value" +} + +# One row of README's "Version pins" table: `| pi | `0.85.1` | ... |` +read_pin_row() { + sed -n "s/^| $1 | \`\\([^\`]*\`*\\)\` |.*/\\1/p" "$README" | head -1 +} + +fail() { + FAILURES=$((FAILURES + 1)) + echo "::error::$1" +} + +ok() { printf ' OK %s\n' "$1"; } + +echo "Checking hand-maintained doc claims against the build files they describe." +echo + +# --------------------------------------------------------------------------- +# 1-3. README's version-pin table vs the ARGs it names by name. +# --------------------------------------------------------------------------- +check_pin() { + local label="$1" documented="$2" actual="$3" where="$4" + if [ -z "$documented" ]; then + fail "README.md: no '| $label |' row found in the version-pin table. Either the + table was restructured (update this gate) or the row was dropped (restore it)." + return + fi + if [ "$documented" != "$actual" ]; then + fail "README.md version-pin table is stale for $label: says '$documented', + $where says '$actual'. Fix the table — it is the reviewable record of what + this repo deliberately freezes, so a wrong row defeats its only purpose." + return + fi + ok "README pin $label = $actual" +} + +PI_ACTUAL="$(read_arg "$DF_VARIANT" PI_VERSION)" +ATELIER_ACTUAL="$(read_arg "$DF_VARIANT" PI_ATELIER_REF)" +MEMPALACE_ACTUAL="$(read_arg "$DF_BASE" MEMPALACE_VERSION)" + +check_pin pi "$(read_pin_row pi)" "$PI_ACTUAL" "ARG PI_VERSION in $DF_VARIANT" +check_pin pi-atelier "$(read_pin_row pi-atelier)" "$ATELIER_ACTUAL" "ARG PI_ATELIER_REF in $DF_VARIANT" +check_pin mempalace "$(read_pin_row mempalace)" "$MEMPALACE_ACTUAL" "ARG MEMPALACE_VERSION in $DF_BASE" + +# --------------------------------------------------------------------------- +# 4. DOCKER_HUB.md's Node claim vs ARG NODE_VERSION. This is the published page, +# so it is the one whose staleness reaches users. +# --------------------------------------------------------------------------- +NODE_ACTUAL="$(read_arg "$DF_BASE" NODE_VERSION)" +NODE_DOCUMENTED="$(sed -n 's/.*\*\*Node\.js\*\* v\([0-9][0-9]*\).*/\1/p' "$HUB" | head -1)" +if [ -z "$NODE_DOCUMENTED" ]; then + fail "$HUB: could not find a '**Node.js** vNN' claim. If the wording changed, + update this gate; do not leave the published page unverified." +elif [ "$NODE_DOCUMENTED" != "$NODE_ACTUAL" ]; then + fail "$HUB claims Node v$NODE_DOCUMENTED but ARG NODE_VERSION=$NODE_ACTUAL. + This file is PUBLISHED to Docker Hub by update-description on every tag, + and it is read from the TAG — so fix it before tagging, not after." +else + ok "$HUB Node claim = v$NODE_ACTUAL" +fi + +# --------------------------------------------------------------------------- +# 5. Placeholders CI will not substitute. docker-publish.yml substitutes exactly +# {{PI_VERSION}} and then greps for leftovers of that ONE token, so any other +# {{...}} sails through the guard and is published literally. +# --------------------------------------------------------------------------- +UNKNOWN_PLACEHOLDERS="$(grep -o '{{[A-Za-z0-9_]*}}' "$HUB" | sort -u | grep -v '^{{PI_VERSION}}$' || true)" +if [ -n "$UNKNOWN_PLACEHOLDERS" ]; then + fail "$HUB contains placeholders CI does not substitute, which would be + published verbatim: $(echo "$UNKNOWN_PLACEHOLDERS" | tr '\n' ' ') + docker-publish.yml only fills {{PI_VERSION}}; add substitution there first." +else + ok "$HUB has no placeholders beyond {{PI_VERSION}}" +fi + +# Match only the UPPER_SNAKE placeholder convention CI uses. A bare '{{' search +# is WRONG here, and the first version of this check proved it by failing on +# README.md:900 — `docker inspect --format '{{json .Config.Labels}}'`, a Go +# template in a legitimate example, not a placeholder. The gate was wrong, not +# the doc. Keep this anchored to [A-Z] so Go/Jinja/Handlebars examples pass. +README_PLACEHOLDERS="$(grep -o '{{[A-Z][A-Z0-9_]*}}' "$README" | sort -u || true)" +if [ -n "$README_PLACEHOLDERS" ]; then + fail "$README contains placeholder(s) nothing substitutes, so they would render + literally for every reader: $(echo "$README_PLACEHOLDERS" | tr '\n' ' ') + Only DOCKER_HUB.md gets substitution, and only for {{PI_VERSION}}." +else + ok "$README has no unsubstituted placeholders" +fi + +# --------------------------------------------------------------------------- +# 6. Hub full_description length. +# --------------------------------------------------------------------------- +HUB_CHARS="$(wc -c < "$HUB" | tr -d ' ')" +if [ "$HUB_CHARS" -gt "$HUB_MAX_CHARS" ]; then + fail "$HUB is $HUB_CHARS chars, over Docker Hub's $HUB_MAX_CHARS-char + full_description limit. update-description would fail with a non-200 AFTER + the full build. Trim it — this file is the essentials-only page, and + README.md is the long form on purpose." +else + ok "$HUB is $HUB_CHARS chars (limit $HUB_MAX_CHARS)" +fi + +# --------------------------------------------------------------------------- +# 7. Stale "Unreleased" pointers. "Unreleased" is a CHANGELOG-only concept; in +# a user-facing doc it is always a pointer that outlived what it pointed at. +# This class has now bitten five times, hence a gate rather than vigilance. +# --------------------------------------------------------------------------- +STALE_MARKERS="$(grep -n 'Unreleased' "$README" "$HUB" || true)" +if [ -n "$STALE_MARKERS" ]; then + fail "'Unreleased' appears in a user-facing doc, which is always a stale + pointer once the thing ships (it has happened five times here): +${STALE_MARKERS//$'\n'/$'\n' } + State the fact directly, or move it to CHANGELOG.md where 'Unreleased' means something." +else + ok "no stale 'Unreleased' pointers in $README or $HUB" +fi + +echo +if [ "$FAILURES" -eq 0 ]; then + echo "OK: every checked doc claim matches the build files." + exit 0 +fi + +echo "::error::$FAILURES doc claim(s) have drifted from the build files." +echo +echo "Docs are read from the TAG, not from main: docker-publish.yml checks out" +echo "github.ref, so a fix pushed after tagging does not reach the release or the" +echo "Hub page. Update the docs BEFORE you tag." + +if [ "$WARN_ONLY" -eq 1 ]; then + echo "(--warn-only: exiting 0 anyway)" + exit 0 +fi +exit 1