diff --git a/CHANGELOG.md b/CHANGELOG.md index 44ab6d1..0ad73a3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,66 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`). --- +## Unreleased + +**The size numbers on the Docker Hub page were the only claim in these docs with +nothing in the repo to check them against, and they had gone 20% wrong across +eight releases.** Every other claim `scripts/check-doc-drift.sh` guards is +anchored to a build file — a pin, an `ARG`, a placeholder — so it cannot rot +without someone editing the thing it describes. Nothing in this repo states the +image size, so `DOCKER_HUB.md`'s `~1.1 GB` simply drifted while the image grew, +and `update-description` POSTed it to Docker Hub every release. It is the first +number a stranger reads about this image. + +Corrected against Docker Hub's measured `full_size`, 2026-09-14 after v1.9.2 +published (amd64 / arm64, compressed): + +| Row | Claimed | Measured | Now says | +|---|---|---|---| +| `:latest` | ~1.1 GB | 1.228 / 1.211 | ~1.23 GB | +| `:latest-studio` | ~1.15 GB | 1.255 / 1.238 | ~1.25 GB | +| `:base-latest`, `:base-` | ~1.0 GB | 1.167 / 1.151 | ~1.17 GB | + +`full_size` is the right field because it tracks the **first manifest entry** +(amd64), not the sum across architectures — measured on v1.9.2: +`full_size=1.228`, `amd64=1.228`, `arm64=1.211`, `sum=2.439`. That matches the +table's per-arch "Size (compressed)" column. + +**New check 8 in `scripts/check-doc-drift.sh`: size claims vs Hub's measured +`full_size`,** so this class cannot rot silently again. It fails on drift beyond +tolerance, and **skips loudly** — a new `skip()` helper, counted and named in the +summary — when `curl`/`python3` are missing, the API is unreachable, or +`SKIP_SIZE_CHECK=1`. Skips are deliberately neither `OK` nor a failure: printing +an unverified claim as OK is the habit this file exists to break, while failing +on Docker Hub's uptime would make every release hostage to a third party. Not in +`hooks/pre-push` (that runs `lint-shell.sh` only), so pushes do not hit the +network. + +Two bugs were caught while building it, both by writing the expected exit code +down *before* running the check: + +- **The tolerance would have missed its own motivating case.** The percentage is + computed against the *measured* size, but the 20% first chosen came from the + claim-relative figure. The real drift was `|1.1 − 1.37| / 1.37 = 19.7%` — it + would have passed. Now 15%, sitting inside a window whose bounds are both + measured: above the largest legitimate skew (a claim describing the published + release while the next tag changes the size — v1.9.1's 1.37 against v1.9.2's + 1.23 = 11.4%) and below the rot it exists to catch (19.7%). +- **A `|| true` on the python invocation made the gate fail open.** It printed + `DRIFT` and exited 0 — a gate that reports the defect and passes anyway. + Removed; the outer `|| SIZE_RC=$?` is what satisfies `set -e` without + swallowing the code. Verified two-sided afterwards: a 19% drift exits 1 at the + default tolerance and 0 at `SIZE_TOLERANCE_PCT=25`, so the threshold is doing + the work rather than the ordering. + +**Explicitly NOT covered:** `README.md`'s `~3.2 GB` figures are *uncompressed* +on-disk sizes, and the registry exposes compressed sizes only (manifest layer +sizes are compressed; the config blob carries no uncompressed totals). Measuring +them needs a real pull, so they remain unverified — a green check 8 says nothing +about them, and the script says so where a reader will see it. + +--- + ## v1.9.2 — 2026-09-14 **The v1.9.1 residual is attributed and fixed: it was mostly npm's own download diff --git a/DOCKER_HUB.md b/DOCKER_HUB.md index 927ad7c..aeddc98 100644 --- a/DOCKER_HUB.md +++ b/DOCKER_HUB.md @@ -8,12 +8,12 @@ A self-contained Docker container for the [pi coding-agent](https://github.com/e | Tag | Architectures | Size (compressed) | What you get | |---|---|---|---| -| `joakimp/pi-devbox:latest` | amd64, arm64 | ~1.1 GB | Self-contained: base + pi `{{PI_VERSION}}` + companions | +| `joakimp/pi-devbox:latest` | amd64, arm64 | ~1.23 GB | Self-contained: base + pi `{{PI_VERSION}}` + companions | | `joakimp/pi-devbox:vX.Y.Z` | amd64, arm64 | same | Pinned semver release | -| `joakimp/pi-devbox:latest-studio` | amd64, arm64 | ~1.15 GB | `latest` + [pi-studio](https://github.com/omaclaren/pi-studio): browser prompt editor, KaTeX/Mermaid preview, tmux-backed literate REPLs | +| `joakimp/pi-devbox:latest-studio` | amd64, arm64 | ~1.25 GB | `latest` + [pi-studio](https://github.com/omaclaren/pi-studio): browser prompt editor, KaTeX/Mermaid preview, tmux-backed literate REPLs | | `joakimp/pi-devbox:vX.Y.Z-studio` | amd64, arm64 | same | Pinned semver studio release | -| `joakimp/pi-devbox:base-latest` | amd64, arm64 | ~1.0 GB | Base layer alias (internal building block; pull `:latest` instead) | -| `joakimp/pi-devbox:base-` | amd64, arm64 | ~1.0 GB | Content-addressed base; immutable. Stable parent for variant rebuilds. | +| `joakimp/pi-devbox:base-latest` | amd64, arm64 | ~1.17 GB | Base layer alias (internal building block; pull `:latest` instead) | +| `joakimp/pi-devbox:base-` | amd64, arm64 | ~1.17 GB | Content-addressed base; immutable. Stable parent for variant rebuilds. | > **pi-studio (`-studio` tags):** launch with `/studio --no-browser --port 8765` inside a pi session. The server binds `127.0.0.1` **inside the container**, so reach it via host networking or a loopback bridge (and `ssh -L` for a remote host; mosh needs a parallel `ssh -L`). Full recipe: [README → Using pi-studio](https://gitea.jordbo.se/joakimp/pi-devbox#using-pi-studio--studio-variant). diff --git a/scripts/check-doc-drift.sh b/scripts/check-doc-drift.sh index 52bd57e..251a6f9 100755 --- a/scripts/check-doc-drift.sh +++ b/scripts/check-doc-drift.sh @@ -69,6 +69,22 @@ HUB_MAX_CHARS=25000 WARN_ONLY=0 FAILURES=0 +SKIPS=0 + +# Tolerance for the published size claims (check 8), as a percentage OF THE +# MEASURED SIZE. The denominator matters: against the claim instead, the same +# drift reads as a different number, and an early draft of this gate took 20% +# from the claim-relative figure and would therefore have MISSED its own +# motivating case. Both bounds are measured, not guessed: +# - the rot that motivated this check: claimed 1.1 GB vs measured 1.37 GB +# = 19.7% off, so the threshold must sit BELOW that or the gate is theatre. +# - the largest legitimate skew, i.e. a claim describing the currently-published +# release while the next tag changes the size: v1.9.1's 1.37 GB against +# v1.9.2's measured 1.23 GB = 11.4% off, so the threshold must sit ABOVE that +# or every size-changing release trips it. +# 15% sits in that 11.4%-19.7% window. Widen it only with a measured reason, and +# re-derive both bounds if you do. +SIZE_TOLERANCE_PCT="${SIZE_TOLERANCE_PCT:-15}" usage() { cat <<'EOF' @@ -124,6 +140,13 @@ fail() { ok() { printf ' OK %s\n' "$1"; } +# A check that could not be EVALUATED, as distinct from one that passed. +# Deliberately neither ok() nor fail(): printing it as OK would launder an +# unmeasured claim into a passing one (the exact habit this file exists to +# break), while failing on a third party's uptime would make every release +# hostage to Docker Hub's API. Loud, counted, and surfaced in the summary. +skip() { SKIPS=$((SKIPS + 1)); printf ' SKIP %s\n' "$1"; } + echo "Checking hand-maintained doc claims against the build files they describe." echo @@ -227,9 +250,133 @@ else ok "no stale 'Unreleased' pointers in $README or $HUB" fi +# --------------------------------------------------------------------------- +# 8. Published size claims vs Docker Hub's MEASURED full_size. +# +# Why this exists: every other claim in these docs is checked against a file +# in this repo, so it cannot rot without someone editing the thing it +# describes. The size claims had no such anchor -- nothing in the repo states +# the image size -- so they quietly went 24% wrong across eight releases +# (DOCKER_HUB.md said ~1.1 GB; :latest measured 1.37 GB on 2026-09-14). +# DOCKER_HUB.md is POSTed to Docker Hub by update-description, so that number +# is the first thing a stranger reads about this image. +# +# Hub's `full_size` tracks the FIRST manifest entry (amd64 here), NOT the sum +# across architectures -- measured: v1.9.2 full_size=1.228 GB, amd64=1.228, +# arm64=1.211, sum=2.439. That matches the table's per-arch "Size +# (compressed)" column, which is why full_size is the right field. +# +# NOT COVERED, deliberately: README.md's ~3.2 GB figures are UNCOMPRESSED +# on-disk sizes, and the registry API exposes compressed sizes only (layer +# sizes in a manifest are compressed; the config blob carries no uncompressed +# totals). Measuring them needs a real pull, so they are out of scope here -- +# do not read a green check 8 as covering them. +# --------------------------------------------------------------------------- +if [ "${SKIP_SIZE_CHECK:-0}" = "1" ]; then + skip "size claims -- SKIP_SIZE_CHECK=1 was set" +elif ! command -v curl >/dev/null 2>&1 || ! command -v python3 >/dev/null 2>&1; then + skip "size claims -- need both curl and python3 to measure them" +else + # Derive the repo from the doc's own rows rather than hardcoding it, so a + # rename cannot leave this check silently probing a repo nobody publishes to. + # shellcheck disable=SC2016 # single quotes are deliberate: this is a sed + # script, and its \( \) groups and \1 backreference must reach sed unexpanded. + HUB_REPO_PATH="$(sed -n 's/^| `\([^:`]*\):[^`]*`.*/\1/p' "$HUB" | head -1)" + if [ -z "$HUB_REPO_PATH" ]; then + skip "size claims -- found no \`repo:tag\` image rows in $HUB to check" + else + HUB_TAGS_JSON="$(curl -sS -m 20 \ + "https://hub.docker.com/v2/repositories/${HUB_REPO_PATH}/tags/?page_size=100" \ + 2>/dev/null || true)" + if [ -z "$HUB_TAGS_JSON" ]; then + skip "size claims -- Docker Hub API unreachable (offline?); NOT verified" + else + SIZE_RC=0 + # NO `|| true` on the python invocation: an early draft had one, and it + # swallowed the exit code so a printed DRIFT line still exited 0 -- a gate + # that reports the defect and passes anyway. The outer `|| SIZE_RC=$?` is + # what keeps `set -e` happy while preserving the code. + SIZE_OUT="$(HUB_MD="$HUB" HUB_JSON="$HUB_TAGS_JSON" TOL="$SIZE_TOLERANCE_PCT" \ + python3 <<'PYEOF' +import json, os, re, sys + +try: + data = json.loads(os.environ["HUB_JSON"]) +except (ValueError, KeyError) as exc: + print(" SKIP size claims -- Hub API returned unparseable JSON (%s)" % exc) + sys.exit(3) + +# full_size == first manifest entry (amd64), which is the per-arch number the +# table's "Size (compressed)" column claims. Verified against .images[] sizes. +sizes = { + r["name"]: r["full_size"] / 1e9 + for r in data.get("results", []) + if isinstance(r.get("full_size"), int) and r.get("name") +} +if not sizes: + print(" SKIP size claims -- Hub API returned no usable tags") + sys.exit(3) + +tol = float(os.environ["TOL"]) +row = re.compile(r"^\|\s*`([^`:]+):([^`]+)`\s*\|[^|]*\|\s*~?([0-9]+(?:\.[0-9]+)?)\s*GB\s*\|") +checked = drift = 0 + +with open(os.environ["HUB_MD"], encoding="utf-8") as fh: + for line in fh: + m = row.match(line) + if not m: + continue # rows saying "same", and every non-image row + _repo, tag, claimed = m.group(1), m.group(2), float(m.group(3)) + if "X.Y.Z" in tag: + continue # placeholder row; the concrete tag is checked instead + # base- is content-addressed and immutable, so its size is + # base-latest's by construction -- probe the alias that always exists. + probe = "base-latest" if tag.startswith("base-") else tag + actual = sizes.get(probe) + if actual is None: + print(" SKIP size %s -- tag '%s' not present on Hub" % (tag, probe)) + continue + checked += 1 + off = abs(claimed - actual) / actual * 100 + if off <= tol: + print(" OK size %s claims ~%.2f GB, Hub measures %.2f GB (%.0f%% off)" + % (tag, claimed, actual, off)) + else: + drift += 1 + print(" DRIFT size %s claims ~%.2f GB but Hub measures %.2f GB" + " (%.0f%% off, tolerance %.0f%%)" % (tag, claimed, actual, off, tol)) + +if checked == 0: + print(" SKIP size claims -- no checkable rows resolved to a published tag") + sys.exit(3) +sys.exit(1 if drift else 0) +PYEOF +)" || SIZE_RC=$? + printf '%s\n' "$SIZE_OUT" + case "$SIZE_RC" in + 0) : ;; + 3) SKIPS=$((SKIPS + 1)) ;; + *) + fail "a published size claim in $HUB has drifted from what Docker Hub + actually serves (see DRIFT above). This page is POSTed to Docker Hub by + update-description, so it is the first size a stranger sees. Re-measure and + update the table: + curl -sS 'https://hub.docker.com/v2/repositories/${HUB_REPO_PATH}/tags/?page_size=100' | + jq -r '.results[] | \"\\(.name) \\(.full_size/1e9)\"'" + ;; + esac + fi + fi +fi + echo if [ "$FAILURES" -eq 0 ]; then - echo "OK: every checked doc claim matches the build files." + if [ "$SKIPS" -gt 0 ]; then + echo "OK: every checked doc claim matches the build files" \ + "($SKIPS check(s) SKIPPED and therefore NOT verified -- see SKIP above)." + else + echo "OK: every checked doc claim matches the build files." + fi exit 0 fi