fix(size): the v1.9.1 residual was npm's own cache, not the platform binaries
Lint / doc-drift (push) Successful in 6s
Lint / hadolint (push) Successful in 10s
Lint / skill-floor (push) Successful in 9s
Lint / actionlint (push) Successful in 4m26s

v1.9.1's @esbuild prune fixed the 431 MB size-gate failure but still shipped
+131 MB compressed over v1.8.14, nearly all of it in the pi/extensions install
layer (87 -> 206 MB). That leftover was filed as an open item with an explicit
hypothesis — the @mariozechner/clipboard-* family, same npm 11 behaviour, a
different package — and an explicit warning that the hypothesis was not a
measured cause. Measured now, after recreating onto v1.9.1, and the hypothesis
accounted for one sixth of it:

  +110 MB  /root/.npm/_cacache  (35.2 -> 145.3 MB)   the build's npm cache
  + 21 MB  clipboard foreign platform packages, both install sites
  = 131 MB  i.e. the whole delta, no unexplained remainder

Method, since there is no docker CLI inside the container: pulled both variant
layer blobs straight from the registry with a token + manifest + blob fetch and
listed the tarballs (29 789 vs 29 889 entries, 270.8 vs 401.9 MB uncompressed),
then aggregated per package. The file COUNT barely moved, which is what said
"few large files", not "npm installed more packages".

  - purge_build_caches: npm cache clean --force + rm -rf /root/.npm, in the SAME
    layer as the installs, in both the main RUN and the studio RUN. npm 11
    caches every platform tarball it downloads, including the ones the prune
    then deletes, so the cache grew faster than the tree. Nothing at runtime
    reads it: build is root, container is developer with its own cache in $HOME.
  - prune_foreign_esbuild -> prune_foreign_natives: now covers both MEASURED
    families. Clipboard keeps linux-$arch-gnu AND -musl because its napi-rs
    loader picks between them at runtime via its own isMusl() probe; the musl
    package is a 420-byte stub. The bare @mariozechner/clipboard wrapper has no
    hyphen suffix and cannot match the pattern.

Verified on arm64 before writing the glob — a widened rm -rf against a tree you
cannot inspect is the one change shape not to write blind, which is why the
order was update-then-patch. Exercised against a copy of the real trees with
foreign dirs fabricated back in (aix-ppc64, android-arm64, darwin-arm64,
win32-x64, linux-x64): all removed, host linux-arm64 kept at both sites, 21 MB
freed, require('@mariozechner/clipboard') still loads and exports all 18
functions, esbuild.transformSync still compiles TS at both sites. v1.9.1's own
arm64 validation also passed here; CI could only smoke amd64.

Two sentinel assertions, because the size gate did not catch this: it has
~225 MB of margin, so 131 MB of residue stayed green. Both were verified RED
against the running v1.9.1 image and GREEN against a pruned tree. The cache one
refuses to run as non-root: test ! -d /root/.npm on mode-700 /root would
otherwise pass for the wrong reason. Size-failure diagnostics now list cache
paths too — they previously enumerated only node_modules and /opt, where these
bytes were not.

Also fixed: -printf '%f\\n' reaches the shell with both backslashes (confirmed
from the published image's recorded created_by), so v1.9.1's progress line
printed a mangled "li ux-arm64" — find emitted a literal backslash and tr ate
the n out of the name. Single backslash now.

Deliberately not purged: /tmp/node-compile-cache (1.3 MB). The manifest RUN
calls pi --version again, so deleting it earlier only relocates those bytes into
that layer — today's manifest layer is 128 kB precisely because it finds the
cache warm.

Dockerfile.base untouched, so no base rebuild: this rides the next release.
This commit is contained in:
Joakim Persson
2026-09-11 09:39:54 +02:00
parent 42bd29d654
commit 1baba79c96
4 changed files with 209 additions and 29 deletions
+73
View File
@@ -11,6 +11,79 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`).
---
## Unreleased
**The v1.9.1 residual is attributed and fixed: it was mostly npm's own download
cache, not the platform binaries it looked like.** v1.9.1 pruned the 26 foreign
`@esbuild/<platform>` directories that npm 11 installs, which fixed the 431 MB
size-gate failure — but the image still shipped **+131 MB compressed over
v1.8.14**, nearly all of it in the single pi/extensions install layer (87 MB →
206 MB). v1.9.1's notes recorded the leftover as an open item with an explicit
hypothesis (the `@mariozechner/clipboard-*` family, same npm 11 behaviour, a
different package) and an explicit warning that the hypothesis was **not** a
measured cause. It was measured on 2026-09-11, after recreating onto v1.9.1, and
the hypothesis accounted for only a sixth of it:
| item | v1.8.14 | v1.9.1 | delta |
|---|---|---|---|
| `/root/.npm/_cacache` — the build's npm download cache | 35.2 MB | 145.3 MB | **+110 MB** |
| `@mariozechner/clipboard-*` foreign platform packages (2 sites) | 0 MB | 21.1 MB | **+21 MB** |
| variant install layer, uncompressed total | 270.8 MB | 401.9 MB | +131 MB |
That is the whole delta with no unexplained remainder. Both items are now
deleted in the **same layer** that creates them, in both the main install RUN and
the studio RUN:
- **`purge_build_caches`** — `npm cache clean --force` plus `rm -rf /root/.npm`.
npm 11 caches every platform tarball it downloads, including the ones the
prune then deletes, so the cache grew far faster than the installed tree.
Nothing at runtime reads it: the build runs as root, the container runs as
`developer` with its own cache under `$HOME`.
- **`prune_foreign_esbuild` → `prune_foreign_natives`** — now covers both
measured families. For clipboard the keep-set is `clipboard-linux-$arch-gnu`
**and** `-musl`, because its napi-rs loader chooses between them at runtime
from its own `isMusl()` probe; the musl package is a 420-byte stub, so keeping
it is free insurance. The bare `@mariozechner/clipboard` wrapper has no
hyphen suffix and cannot match the pattern.
**Verified on arm64 before writing the patch, which is why the order was
update-then-patch:** a widened `rm -rf` glob is the worst possible change to
write against a tree you cannot inspect, and there is no docker CLI inside the
container — but after a recreate the container *is* the image. The prune was
exercised against a copy of the real trees with foreign directories fabricated
back in (aix-ppc64, android-arm64, darwin-arm64, win32-x64, linux-x64): all
removed, host `linux-arm64` kept at both sites, 21 MB freed, and
`require('@mariozechner/clipboard')` still loads and exports all 18 functions.
`esbuild.transformSync` still compiles TS at both sites. v1.9.1's own arm64
validation of the esbuild prune also passed here — CI could only smoke-test
amd64.
**Two sentinel assertions in `smoke-test.sh`, because the size gate did not
catch this.** The gate has ~225 MB of deliberate margin, so 131 MB of pure
build residue stayed green. There are now named PASS/FAIL checks for *foreign
platform packages beyond the host arch* and for *`/root/.npm` being shipped* —
the latter deliberately refuses to run as non-root, because `test ! -d
/root/.npm` on mode-700 `/root` would otherwise pass for the wrong reason. The
size-failure diagnostics now also list cache paths: they previously enumerated
only `node_modules` and `/opt`, where these bytes were not.
**Also fixed: the prune's own progress line was mangled.** `-printf '%f\\n'`
reaches the shell with both backslashes (confirmed from the published image's
recorded `created_by`), so `find` emitted a literal backslash and `tr` then ate
the `n` out of the name — v1.9.1 printed `esbuild platform dirs kept: li
ux-arm64`. Single backslash now.
Deliberately **not** changed: `/tmp/node-compile-cache` (1.3 MB). The manifest
RUN at the end of `Dockerfile.variant` calls `pi --version` again, so deleting
it earlier only relocates those bytes into that layer — today's manifest layer
is 128 kB precisely because it finds the cache warm.
Touches `Dockerfile.variant` and `scripts/smoke-test.sh` only: `Dockerfile.base`
is unchanged, so this needs no base rebuild and should **ride the next release**
rather than burn a cycle of its own.
---
## v1.9.1 — 2026-09-10
**v1.9.0 was tagged but never published: its own smoke gate stopped it, and it