fix(size): the v1.9.1 residual was npm's own cache, not the platform binaries
v1.9.1's @esbuild prune fixed the 431 MB size-gate failure but still shipped
+131 MB compressed over v1.8.14, nearly all of it in the pi/extensions install
layer (87 -> 206 MB). That leftover was filed as an open item with an explicit
hypothesis — the @mariozechner/clipboard-* family, same npm 11 behaviour, a
different package — and an explicit warning that the hypothesis was not a
measured cause. Measured now, after recreating onto v1.9.1, and the hypothesis
accounted for one sixth of it:
+110 MB /root/.npm/_cacache (35.2 -> 145.3 MB) the build's npm cache
+ 21 MB clipboard foreign platform packages, both install sites
= 131 MB i.e. the whole delta, no unexplained remainder
Method, since there is no docker CLI inside the container: pulled both variant
layer blobs straight from the registry with a token + manifest + blob fetch and
listed the tarballs (29 789 vs 29 889 entries, 270.8 vs 401.9 MB uncompressed),
then aggregated per package. The file COUNT barely moved, which is what said
"few large files", not "npm installed more packages".
- purge_build_caches: npm cache clean --force + rm -rf /root/.npm, in the SAME
layer as the installs, in both the main RUN and the studio RUN. npm 11
caches every platform tarball it downloads, including the ones the prune
then deletes, so the cache grew faster than the tree. Nothing at runtime
reads it: build is root, container is developer with its own cache in $HOME.
- prune_foreign_esbuild -> prune_foreign_natives: now covers both MEASURED
families. Clipboard keeps linux-$arch-gnu AND -musl because its napi-rs
loader picks between them at runtime via its own isMusl() probe; the musl
package is a 420-byte stub. The bare @mariozechner/clipboard wrapper has no
hyphen suffix and cannot match the pattern.
Verified on arm64 before writing the glob — a widened rm -rf against a tree you
cannot inspect is the one change shape not to write blind, which is why the
order was update-then-patch. Exercised against a copy of the real trees with
foreign dirs fabricated back in (aix-ppc64, android-arm64, darwin-arm64,
win32-x64, linux-x64): all removed, host linux-arm64 kept at both sites, 21 MB
freed, require('@mariozechner/clipboard') still loads and exports all 18
functions, esbuild.transformSync still compiles TS at both sites. v1.9.1's own
arm64 validation also passed here; CI could only smoke amd64.
Two sentinel assertions, because the size gate did not catch this: it has
~225 MB of margin, so 131 MB of residue stayed green. Both were verified RED
against the running v1.9.1 image and GREEN against a pruned tree. The cache one
refuses to run as non-root: test ! -d /root/.npm on mode-700 /root would
otherwise pass for the wrong reason. Size-failure diagnostics now list cache
paths too — they previously enumerated only node_modules and /opt, where these
bytes were not.
Also fixed: -printf '%f\\n' reaches the shell with both backslashes (confirmed
from the published image's recorded created_by), so v1.9.1's progress line
printed a mangled "li ux-arm64" — find emitted a literal backslash and tr ate
the n out of the name. Single backslash now.
Deliberately not purged: /tmp/node-compile-cache (1.3 MB). The manifest RUN
calls pi --version again, so deleting it earlier only relocates those bytes into
that layer — today's manifest layer is 128 kB precisely because it finds the
cache warm.
Dockerfile.base untouched, so no base rebuild: this rides the next release.
This commit is contained in:
+40
-2
@@ -28,6 +28,8 @@
|
||||
# (human, --json, --quiet)
|
||||
# - (studio variant only, auto-detected) pi-studio cloned + prebuilt
|
||||
# client bundle present + registered via `pi install`
|
||||
# - no foreign npm-11 platform packages (@esbuild, clipboard) beyond the host
|
||||
# - no build-time npm cache (/root/.npm) shipped in the image
|
||||
# - image size within threshold
|
||||
|
||||
set -euo pipefail
|
||||
@@ -879,6 +881,34 @@ exec_test "pi-fork extensions floor is [] (forks cannot write to the palace)" \
|
||||
exec_test "/tmp/sshcm dir mode 700 (ssh ControlMaster)" \
|
||||
'test -d /tmp/sshcm && [ "$(stat -c %a /tmp/sshcm)" = "700" ] && echo ok'
|
||||
|
||||
# ── Build-time leftovers (npm 11 bloat sentinels) ─────────────────────
|
||||
# Both of these are worth a PASS/FAIL assertion rather than a size-gate
|
||||
# diagnostic, because the size gate has ~225 MB of deliberate margin: v1.9.1
|
||||
# shipped +131 MB of pure build residue and stayed green. These name the
|
||||
# residue directly, so a regression is legible instead of merely "bigger".
|
||||
echo ""
|
||||
echo "── Build-time leftovers ──"
|
||||
|
||||
# npm 11 installs EVERY optional platform package of a native dependency, not
|
||||
# just the host's (it ignores os/cpu, --os/--cpu and npmrc os=/cpu=). Two
|
||||
# families are affected and pruned in Dockerfile.variant: @esbuild/<platform>
|
||||
# and @mariozechner/clipboard-<triple>. Keep-set is the host arch only, plus
|
||||
# clipboard's gnu AND musl (its napi loader picks between them at runtime).
|
||||
# Runs as root because the image declares no USER; that is also what lets the
|
||||
# cache assertion below read /root.
|
||||
run "no foreign platform packages (npm 11 sentinel)" \
|
||||
'arch=$(node -p process.arch); bad=$(find /usr/lib/node_modules /opt -type d \( -regex ".*/@esbuild/[^/]+" -o -regex ".*/@mariozechner/clipboard-[^/]+" \) ! -name "linux-$arch" ! -name "clipboard-linux-$arch-gnu" ! -name "clipboard-linux-$arch-musl" -prune -print 2>/dev/null); if [ -n "$bad" ]; then echo "foreign platform dirs shipped:" >&2; echo "$bad" >&2; du -sm $bad 2>/dev/null | sort -rn | head -5 >&2; exit 1; fi; echo ok'
|
||||
|
||||
# The build's own npm download cache is not free: it lands in the layer that
|
||||
# created it. v1.9.1 shipped 145 MB of /root/.npm/_cacache (35 MB in v1.8.14)
|
||||
# — the largest single item in its +131 MB residual, and invisible to the
|
||||
# size-gate diagnostics because those only looked under node_modules and /opt.
|
||||
# Nothing at runtime reads it: root's cache, while the container runs as
|
||||
# `developer`. NOTE the assertion must run as root or a permission error on
|
||||
# mode-700 /root would make `test ! -d` pass for the wrong reason.
|
||||
run "no build-time npm cache shipped (/root/.npm)" \
|
||||
'test "$(id -u)" = "0" || { echo "assertion needs root to read /root" >&2; exit 1; }; if [ -e /root/.npm ]; then echo "/root/.npm shipped: $(du -sm /root/.npm | cut -f1) MB" >&2; exit 1; fi; echo ok'
|
||||
|
||||
# ── Image size ────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "── Image size ──"
|
||||
@@ -920,9 +950,17 @@ else
|
||||
docker run --rm --entrypoint sh "$IMAGE" -c \
|
||||
'du -sm /usr/lib/node_modules/* /opt/* /usr/local/share/ms-playwright 2>/dev/null | sort -rn | head -12' \
|
||||
2>/dev/null | sed 's/^/ /' || echo " (could not inspect directories)"
|
||||
echo " ── @esbuild platform dirs (npm 11 regression sentinel) ──"
|
||||
echo " ── build caches that should not be in the image ──"
|
||||
# v1.9.1's residual was 145 MB of npm cache under /root, and the du list
|
||||
# above cannot see it: it enumerates node_modules and /opt only. A gate whose
|
||||
# diagnostic looks only where the bytes were LAST time sends the next reader
|
||||
# spelunking again, so name the cache paths explicitly.
|
||||
docker run --rm --entrypoint sh "$IMAGE" -c \
|
||||
'find /usr/lib/node_modules /opt -type d -regex ".*/@esbuild/[^/]+" -printf "%f\n" 2>/dev/null | sort | uniq -c | sort -rn | head' \
|
||||
'du -sm /root/.npm /root/.cache /tmp/node-compile-cache /home/developer/.npm 2>/dev/null | sort -rn' \
|
||||
2>/dev/null | sed 's/^/ /' || true
|
||||
echo " ── foreign platform dirs (npm 11 regression sentinel) ──"
|
||||
docker run --rm --entrypoint sh "$IMAGE" -c \
|
||||
'find /usr/lib/node_modules /opt -type d \( -regex ".*/@esbuild/[^/]+" -o -regex ".*/@mariozechner/clipboard-[^/]+" \) -printf "%f\n" 2>/dev/null | sort | uniq -c | sort -rn | head' \
|
||||
2>/dev/null | sed 's/^/ /' || true
|
||||
fi
|
||||
|
||||
|
||||
Reference in New Issue
Block a user