fix(size): the v1.9.1 residual was npm's own cache, not the platform binaries
Lint / doc-drift (push) Successful in 6s
Lint / hadolint (push) Successful in 10s
Lint / skill-floor (push) Successful in 9s
Lint / actionlint (push) Successful in 4m26s

v1.9.1's @esbuild prune fixed the 431 MB size-gate failure but still shipped
+131 MB compressed over v1.8.14, nearly all of it in the pi/extensions install
layer (87 -> 206 MB). That leftover was filed as an open item with an explicit
hypothesis — the @mariozechner/clipboard-* family, same npm 11 behaviour, a
different package — and an explicit warning that the hypothesis was not a
measured cause. Measured now, after recreating onto v1.9.1, and the hypothesis
accounted for one sixth of it:

  +110 MB  /root/.npm/_cacache  (35.2 -> 145.3 MB)   the build's npm cache
  + 21 MB  clipboard foreign platform packages, both install sites
  = 131 MB  i.e. the whole delta, no unexplained remainder

Method, since there is no docker CLI inside the container: pulled both variant
layer blobs straight from the registry with a token + manifest + blob fetch and
listed the tarballs (29 789 vs 29 889 entries, 270.8 vs 401.9 MB uncompressed),
then aggregated per package. The file COUNT barely moved, which is what said
"few large files", not "npm installed more packages".

  - purge_build_caches: npm cache clean --force + rm -rf /root/.npm, in the SAME
    layer as the installs, in both the main RUN and the studio RUN. npm 11
    caches every platform tarball it downloads, including the ones the prune
    then deletes, so the cache grew faster than the tree. Nothing at runtime
    reads it: build is root, container is developer with its own cache in $HOME.
  - prune_foreign_esbuild -> prune_foreign_natives: now covers both MEASURED
    families. Clipboard keeps linux-$arch-gnu AND -musl because its napi-rs
    loader picks between them at runtime via its own isMusl() probe; the musl
    package is a 420-byte stub. The bare @mariozechner/clipboard wrapper has no
    hyphen suffix and cannot match the pattern.

Verified on arm64 before writing the glob — a widened rm -rf against a tree you
cannot inspect is the one change shape not to write blind, which is why the
order was update-then-patch. Exercised against a copy of the real trees with
foreign dirs fabricated back in (aix-ppc64, android-arm64, darwin-arm64,
win32-x64, linux-x64): all removed, host linux-arm64 kept at both sites, 21 MB
freed, require('@mariozechner/clipboard') still loads and exports all 18
functions, esbuild.transformSync still compiles TS at both sites. v1.9.1's own
arm64 validation also passed here; CI could only smoke amd64.

Two sentinel assertions, because the size gate did not catch this: it has
~225 MB of margin, so 131 MB of residue stayed green. Both were verified RED
against the running v1.9.1 image and GREEN against a pruned tree. The cache one
refuses to run as non-root: test ! -d /root/.npm on mode-700 /root would
otherwise pass for the wrong reason. Size-failure diagnostics now list cache
paths too — they previously enumerated only node_modules and /opt, where these
bytes were not.

Also fixed: -printf '%f\\n' reaches the shell with both backslashes (confirmed
from the published image's recorded created_by), so v1.9.1's progress line
printed a mangled "li ux-arm64" — find emitted a literal backslash and tr ate
the n out of the name. Single backslash now.

Deliberately not purged: /tmp/node-compile-cache (1.3 MB). The manifest RUN
calls pi --version again, so deleting it earlier only relocates those bytes into
that layer — today's manifest layer is 128 kB precisely because it finds the
cache warm.

Dockerfile.base untouched, so no base rebuild: this rides the next release.
This commit is contained in:
Joakim Persson
2026-09-11 09:39:54 +02:00
parent 42bd29d654
commit 1baba79c96
4 changed files with 209 additions and 29 deletions
+81 -25
View File
@@ -196,24 +196,70 @@ RUN set -e && \
done; \
return 1; \
} && \
# prune_foreign_esbuild: npm 11 (shipped with Node 24) installs EVERY
# @esbuild/<platform> optional binary instead of only the one matching the
# host — 26 platform dirs, 284 MB, for aix-ppc64/android/darwin/freebsd/
# netbsd/openbsd/win32/s390x/riscv64/... that this image can never execute.
# Measured on pi-fork's tree: npm 10.9.8 -> 165 MB, npm 11.19.0 -> 449 MB,
# and the 165 MB figure reproduces what v1.9.0's predecessor actually shipped.
# prune_foreign_natives: npm 11 (shipped with Node 24) installs EVERY optional
# platform package of a native dependency, not just the one matching the host.
# TWO families are affected in this image, and BOTH have been measured — add a
# family here only after measuring it, never by widening the pattern on a hunch:
#
# @esbuild/<platform> 26 dirs, 284 MB (found first, v1.9.1)
# @mariozechner/clipboard-<triple> 11 dirs, 12 MB per site, 10 MB foreign
#
# esbuild declares those with os/cpu constraints, but npm 11 ignores them and
# ALSO ignores --os/--cpu and an npmrc carrying os=/cpu= (all three measured).
# So prune explicitly, keeping only linux-$(node -p process.arch) so the same
# line is correct on amd64 and arm64. Verified after pruning that esbuild still
# works (transformSync compiles TS), i.e. this removes dead weight, not function.
# So prune explicitly, keeping only the host platform, computed from
# `node -p process.arch` so one line stays correct on amd64 and arm64.
# Measured on pi-fork's tree: npm 10.9.8 -> 165 MB, npm 11.19.0 -> 449 MB,
# and the 165 MB figure reproduces what v1.9.0's predecessor actually shipped.
#
# WHY THE CLIPBOARD FAMILY WAS ADDED (2026-09-11): v1.9.1 pruned @esbuild only
# and still shipped +131 MB compressed over v1.8.14. That residual was
# attributed by listing the PUBLISHED arm64 layer tarballs straight from the
# registry (there is no docker CLI inside the container, so `docker history`
# was not available): +110 MB /root/.npm/_cacache (purged below) and +21 MB of
# clipboard platform packages across the two install sites — 131 MB total, so
# the delta is now fully accounted for with no unexplained remainder.
#
# Keeping linux-$arch-{gnu,musl} is deliberate: clipboard's napi-rs loader
# tries ./<name>.node then the platform package, per platform in try/catch, and
# chooses gnu vs musl at runtime from its own isMusl() probe — so both host-arch
# branches must survive. The musl package is a 420-byte stub, i.e. free. The
# bare wrapper `@mariozechner/clipboard` has no hyphen suffix and therefore
# cannot match the regex below. Verified on arm64 against a copy of the real
# tree before this was written: after pruning to those two,
# require('@mariozechner/clipboard') still loads and exports all 18 functions.
# esbuild likewise still compiles TS via transformSync at both install sites.
# This removes dead weight, not function.
#
# MUST run in the SAME layer as the npm installs above: deleting in a later RUN
# leaves the bytes in this layer and shrinks the image by nothing.
prune_foreign_esbuild() { \
keep="linux-$(node -p process.arch)"; \
# NOTE the single backslash in -printf '%f\n': Docker passes '\\n' through
# verbatim, so v1.9.1's doubled version printed a mangled "li ux-arm64"
# (find emitted a literal backslash, then `tr` translated the n out of the
# name). Confirmed from the published image's own recorded created_by.
prune_foreign_natives() { \
arch="$(node -p process.arch)"; \
find /usr/lib/node_modules /opt -type d -regex '.*/@esbuild/[^/]+' \
! -name "$keep" -prune -exec rm -rf {} + ; \
echo "esbuild platform dirs kept: $(find /usr/lib/node_modules /opt -type d -regex '.*/@esbuild/[^/]+' -printf '%f\\n' 2>/dev/null | sort -u | tr '\\n' ' ')"; \
! -name "linux-$arch" -prune -exec rm -rf {} + ; \
find /usr/lib/node_modules /opt -type d -regex '.*/@mariozechner/clipboard-[^/]+' \
! -name "clipboard-linux-$arch-gnu" ! -name "clipboard-linux-$arch-musl" \
-prune -exec rm -rf {} + ; \
echo "native platform dirs kept: $(find /usr/lib/node_modules /opt -type d \( -regex '.*/@esbuild/[^/]+' -o -regex '.*/@mariozechner/clipboard-[^/]+' \) -printf '%f\n' 2>/dev/null | sort | uniq -c | tr '\n' ' ')"; \
} && \
# purge_build_caches: the build's own download caches are NOT free — they land
# in whichever layer created them. Measured on the published v1.9.1 arm64
# variant layer: root/.npm/_cacache was 145.2 MB of a 401.9 MB layer (35.2 MB
# in v1.8.14), the single biggest item in the +131 MB residual, because npm 11
# caches every platform tarball it fetched — including the ones just pruned.
# Nothing at runtime reads it: the build runs as root, the container runs as
# `developer` with its own cache under $HOME (and $HOME/.pi is a volume).
# DELIBERATELY NOT purged here: /tmp/node-compile-cache (1.3 MB, written by
# `pi --version` below). The manifest RUN at the end of this file calls
# `pi --version` again, so deleting it here only relocates those bytes into
# that layer instead of removing them from the image — measured, not assumed:
# today the manifest layer is 128 kB precisely because it finds the cache warm.
purge_build_caches() { \
npm cache clean --force >/dev/null 2>&1 || true; \
rm -rf /root/.npm; \
} && \
if [ "${PI_VERSION}" = "latest" ]; then \
NPM_CONFIG_PREFIX=/usr npm install -g @earendil-works/pi-coding-agent ; \
@@ -228,7 +274,8 @@ RUN set -e && \
git_fetch_ref "${PI_ATELIER_REPO}" "${PI_ATELIER_REF}" /opt/pi-atelier && \
(cd /opt/pi-fork && npm install --omit=dev --no-audit --no-fund) && \
(cd /opt/pi-observational-memory && npm install --omit=dev --no-audit --no-fund) && \
prune_foreign_esbuild && \
prune_foreign_natives && \
purge_build_caches && \
echo "pi-toolkit at $(cd /opt/pi-toolkit && git rev-parse --short HEAD)" && \
echo "pi-extensions at $(cd /opt/pi-extensions && git rev-parse --short HEAD)" && \
echo "pi-fork at $(cd /opt/pi-fork && git rev-parse --short HEAD)" && \
@@ -329,17 +376,25 @@ ARG PI_STUDIO_REF=main
ARG PI_STUDIO_VERSION=none
RUN if [ "${INSTALL_STUDIO}" = "true" ]; then \
set -e; \
# Same esbuild prune as the main install RUN — see the comment there. It has
# to be redefined because shell functions do not survive across layers, and
# it has to run in THIS layer because pi-studio's npm install happens here:
# deleting in a later RUN would leave the bytes in this layer and shrink
# nothing. pi-studio pulls its own pi-coding-agent copy, so it is a third
# ~274 MB site on top of the two in the non-studio variant.
prune_foreign_esbuild() { \
keep="linux-$(node -p process.arch)"; \
# Same prune + cache purge as the main install RUN — see the comments there.
# They have to be redefined because shell functions do not survive across
# layers, and they have to run in THIS layer because pi-studio's npm install
# happens here: deleting in a later RUN would leave the bytes in this layer
# and shrink nothing. pi-studio pulls its own pi-coding-agent copy, so it is
# a third ~274 MB site on top of the two in the non-studio variant — and its
# npm install refills /root/.npm, which the main RUN emptied in ITS layer.
prune_foreign_natives() { \
arch="$(node -p process.arch)"; \
find /usr/lib/node_modules /opt -type d -regex '.*/@esbuild/[^/]+' \
! -name "$keep" -prune -exec rm -rf {} + ; \
echo "esbuild platform dirs kept: $(find /usr/lib/node_modules /opt -type d -regex '.*/@esbuild/[^/]+' -printf '%f\\n' 2>/dev/null | sort -u | tr '\\n' ' ')"; \
! -name "linux-$arch" -prune -exec rm -rf {} + ; \
find /usr/lib/node_modules /opt -type d -regex '.*/@mariozechner/clipboard-[^/]+' \
! -name "clipboard-linux-$arch-gnu" ! -name "clipboard-linux-$arch-musl" \
-prune -exec rm -rf {} + ; \
echo "native platform dirs kept: $(find /usr/lib/node_modules /opt -type d \( -regex '.*/@esbuild/[^/]+' -o -regex '.*/@mariozechner/clipboard-[^/]+' \) -printf '%f\n' 2>/dev/null | sort | uniq -c | tr '\n' ' ')"; \
}; \
purge_build_caches() { \
npm cache clean --force >/dev/null 2>&1 || true; \
rm -rf /root/.npm; \
}; \
rm -rf /opt/pi-studio && mkdir -p /opt/pi-studio && \
git -C /opt/pi-studio init -q && \
@@ -352,7 +407,8 @@ RUN if [ "${INSTALL_STUDIO}" = "true" ]; then \
done; \
[ "$ok" = "1" ] && \
(cd /opt/pi-studio && npm install --omit=dev --no-audit --no-fund) && \
prune_foreign_esbuild && \
prune_foreign_natives && \
purge_build_caches && \
echo "pi-studio at $(cd /opt/pi-studio && git rev-parse --short HEAD)"; \
fi