ci: pin the smoke image with a running container during the test
The label only protects against reclaims that honour it. The runner hosts also run jobs we do not control: runner-2 has a second act_runner registered with gitea.egl.lan on the same Docker daemon, and opencode-devbox's workflows run raw `docker system prune -af` in 7 places. No prune removes an image a running container uses, so the smoke step now starts a holder (`ci-reclaim-docker.sh hold`: --rm, --network none, sleep 3600) right after the preflight, and the always() cleanup step runs `release` (holder + image). Unprotected now: only the seconds between the build step's load and the holder starting. Verified on an isolated docker:29-dind: an unlabelled held image survives raw system prune and image prune -af (control without holder: deleted); hold fails on a missing image; holder is AutoRemove + network none; release removes holder and image; reclaim behaviour unchanged.
This commit is contained in:
+17
-6
@@ -17,9 +17,11 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`).
|
||||
|
||||
- **CI: a smoke job's image could be deleted mid-test by another job's disk
|
||||
reclaim.** This is why v1.12.2's `-studio` tags were not published by its
|
||||
tag run. Jobs that land on the same runner share its Docker daemon (in run
|
||||
740, runner-a1 ran `smoke-studio` and `build-variant` at the same time), and
|
||||
each job began with `docker system prune -af --volumes`, which removes every
|
||||
tag run. Everything on one runner host shares its Docker daemon: concurrent
|
||||
jobs of a runner with capacity > 1 (in run 740, runner-a1 ran `smoke-studio`
|
||||
and `build-variant` at the same time), and the jobs of other act_runner
|
||||
containers on the same VM (runner-2 has one for gitea.jordbo.se and one for
|
||||
gitea.egl.lan). Each pi-devbox job began with `docker system prune -af --volumes`, which removes every
|
||||
image that no container is using at that moment. A smoke image is unused
|
||||
between `smoke-test.sh`'s one-shot `docker run`s. In run 740, `smoke`
|
||||
passed and `build-variant` started. Its prune logged
|
||||
@@ -33,8 +35,14 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`).
|
||||
volumes, build cache) but never prunes an image labelled
|
||||
`se.jordbo.pi-devbox.ci-smoke-image`. The smoke builds apply that label, a
|
||||
preflight fails if it is missing, and each smoke job removes its own image at
|
||||
the end. Smoke images left by a killed job are swept once they are older
|
||||
than 6 h. Smoke tags carry the run id (`pi-devbox:smoke-<run_id>`), so two
|
||||
the end. Jobs we do not control can still run a raw prune on the same
|
||||
daemon: opencode-devbox's workflows do, in 7 places, and so may whatever
|
||||
gitea.egl.lan runs. So the smoke step also pins its image with a running
|
||||
container (`ci-reclaim-docker.sh hold`) for the whole test, because no prune
|
||||
removes an image a running container uses. Only the few seconds between the
|
||||
build step loading the image and the pin starting are left unprotected. The
|
||||
holder exits by itself after 1 h, and smoke images left by a killed job are
|
||||
swept once they are older than 6 h. Smoke tags carry the run id (`pi-devbox:smoke-<run_id>`), so two
|
||||
runs on different refs cannot test or delete each other's image. `lint.yml`
|
||||
fails on any raw `docker system|image prune` in a workflow. Job order is
|
||||
unchanged: the studio jobs still gate only the `-studio` tags. Reordering
|
||||
@@ -45,7 +53,10 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`).
|
||||
fresh labelled image, deletes an unlabelled one and a labelled one created
|
||||
in 1970, keeps an image a running container uses, and prints no errors.
|
||||
`assert-protected` passes the labelled image and fails both an unlabelled
|
||||
and a missing one.
|
||||
and a missing one. With `hold`, an unlabelled image survives both a raw
|
||||
`docker system prune -af --volumes` and `docker image prune -af` (control:
|
||||
without the holder, the same prune deletes it). `hold` fails on a missing
|
||||
image, and `release` removes the holder and the image.
|
||||
|
||||
## v1.12.2 — 2026-10-22
|
||||
|
||||
|
||||
Reference in New Issue
Block a user