v1.15.0: opencode bump + git clone retry + pi-devbox sibling mention

- Bump OPENCODE_VERSION 1.14.50 -> 1.15.0 in Dockerfile.variant.
- Wrap pi-toolkit/pi-extensions git clone in Dockerfile.variant in a
  5-attempt retry loop with linear backoff (matches pi-devbox pattern).
  gitea.jordbo.se occasionally returns transient HTTP 500s that
  previously broke with-pi/omos-with-pi variant builds.
- Add 'Sibling images' section to DOCKER_HUB.md mentioning
  joakimp/pi-devbox as the pi-only counterpart.
- CHANGELOG entry for v1.15.0 with full notes.

.gitea/workflows/validate.yml

@@ -4,6 +4,22 @@ name: Validate
 # runs the smoke test, and checks image size — without pushing anything
 # to Docker Hub. Tag pushes are handled by docker-publish-split.yml which
 # does the full multi-arch split-base build-and-push.
+#
+# Trade-off: variant builds here use the published `base-latest` image
+# from Docker Hub as their parent, NOT a locally-built base. This is
+# because `docker/build-push-action@v7` runs each invocation in its own
+# buildx container context, so an image loaded into the host docker
+# daemon by step N is not visible to step N+1's buildx invocation.
+# Building base + variant in the same job would require either pushing
+# the base to a registry or sharing a buildx instance across steps — both
+# significantly more complex than just using the published base.
+#
+# Consequence: PRs/pushes that change Dockerfile.base, rootfs/, or
+# entrypoint*.sh are NOT exercised by this workflow. The release path
+# (docker-publish-split.yml on tag push) does build the new base, so
+# release tags are the gate that fully validates base-image changes.
+# The base-change-warning job below surfaces a runtime warning when this
+# blind-spot applies.
 
 on:
   push:
@@ -34,6 +50,33 @@ jobs:
         run: |
           python3 scripts/generate-dockerhub-md.py --check
 
+  base-change-warning:
+    # Surfaces a warning when this commit changes base-image inputs
+    # (Dockerfile.base, rootfs/, entrypoint*.sh). validate.yml uses
+    # Hub's base-latest as the parent for variant builds, so changes to
+    # those files are NOT exercised here — only release tags rebuild the
+    # base via docker-publish-split.yml.
+    runs-on: ubuntu-latest
+    container:
+      image: catthehacker/ubuntu:act-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 2
+
+      - name: Detect base-input changes
+        run: |
+          set -e
+          if ! git diff --name-only HEAD~1 HEAD 2>/dev/null \
+              | grep -qE '^(Dockerfile\.base|rootfs/|entrypoint.*\.sh)$'; then
+            echo "No base-image inputs changed in this commit — validate.yml fully exercises the published base-latest."
+            exit 0
+          fi
+          echo "::warning::This commit changes base-image inputs (Dockerfile.base, rootfs/, or entrypoint*.sh). validate.yml uses Hub's base-latest as the parent for variant builds, so the new base is NOT exercised by this workflow. Cut a release tag, or run a workflow_dispatch of docker-publish-split.yml against a test tag (e.g. v0.0.0-base-test, promote_latest=false) for end-to-end validation of the new base."
+          echo "Changed base-input files:"
+          git diff --name-only HEAD~1 HEAD | grep -E '^(Dockerfile\.base|rootfs/|entrypoint.*\.sh)$'
+
   validate-base:
     runs-on: ubuntu-latest
     container:
@@ -83,9 +126,12 @@ jobs:
         uses: docker/build-push-action@v7
         with:
           context: .
+          file: Dockerfile.variant
           platforms: linux/amd64
           push: false
           load: true
+          build-args: |
+            BASE_IMAGE=joakimp/opencode-devbox:base-latest
           tags: opencode-devbox:ci-base
 
       - name: Smoke test
@@ -137,10 +183,12 @@ jobs:
         uses: docker/build-push-action@v7
         with:
           context: .
+          file: Dockerfile.variant
           platforms: linux/amd64
           push: false
           load: true
           build-args: |
+            BASE_IMAGE=joakimp/opencode-devbox:base-latest
             INSTALL_OMOS=true
           tags: opencode-devbox:ci-omos
 
@@ -193,10 +241,12 @@ jobs:
         uses: docker/build-push-action@v7
         with:
           context: .
+          file: Dockerfile.variant
           platforms: linux/amd64
           push: false
           load: true
           build-args: |
+            BASE_IMAGE=joakimp/opencode-devbox:base-latest
             INSTALL_PI=true
           tags: opencode-devbox:ci-with-pi
 
@@ -249,10 +299,12 @@ jobs:
         uses: docker/build-push-action@v7
         with:
           context: .
+          file: Dockerfile.variant
           platforms: linux/amd64
           push: false
           load: true
           build-args: |
+            BASE_IMAGE=joakimp/opencode-devbox:base-latest
             INSTALL_OMOS=true
             INSTALL_PI=true
           tags: opencode-devbox:ci-omos-with-pi

CHANGELOG.md

@@ -8,6 +8,49 @@ Tags follow `v{opencode_version}[letter]` — bare tag for the first build on a
 
 ## Unreleased
 
+## v1.15.0 — 2026-05-15
+
+opencode 1.14.50 → 1.15.0 bump (upstream minor release).
+
+- **Bump:** opencode 1.14.50 → 1.15.0 (`OPENCODE_VERSION` in `Dockerfile.variant`).
+- **Resilience:** `git clone` for pi-toolkit and pi-extensions in `Dockerfile.variant` is now wrapped in a 5-attempt retry loop with linear backoff (5s, 10s, 15s, 20s, 25s = up to ~75s total). gitea.jordbo.se occasionally returns transient HTTP 500s on the first request after idle, which previously broke the with-pi and omos-with-pi variant builds. Same pattern landed in pi-devbox repo concurrently.
+- **Docs:** `DOCKER_HUB.md` mentions `joakimp/pi-devbox` as a sibling image — the pi-only build that uses this image's base layer as its parent. Generator template (`scripts/generate-dockerhub-md.py`) updated and regenerated. Hub size: 5905 bytes (well under the 25 kB limit).
+- **Recovery from v1.14.50c partial publish:** the `latest-omos`, `v1.14.50c-omos` Hub gap is closed by this release — `latest-omos` will move forward to v1.15.0 once all four variants publish cleanly. Users on the floating tag were unaffected (still pointing at v1.14.41b until now).
+
+## v1.14.50c — 2026-05-14
+
+Recovery release for v1.14.50b's missing variants. v1.14.50b shipped only the `base` variant; `omos`, `with-pi`, and `omos-with-pi` were lost to a runner-fleet incident (see postmortem below).
+
+No container-side changes. This is a tag-only retag to re-run the build on a now-healthy runner fleet. Same `base-35ee5fe7861a` from v1.14.50b is reused via hash-cache hit; only the four variant deltas are rebuilt and published.
+
+### Postmortem: v1.14.50 / v1.14.50b runner-fleet incident
+
+Two orthogonal runner-host issues compounded across runs 285–291:
+
+1. **AVX-less runner shadowing the new fleet.** A pre-migration `act_runner` container on `nyvaken` (Sandy Bridge E3-12xx, has AVX but no AVX2; 4 weeks old, name `act_runner-runner-1`) collided with the orchestrator's freshly deployed `runner-1` VM (Broadwell-EP host, fully AVX2-capable). Gitea scheduled jobs to both. Jobs landing on the nyvaken container `npm install -g opencode-ai@1.14.50` succeeded, then ran `opencode --version` postinstall → the bundled Bun (v1.3.13 baseline) emitted `CPU lacks AVX support`, panicked, and SIGILLed (exit code 132).
+2. **Containerd shared-state race at `capacity: 2`.** The new VM-based runners initially ran `act_runner` with `capacity: 2`, scheduling two concurrent jobs on a single host. Both jobs would invoke `docker/setup-buildx-action@v4`, which pulls `moby/buildkit:buildx-stable-1`. Containerd's content store raced on identical sha256 ingestion, surfacing as `commit failed: rename .../ingest/.../data .../blobs/sha256/...: no such file or directory` or `failed to extract layer: failed to Lchown ...`.
+
+A secondary issue surfaced: **Proxmox VM `cpu:` field defaults mask AVX**. The newly-cloned runner VMs had no explicit `cpu:` line in `qm config` and inherited Proxmox's recent default `x86-64-v2-AES`, which excludes AVX even though the Broadwell-EP host silicon has full `avx2`. Fix: `qm set <vmid> --cpu x86-64-v3` (or `host` for full passthrough), then `qm shutdown` + `qm start` (live reboot is not enough). Verified inside guest with `grep -m1 -oE 'avx[2]?' /proc/cpuinfo`.
+
+Additionally, when `promote-base-latest`'s `needs:` graph requires *all four* `build-variant-*` jobs to succeed, partial publishes leave the `base-latest` Hub alias never advancing. Workaround used during recovery: manually re-tag the new base hash via Docker Hub registry manifest API (`PUT /v2/<repo>/manifests/base-latest` with the body of `GET /v2/<repo>/manifests/base-<sha>`) using a granular Hub PAT. No blob copy needed since blobs are content-addressed.
+
+### Recovery actions taken (orchestrator + this repo)
+
+- Orchestrator (cloud-init + ansible repos): set explicit `cpu_type: x86-64-v3` in all runner host yaml files; provision.sh now applies `qm set --cpu` after clone; added runner-3 on proxmox003 for anti-affinity (one runner per Proxmox node); dropped `capacity: 2 → 1` on all runners; bumped `act_runner` 0.3.1 → 0.6.1 across the fleet; documented the CPU-type gotcha as gotcha #9 in cloud-init AGENTS.md and a section in proxmox-guide.md.
+- User: retired the legacy `act_runner-runner-1` container on nyvaken; cleaned up stale runner registrations in Gitea Site Admin → Actions → Runners.
+- This repo: no changes needed in Dockerfile.base / Dockerfile.variant; v1.14.50c is a tag-only retag.
+
+### Fleet state at v1.14.50c
+
+3 runners (runner-1@proxmox001, runner-2@proxmox002, runner-3@proxmox003), all `act_runner` v0.6.1, all `capacity: 1`, all expose AVX + AVX2 to the guest. No name collisions. Estimated wall clock for v1.14.50c (cache-hit base, 4 variant deltas across 3 runners with capacity:1): ~40–50 min.
+
+## v1.14.50b — 2026-05-14
+
+Rebuild of v1.14.50 with two fixes — the v1.14.50 release was incomplete (smokes failed under containerd contention; build-variant jobs skipped; base-latest never promoted to Docker Hub).
+
+- **Force fresh base rebuild.** Added a `BASE_REBUILD_DATE` comment header to `Dockerfile.base` to invalidate the content hash and trigger a full base rebuild. Picks up ~5 days of Debian trixie security updates and other apt-tracked packages. The comment also documents the pattern for future intentional base-rebuilds without other code changes (recommended cadence: once per release).
+- **First publish of `base-latest` alias.** `promote-base-latest` runs unconditionally on tag push (`PROMOTE_LATEST=true`), so this release is the first to put `joakimp/opencode-devbox:base-latest` on Docker Hub. Required before pi-devbox (and any other downstream image FROMing the base) can build.
+
 ## v1.14.50 — 2026-05-14
 
 opencode 1.14.44 → 1.14.50 bump. First release on the split-base build pipeline.

DOCKER_HUB.md

@@ -86,6 +86,10 @@ Full persistence reference, including multi-user (`SIGNUM`) isolation and host b
 - **Issues / source / docker-compose templates:** <https://gitea.jordbo.se/joakimp/opencode-devbox>
 - **Agent-facing internals** (for future maintainers / coding agents working in the repo): <https://gitea.jordbo.se/joakimp/opencode-devbox/src/branch/main/AGENTS.md>
 
+## Sibling images
+
+- **[`joakimp/pi-devbox`](https://hub.docker.com/r/joakimp/pi-devbox)** — pi-only image built on top of this image's base layer. Smaller (~700 MB) and version-tracks the [pi npm package](https://www.npmjs.com/package/@earendil-works/pi-coding-agent) directly. Use this if you want pi without opencode. Source: <https://gitea.jordbo.se/joakimp/pi-devbox>
+
 ## License
 
 MIT. See <https://gitea.jordbo.se/joakimp/opencode-devbox/src/branch/main/LICENSE>.

Dockerfile.base

@@ -11,6 +11,13 @@
 # changes (rootfs/, entrypoint*.sh). Version bumps to OPENCODE_VERSION,
 # OMOS_VERSION, PI_VERSION, etc. do NOT trigger a base rebuild.
 #
+# To force a base rebuild for fresh apt packages without other code
+# changes, bump the BASE_REBUILD_DATE comment below. The hash is
+# content-addressed over this file, so any byte change invalidates the
+# cache. Recommended cadence: once per release for security updates.
+#
+# BASE_REBUILD_DATE: 2026-05-14 (v1.14.50b — fresh apt + first promote-base-latest)
+#
 # See the project README's "Build pipeline" section for the rationale.
 
 ARG DEBIAN_VERSION=trixie-slim

Dockerfile.variant

@@ -32,7 +32,7 @@ ARG USER_NAME=developer
 
 # ── Install opencode via npm ─────────────────────────────────────────
 ARG INSTALL_OPENCODE=true
-ARG OPENCODE_VERSION=1.14.50
+ARG OPENCODE_VERSION=1.15.0
 RUN if [ "${INSTALL_OPENCODE}" = "true" ]; then \
       NPM_CONFIG_PREFIX=/usr npm install -g opencode-ai@${OPENCODE_VERSION} && \
       opencode --version ; \
@@ -47,16 +47,25 @@ ARG PI_VERSION=latest
 ARG PI_TOOLKIT_REF=main
 ARG PI_EXTENSIONS_REF=main
 RUN if [ "${INSTALL_PI}" = "true" ]; then \
+      set -e && \
+      git_clone_retry() { \
+        url="$1"; ref="$2"; dest="$3"; \
+        for i in 1 2 3 4 5; do \
+          if git clone --depth 1 --branch "$ref" "$url" "$dest"; then return 0; fi; \
+          rm -rf "$dest"; \
+          echo "git clone $url failed (attempt $i/5), retrying in $((i*5))s..."; \
+          sleep $((i*5)); \
+        done; \
+        return 1; \
+      } && \
       if [ "${PI_VERSION}" = "latest" ]; then \
         NPM_CONFIG_PREFIX=/usr npm install -g @earendil-works/pi-coding-agent ; \
       else \
         NPM_CONFIG_PREFIX=/usr npm install -g @earendil-works/pi-coding-agent@${PI_VERSION} ; \
       fi && \
       pi --version && \
-      git clone --depth 1 --branch "${PI_TOOLKIT_REF}" \
-        https://gitea.jordbo.se/joakimp/pi-toolkit.git /opt/pi-toolkit && \
-      git clone --depth 1 --branch "${PI_EXTENSIONS_REF}" \
-        https://gitea.jordbo.se/joakimp/pi-extensions.git /opt/pi-extensions && \
+      git_clone_retry https://gitea.jordbo.se/joakimp/pi-toolkit.git "${PI_TOOLKIT_REF}" /opt/pi-toolkit && \
+      git_clone_retry https://gitea.jordbo.se/joakimp/pi-extensions.git "${PI_EXTENSIONS_REF}" /opt/pi-extensions && \
       echo "pi-toolkit at $(cd /opt/pi-toolkit && git rev-parse --short HEAD)" && \
       echo "pi-extensions at $(cd /opt/pi-extensions && git rev-parse --short HEAD)" ; \
     fi

scripts/generate-dockerhub-md.py

@@ -140,6 +140,10 @@ Full persistence reference, including multi-user (`SIGNUM`) isolation and host b
 - **Issues / source / docker-compose templates:** <{GITEA}>
 - **Agent-facing internals** (for future maintainers / coding agents working in the repo): <{GITEA}/src/branch/main/AGENTS.md>
 
+## Sibling images
+
+- **[`joakimp/pi-devbox`](https://hub.docker.com/r/joakimp/pi-devbox)** — pi-only image built on top of this image's base layer. Smaller (~700 MB) and version-tracks the [pi npm package](https://www.npmjs.com/package/@earendil-works/pi-coding-agent) directly. Use this if you want pi without opencode. Source: <https://gitea.jordbo.se/joakimp/pi-devbox>
+
 ## License
 
 MIT. See <{GITEA}/src/branch/main/LICENSE>.