Reclaim runner disk before load: true smoke builds

v1.14.31 publish and validate both hit 'No space left on device' on
single-arch amd64 smoke/validate builds. The image has crossed ~3 GB
and the runner's ~40 GB overlay starts ~70% full, so 'load: true'
peak disk (tarball + unpacked image + buildx cache) no longer fits.

Add a 'Reclaim runner disk' step to validate-base, validate-omos,
smoke-base, smoke-omos. Strips catthehacker-resident toolchains we
never use (hosted-tool-cache, dotnet, android, powershell, swift,
ghc, jvm, microsoft, chromium, boost), then runs 'docker system
prune -af --volumes' + 'docker builder prune -af' against the
runner's dockerd before setup-buildx-action. Expected reclaim is
6-12 GB depending on what's resident.

Deliberately NOT in the per-arch matrix build jobs — push-by-digest
doesn't need it and pruning in parallel jobs risks one job nuking
another's in-flight buildx cache. Also add workflow-level
concurrency on docker-publish.yml so concurrent tag pushes serialize
cleanly.

.gitea/workflows/docker-publish.yml

@@ -5,8 +5,29 @@ on:
     tags:
       - 'v*'
 
+# Serialize concurrent runs of the same workflow on the same ref so the
+# matrix build jobs can't race `docker system prune` in the smoke gates
+# (pruning from one job can nuke another job's in-flight buildx cache).
+# cancel-in-progress: false — tag pushes are release events, we never
+# want to silently drop one.
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: false
+
+# Runner disk pressure notes:
+# Gitea Actions runners use `catthehacker/ubuntu:act-latest` on a shared host
+# with limited overlay space (~40 GB, often 70%+ used at start). Building both
+# architectures of both variants on a single runner exhausted disk around the
+# nodejs dpkg unpack / git-lfs layer export. To fix this:
+#   * smoke test (amd64 only, load into daemon) runs on its own runner
+#   * each push target (variant × arch) runs on its own runner, pushes by
+#     digest (no local image store), uploads digest as an artifact
+#   * a merge job composes the multi-arch manifest with `imagetools create`
+# Per-runner disk pressure is now one-quarter of the old single-job peak.
+
 jobs:
-  build-base:
+  # ── Smoke test (amd64 only, gates the push jobs) ────────────────────
+  smoke-base:
     runs-on: ubuntu-latest
     container:
       image: catthehacker/ubuntu:act-latest
@@ -15,30 +36,41 @@ jobs:
         uses: actions/checkout@v4
 
       - name: Force IPv4 for Docker Hub
-        run: |
+        run: echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
-          # Prefer IPv4 to avoid intermittent IPv6 connectivity failures
-          echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
 
-      - name: Set up QEMU
+      # See docker-publish.yml preamble. `load: true` peak disk = tarball
-        uses: docker/setup-qemu-action@v4
+      # + unpacked image + buildx cache; the image now crosses the 40 GB
+      # runner overlay's starting headroom. Strip catthehacker-resident
+      # toolchains and any stale docker state up front.
+      - name: Reclaim runner disk
+        run: |
+          set -x
+          df -h / || true
+          rm -rf \
+            /opt/hostedtoolcache \
+            /opt/microsoft \
+            /opt/az \
+            /opt/ghc \
+            /usr/local/.ghcup \
+            /usr/share/dotnet \
+            /usr/share/swift \
+            /usr/local/lib/android \
+            /usr/local/share/powershell \
+            /usr/local/share/chromium \
+            /usr/local/share/boost \
+            /usr/lib/jvm 2>/dev/null || true
+          apt-get clean || true
+          rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* || true
+          docker system df || true
+          docker system prune -af --volumes || true
+          docker builder prune -af || true
+          df -h / || true
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4
         with:
           driver-opts: network=host
 
-      - name: Login to Docker Hub
-        uses: docker/login-action@v4
-        with:
-          username: ${{ vars.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Extract version from tag
-        id: version
-        run: |
-          VERSION=${GITHUB_REF#refs/tags/}
-          echo "version=${VERSION}" >> $GITHUB_OUTPUT
-
       - name: Build and load amd64 image for smoke test
         uses: docker/build-push-action@v7
         with:
@@ -49,20 +81,9 @@ jobs:
           tags: opencode-devbox:smoke-base
 
       - name: Smoke test (amd64)
-        run: |
+        run: bash scripts/smoke-test.sh opencode-devbox:smoke-base --variant base
-          bash scripts/smoke-test.sh opencode-devbox:smoke-base --variant base
 
-      - name: Build and push (base)
+  smoke-omos:
-        uses: docker/build-push-action@v7
-        with:
-          context: .
-          platforms: linux/amd64,linux/arm64
-          push: true
-          tags: |
-            ${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox:${{ steps.version.outputs.version }}
-            ${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox:latest
-
-  build-omos:
     runs-on: ubuntu-latest
     container:
       image: catthehacker/ubuntu:act-latest
@@ -71,30 +92,37 @@ jobs:
         uses: actions/checkout@v4
 
       - name: Force IPv4 for Docker Hub
-        run: |
+        run: echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
-          # Prefer IPv4 to avoid intermittent IPv6 connectivity failures
-          echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
 
-      - name: Set up QEMU
+      - name: Reclaim runner disk
-        uses: docker/setup-qemu-action@v4
+        run: |
+          set -x
+          df -h / || true
+          rm -rf \
+            /opt/hostedtoolcache \
+            /opt/microsoft \
+            /opt/az \
+            /opt/ghc \
+            /usr/local/.ghcup \
+            /usr/share/dotnet \
+            /usr/share/swift \
+            /usr/local/lib/android \
+            /usr/local/share/powershell \
+            /usr/local/share/chromium \
+            /usr/local/share/boost \
+            /usr/lib/jvm 2>/dev/null || true
+          apt-get clean || true
+          rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* || true
+          docker system df || true
+          docker system prune -af --volumes || true
+          docker builder prune -af || true
+          df -h / || true
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4
         with:
           driver-opts: network=host
 
-      - name: Login to Docker Hub
-        uses: docker/login-action@v4
-        with:
-          username: ${{ vars.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Extract version from tag
-        id: version
-        run: |
-          VERSION=${GITHUB_REF#refs/tags/}
-          echo "version=${VERSION}" >> $GITHUB_OUTPUT
-
       - name: Build and load amd64 image for smoke test
         uses: docker/build-push-action@v7
         with:
@@ -107,24 +135,225 @@ jobs:
           tags: opencode-devbox:smoke-omos
 
       - name: Smoke test (amd64)
-        run: |
+        run: bash scripts/smoke-test.sh opencode-devbox:smoke-omos --variant omos
-          bash scripts/smoke-test.sh opencode-devbox:smoke-omos --variant omos
 
-      - name: Build and push (omos)
+  # ── Per-arch push (by digest, no local image) ───────────────────────
+  build-base:
+    runs-on: ubuntu-latest
+    needs: smoke-base
+    container:
+      image: catthehacker/ubuntu:act-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        platform:
+          - linux/amd64
+          - linux/arm64
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Force IPv4 for Docker Hub
+        run: echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
+
+      - name: Derive platform slug
+        id: platform
+        run: |
+          PLATFORM_PAIR="${{ matrix.platform }}"
+          echo "pair=${PLATFORM_PAIR//\//-}" >> $GITHUB_OUTPUT
+
+      - name: Set up QEMU
+        if: matrix.platform != 'linux/amd64'
+        uses: docker/setup-qemu-action@v4
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+        with:
+          driver-opts: network=host
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@v4
+        with:
+          username: ${{ vars.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - name: Build and push by digest
+        id: build
         uses: docker/build-push-action@v7
         with:
           context: .
-          platforms: linux/amd64,linux/arm64
+          platforms: ${{ matrix.platform }}
-          push: true
+          outputs: type=image,name=${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox,push-by-digest=true,name-canonical=true,push=true
+
+      - name: Export digest
+        run: |
+          mkdir -p /tmp/digests
+          digest="${{ steps.build.outputs.digest }}"
+          touch "/tmp/digests/${digest#sha256:}"
+
+      - name: Upload digest
+        uses: actions/upload-artifact@v4
+        with:
+          name: digests-base-${{ steps.platform.outputs.pair }}
+          path: /tmp/digests/*
+          if-no-files-found: error
+          retention-days: 1
+
+  build-omos:
+    runs-on: ubuntu-latest
+    needs: smoke-omos
+    container:
+      image: catthehacker/ubuntu:act-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        platform:
+          - linux/amd64
+          - linux/arm64
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Force IPv4 for Docker Hub
+        run: echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
+
+      - name: Derive platform slug
+        id: platform
+        run: |
+          PLATFORM_PAIR="${{ matrix.platform }}"
+          echo "pair=${PLATFORM_PAIR//\//-}" >> $GITHUB_OUTPUT
+
+      - name: Set up QEMU
+        if: matrix.platform != 'linux/amd64'
+        uses: docker/setup-qemu-action@v4
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+        with:
+          driver-opts: network=host
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@v4
+        with:
+          username: ${{ vars.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - name: Build and push by digest
+        id: build
+        uses: docker/build-push-action@v7
+        with:
+          context: .
+          platforms: ${{ matrix.platform }}
           build-args: |
             INSTALL_OMOS=true
-          tags: |
+          outputs: type=image,name=${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox,push-by-digest=true,name-canonical=true,push=true
+
+      - name: Export digest
+        run: |
+          mkdir -p /tmp/digests
+          digest="${{ steps.build.outputs.digest }}"
+          touch "/tmp/digests/${digest#sha256:}"
+
+      - name: Upload digest
+        uses: actions/upload-artifact@v4
+        with:
+          name: digests-omos-${{ steps.platform.outputs.pair }}
+          path: /tmp/digests/*
+          if-no-files-found: error
+          retention-days: 1
+
+  # ── Merge per-arch digests into multi-arch tags ─────────────────────
+  merge-base:
+    runs-on: ubuntu-latest
+    needs: build-base
+    container:
+      image: catthehacker/ubuntu:act-latest
+    steps:
+      - name: Force IPv4 for Docker Hub
+        run: echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
+
+      - name: Download digests
+        uses: actions/download-artifact@v4
+        with:
+          path: /tmp/digests
+          pattern: digests-base-*
+          merge-multiple: true
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+        with:
+          driver-opts: network=host
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@v4
+        with:
+          username: ${{ vars.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - name: Extract version from tag
+        id: version
+        run: echo "version=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
+
+      - name: Create manifest list and push
+        working-directory: /tmp/digests
+        run: |
+          docker buildx imagetools create \
+            -t ${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox:${{ steps.version.outputs.version }} \
+            -t ${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox:latest \
+            $(printf '${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox@sha256:%s ' *)
+
+      - name: Inspect image
+        run: |
+          docker buildx imagetools inspect \
+            ${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox:${{ steps.version.outputs.version }}
+
+  merge-omos:
+    runs-on: ubuntu-latest
+    needs: build-omos
+    container:
+      image: catthehacker/ubuntu:act-latest
+    steps:
+      - name: Force IPv4 for Docker Hub
+        run: echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
+
+      - name: Download digests
+        uses: actions/download-artifact@v4
+        with:
+          path: /tmp/digests
+          pattern: digests-omos-*
+          merge-multiple: true
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+        with:
+          driver-opts: network=host
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@v4
+        with:
+          username: ${{ vars.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - name: Extract version from tag
+        id: version
+        run: echo "version=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
+
+      - name: Create manifest list and push
+        working-directory: /tmp/digests
+        run: |
+          docker buildx imagetools create \
+            -t ${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox:${{ steps.version.outputs.version }}-omos \
+            -t ${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox:latest-omos \
+            $(printf '${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox@sha256:%s ' *)
+
+      - name: Inspect image
+        run: |
+          docker buildx imagetools inspect \
             ${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox:${{ steps.version.outputs.version }}-omos
-            ${{ vars.DOCKERHUB_USERNAME }}/opencode-devbox:latest-omos
 
   update-description:
     runs-on: ubuntu-latest
-    needs: [build-base, build-omos]
+    needs: [merge-base, merge-omos]
     container:
       image: catthehacker/ubuntu:act-latest
     steps:

.gitea/workflows/validate.yml

@@ -46,6 +46,34 @@ jobs:
         run: |
           echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
 
+      # The runner's overlay disk starts ~70% full. `load: true` peak disk
+      # is tarball + unpacked image + buildx cache, which tips it over
+      # once the image crosses ~3 GB. Strip catthehacker-resident
+      # toolchains we never use and any stale docker state up front.
+      - name: Reclaim runner disk
+        run: |
+          set -x
+          df -h / || true
+          rm -rf \
+            /opt/hostedtoolcache \
+            /opt/microsoft \
+            /opt/az \
+            /opt/ghc \
+            /usr/local/.ghcup \
+            /usr/share/dotnet \
+            /usr/share/swift \
+            /usr/local/lib/android \
+            /usr/local/share/powershell \
+            /usr/local/share/chromium \
+            /usr/local/share/boost \
+            /usr/lib/jvm 2>/dev/null || true
+          apt-get clean || true
+          rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* || true
+          docker system df || true
+          docker system prune -af --volumes || true
+          docker builder prune -af || true
+          df -h / || true
+
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4
         with:
@@ -76,6 +104,30 @@ jobs:
         run: |
           echo 'precedence ::ffff:0:0/96  100' >> /etc/gai.conf
 
+      - name: Reclaim runner disk
+        run: |
+          set -x
+          df -h / || true
+          rm -rf \
+            /opt/hostedtoolcache \
+            /opt/microsoft \
+            /opt/az \
+            /opt/ghc \
+            /usr/local/.ghcup \
+            /usr/share/dotnet \
+            /usr/share/swift \
+            /usr/local/lib/android \
+            /usr/local/share/powershell \
+            /usr/local/share/chromium \
+            /usr/local/share/boost \
+            /usr/lib/jvm 2>/dev/null || true
+          apt-get clean || true
+          rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* || true
+          docker system df || true
+          docker system prune -af --volumes || true
+          docker builder prune -af || true
+          df -h / || true
+
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4
         with:

CHANGELOG.md

@@ -6,6 +6,28 @@ Tags follow `v{opencode_version}[letter]` — bare tag for the first build on a
 
 ---
 
+## v1.14.31b — 2026-05-01
+
+**CI: reclaim runner disk before `load: true` smoke builds.**
+
+- **Fix:** v1.14.31's publish workflow and the `validate` workflow both hit `No space left on device` on the single-arch amd64 smoke/validate builds (`/opt/uv-tools/mempalace/lib/python3.13/site-packages/hf_xet/hf_xet.abi3.so`, `/usr/local/bin/git-lfs`). Root cause is not the build itself but the `load: true` step: peak disk during export equals tarball + unpacked image + buildx cache, and the image has crossed the ~3 GB threshold where this no longer fits in the ~12 GB of free space the runner container starts with. The v1.14.30c refactor split multi-arch into per-arch push-by-digest jobs (which don't `load`), but the smoke gates still do and still hit the wall.
+  - Added a `Reclaim runner disk` step to all four `load: true` jobs (`validate-base`, `validate-omos`, `smoke-base`, `smoke-omos`). The step strips `catthehacker/ubuntu:act-latest`-resident toolchains we never use (hosted-tool-cache, dotnet, android, powershell, swift, ghc, jvm, microsoft, chromium, boost) and runs `docker system prune -af --volumes` + `docker builder prune -af` against the runner's dockerd before `setup-buildx-action`. Expected reclaim is 6–12 GB depending on what's resident.
+  - Added workflow-level `concurrency: { group: ..., cancel-in-progress: false }` on `docker-publish.yml` so concurrent tag pushes can't race `docker system prune` in one job against an in-flight buildx cache in another.
+  - Pruning is deliberately kept out of the per-arch matrix push-by-digest jobs (`build-base`/`build-omos`) — those don't need it (no `load: true`), and pruning in parallel jobs risks one job nuking another's cache.
+- **Follow-up** (not in this release): image-size reduction via a dedicated `uv tool install mempalace` build stage (strips uv's cache from the final image), pinning `mempalace-toolkit` to a commit SHA with `--depth=1 --filter=blob:none`, and auditing whether `hf_xet` is actually required by mempalace at runtime. These will ship in the next release that rebases on a new opencode version.
+- No image changes. Rebuild of v1.14.31 content only.
+
+## v1.14.31 — 2026-05-01
+
+Bump opencode to 1.14.31.
+
+**CI infrastructure: split multi-arch publish across separate runners.**
+
+- **Fix:** The `publish` workflow exhausted runner disk space on `v1.14.30b` and would have hit the same wall on any subsequent release. Both variants built both architectures on a single `catthehacker/ubuntu:act-latest` container with ~40 GB of shared overlay space, and the peak disk footprint during the nodejs dpkg unpack / git-lfs layer export pushed it over the edge (`No space left on device`). The mempalace-toolkit bake-in from v1.14.30b added the final straw; the underlying issue is that QEMU-emulated arm64 layers were stored alongside the amd64 build on the same runner.
+  - `docker-publish.yml` refactored to the canonical `push-by-digest` + manifest-merge pattern: smoke test (amd64) runs on its own runner, each `(variant × arch)` push target runs on its own fresh runner with `outputs: type=image,...,push-by-digest=true,push=true` (no local image store), then a tiny merge job assembles the multi-arch manifest with `docker buildx imagetools create` from digest artifacts.
+  - Per-runner disk peak is now roughly one-quarter of the old single-job peak. The four Docker Hub tags produced per release (`vX.Y.Z[n]`, `latest`, `vX.Y.Z[n]-omos`, `latest-omos`) are unchanged.
+  - Also parallelizes the amd64 and arm64 builds, so wall-clock time for a release should drop noticeably despite the added merge hop.
+
 ## v1.14.30b — 2026-04-30
 
 **Bake mempalace-toolkit wrappers into the image.**

Dockerfile

@@ -5,7 +5,7 @@ ARG DEBIAN_VERSION=trixie-slim
 FROM debian:${DEBIAN_VERSION} AS base
 
 ARG TARGETARCH
-ARG OPENCODE_VERSION=1.14.30
+ARG OPENCODE_VERSION=1.14.31
 
 LABEL maintainer="joakimp"
 LABEL description="Portable opencode developer container"