From ecfd2fc2e5c907aa864ae0a46a7e216d09469b83 Mon Sep 17 00:00:00 2001 From: Joakim Persson Date: Thu, 10 Sep 2026 18:56:11 +0200 Subject: [PATCH] feat: bake dig/ldapsearch/xxd and refresh the stale pi-extensions rootfs floor Two changes that share one forced base rebuild, hence one commit. 1. THREE PACKAGES, each closing a capability gap measured during the gitea.egl.lan/FreeIPA work on 2026-09-09..10 rather than a preference: bind9-dnsutils (~6.1 MB measured) -- dig/host/nslookup were ALL absent, so the container could resolve names but had no way to interrogate a SPECIFIC nameserver. `getent hosts` only follows the resolver's default path, so diagnosing "gateway 172.16.88.1 NXDOMAINs the egl.lan zone while 10.20.253.1 is authoritative for it" had to be hand-rolled in python3. Split-horizon DNS is a recurring class of bug on this fleet. Note the package name: plain `dnsutils` is transitional in trixie. ldap-utils (1244 KB, pulls nothing extra) -- the fleet authenticates against FreeIPA, yet every LDAP probe had to be run by SSHing to an already-enrolled host. Simple binds only; GSSAPI would additionally need krb5-user + libsasl2-modules-gssapi-mit, deliberately not added as that is a Kerberos-client decision, not a tool. xxd (198 KB) -- convenience for verifying git-crypt blob magic in myconfigs; `od -c` from coreutils already does the same job. netcat-openbsd was in the original proposal and is deliberately NOT here: measured redundant, because socat is already baked and bash's /dev/tcp does reachability checks with zero packages (verified against gitea.egl.lan:3000). Recorded in the Dockerfile so the omission reads as a decision rather than an oversight. 2. ROOTFS FLOOR REFRESH: rootfs/.../pi-extensions/SKILL.md was 34284 B, unchanged since fa04d20 (2026-07-30), while the canonical package copy is 38973 B. Dockerfile.variant copies the fresh package copy over the SERVED path at build time but never writes back to this floor, so the floor is a silent fallback: if that build-time copy is ever absent it ships the July skill with no log line or manifest flag to say which version deployed. Refreshed from pi-extensions@c64c122, verified byte-identical to both the canonical and the runtime-served copies. Why one commit: the base_tag hash folds in `cat Dockerfile.base` AND `find rootfs -type f | xargs cat` (.gitea/workflows/docker-publish.yml), so either change alone forces the same full base rebuild -- and that rebuild is precisely what re-bakes rootfs/ as it then stands. Emulating the workflow hash with a fixed toolkit ref: f3d6462c7416 -> fc4edda03c54. Verified: scripts/check-base-hash.sh passes (no new ARG *_REF added), and no shell scripts are touched so the lint-shell gate is unaffected. Sizes and dependency fan-out measured via apt-get --no-install-recommends --dry-run on Debian 13 trixie. --- Dockerfile.base | 43 +++++++++++ .../pi-devbox/skills/pi-extensions/SKILL.md | 73 ++++++++++++++++++- 2 files changed, 114 insertions(+), 2 deletions(-) diff --git a/Dockerfile.base b/Dockerfile.base index 6f66c85..a5cf161 100644 --- a/Dockerfile.base +++ b/Dockerfile.base @@ -122,6 +122,46 @@ ENV DEBIAN_FRONTEND=noninteractive # are already present. NOTE this file feeds the base-decide # hash (Dockerfile.base + rootfs/), so adding it forces one # full base rebuild. +# bind9-dnsutils — `dig` and `nslookup`. Added 2026-09-10 to close a +# DIAGNOSTIC gap measured during the gitea.egl.lan/FreeIPA +# work: the container could resolve names but had NO way to +# ask a SPECIFIC nameserver anything. `getent hosts` only +# follows the resolver's default path, so the whole "gateway +# 172.16.88.1 returns NXDOMAIN for the egl.lan zone while +# 10.20.253.1 is authoritative for it" diagnosis had to be +# hand-rolled in python3 — dig, host AND nslookup were all +# absent. `dig @10.20.253.1 freeipa-4.egl.lan` is the +# one-liner that replaces it, and split-horizon DNS is a +# recurring class of bug on this fleet, not a one-off. NOTE +# the package to name is bind9-dnsutils: plain `dnsutils` is +# a transitional package in trixie. ~6.1 MB total (6210 KB +# measured): bind9-dnsutils 721 KB + bind9-host 161 KB + +# bind9-libs 3804 KB plus 7 small libs (libfstrm0, +# libjson-c5, liblmdb0, libmaxminddb0, libprotobuf-c1, +# liburcu8t64, libuv1t64) under --no-install-recommends. +# ldap-utils — `ldapsearch`/`ldapmodify`. Added 2026-09-10. This fleet +# authenticates against FreeIPA (EGL.LAN), and every LDAP +# probe during the Gitea auth work had to be run by SSHing to +# an already-enrolled host because the container had no LDAP +# client at all. 1244 KB and pulls NOTHING extra under +# --no-install-recommends — its deps (libldap, libsasl2) are +# already present. CAVEAT: this gives SIMPLE binds only, +# which is what Gitea itself uses and what most probes need. +# GSSAPI binds (`ldapsearch -Y GSSAPI`) additionally require +# krb5-user + libsasl2-modules-gssapi-mit, deliberately NOT +# added here — that is a Kerberos-client decision with +# /etc/krb5.conf implications, not just a tool. +# xxd — hex dump. 198 KB, no extra deps. Convenience, and honestly +# marginal: `od -c` from coreutils is always present and does +# the same job. Earned its place because verifying that +# git-crypt actually encrypted a staged blob (the \0GITCRYPT\0 +# magic) is a recurring check in myconfigs and xxd is the +# muscle-memory command for it. +# NOT added — netcat-openbsd (133 KB): measured redundant on +# 2026-09-10, because socat is already baked above AND bash's +# /dev/tcp does reachability checks with zero packages +# (verified against gitea.egl.lan:3000). Recorded here so the +# omission reads as a decision rather than an oversight. RUN apt-get update && \ apt-get upgrade -y --no-install-recommends && \ apt-get install -y --no-install-recommends \ @@ -163,6 +203,9 @@ RUN apt-get update && \ kitty-terminfo \ ncurses-term \ iproute2 \ + bind9-dnsutils \ + ldap-utils \ + xxd \ && ln -s /usr/bin/fdfind /usr/local/bin/fd \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* diff --git a/rootfs/usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md b/rootfs/usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md index 2cc25fe..e33b08f 100644 --- a/rootfs/usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md +++ b/rootfs/usr/local/share/pi-devbox/skills/pi-extensions/SKILL.md @@ -1,7 +1,7 @@ --- name: pi-extensions description: >- - Use the pi extensions (pi-fork, pi-observational-memory, ssh-controlmaster) effectively in the pi coding agent harness. Load this skill only when running inside pi (detection - `fork` and `recall` are present in your tool list, or `pi --ssh` was used to start the session). pi-fork dispatches focused subtasks to forked agents at fast/balanced/deep effort tiers; pi-observational-memory compacts long sessions into recallable observations + reflections; ssh-controlmaster rewires pi's read/write/edit/bash tools to execute on a remote host over a multiplexed SSH connection. This skill covers tier selection, task design, boundary discipline, when to use recall, and remote-pi mechanics. + Use the pi extensions (pi-fork, pi-observational-memory, ssh-controlmaster) effectively in the pi coding agent harness. Load this skill only when running inside pi (detection - `fork` and `recall` are present in your tool list, or `pi --ssh` was used to start the session). pi-fork dispatches focused subtasks to forked agents at fast/balanced/deep effort tiers; pi-observational-memory compacts long sessions into recallable observations + reflections; ssh-controlmaster rewires pi's read/write/edit/bash tools to execute on a remote host over a multiplexed SSH connection. Also covers the context ladder L0-L4 and when to reach for the separate `pi-task` CLI instead of `fork` - isolated child, immutable spec, machine-checked envelope, write-boundary diff. This skill covers tier selection, task design, boundary discipline, when to use recall, and remote-pi mechanics. --- # Pi Extensions: pi-fork, pi-observational-memory, ssh-controlmaster @@ -161,6 +161,68 @@ The "three" things it completed were exactly the main thread's pending todos, vi - Distrust **quantities** and **provenance claims** in fork prose specifically ("all N sessions", "shipped with the image", "as expected") — those are the slots confabulation fills. - The fact that the fork was "right anyway" is not the same as the fork having followed instructions. +### The context ladder — and the second dispatch mechanism (`pi-task`) + +Everything above describes a child that inherits everything. That is not a fixed +cost of delegation — **how much context a child gets is a choice**, and `fork` +sits at one extreme of it. Five rungs: + +| rung | what the child sees | mechanism | built? | +|---|---|---|---| +| **L0** | nothing but the goal | `pi-task` default: fresh `--session-id pitask--` in a private `--session-dir` | yes | +| **L1** | goal + **names** of files/commands to read itself | `pi-task` spec `context.files` / `context.commands` (`bin/pi-task:154,157`) | yes | +| **L2** | goal + an **excerpt the parent curated** | `pi-task` spec `context.facts`, pasted verbatim (`bin/pi-task:151`) | yes | +| **L3** | a **truncated tail** of the parent branch | *nothing implements this* — would need a new spec key plus `--session ` | **no** | +| **L4** | the **entire** parent branch | `fork(task=…)` — `getHeader()+getBranch()`, no offset or limit anywhere in the call chain | yes | + +**`pi-task` is a CLI, not an extension — it will never appear in your tool list.** +Invoke it with `bash`: `/opt/pi-toolkit/bin/pi-task run ` (source at +`/workspace/pi-toolkit/bin/pi-task`, `schema` subcommand prints the spec fields). +It reads an immutable JSON spec, and "inherit the session" is not expressible in +that schema — the isolation is structural, not a request. + +**Choose the lowest rung that can do the job:** + +- **`fork` (L4)** when the subtask only makes sense against this conversation, + when you want several independent opinions in parallel from one message, or for + read-only exploration whose detail you will discard. Everything in "Boundary + discipline" above applies in full. +- **`pi-task` (L0–L2)** when the brief contains a **prohibition** (the inherited + transcript is exactly what overrides those), when you want a **pass/fail** + result instead of prose, when you need an **audit trail**, or when writes + outside an authorised set must be caught. +- **Neither** for trivial work, iterative work (both are one-shot), or judgement + that needs context only you have. + +**What `pi-task` gets you that no brief can.** The envelope must parse or the run +FAILED, however fluent the prose. `roots[]` is the WATCHED set and +`write_allowed` the CHANGEABLE subset, diffed before and after with git +`--porcelain --ignored`. That `--ignored` flag is load-bearing: in the T4 test the +child obeyed its brief perfectly and still tripped the detector, because +`py_compile` wrote `__pycache__` into a watched-but-not-writable root — a +gitignored path that plain `--porcelain` reports as clean. Note the structural +point that test exposed: under `read_only: true` a write is *defiance*, so a +well-behaved child never produces a delta and the detector is never exercised. +Splitting WATCHED from WRITABLE is what lets an **obedient** child reveal a +violation, which is the realistic hazard. + +**What it does not fix.** `--no-extensions` removes extensions, not the core +`read`/`write`/`edit`/`bash` tools — exactly as described above — so the boundary +diff is post-hoc **detection, not prevention**. And a fresh L0 context removes the +*narrative* failures (parent voice, invented continuity) without removing +confabulation: given an under-specified spec built on a false premise, the child +still filled the `deliverable` slot with a confident shape. The envelope's own +structure creates that pressure. Verify decisive claims from the filesystem +regardless of which rung you used. + +**Trap — the capability floor is inverted from intuition.** `runner.ts:188` reads +`if (extensions !== null) args.push("--no-extensions")`. So `pi-fork.extensions: +[]` passes the flag and the floor is **on**; setting it to `null` — documented in +`settings.json` as the way to "restore normal extension loading" — passes nothing +and the floor is **off**, restoring palace writes inside every fork child. +Changing `[]` to `null` as a tidy-up re-arms what was deliberately disarmed. +`pi-task` hardcodes the flag and cannot drift this way. + ### Anti-patterns - **Forking trivial work.** A fork has overhead. If the task takes < 30 seconds in your main thread, just do it. @@ -230,7 +292,7 @@ When entries conflict, **the most recent observation reflects the latest known s ## Quick Reference ``` -fork(task=..., effort=fast|balanced|deep) +fork(task=..., effort=fast|balanced|deep) # L4: child inherits your WHOLE branch - state decision authority explicitly - pass verified context up front - specify deliverable shape @@ -240,6 +302,13 @@ fork(task=..., effort=fast|balanced|deep) - write-capable? demand "What I did NOT do", then verify from git/fs, not the report - prohibition in the brief => not a `fast` task +bash: /opt/pi-toolkit/bin/pi-task run # L0-L2: isolated child, NOT a tool + - schema | selftest | run [--dry-run] + - context.facts (pasted) / .files (names only) / .commands + - roots[] = WATCHED, write_allowed[] = CHANGEABLE subset + - envelope must parse or the run FAILED + - audit + cost: ~/.pi/agent/pi-task/-/result.json + recall(id=<12-char-hex>) - only when stakes justify the cost - id must already be visible in your context