Compare commits
10 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| a2846a5f7e | |||
| 58c22afb04 | |||
| 30094782df | |||
| 9b5783f9dd | |||
| d9a7fe101b | |||
| 36e65fe657 | |||
| f0ebea2d98 | |||
| b615571913 | |||
| 495b7e3859 | |||
| 45850bc973 |
@@ -146,6 +146,14 @@ GIT_USER_EMAIL=
|
||||
# Detection is automatic if the skillset lives at WORKSPACE_PATH/skillset.
|
||||
# SKILLSET_CONTAINER_PATH=
|
||||
|
||||
# ── cli_utils (standalone commands from a mounted checkout) ──────────
|
||||
# If a cli_utils repo is mounted, the entrypoint symlinks its bin/ commands
|
||||
# into ~/.local/bin on every start, so they survive container recreate and
|
||||
# resolve in non-interactive shells too (docker exec, agent tool shells).
|
||||
# Detection is automatic at WORKSPACE_PATH/cli_utils (or one level below).
|
||||
# CLI_UTILS_CONTAINER_PATH=
|
||||
# CLI_UTILS_LINK=0 # disable the linking entirely
|
||||
|
||||
# ── Locale ───────────────────────────────────────────────────────────
|
||||
# LANG=sv_SE.UTF-8
|
||||
# LANGUAGE=sv_SE:sv
|
||||
|
||||
+469
@@ -11,6 +11,475 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`).
|
||||
|
||||
---
|
||||
|
||||
## v1.8.12 — 2026-08-31
|
||||
|
||||
**`pi` `0.84.3` → `0.84.4`, and `pi-atelier` `v0.8.2` → `v0.10.0`.** Both audited
|
||||
by the routine in `Dockerfile.variant` rather than adopted on sight, and the
|
||||
audit notes live next to the pins where the next reader will meet them.
|
||||
|
||||
**pi 0.84.4 (published 2026-08-28) carries no `Breaking Changes` and no
|
||||
`Removed` heading** — checked by grepping the section, 0 matches, which is worth
|
||||
stating because 0.84.3 *did* have one. It was adopted for three fixes that land
|
||||
on machinery this fleet runs every day, not for the feature list:
|
||||
|
||||
- **#6879** — a large tool result crossing the auto-compaction threshold used to
|
||||
be sent to the provider *before* compaction. Pi now compacts between tool
|
||||
execution and the next assistant response inside the same run. That is the
|
||||
shape of nearly every session on these boxes, where a single `event_list` or
|
||||
palace search returns hundreds of KB.
|
||||
- **#8345** — a resumed session corrupted its next appended entry when the JSONL
|
||||
file lacked a trailing newline. That file is the memory feeder's *input*, so
|
||||
the failure would have surfaced as unexplained gaps in `wing_conversations`
|
||||
rather than as an error. Measured on tor-ms22 before bumping: 49/49
|
||||
transcripts end in a newline and 0 lines fail `json.loads` — this corpus was
|
||||
never bitten, and we now know that rather than hope it.
|
||||
- **#8537** — extension messages sent with `triggerTurn: false` *while the agent
|
||||
is running* were inserted between a tool call and its result, so
|
||||
order-validating providers rejected the replayed history. **The mempalace
|
||||
mailbox is outside that precondition**: it delivers at `agent_settled`, when
|
||||
no inference is in flight, with `{deliverAs: "steer"}` and deliberately no
|
||||
`triggerTurn`. 0.84.4 also leaves the documented steer semantics untouched
|
||||
("delivered after the current assistant turn finishes executing its tool
|
||||
calls, before the next LLM call"), so RFC 003 §7.11 stands as written. Recorded
|
||||
because this fix is precisely what would make a *mid-run* delivery safe, which
|
||||
is the only reason we would ever change that call.
|
||||
|
||||
Also new and relevant, though nothing here uses them yet: `ui_prompt_start` /
|
||||
`ui_prompt_end` extension events (the `docs/extensions.md` diff is add-only — no
|
||||
steer or `triggerTurn` semantics moved), and an RPC `clear_queue` that returns
|
||||
and removes queued steering messages. The second one can discard an
|
||||
already-delivered but unconsumed mailbox steer; that is survivable because the
|
||||
mailbox re-delivers on `MEMPALACE_MAILBOX_RESURFACE_MS` (default 3600000), and it
|
||||
is written down here so a future "the mailbox lost a message" report has a
|
||||
candidate cause. The three new `PI_HYPERLINKS` / `PI_IMAGE_PROTOCOL` /
|
||||
`PI_TRUE_COLOR` environment variables were grepped against this whole repo: no
|
||||
collisions with anything the image sets.
|
||||
|
||||
**The bump moved one documented mechanism, so `docs/observational-memory.md` §3
|
||||
moved with it.** Pi's own `docs/compaction.md` gained exactly one paragraph in
|
||||
0.84.4: the `autoCompact` threshold is now *also* checked mid-run, after a tool
|
||||
batch's results are appended and before the next assistant response, skipped only
|
||||
when that batch ends the run and no queued message needs another response. Our
|
||||
doc said compaction is "checked when pi goes idle, so it never interrupts a
|
||||
turn". That was only ever true of observational-memory's **own** trigger
|
||||
(`compaction-trigger.ts` hooks `agent_settled`); read as a statement about pi it
|
||||
is now false. `session_before_compact` (`compaction-hook.ts`) therefore has
|
||||
**two** entry points and the second can fire inside a turn — harmless for the
|
||||
ledger fold, which makes no model call, but a doc that ships a false promise
|
||||
about when a hook runs is worse than one that admits two paths. The §3 mermaid
|
||||
diagram gained the second edge, and the whole file re-passes the bundled mermaid
|
||||
checker (6 blocks, 44 labels, 0 soft-wrapped, no cut glyphs at 1280px and
|
||||
800px).
|
||||
|
||||
**pi-atelier `v0.8.2` → `v0.10.0` is two minor releases and both are UI-only** —
|
||||
Sidebar kept calm during an active Turn, composer frame and Status Rail polish,
|
||||
fullscreen-copy-safe Sidebar, Windows path normalisation, Workspace Pulse
|
||||
deferred until pi trusts the project. Neither release carries a BREAKING notice.
|
||||
The coupling that matters runs the *opposite* way to this pin's hard-earned
|
||||
floor: v0.9.0 renders the Sidebar as a separate split-layout child and therefore
|
||||
"raises the minimum supported Pi version to 0.84.0", and — unlike the
|
||||
0.7.1-under-pi-0.84 startup-hang precedent, which its metadata never encoded —
|
||||
this time `peerDependencies` says so (`>=0.84.0`, up from `>=0.80.7`). Satisfied
|
||||
with room to spare by `PI_VERSION=0.84.4`. It also pairs deliberately with a
|
||||
0.84.4 feature: atelier keeps Sidebar content out of the fullscreen transcript
|
||||
selection while pi adds `fullscreenCopyOnSelect` and Ctrl+X for the selection
|
||||
itself. Both executable floors (`scripts/smoke-test.sh`,
|
||||
`scripts/recreate-sanity-check.sh`) compare with `sort -V`, so `0.10.0 >= 0.7.1`
|
||||
is evaluated correctly — verified by running the comparison, because the string
|
||||
form of that test reads `0.10.0` as *older* than `0.7.1`.
|
||||
|
||||
**While bumping the pins, the README's own pin table turned out to have been
|
||||
wrong since v1.8.6.** It advertised pi `0.84.2` and mempalace `3.7.1` in the very
|
||||
table whose purpose is to tell a reader what is pinned and where. Both rows went
|
||||
stale in the *same* commit — `93f986e` (v1.8.6, "adopt pi 0.84.3 + mempalace
|
||||
3.8.0") moved both `ARG`s and neither table row; the rows themselves date from
|
||||
`29b6209` (v1.8.0) and `2ebf00d` (v1.8.4). Only atelier's row was still true.
|
||||
All three corrected now, and the `--expected-version 0.84.3` example in the
|
||||
recreate-sanity section updated too, since that one is a copy-pasteable command
|
||||
that would now fail against a 0.84.4 image. Worth noting how it survived two
|
||||
releases: nothing checks prose against the `ARG`s, so this table has to be
|
||||
remembered by hand on every pin bump, and once it was not.
|
||||
|
||||
**`credential-incident-response` gained the section its own guidance had been
|
||||
missing, and §2 gained a precondition it should always have carried.** Docs only;
|
||||
no image behaviour moves. Both changes came out of a session where three separate
|
||||
detectors reported *clean* over secrets that were really there — the skill was
|
||||
the artifact that had taught two agents the pattern, so the fix belongs here
|
||||
rather than in either operator's private notes.
|
||||
|
||||
**§2 previously said an 8-hex fingerprint lets you compare a credential "without
|
||||
ever materialising the secret", with no condition attached.** That is true only
|
||||
when the *input space* is unreachable. A fingerprint is 32 bits over whatever it
|
||||
was computed from, so publishing `fp8(x)` hands anyone a **membership oracle**:
|
||||
they can test `x == v` for every candidate `v` they can generate. For a 40-char
|
||||
random token, fine. For a hostname, username, e-mail, port, path, commit SHA or
|
||||
weak password, that candidate set is a wordlist — and note that "high entropy" is
|
||||
the usual sufficient condition, not the test: a commit SHA is 160-bit and still
|
||||
fully enumerable from the repo. Two agents on this fleet published fingerprints of
|
||||
`GIT_USER_EMAIL`-class values while following this section as written; harmless in
|
||||
that instance, because those values sit in every commit trailer already, but the
|
||||
guidance licensed it. §2 now states the precondition, adds that candidate
|
||||
fingerprints are working memory and never output (a scanner hashes hostnames and
|
||||
paths too, so "print what it saw" leaks wholesale), and names what a fingerprint
|
||||
register *is* — a confirmation oracle for anyone already holding a candidate
|
||||
corpus, which is exactly how a retired token gets identified in old transcripts,
|
||||
and works the same way for someone else holding those files.
|
||||
|
||||
**New §6, "Proving absence: instrument strength, and four ways a scan lies
|
||||
clean".** Deliberately placed next to §5, because §5 optimises against false
|
||||
*positives* (name-anchoring, provenance — what stops a triage sweep drowning in
|
||||
session UUIDs) and every failure in §6 is a false *negative*. Triage optimises
|
||||
precision; a gate optimises recall, and conflating the two is what produced the
|
||||
clean reports. It carries: an instrument-strength ranking (exact-byte value search
|
||||
> class/structure pass > fingerprint census) with the standing instruction to say
|
||||
which one produced your zero; census and class passes answering different
|
||||
questions, with both failure modes measured here — a class-only pre-commit hook
|
||||
passed plaintext UUID API credentials to a shared repo twice because a UUID has no
|
||||
key header, while a census-only gate reported 0 hits with freshly-synced SSH
|
||||
private keys in the tree because no key is in the census; the tokenisation trap,
|
||||
where maximal-run extraction swallows an unquoted `VAR=<uuid>` so the value is
|
||||
never hashed alone while a *quoted* one is found, meaning quoting alone decided
|
||||
detectability; scan the index or the pushed tree, never the working tree, plus why
|
||||
a repo-only fix on an rsync-published mirror is temporary rather than weaker; git
|
||||
filters never running on symlinks, where `check-attr` answers `git-crypt` for a
|
||||
path it can never encrypt, so a coverage audit must join the attribute against the
|
||||
file mode and verify the blob magic; two-sided self-tests that abort, including
|
||||
the fixture-interaction artifact where a quoted and unquoted probe share one
|
||||
buffer and make the weak extractor look as strong as the union; and row-gone is
|
||||
not bytes-gone, since a correct sqlite DELETE leaves the payload in freelist pages
|
||||
until VACUUM.
|
||||
|
||||
Findings contributed by `pi@emb-7kj4vr4g` (the census/class split, and the
|
||||
instrument ranking's provenance) and `pi@tor-ms22` (exact-byte value search over
|
||||
index blobs). The description's trigger list grew accordingly and is 1022/1024
|
||||
characters — **it has almost no headroom, so trim before adding to it**, or the
|
||||
skill silently fails to load.
|
||||
|
||||
**Deployment:** the skill is baked at
|
||||
`/usr/local/share/pi-devbox/skills/credential-incident-response/`, so this needs
|
||||
an image rebuild **and** a container recreate to reach any running container.
|
||||
|
||||
**Two vendored skills changed, and one of the changes is a correction rather than
|
||||
an addition.** Nothing about the image's behaviour moves; this is entirely about
|
||||
what the next agent reads before it acts.
|
||||
|
||||
**`pi-devbox-environment` §2 had a rule that was half wrong, and the wrong half
|
||||
cost five findings in one session.** The section "A negative result is usually
|
||||
your own filter" closed with *"a positive result needs no such scepticism — it
|
||||
carries its own evidence."* That sentence is false. A positive result is evidence
|
||||
about the question your command *actually posed*, which may not be the question
|
||||
you meant — and the failure is invisible precisely because the command succeeded.
|
||||
Three measured instances, all from 2026-08-29, all filed as fact before being
|
||||
caught: an SSH handshake that succeeded and greeted the agent as `joakimp` while
|
||||
it believed it was probing `gitea.egl.lan` (a `Host gitea*` block had rewritten
|
||||
`HostName`, so it authenticated to the wrong Gitea instance); a `401` that was a
|
||||
genuine answer from an issuer which had never minted the credential being tested;
|
||||
and a "regression" produced by diffing `ssh -G` output against a `2222` that the
|
||||
agent's own earlier `-p 2222` flag had supplied. The section now carries a
|
||||
counterpart, *"…and a positive result only proves what you actually asked"*, plus
|
||||
the three false-negative rows that session added (a palace scan that queried
|
||||
`embedding_metadata` while documents live in `embedding_fulltext_search_content`;
|
||||
a token declared dead on a 401 from the wrong issuer; a host declared unreachable
|
||||
after trying two of its three open ports, with the port written in an environment
|
||||
variable the agent already held).
|
||||
|
||||
**The cross-cutting form of that rule went into `pi-global-AGENTS.append.md`, not
|
||||
into the skill — deliberately, and this is the whole point of the change.** The
|
||||
rule *already existed* in the baked skill, authored by an earlier session,
|
||||
symlinked into `~/.agents/skills/` at every container start. It survived every
|
||||
recreate, was available for the entire session that broke it, and was violated
|
||||
five times anyway. So the gap was never persistence; it was **activation**.
|
||||
A reasoning rule that only loads when a task description happens to match it
|
||||
cannot fire on the occasions that need it, because "I am about to state something
|
||||
false" is not a recognisable task type. The always-appended block is read by every
|
||||
agent in every container without being asked for, which is the only property that
|
||||
matters here. Writing a sixth document restating the rule would have felt like
|
||||
progress and changed nothing.
|
||||
|
||||
**New baked skill: `credential-incident-response`.** Authored here, so the baked
|
||||
copy is canonical and it is *not* listed in `skillset-owned.txt`. It carries the
|
||||
*facts* a two-day credential incident produced, on the theory that facts transfer
|
||||
between sessions where exhortations do not: probe the issuing provider **first**
|
||||
(11 of 13 "exposed" credentials in that sweep turned out to be already dead at the
|
||||
provider — five HTTP requests would have established it, and nobody asked);
|
||||
`sha256[:8]` fingerprints as leak-free credential identity; the `403`-vs-`401`
|
||||
trap that scoped tokens introduce into liveness probes, where a live token looks
|
||||
revoked on `/api/v1/user`; **revocation beats deletion** for anything already
|
||||
replicated, because deletion is best-effort over an unbounded copy set (FTS shadow
|
||||
rows, per-host feed inboxes, sqlite free pages, mesh replicas, backups) while
|
||||
revocation invalidates copies nobody enumerated; the three places a secret hides
|
||||
in a Chroma palace, in coverage order; deriving least-privilege scopes from
|
||||
*measured* consumers; and the exposures rotation does not fix (cleartext channels,
|
||||
git history, agent-authored drawers).
|
||||
|
||||
**Three smoke assertions extended** so a rebuild cannot silently drop the new
|
||||
skill: baked-file existence, resolves-to-the-baked-tree, and reported as `baked`
|
||||
by `pi-devbox-version`. Skill directories are picked up by a glob in
|
||||
`entrypoint-user.sh`, so no registration was needed — verified rather than
|
||||
assumed, since an enumerated list would have left the skill inert, which would
|
||||
have been a fitting way for *this* skill to fail.
|
||||
|
||||
Neither skills change reaches a running container until the image is rebuilt **and**
|
||||
the container recreated: `~/.agents/skills/` and the global `AGENTS.md` both live in
|
||||
the image, not in a volume or a mount.
|
||||
|
||||
**`cli_utils`' shell *functions* are now sourced, closing the half of that wiring
|
||||
the image never did.** v1.8.11 linked the repo's `bin/` **commands** into
|
||||
`~/.local/bin` so they resolve in non-interactive shells; nothing ever sourced
|
||||
`cli_utils.sh`, so its 14 **functions** (`fgit`, `fhist`, `fssh`, `fdocker`,
|
||||
`fmark`, `fproc`, `fex`, `fenv`, `extract`, `mkcd`, `pathls`, `portcheck`,
|
||||
`agents-sync`, `up`) were missing from every interactive shell whose `$HOME` had
|
||||
no zsh rc. That is the normal case, not an edge case: the container's interactive
|
||||
shell is bash and **zsh is not installed in the image**. A symlink cannot carry a
|
||||
shell function and a function cannot be reached from a non-interactive shell, so
|
||||
the two mechanisms are disjoint and both are required — the image had been paying
|
||||
this layer's dependency cost (`fzf`, `bat`, `fd`, `rg`, `jq` are baked partly *for*
|
||||
these functions) while delivering none of its benefit. Now sourced from
|
||||
`/etc/skel-devbox/.bash_aliases`, with the same detection order as the symlink
|
||||
block so commands and functions can never come from two different clones.
|
||||
`CLI_UTILS_SOURCE=0` opts out, deliberately independent of `CLI_UTILS_LINK=0`
|
||||
because the two disable independent mechanisms. Measured: all 14 resolve in a
|
||||
freshly-seeded `$HOME`, the opt-out is honoured, an absent checkout is a genuinely
|
||||
silent no-op (no output, no leaked `_cu` variable), and interactive shell startup
|
||||
goes from 12 ms to 17 ms.
|
||||
|
||||
**Named explicitly, per this repo's own floating-ref rule: `/workspace/cli_utils`
|
||||
is a host bind mount, not a pinned ref.** Sourcing it means the image now executes
|
||||
content it does not pin, on every interactive shell, on every device. It is
|
||||
bash-safe today and that was measured rather than assumed — sourcing under
|
||||
`bash --noprofile --norc` exits 0 and defines all 14 despite the `*.zsh`
|
||||
filenames, the functions run, and the tree's single zsh-only construct (`print -z`
|
||||
in `fzf/fhist.zsh`) is already guarded by `[[ -n $ZSH_VERSION ]]` with a bash
|
||||
fallback. The residual risk is future content: a cli_utils commit adding a
|
||||
genuinely zsh-only file would surface as parse errors at every prompt, fleet-wide.
|
||||
Errors are therefore left visible rather than sent to `/dev/null`, so the failure
|
||||
is diagnosable, and `CLI_UTILS_SOURCE=0` is the one-line escape hatch.
|
||||
|
||||
**`iproute2` is installed, so the container can answer "what is listening in
|
||||
here".** Neither `ss` nor `ip` was present in any image up to and including
|
||||
v1.8.11 — nor `lsof`, nor `netstat` — which made `cli_utils`' `portcheck` a hard
|
||||
stub that printed `portcheck requires at least one of: ss, lsof, netstat` and
|
||||
exited. `ss` satisfies its preferred branch (`ss -tlnp`), which is also the only
|
||||
branch that reports the owning PID. `net-tools` is deliberately **not** added
|
||||
(`netstat` is deprecated and only a fallback path) and neither is `lsof` (~500 KB
|
||||
for a third route to the same answer). Cost measured, not estimated: ~5.5 MB total
|
||||
— `iproute2` is 4.2 MB and pulls six libs under `--no-install-recommends`
|
||||
(`libbpf1`, `libmnl0`, `libtirpc-common`, `libtirpc3t64`, `libxtables12`,
|
||||
`libcap2-bin`; `libpam-cap` is a Recommends and is correctly dropped). Verified in
|
||||
a live container: `ss` at `/usr/bin/ss`, `ip` at `/usr/sbin/ip`, both already on
|
||||
the developer `PATH`, and `portcheck --all` then correctly identifies the `socat`
|
||||
listener on 8765.
|
||||
|
||||
The two changes above also need a rebuild **and** a recreate, for a different
|
||||
reason than the skills: `$HOME` is the container's writable layer rather than a
|
||||
named volume (verified — `~/.bash_aliases` carries the container's start mtime
|
||||
while `~/.bashrc` carries the image's), so the skel file is re-seeded on every
|
||||
recreate. A `$HOME/.bash_aliases` that is bind-mounted from the host is still
|
||||
never overwritten, which is the existing contract.
|
||||
|
||||
### Dependency audit (2026-08-31)
|
||||
|
||||
Every component checked against upstream by direct command, not assumed:
|
||||
|
||||
| Component | Baked in v1.8.11 | Upstream now | Action |
|
||||
|---|---|---|---|
|
||||
| **pi** | `0.84.3` (pinned) | **`0.84.4`** is npm latest | bumped + audited (above) |
|
||||
| **pi-atelier** | `v0.8.2` (pinned) | **`v0.10.0`** highest tag | bumped + audited (above) |
|
||||
| mempalace | `3.8.0` (pinned) | `3.8.0` is PyPI latest | none |
|
||||
| skillset (mempalace fallback snapshot) | `a12fe5e` | `a12fe5e` == `origin/main`, 0 commits since | none — `--check` reports OK, no NOTICE |
|
||||
| mempalace-toolkit | `21023e7` | `21023e7` | none |
|
||||
| pi-toolkit | `0e1369e` | `0e1369e` | none |
|
||||
| pi-extensions | `2022887` | `2022887` | none |
|
||||
| pi-fork | `bf702b4` | `bf702b4` | none |
|
||||
| pi-observational-memory | `ce9fc98` | `ce9fc98` (v3.0.4, peerDeps `*` → no pi floor to clear) | none |
|
||||
| pi-studio (studio variant) | `3328b3d` | `3328b3d` | none |
|
||||
| floating `*_VERSION=latest` tools (16) | — | 14 already at latest; `git-lfs` `3.7.1`→`3.8.0` (feature, no breaking section), `uv` `0.12.6`→`0.12.7` (patch) | adopted implicitly by the rebuild; named here per this repo's floating-ref rule |
|
||||
| node | major pin `22`, installed `v22.23.2` | `v22.23.2` is the newest 22.x | none — a newer LTS *line* (24.x) exists and is deliberately not tracked |
|
||||
|
||||
Two method notes, because both would have produced a confident wrong answer:
|
||||
|
||||
- **An annotated tag's `ls-remote` SHA is the tag object, not the commit.**
|
||||
`refs/tags/v0.8.2` is `6e07bf85` while `refs/tags/v0.8.2^{}` is `159f34cf` —
|
||||
the value actually baked. Comparing the un-dereferenced form reported
|
||||
`pi-atelier` as *drifted from its own pin*, which would have been a false
|
||||
integrity alarm about the one component whose pin is load-bearing. Always
|
||||
deref with `^{}` before calling a pin broken.
|
||||
- **`git ls-remote --tags | sort -V | tail` is not a "latest release" proxy.**
|
||||
`typst/typst` carries date-style tags (`v23-03-28`) and `mikefarah/yq` carries
|
||||
`vTestA`/`vTestB`; both sort *after* the real releases. `Dockerfile.base`
|
||||
itself resolves `latest` by reading the `Location` of
|
||||
`curl -sI …/releases/latest`, so replaying that exact step is both
|
||||
noise-immune and the same source of truth the build will see.
|
||||
|
||||
---
|
||||
|
||||
## v1.8.11 — 2026-08-27
|
||||
|
||||
**Shell state that the writable layer eats on every recreate now gets rebuilt at
|
||||
start.** Two additions to `entrypoint-user.sh`, both idempotent, both silent
|
||||
no-ops when the thing they wire up is absent.
|
||||
|
||||
**`cli_utils` commands are linked onto `PATH`.** If a `cli_utils` checkout is
|
||||
mounted, every executable in its `bin/` is symlinked into `~/.local/bin` at
|
||||
container start — `git-status-all`, `git-pull-all`, `devbox-sanity`,
|
||||
`pi-devbox-sanity`, `pi-session-repair`, `docker-clean`, `vpn-status`. Detection:
|
||||
`CLI_UTILS_CONTAINER_PATH` → `/workspace/cli_utils` → `$HOME/cli_utils` →
|
||||
`/workspace/*/cli_utils`; `CLI_UTILS_LINK=0` disables it.
|
||||
|
||||
The reason this is an *image* concern and not the user's problem to re-solve: on a
|
||||
host, `cli_utils/install.sh` puts those commands on `PATH` by symlinking them into
|
||||
`~/.local/bin`, which is persistent there — and **ephemeral here**. Same installer,
|
||||
same repo, opposite durability, so the fix died on every `--force-recreate` and the
|
||||
next session was back to typing `/workspace/cli_utils/bin/git-status-all`. Running
|
||||
`install.sh` *inside* a container is the trap rather than the fix: it re-creates
|
||||
the same disposable state.
|
||||
|
||||
**Symlinks rather than a `PATH` edit in an rc file, deliberately.** `~/.local/bin`
|
||||
is already ahead of `/usr/local/bin` in `ENV PATH`, so links resolve in
|
||||
**non-interactive** shells too — `docker exec <c> git-status-all`, agent tool
|
||||
shells, scripts. An rc-file `PATH` edit cannot reach those: `~/.bashrc` returns
|
||||
early when the shell is not interactive. Measured on tor-ms22 2026-08-27,
|
||||
`command -v git-status-all` failed in a non-interactive shell while succeeding in
|
||||
an interactive one, from exactly that asymmetry. Guards, because `~/.local/bin` is
|
||||
shared with other tooling: a real file is never clobbered, a symlink pointing
|
||||
somewhere else is never stolen, our own links are refreshed, and links into a
|
||||
`cli_utils/bin` whose target vanished are pruned — a dangling link on `PATH`
|
||||
reports "No such file or directory" and reads as a broken container rather than a
|
||||
removed script.
|
||||
|
||||
**A per-device boot hook: `~/.config/devbox-shell/init.sh`.** If the host provides
|
||||
one, it runs once at start with output to `~/.pi/agent/devbox-init.log`. That
|
||||
directory is the host-owned bind-mount already sourced into every interactive
|
||||
shell by `/etc/skel-devbox/.bash_aliases`, so this is its boot-time twin — the
|
||||
same ownership and the same persistence, but running *before any shell*, which is
|
||||
what non-interactive fixups (symlinks, directories, one-off migrations) need. **It
|
||||
introduces no new trust boundary**: that path is already arbitrary code from the
|
||||
same owner; only *when* it runs is new. Invoked as `bash <file>`, never sourced,
|
||||
and its exit status is ignored — a hook must not be able to mutate the
|
||||
entrypoint's own shell state or stop a container from starting.
|
||||
|
||||
With the hook in place, the next "can this run on every recreate?" question needs
|
||||
no image change at all — which is the point, given what the next paragraph costs.
|
||||
|
||||
**This moves the base hash.** `base-decide` folds `cat entrypoint.sh
|
||||
entrypoint-user.sh` into it, so this change forces the ~40-minute base rebuild at
|
||||
the next tag whether or not anything else in the base moved. It is a rider, not a
|
||||
reason to tag.
|
||||
|
||||
**How it was validated, since CI cannot.** `docker-publish.yml` runs only on
|
||||
`push: tags: v*`, and `lint.yml` runs `actionlint` over workflow `run:` steps —
|
||||
neither one executes `entrypoint-user.sh`. So both sections were extracted and run
|
||||
against fixtures in a throwaway `$HOME` before commit: real file not clobbered,
|
||||
foreign symlink respected, stale link pruned, new command picked up, second run
|
||||
byte-identical, `CLI_UTILS_LINK=0` honoured, and "no `cli_utils` anywhere" a silent
|
||||
`exit 0`. Then run for real in a live v1.8.10 container, after which
|
||||
`command -v git-status-all` resolved in a *non-interactive* shell. No
|
||||
`smoke-test.sh` assertion was added on purpose: the positive path needs a
|
||||
`/workspace` mount that smoke does not have, and asserting it there would repeat
|
||||
the v1.8.0 mistake of a smoke assertion written against a stage that does not
|
||||
exist at run time. `workflow_dispatch` with `smoke_only` remains the way to
|
||||
exercise this against `HEAD` before a tag.
|
||||
|
||||
**Also carried by the floating `mempalace-toolkit` main ref** (resolved at build
|
||||
time, not by a pi-devbox commit — `MEMPALACE_TOOLKIT_REF=main`):
|
||||
|
||||
**A scrubbed re-export of a dormant session could silently never reach the
|
||||
palace host.** `bin/mempalace-pi-session` ships to the palace with
|
||||
`rsync -a --update`, and the stage file's mtime is deliberately the SOURCE
|
||||
transcript's mtime (`os.utime()`, "preserve session mtime for dedup
|
||||
stability"). Re-exporting a session that has not been appended to since its
|
||||
last ship therefore produces a mtime that is *not newer* than the receiver's —
|
||||
exactly the case a redactor upgrade needs to ship, since content differs while
|
||||
mtime does not. `--update` reported success and sent nothing. Found and
|
||||
patched by `pi@mbp-m1-2020` (mempalace-toolkit `a361b71`): `--update` →
|
||||
`--checksum`, which compares content and ignores size/mtime entirely.
|
||||
Dropping `--update` outright was considered and rejected — rsync's default
|
||||
quick check already transfers on a size difference alone, which would have
|
||||
masked the *next* instance of this (a redaction whose placeholder happens to
|
||||
match the secret's length) as fixed. `os.utime()` is untouched; its backdating
|
||||
is a separate, load-bearing design call for dedup stability. New regression
|
||||
test, `scripts/test-rsync-ship-idempotency.sh`, runs fully offline (a local
|
||||
rsync destination exercises the same size/mtime/checksum comparison as the ssh
|
||||
transfer) and is built to *discriminate*: it must fail against `--update` and
|
||||
pass against `--checksum,` not merely exercise the code path — the first draft
|
||||
of the test used fixture strings of different lengths and passed for the wrong
|
||||
reason (rsync's quick check transfers on size difference alone regardless of
|
||||
`--update`), which is the same trap the patch itself was written to avoid.
|
||||
**Acceptance line for this class of change going forward:** "receiver sha256
|
||||
matches sender for every staged file", not "local stage is clean" — a clean
|
||||
local stage says nothing about what a dormant session already sent.
|
||||
|
||||
**An event addressed to an identity no session runs as is delivered to
|
||||
nobody, and this fleet has now hit it three separate ways.** RFC 003 gains
|
||||
§7.13 and open-decision 10 (mempalace-toolkit `21023e7`, docs only, no image
|
||||
behaviour change): the owed-set derivation — the log's only push channel — is
|
||||
keyed on `to_agent`, and a reply is always addressed back to whatever string
|
||||
the *original writer* put in `from_agent`. Nothing validates that string
|
||||
against a live session identity, so authoring under a synthetic or foreign
|
||||
name makes every reply to that event write-only. Measured cost this cycle: a
|
||||
directed ask planted under a synthetic sender drew a correct reply containing
|
||||
an urgent security finding, and it sat unread for ~2h20m, found only because a
|
||||
human asked whether mail had arrived. Permitted exception, unchanged: a
|
||||
synthetic sender is fine for a deliberate control experiment, provided the
|
||||
body names the real identity to reply to.
|
||||
|
||||
**Also carried by the live `skillset` mount** (each device's own clone, not
|
||||
baked — except the `mempalace` skill's fallback snapshot, re-vendored below):
|
||||
|
||||
**The mermaid-diagrams checker's cut gate moved from client pixels to a
|
||||
per-SVG user-space unit.** `CUT_PX` was calibrated against one live page at
|
||||
one render scale; sweeping `--viewport` 500→1600 on an *unchanged* document
|
||||
moved the worst overflow −1.0px → −3.0px, near-proportional to the viewport,
|
||||
i.e. a constant geometric overflow viewed through a changing scale. `cutU =
|
||||
cutPx / scale` (scale taken per-SVG, never a page average — one page mixes
|
||||
scales 0.643–0.988) recovers that invariant: the sweep now collapses to
|
||||
exactly −3.0u at every width. Re-deriving the threshold against the live host
|
||||
surfaced a real false negative the old pixel gate had: a label at
|
||||
`cutPx=0.4, scale=0.678` read as healthy under `CUT_PX=0.5` but is `0.59u` —
|
||||
a genuine cut hiding behind a compressed render scale. `CUT_U` stays `0.5`;
|
||||
`cutPx` and `scale` are still printed on every issue so a devtools ruler still
|
||||
confirms the number on the actual page. A new, explicitly-deferred finding
|
||||
from the same review: `cut` only measures vertically, so an unbreakable token
|
||||
wider than its box (a long URL, a `snake_case` identifier) is invisible to
|
||||
soft-wrap, tall, *and* cut simultaneously — filed as a backlog item, not
|
||||
implemented, pending a fifth acceptance control.
|
||||
|
||||
**The `from_agent`-identity finding above is also now in the `mempalace`
|
||||
skill itself** ("Writing to another machine", and Anti-Patterns), and the
|
||||
baked fallback snapshot of that skill was refreshed to match
|
||||
(`vendor-mempalace-skill.sh`, `6eb20af` → `a12fe5e`) — sanctioned to skip on
|
||||
its own (`--check` reported stale-but-truthful), done anyway because this
|
||||
release's point is getting today's fixes live, and the base rebuild below was
|
||||
already forced regardless.
|
||||
|
||||
### Dependency audit (2026-08-27)
|
||||
|
||||
Every component checked against upstream by direct command, not assumed:
|
||||
|
||||
| Component | Baked in v1.8.10 | Upstream now | Action |
|
||||
|---|---|---|---|
|
||||
| **mempalace-toolkit** | `b2b50af` | **`21023e7`** | ships the rsync ship-fix + RFC 003 §7.13 (both above) |
|
||||
| **skillset** (mempalace fallback snapshot) | `6eb20af` | **`a12fe5e`** | re-vendored (above); live-mounted devices already had it |
|
||||
| pi | `0.84.3` (pinned) | `0.84.3` is npm latest | none |
|
||||
| mempalace | `3.8.0` (pinned) | `3.8.0` is PyPI latest | none |
|
||||
| pi-atelier | `v0.8.2` (pinned) | `v0.8.2` highest tag | none |
|
||||
| pi-studio (studio variant) | `v0.9.52` | `v0.9.52` — `main`'s commit and the tag's commit are identical (0 either direction) | none |
|
||||
| pi-toolkit | `0e1369e` | `0e1369e` (local clone HEAD == `origin/main`) | none |
|
||||
| pi-extensions | `2022887` | `2022887` (local clone HEAD == `origin/main`) | none |
|
||||
| pi-fork | `bf702b4` | `bf702b4` | none |
|
||||
| pi-observational-memory | `ce9fc98` | `ce9fc98` | none |
|
||||
|
||||
pi-toolkit / pi-extensions checked against their actual Gitea origin (the
|
||||
Dockerfile's `PI_TOOLKIT_REPO` / `PI_EXTENSIONS_REPO`), not a GitHub mirror —
|
||||
querying `api.github.com` for those two returned nothing (rate-limited or
|
||||
blocked; not investigated, the local clones are the source of truth anyway).
|
||||
No SHA above is fork-supplied; a fork inventing plausible-looking upstream SHAs
|
||||
is a recorded failure mode (v1.8.9), so every value here came from
|
||||
`git ls-remote`, a local clone's own `origin/HEAD`, `npm view`/registry JSON,
|
||||
or the PyPI JSON API, run directly.
|
||||
|
||||
---
|
||||
|
||||
## v1.8.10 — 2026-08-27
|
||||
|
||||
**This tag exists to deploy a fix and a safety net that are currently running on
|
||||
|
||||
@@ -83,6 +83,24 @@ ENV DEBIAN_FRONTEND=noninteractive
|
||||
# above); TERM=xterm-ghostty is compiled from an alias further
|
||||
# down (ncurses ships `ghostty`, not `xterm-ghostty`). iTerm2
|
||||
# defaults to xterm-256color (ncurses-base), so needs nothing.
|
||||
# iproute2 — `ss` (socket statistics) and `ip`. Measured 2026-08-30 on
|
||||
# v1.8.11: NEITHER was present, so the container could not
|
||||
# answer "what is listening in here" by any means, and
|
||||
# cli_utils' `portcheck` was a hard stub — it prints
|
||||
# "portcheck requires at least one of: ss, lsof, netstat" and
|
||||
# all three were absent. `ss` satisfies its preferred branch
|
||||
# (`ss -tlnp`), which is also the branch that reports the
|
||||
# owning PID, so nothing further is needed: net-tools is
|
||||
# deliberately NOT added (`netstat` is deprecated and only a
|
||||
# fallback branch) and neither is lsof (~500 KB for a third
|
||||
# path to the same answer). ~5.5 MB total: iproute2 itself is
|
||||
# 4.2 MB and pulls 6 libs under --no-install-recommends
|
||||
# (libbpf1, libmnl0, libtirpc-common, libtirpc3t64,
|
||||
# libxtables12, libcap2-bin — libpam-cap is a Recommends and
|
||||
# is correctly dropped). Verified end-to-end in a live
|
||||
# container: `ss` lands at /usr/bin/ss, `ip` at /usr/sbin/ip
|
||||
# (both already on the developer PATH), and `portcheck --all`
|
||||
# then correctly identifies the socat listener on 8765.
|
||||
RUN apt-get update && \
|
||||
apt-get upgrade -y --no-install-recommends && \
|
||||
apt-get install -y --no-install-recommends \
|
||||
@@ -122,6 +140,7 @@ RUN apt-get update && \
|
||||
nano \
|
||||
kitty-terminfo \
|
||||
ncurses-term \
|
||||
iproute2 \
|
||||
&& ln -s /usr/bin/fdfind /usr/local/bin/fd \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
+46
-6
@@ -57,6 +57,32 @@ ARG USER_NAME=developer
|
||||
# v0.74.0..v0.75.5; discovered + fixed in v0.75.5b, 2026-05-23). The `latest`
|
||||
# branch below is kept only for a deliberate local `docker build` override.
|
||||
#
|
||||
# AUDITED AT 0.84.4 (2026-08-31, was 0.84.3): NO "Breaking Changes" and no
|
||||
# "Removed" heading in the 0.84.4 section (grepped, 0 matches) — unlike 0.84.3,
|
||||
# whose heading is described in the paragraph below and stays audited. Adopted
|
||||
# for three fixes that land on machinery this fleet actually runs:
|
||||
# - #6879 large tool results crossing the auto-compaction threshold were sent
|
||||
# to the provider BEFORE compacting; pi now compacts between tool execution
|
||||
# and the next assistant response in the same run. This is the shape of
|
||||
# nearly every session here (multi-hundred-KB logstream/palace tool output).
|
||||
# - #8345 a resumed session corrupted its next appended entry when the JSONL
|
||||
# lacked a trailing newline. That file is the memory feeder's own input.
|
||||
# Measured on tor-ms22 before the bump: 49/49 transcripts end in a newline,
|
||||
# 0 lines fail json.loads — the bug had not bitten this corpus.
|
||||
# - #8537 extension messages sent with `triggerTurn: false` WHILE THE AGENT IS
|
||||
# RUNNING were inserted between a tool call and its result, so
|
||||
# order-validating providers rejected the replayed history. The mempalace
|
||||
# mailbox is outside that precondition — it delivers at `agent_settled`
|
||||
# (idle) with `{deliverAs:"steer"}` and deliberately no `triggerTurn` — and
|
||||
# 0.84.4 leaves the documented steer semantics unchanged, so RFC 003 §7.11
|
||||
# still holds. Recorded because the fix is what would make a future mid-run
|
||||
# delivery safe, which is the only reason we would ever change that call.
|
||||
# One doc consequence, fixed in this same release: pi's own docs/compaction.md
|
||||
# gained exactly one paragraph — the autoCompact threshold is now ALSO checked
|
||||
# mid-run, after a tool batch's results are appended. See
|
||||
# docs/observational-memory.md §3, which had said compaction is only checked
|
||||
# when pi goes idle.
|
||||
#
|
||||
# AUDITED AT 0.84.3 (2026-08-25, was 0.84.2): upstream's notes carry a
|
||||
# "Breaking Changes" heading — `GoogleThinkingLevel` renamed to
|
||||
# `GoogleApiThinkingLevel`. INERT FOR THIS IMAGE: all four vendored companions
|
||||
@@ -69,9 +95,7 @@ ARG USER_NAME=developer
|
||||
# `.agents/skills/<group>/` directories were not discovered, and root Markdown
|
||||
# files such as README.md / AGENTS.md inside a skill dir were reported as
|
||||
# broken skills unless they declared valid skill frontmatter.
|
||||
# pi-atelier needs no companion bump: v0.8.2 clears the >=0.7.1 floor that
|
||||
# pi >= 0.84 requires (see PI_ATELIER_REF below).
|
||||
ARG PI_VERSION=0.84.3
|
||||
ARG PI_VERSION=0.84.4
|
||||
ARG PI_TOOLKIT_REF=main
|
||||
ARG PI_EXTENSIONS_REF=main
|
||||
# Repo URLs default to the canonical gitea origin but are overridable so a
|
||||
@@ -101,15 +125,31 @@ ARG PI_OBSMEM_REF=master
|
||||
# pin and PI_VERSION together, checking atelier's CHANGELOG for the pi
|
||||
# version it claims to track.
|
||||
#
|
||||
# AUDITED AT v0.10.0 (2026-08-31, was v0.8.2 — two minor releases): no
|
||||
# BREAKING notice in either release, and both are UI-only (Sidebar calm during
|
||||
# an active Turn, composer frame + Status Rail, fullscreen-copy-safe Sidebar,
|
||||
# Windows path normalisation, Workspace Pulse deferred until pi trusts the
|
||||
# project). The one coupling that matters runs the OPPOSITE way to the floor
|
||||
# above: v0.9.0 renders the Sidebar as a separate split-layout child and
|
||||
# therefore "raises the minimum supported Pi version to 0.84.0", which its
|
||||
# peerDependencies do encode this time (`>=0.84.0`, up from `>=0.80.7`).
|
||||
# Satisfied with room to spare by PI_VERSION 0.84.4 above — and note that both
|
||||
# executable floors (scripts/smoke-test.sh, scripts/recreate-sanity-check.sh)
|
||||
# compare with `sort -V`, so 0.10.0 >= 0.7.1 is evaluated correctly rather than
|
||||
# as the string comparison that would read 0.10.0 as older than 0.7.1.
|
||||
# Pairs deliberately with pi 0.84.4's own fullscreen selection-copy controls:
|
||||
# atelier keeps Sidebar content out of the transcript selection, pi adds
|
||||
# `fullscreenCopyOnSelect` + Ctrl+X for the selection itself.
|
||||
#
|
||||
# No `npm install` step, unlike pi-fork/pi-observational-memory/pi-studio:
|
||||
# pi-atelier declares ZERO runtime dependencies (only peerDeps, satisfied by
|
||||
# the baked pi) and has no build step — pi loads its TypeScript directly from
|
||||
# the /opt checkout. Adding an install here would be a no-op that only costs
|
||||
# build time.
|
||||
ARG PI_ATELIER_REPO=https://github.com/michaelmjhhhh/pi-atelier.git
|
||||
ARG PI_ATELIER_REF=v0.8.2
|
||||
ARG PI_ATELIER_REF=v0.10.0
|
||||
# Human-readable tag PI_ATELIER_REF was resolved from; recorded as a label.
|
||||
ARG PI_ATELIER_VERSION=v0.8.2
|
||||
ARG PI_ATELIER_VERSION=v0.10.0
|
||||
|
||||
RUN set -e && \
|
||||
# git_fetch_ref: clone-equivalent helper that accepts EITHER a branch name
|
||||
@@ -305,7 +345,7 @@ ARG MEMPALACE_TOOLKIT_REF=main
|
||||
# no ~67-minute base rebuild. (scripts/check-base-hash.sh scans only
|
||||
# Dockerfile.base, so no folding into the base hash is required — nor would
|
||||
# it be correct, since this ARG changes nothing about the base's contents.)
|
||||
ARG SKILLSET_SNAPSHOT_REF=6eb20af181f0147cb8c1377f6e36a6a47a68e8e5
|
||||
ARG SKILLSET_SNAPSHOT_REF=a12fe5ecc71e60feb24791e3e33571105f1afba7
|
||||
|
||||
# Dockerfile.base sets description="pi-devbox — base image (variant-independent)"
|
||||
# and every variant INHERITS it, so both published images used to advertise
|
||||
|
||||
@@ -538,6 +538,35 @@ to refresh.
|
||||
Anything not on a volume is on the writable layer and is lost on
|
||||
container recreate.
|
||||
|
||||
### Rebuilding ephemeral shell state at start
|
||||
|
||||
Two entrypoint steps put back the kind of state that the writable layer eats, so a
|
||||
recreate does not cost you a manual re-install:
|
||||
|
||||
- **`cli_utils` commands.** If a `cli_utils` checkout is mounted, every
|
||||
executable in its `bin/` is symlinked into `~/.local/bin` on start, so
|
||||
`git-status-all` and friends are on `PATH` without a path prefix. Detection:
|
||||
`CLI_UTILS_CONTAINER_PATH` → `/workspace/cli_utils` → `$HOME/cli_utils` →
|
||||
`/workspace/*/cli_utils`. Set `CLI_UTILS_LINK=0` to disable. Existing real files
|
||||
in `~/.local/bin` and symlinks pointing elsewhere are left alone, so a
|
||||
deliberate override still wins; links whose target disappeared are pruned.
|
||||
Do **not** run a host installer's `install.sh` inside the container to achieve
|
||||
this — it writes to the ephemeral home and dies on the next recreate.
|
||||
- **A per-device boot hook.** If `~/.config/devbox-shell/init.sh` exists it is run
|
||||
once at start (`bash`, never sourced, exit status ignored), with output in
|
||||
`~/.pi/agent/devbox-init.log`. `~/.config/devbox-shell/` is the host-owned
|
||||
bind-mount whose `bash_aliases` is already sourced into every interactive shell,
|
||||
so a hook there persists across recreates with no image change. Use it for
|
||||
fixups that must exist *before any shell* — symlinks, directories, one-off
|
||||
migrations.
|
||||
|
||||
The distinction that decides which mechanism you want: `~/.local/bin` is on `ENV
|
||||
PATH`, so symlinks there work in **non-interactive** shells too (`docker exec <c>
|
||||
<cmd>`, agent tool shells, scripts). A `PATH` edit in `bash_aliases` reaches only
|
||||
*interactive* shells, because `~/.bashrc` returns early when non-interactive —
|
||||
which is also why shell **functions** (fzf helpers and the like) can only come
|
||||
from the sourced file, never from a symlink.
|
||||
|
||||
## MemPalace integration
|
||||
|
||||
MemPalace is installed in the base image and pre-warmed with the
|
||||
@@ -1064,7 +1093,7 @@ persisted volumes survived, and pi runtime wiring is intact:
|
||||
```bash
|
||||
./scripts/recreate-sanity-check.sh # auto-detects variant
|
||||
./scripts/recreate-sanity-check.sh --expected-image-version 1.8.9 # assert the pi-devbox release tag
|
||||
./scripts/recreate-sanity-check.sh --expected-version 0.84.3 # assert the pi coding agent version
|
||||
./scripts/recreate-sanity-check.sh --expected-version 0.84.4 # assert the pi coding agent version
|
||||
```
|
||||
|
||||
Those are **two different versions**, and the flags are not interchangeable:
|
||||
@@ -1103,9 +1132,9 @@ resolved to `latest` at build time:
|
||||
|
||||
| Component | Pin | Where |
|
||||
|---|---|---|
|
||||
| pi | `0.84.2` | `ARG PI_VERSION` — `Dockerfile.variant` |
|
||||
| pi-atelier | `v0.8.2` | `ARG PI_ATELIER_REF` — `Dockerfile.variant` |
|
||||
| mempalace | `3.7.1` | `ARG MEMPALACE_VERSION` — `Dockerfile.base` |
|
||||
| pi | `0.84.4` | `ARG PI_VERSION` — `Dockerfile.variant` |
|
||||
| pi-atelier | `v0.10.0` | `ARG PI_ATELIER_REF` — `Dockerfile.variant` |
|
||||
| mempalace | `3.8.0` | `ARG MEMPALACE_VERSION` — `Dockerfile.base` |
|
||||
|
||||
The objective is **not** to freeze versions. Bumping is routine — usually one
|
||||
line plus a changelog note. The objective is that adopting a new upstream
|
||||
|
||||
@@ -21,8 +21,9 @@ palace, see
|
||||
> Verified on pi-devbox **v1.8.9** (`release_tag v1.8.9`, source `aac4a1c`),
|
||||
> which bakes pi-observational-memory **v3.0.4** at commit `ce9fc98` — the value
|
||||
> in `/etc/pi-devbox/build-manifest.json` → `components.pi-observational-memory`.
|
||||
> Every number below was read from that tree, from pi 0.84.3's own docs, or from
|
||||
> the live container.
|
||||
> Every number below was read from that tree, from pi's own docs, or from the
|
||||
> live container. The pi-side mechanics were first read at pi **0.84.3** and
|
||||
> re-checked at **0.84.4** (v1.8.12), which moved one of them — see §3.
|
||||
|
||||
---
|
||||
|
||||
@@ -93,6 +94,7 @@ flowchart TD
|
||||
S(["agent_settled"]) --> C{"81k tokens<br/>since compacting?"}
|
||||
C -- yes --> CP["ctx.compact()"]
|
||||
CP --> H(["session_before_compact"])
|
||||
A(["pi autoCompact<br/>idle, or mid-run<br/>after a tool batch"]) --> H
|
||||
H --> F["fold the ledger<br/>no model call"]
|
||||
F --> VIS["compacted memory"]
|
||||
```
|
||||
@@ -105,10 +107,16 @@ flowchart TD
|
||||
a *successful same-turn* reflection **and** an active pool above
|
||||
`observationsPoolTargetTokens` [10000]. Not a third worker on a third
|
||||
threshold.
|
||||
- **compaction** — `compactAfterTokens` [81000], checked when pi goes idle, so it
|
||||
never interrupts a turn. Pi will also compact on its own when the context is
|
||||
nearly full (`contextTokens > contextWindow - reserveTokens`, `reserveTokens`
|
||||
[16384]).
|
||||
- **compaction** — `compactAfterTokens` [81000], checked at `agent_settled`, so
|
||||
*this* trigger never interrupts a turn. Pi will also compact on its own when
|
||||
the context is nearly full (`contextTokens > contextWindow - reserveTokens`,
|
||||
`reserveTokens` [16384]), and **from pi 0.84.4 that check also runs mid-run** —
|
||||
after a tool batch's results are appended, before the next assistant response,
|
||||
skipped only when the batch ends the run and no queued message needs another
|
||||
response. So `session_before_compact` has **two** entry points and the second
|
||||
one can fire *inside* a turn. Harmless for the fold itself, which makes no
|
||||
model call, but worth stating plainly: "never interrupts a turn" was only ever
|
||||
true of the observational-memory trigger, and reads as a promise about pi's.
|
||||
|
||||
## 4. What compaction actually does to your context
|
||||
|
||||
|
||||
@@ -188,6 +188,111 @@ if [ "${MEMPALACE_FEED:-1}" != "0" ] && [ -n "$MEMPALACE_FEEDER" ]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── cli_utils: link workspace bin/ commands onto PATH ────────────────
|
||||
# Standalone commands from a mounted cli_utils checkout (git-status-all,
|
||||
# git-pull-all, devbox-sanity, pi-session-repair, ...) live in <repo>/bin. On a
|
||||
# host they reach PATH via cli_utils' own install.sh, whose install_bin step
|
||||
# symlinks them into ~/.local/bin — but that home is on the container's WRITABLE
|
||||
# LAYER, so every recreate loses them and the human is back to typing
|
||||
# /workspace/cli_utils/bin/git-status-all. This is the container equivalent of
|
||||
# that install step, re-run at every start.
|
||||
#
|
||||
# WHY SYMLINKS RATHER THAN A PATH EDIT IN AN rc FILE: ~/.local/bin is already
|
||||
# ahead of /usr/local/bin in ENV PATH (Dockerfile.base), so links here resolve in
|
||||
# NON-interactive shells too — `docker exec <c> git-status-all`, agent tool
|
||||
# shells, scripts. An rc-file PATH edit cannot reach those, because ~/.bashrc
|
||||
# returns early when the shell is not interactive. Measured 2026-08-27 on
|
||||
# tor-ms22: `command -v git-status-all` failed in a non-interactive shell while
|
||||
# working in an interactive one, from exactly that asymmetry.
|
||||
#
|
||||
# Detection order (first hit wins):
|
||||
# 1. CLI_UTILS_CONTAINER_PATH explicit, for non-standard layouts
|
||||
# 2. /workspace/cli_utils repo directly in the workspace root
|
||||
# 3. $HOME/cli_utils dedicated mount
|
||||
# 4. /workspace/*/cli_utils workspace root holds several repo groups
|
||||
# CLI_UTILS_LINK=0 disables. Absent repo = silent no-op, which is the common
|
||||
# case for anyone who does not use cli_utils.
|
||||
if [ "${CLI_UTILS_LINK:-1}" != "0" ]; then
|
||||
CLI_UTILS_BIN=""
|
||||
if [ -n "${CLI_UTILS_CONTAINER_PATH:-}" ] && [ -d "${CLI_UTILS_CONTAINER_PATH}/bin" ]; then
|
||||
CLI_UTILS_BIN="${CLI_UTILS_CONTAINER_PATH}/bin"
|
||||
elif [ -d /workspace/cli_utils/bin ]; then
|
||||
CLI_UTILS_BIN=/workspace/cli_utils/bin
|
||||
elif [ -d "$HOME/cli_utils/bin" ]; then
|
||||
CLI_UTILS_BIN="$HOME/cli_utils/bin"
|
||||
else
|
||||
# `if` bodies, not `&&` chains: under `set -e` a loop whose LAST command is a
|
||||
# false test exits non-zero and would abort the entrypoint. With no match the
|
||||
# glob stays literal, so that is the normal case on any machine without this
|
||||
# repo — i.e. the bug would have been "container will not start", not "links
|
||||
# missing".
|
||||
for _cu in /workspace/*/cli_utils/bin; do
|
||||
if [ -d "$_cu" ]; then
|
||||
CLI_UTILS_BIN="$_cu"
|
||||
break
|
||||
fi
|
||||
done
|
||||
unset _cu
|
||||
fi
|
||||
|
||||
if [ -n "$CLI_UTILS_BIN" ]; then
|
||||
mkdir -p "$HOME/.local/bin" 2>/dev/null || true
|
||||
# Never clobber a real file, and never steal a link that points elsewhere: a
|
||||
# deliberate user override in ~/.local/bin must win, and silently shadowing
|
||||
# an image-provided command is worse than the missing command.
|
||||
for _f in "$CLI_UTILS_BIN"/*; do
|
||||
if [ ! -f "$_f" ] || [ ! -x "$_f" ]; then
|
||||
continue
|
||||
fi
|
||||
_link="$HOME/.local/bin/$(basename "$_f")"
|
||||
if [ -e "$_link" ] && [ ! -L "$_link" ]; then
|
||||
continue
|
||||
fi
|
||||
if [ -L "$_link" ]; then
|
||||
case "$(readlink "$_link")" in
|
||||
"$CLI_UTILS_BIN"/*) ;;
|
||||
*) continue ;;
|
||||
esac
|
||||
fi
|
||||
ln -sf "$_f" "$_link" 2>/dev/null || true
|
||||
done
|
||||
# Prune links we own whose target vanished (command renamed, repo moved),
|
||||
# mirroring the skillset deploy's --prune-stale. A dangling link on PATH
|
||||
# reports "No such file or directory" for a command that simply no longer
|
||||
# exists, which reads as a broken container rather than a removed script.
|
||||
for _link in "$HOME/.local/bin"/*; do
|
||||
[ -L "$_link" ] || continue
|
||||
case "$(readlink "$_link")" in
|
||||
*/cli_utils/bin/*) [ -e "$_link" ] || rm -f "$_link" ;;
|
||||
esac
|
||||
done
|
||||
unset _f _link
|
||||
fi
|
||||
unset CLI_UTILS_BIN
|
||||
fi
|
||||
|
||||
# ── Per-device boot hook ─────────────────────────────────────────────
|
||||
# Runs ~/.config/devbox-shell/init.sh if the host provides one. That directory is
|
||||
# the host-owned, bind-mounted shell-sharing dir (see "Volumes and persistence"),
|
||||
# so a hook placed there survives every recreate WITHOUT an image change — the
|
||||
# boot-time twin of the interactive bridge in /etc/skel-devbox/.bash_aliases,
|
||||
# which sources ~/.config/devbox-shell/bash_aliases for every interactive shell.
|
||||
#
|
||||
# NO NEW TRUST BOUNDARY: that same directory is already sourced into every
|
||||
# interactive shell, i.e. it is already arbitrary code from the same owner. What
|
||||
# is new is only WHEN it runs — once at start, before any shell — which is what
|
||||
# non-interactive fixups (symlinks, dirs, one-off migrations) need.
|
||||
#
|
||||
# Deliberately `bash <file>`, not `.` — a hook must not be able to mutate this
|
||||
# entrypoint's own shell state, and its exit status must not matter. Output goes
|
||||
# to a log rather than the container's start output, so a chatty hook cannot
|
||||
# masquerade as a startup error.
|
||||
if [ -r "$HOME/.config/devbox-shell/init.sh" ]; then
|
||||
mkdir -p "$HOME/.pi/agent" 2>/dev/null || true
|
||||
bash "$HOME/.config/devbox-shell/init.sh" \
|
||||
>"$HOME/.pi/agent/devbox-init.log" 2>&1 || true
|
||||
fi
|
||||
|
||||
# ── Git config defaults ──────────────────────────────────────────────
|
||||
if [ -n "${GIT_USER_NAME:-}" ] && ! git config --global user.name &>/dev/null; then
|
||||
git config --global user.name "$GIT_USER_NAME"
|
||||
|
||||
@@ -116,6 +116,50 @@ if command -v fzf >/dev/null 2>&1; then
|
||||
eval "$(fzf --bash)" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# cli_utils — shell FUNCTIONS (fgit, fhist, fssh, portcheck, up, mkcd, extract,
|
||||
# agents-sync, …). This is the OTHER HALF of the cli_utils wiring, and until
|
||||
# v1.8.11 the image shipped only one half. entrypoint-user.sh symlinks the repo's
|
||||
# bin/ COMMANDS into ~/.local/bin, which is what makes them resolve in
|
||||
# NON-interactive shells (docker exec, agent tool shells, scripts). A symlink
|
||||
# cannot carry a shell function, and a function cannot be reached from a
|
||||
# non-interactive shell, so the two mechanisms are disjoint and both are
|
||||
# required. Nothing sourced the loader: measured 2026-08-30 on v1.8.11, all 14
|
||||
# functions were simply missing on a device whose $HOME has no zsh rc — which is
|
||||
# the normal case, since the container's interactive shell is bash and zsh is not
|
||||
# installed in the image. The image was already paying this layer's dependency
|
||||
# cost (fzf, bat, fd, rg, jq are all baked partly FOR these functions) while
|
||||
# delivering none of its benefit.
|
||||
#
|
||||
# Detection order deliberately mirrors the symlink block in entrypoint-user.sh so
|
||||
# that commands and functions can never come from two different clones.
|
||||
# CLI_UTILS_SOURCE=0 opts out. That is independent of CLI_UTILS_LINK=0 on purpose:
|
||||
# they disable independent mechanisms, and someone who wants PATH commands
|
||||
# without 14 extra functions in every prompt (or vice versa) should be able to
|
||||
# say so.
|
||||
#
|
||||
# THE LOADER IS BASH-SAFE, MEASURED, NOT ASSUMED: despite every function file
|
||||
# being named *.zsh, sourcing cli_utils.sh under `bash --noprofile --norc` exits
|
||||
# 0 with no errors and defines all 14, and they run (pathls, mkcd, up, extract,
|
||||
# agents-sync, fhist all verified). The single zsh-only construct in the tree
|
||||
# (`print -z` in fzf/fhist.zsh) is already guarded by [[ -n $ZSH_VERSION ]] with
|
||||
# a bash fallback, and the loader's own header states "bash & zsh compatible".
|
||||
# ACCEPTED RISK, stated plainly: /workspace/cli_utils is a HOST BIND MOUNT, so
|
||||
# unlike a pinned git ref this content floats outside the image's control. A
|
||||
# future cli_utils commit that adds a genuinely zsh-only file would surface as
|
||||
# parse errors at every prompt on every device. Errors are left VISIBLE rather
|
||||
# than sent to /dev/null so that failure is diagnosable instead of mysterious,
|
||||
# and CLI_UTILS_SOURCE=0 is the documented one-line escape hatch.
|
||||
if [ "${CLI_UTILS_SOURCE:-1}" != "0" ]; then
|
||||
for _cu in "${CLI_UTILS_CONTAINER_PATH:-}" /workspace/cli_utils "$HOME/cli_utils" /workspace/*/cli_utils; do
|
||||
[ -n "$_cu" ] || continue
|
||||
if [ -r "$_cu/cli_utils.sh" ]; then
|
||||
. "$_cu/cli_utils.sh" || true
|
||||
break
|
||||
fi
|
||||
done
|
||||
unset _cu
|
||||
fi
|
||||
|
||||
# ── PROMPT_COMMAND: flush history every prompt ───────────────────────
|
||||
# Installed AFTER zoxide init so zoxide's hook is already in place;
|
||||
# we append with a newline separator to avoid the ';;' parse error
|
||||
|
||||
@@ -70,3 +70,41 @@ rather than merely confusing you:
|
||||
local disk, so `mempalace search` can return older and different results than
|
||||
the MCP tools while both look correct. Use the MCP tools for the central
|
||||
palace; the CLI only for a local one.
|
||||
|
||||
## Before you file a finding: second measurement, different route
|
||||
|
||||
This is here rather than in a skill because it has to fire *without* a matching
|
||||
task description, and because the version of it that lived only in a skill was
|
||||
violated five times in one session by an agent that had the skill available.
|
||||
|
||||
**Any claim you are about to record as fact — in a drawer, a diary entry, a
|
||||
coordination event, or a report to the user — needs a second measurement taken
|
||||
by a different route.** Not a re-read of your reasoning: re-reading has caught
|
||||
zero of these. A disagreeing measurement has caught all of them.
|
||||
|
||||
The two shapes that get filed as fact and are not:
|
||||
|
||||
- **A negative result** (`401`, connection refused, zero rows, "not found") is
|
||||
first a claim about *your filter*, not about the world. Wrong host, wrong port,
|
||||
wrong table, capped output.
|
||||
- **A positive result** proves only what your command *actually asked*. An SSH
|
||||
handshake can succeed against the wrong host (`ssh -G` tells you which rule
|
||||
captured the name); a `401` can be a real answer from an issuer that never
|
||||
minted the credential.
|
||||
|
||||
Cheapest habit that works: **write the expected result next to each check before
|
||||
running it**, then diff. Expectations declared up front turn a silent wrong
|
||||
assumption into a visible mismatch. And if you cannot think of a second route to
|
||||
the same fact, you do not have a finding — you have a hypothesis, so label it as
|
||||
one.
|
||||
|
||||
## Handling an exposed credential
|
||||
|
||||
If a task touches a leaked secret, a token rotation, "is this credential still
|
||||
live?", whether to delete stored content, or which scopes a new token needs:
|
||||
**read `~/.agents/skills/credential-incident-response/SKILL.md` first.** One rule
|
||||
is load-bearing enough to state here: **probe the issuing provider before doing
|
||||
anything else** — most "exposed" credentials in a long-lived fleet are already
|
||||
dead, and the ones that are live are often far more privileged than assumed.
|
||||
Severity first, cleanup second, and prefer **revocation over deletion** for
|
||||
anything already replicated.
|
||||
|
||||
@@ -9,6 +9,7 @@ one", which was a bug).
|
||||
| skill | owner | how it gets here |
|
||||
|-------|-------|------------------|
|
||||
| `pi-devbox-environment` | pi-devbox (this repo) | authored here; the canonical copy |
|
||||
| `credential-incident-response` | pi-devbox (this repo) | authored here; the canonical copy |
|
||||
| `pi-extensions` | the `pi-extensions` package repo (`skill/`) | **vendored fallback** + refreshed at build |
|
||||
| `mempalace` | the `skillset` repo | **vendored fallback** (snapshot only) |
|
||||
|
||||
|
||||
@@ -0,0 +1,255 @@
|
||||
---
|
||||
name: credential-incident-response
|
||||
description: >-
|
||||
Respond correctly when a live credential is found where it should not be —
|
||||
in a chat transcript, a MemPalace drawer, a log, a git-tracked config, or an
|
||||
agent-authored note. Load this whenever a task involves a leaked/exposed
|
||||
secret, a token rotation, a "is this credential still live?" question, deciding
|
||||
whether to delete or scrub stored content, proving a corpus is clean, or
|
||||
choosing scopes for a new API token. Covers the mandatory order of operations
|
||||
(probe the issuer FIRST — severity before cleanliness), leak-free identity via
|
||||
sha256[:8] fingerprints and when publishing one is safe,
|
||||
why revocation beats deletion for anything already replicated, scopes derived
|
||||
from measured consumers, the three places a secret hides in a Chroma palace, how to prove ABSENCE rather than assume it (instrument strength,
|
||||
census vs class passes, the tokenisation trap where quoting decides detectability, why git filters never run on symlinks, self-tests that abort),
|
||||
where this fleet's secrets live, and what rotation does NOT fix.
|
||||
---
|
||||
|
||||
# Credential incident response
|
||||
|
||||
A leaked credential is a **severity** question before it is a cleanliness
|
||||
question. Two days of scrubbing, redaction plumbing and deletion planning were
|
||||
once spent on a set of 13 credentials of which **11 were already dead at the
|
||||
provider** — a fact that cost five HTTP requests to establish and was never
|
||||
checked. Meanwhile the two live ones turned out to be instance-owner **admin**
|
||||
tokens, which nobody had looked at either.
|
||||
|
||||
## 1. Order of operations — do not reorder this
|
||||
|
||||
1. **Is it still accepted?** Probe the issuing provider. Dead credential →
|
||||
hygiene item, stop panicking. Live → incident, continue.
|
||||
2. **What can it do?** Read the identity back. `is_admin`, `id=1`, scopes,
|
||||
which account. A read-only repo token and an instance-owner admin token are
|
||||
not the same finding.
|
||||
3. **What consumes it?** Grep for real consumers before assuming breakage.
|
||||
4. **Where does it live?** Enumerate copies (store, palace, transcripts, git).
|
||||
5. **Then** rotate/revoke, and only then consider cleanup.
|
||||
|
||||
Doing 4→3→1 in reverse produces confident, wrong severity calls and wasted
|
||||
cleanup. If you only have time for one step, do step 1.
|
||||
|
||||
## 2. Leak-free identity: fingerprint, never the value
|
||||
|
||||
Publishing an 8-hex fingerprint lets you compare a credential across machines,
|
||||
files, drawers and peers without ever materialising the secret. Same formula as
|
||||
`mempalace_redact.py`:
|
||||
|
||||
```sh
|
||||
printf '%s' "$SECRET" | sha256sum | cut -c1-8 # printf, NOT echo (no newline)
|
||||
printf '%s' 'test' | sha256sum | cut -c1-8 # self-test -> 9f86d081
|
||||
```
|
||||
|
||||
Report as `(variable, fp, length)`. Equal fingerprints across hosts prove a
|
||||
shared credential; that is usually the important part. **Never** paste a live
|
||||
value into a search query, a palace drawer, an event body, or a chat message —
|
||||
in an agent context your own tool output is itself captured and re-filed.
|
||||
|
||||
**Precondition — only fingerprint what an adversary cannot enumerate.** An 8-hex
|
||||
fingerprint is 32 bits over its *input space*, so publishing `fp8(x)` hands
|
||||
anyone a **membership oracle**: they can test `x == v` for every candidate `v`
|
||||
they can generate. For a 40-char random token that space is unreachable. For a
|
||||
hostname, username, e-mail, port, path, commit SHA or weak password it is a
|
||||
wordlist. **If you can imagine writing the wordlist, you cannot publish the
|
||||
fingerprint** — reference those by name and location instead. "High entropy" is
|
||||
the usual *sufficient condition*, not the test: a commit SHA is 160-bit and still
|
||||
fully enumerable from the repo. `sha256("")` = `e3b0c442` is the degenerate case,
|
||||
recognisable on sight precisely because its input space has one member.
|
||||
|
||||
**Candidate fingerprints are working memory, never output.** A scanner that hashes
|
||||
every token in a file also hashes hostnames, paths and e-mails. Print only
|
||||
fingerprints that *matched* a known entry — the tempting debug step when a scan
|
||||
returns zero ("print what it saw") publishes low-entropy fingerprints wholesale.
|
||||
|
||||
And say plainly what a fingerprint register *is*, so nobody rediscovers it later
|
||||
as an alarm: even for an unguessable secret, a published fingerprint is a
|
||||
**confirmation oracle** for anyone who already holds a candidate corpus. That is
|
||||
exactly how a long-retired token gets identified in old transcripts — and it works
|
||||
identically for someone else holding those same files. Net positive, since they
|
||||
would already hold the value; state it rather than leaving it implicit.
|
||||
|
||||
## 3. Liveness probes, and the trap that scoping creates
|
||||
|
||||
```sh
|
||||
# Gitea
|
||||
curl -sS -m 10 -o /dev/null -w '%{http_code}\n' -H "Authorization: token $T" \
|
||||
"$GITEA_HOST/api/v1/repos/<owner>/<repo>/actions/runs?limit=1"
|
||||
# GitHub
|
||||
curl -sS -m 10 -o /dev/null -w '%{http_code}\n' -H "Authorization: token $T" \
|
||||
https://api.github.com/user
|
||||
```
|
||||
|
||||
- `200` live · `401` revoked/invalid · **`403` = wrong question, not a dead token**
|
||||
- **Probe the issuer that minted it.** A 401 from an unrelated instance says
|
||||
nothing. Resolve the host from config (`GITEA_EGL_HOST` etc.), do not assume.
|
||||
- **Under scoped tokens, `/api/v1/user` returns 403 for a perfectly live token**
|
||||
unless `user` scope was granted. So it cannot distinguish *revoked* from
|
||||
*merely scoped*. Use a **repository route the token is authorised for**.
|
||||
- Verify **both directions** after a rotation: old → 401, new → 200. The second
|
||||
check is what catches "deleted the wrong token".
|
||||
- Port/scheme come from config, not habit: one instance here is
|
||||
`http://gitea.egl.lan:3000` — plain HTTP, with 443 refused.
|
||||
|
||||
## 4. Revocation beats deletion — the load-bearing rule
|
||||
|
||||
Once revoked, stored copies are **inert**; you may leave them. Deleting them is
|
||||
best-effort over an *unbounded* copy set: FTS shadow rows, feed inbox `.jsonl`
|
||||
files on every host, sqlite free pages after the delete, mesh replicas that
|
||||
already synced, and backups. **Revocation invalidates every copy everywhere at
|
||||
once, including copies nobody enumerated.**
|
||||
|
||||
So: **rotate + revoke first.** Treat drawer deletion as optional hygiene, never
|
||||
as the remedy. Then record the retired fingerprints as *known-dead* so the next
|
||||
census recognises them instead of reopening the investigation.
|
||||
|
||||
Corollary: never reach for `mempalace_sync` or a bulk `delete_by_source` on a
|
||||
shared palace as incident response. High blast radius, low actual benefit.
|
||||
|
||||
## 5. Finding a secret in a Chroma palace — three targets, in this order
|
||||
|
||||
1. `embedding_fulltext_search_content.c0` — **where document text actually is**
|
||||
2. `embedding_metadata.string_value` — metadata fields only
|
||||
3. raw byte scan of every `*.sqlite3` — backstop, covers FTS pages and free space
|
||||
|
||||
Scanning only (2) is the classic false clean: hundreds of thousands of rows,
|
||||
zero hits, and the secret sitting in (1) the whole time. Semantic search proves
|
||||
nothing about absence — it returns top-k. For completeness, enumerate by filing
|
||||
window (`list_drawers(since=T, before=T+1m)`), since one mine shares a minute.
|
||||
|
||||
Value-agnostic sweeps (uuid / 40-hex / `NAME=VALUE`) drown in false positives at
|
||||
fleet scale — 608 candidates, mostly session UUIDs and git SHAs. Name-anchoring
|
||||
plus entropy plus provenance, applied to **document text**, is what works.
|
||||
|
||||
## 6. Proving absence: instrument strength, and four ways a scan lies clean
|
||||
|
||||
Section 5's warning is about false *positives* — name-anchoring and provenance are
|
||||
what stop a triage sweep drowning in session UUIDs. **A gate is the opposite job.**
|
||||
Triage optimises precision; proving absence optimises recall. Every failure below
|
||||
reported a reassuring zero over a secret that was really there.
|
||||
|
||||
**Rank the instrument, and state which one produced your zero.**
|
||||
|
||||
| Instrument | Needs | Blind to |
|
||||
|---|---|---|
|
||||
| exact-byte value search | you hold the value | nothing — no tokeniser to fool |
|
||||
| class/structure pass | a header pattern | anything without a recognisable shape |
|
||||
| fingerprint census | a fingerprint list | any secret not listed; tokenisation |
|
||||
|
||||
A census is deliberately value-free, so it must *extract candidates and hash them*
|
||||
— which makes its sensitivity a property of the tokeniser, not of the corpus. If
|
||||
you hold the value, search the bytes instead, and search the value's JSON-escaped
|
||||
rendering too when the corpus is `.jsonl`.
|
||||
|
||||
**1. Census and class answer different questions; neither substitutes.** A census
|
||||
answers *"has a KNOWN secret leaked?"*, a class pass *"is there secret-SHAPED
|
||||
material here?"* Both failure modes were measured on this fleet: a class-only
|
||||
pre-commit hook passed plaintext UUID API credentials to a shared repo twice,
|
||||
because a UUID carries no key header — while a census-only gate reported 0 hits
|
||||
with freshly-synced SSH private keys and an age identity in the tree, because no
|
||||
key is in the census. Run both passes.
|
||||
|
||||
**2. Tokenisation — quoting alone can decide detectability.** Maximal-run
|
||||
extraction swallows the value of an *unquoted* assignment:
|
||||
|
||||
```
|
||||
PROXMOX_SECRET=<uuid> # ONE run; the uuid is never hashed alone -> MISS
|
||||
export SECRET="<uuid>" # the quote ends the run; bare uuid hashed -> HIT
|
||||
```
|
||||
|
||||
Take the **union** of three strategies, because each fails in a different
|
||||
direction — (2) is the one that recovers the unquoted case:
|
||||
|
||||
~~~python
|
||||
runs = re.findall(r'[^\s"\'`]{12,}', text) # 1. maximal runs
|
||||
split = [p for r in runs for p in re.split(r'[=!,;:@|()\[\]{}<>]', r) if len(p) >= 12]
|
||||
shape = re.findall(UUID_RE, text) + re.findall(r'[0-9a-f]{32,64}', text)
|
||||
candidates = set(runs) | set(split) | set(shape)
|
||||
~~~
|
||||
|
||||
**3. Scan the index or the pushed tree, never the working tree.** The working tree
|
||||
is not what gets published. And for an rsync-published mirror a repo-only fix is
|
||||
not weaker, it is *temporary*: the next sync re-publishes the live disk. Fix the
|
||||
live file first, verify it clean **by fingerprint**, then sync. Read blobs with
|
||||
`git ls-tree -r <sha>` plus one `git cat-file --batch` (thousands of `git show`
|
||||
calls is the slow way).
|
||||
|
||||
**4. Git filters never run on symlinks — and `check-attr` will not tell you.** A
|
||||
symlink's blob is the *target path*, so `filter=git-crypt` can never encrypt it,
|
||||
yet `git check-attr filter` cheerfully answers `git-crypt` for that path. **A
|
||||
symlinked secret stays plaintext no matter what `.gitattributes` says.** Join the
|
||||
attribute against the **file mode** (`git ls-files -s`, mode `120000`) and verify
|
||||
the index blob really begins `\0GITCRYPT\0`. Report encrypted / symlinked /
|
||||
scanned as three separate numbers and assert they sum — encrypted and symlinked
|
||||
blobs are *skipped*, not certified clean.
|
||||
|
||||
**Self-test two-sided, and abort if it cannot discriminate.** Require a synthetic
|
||||
positive to fire AND a negative to stay silent before trusting any zero. Keep the
|
||||
fixtures in *structurally separate buffers*: put a quoted and an unquoted probe in
|
||||
one buffer and the quote terminates the run, handing the bare token to the weak
|
||||
extractor and making it look as strong as the union — a self-test artifact that
|
||||
has already fooled an agent here. And never gate on `$?` when the tool has a
|
||||
lock-skip or no-op path that also exits 0; judge the reported line.
|
||||
|
||||
**Row-gone is not bytes-gone.** A correct sqlite `DELETE` leaves the payload in
|
||||
freelist pages until `VACUUM`, so deletion effectiveness is *two* numbers: rows
|
||||
removed, and a raw byte scan of the `.sqlite3`. One aggregate figure reported as
|
||||
"erased" has only measured "unretrievable".
|
||||
|
||||
## 7. Choosing scopes: derive them from measured consumers
|
||||
|
||||
Before creating a replacement token, find out what actually uses it:
|
||||
|
||||
```sh
|
||||
git -C <repo> remote get-url origin # ssh:// ? then git needs NO token
|
||||
git config --global --list | grep -iE 'credential|insteadof' # and no helper?
|
||||
grep -rhoE 'api/v1/[A-Za-z0-9/{}$_.-]+' <consumers> | sort -u # exact routes
|
||||
grep -rhoE '\-X [A-Z]+' <consumers> # any writes?
|
||||
```
|
||||
|
||||
Real outcome here: git used SSH keys throughout, and the token's only consumer
|
||||
read three CI-run routes with `GET`. So `repository: Read` and nothing else
|
||||
replaced two admin tokens. **Scoping shrinks the blast radius of the next leak
|
||||
far more than any redaction pipeline does** — a read-only token in a transcript
|
||||
is a hygiene event, not an instance compromise.
|
||||
|
||||
Then prove the scope with an acceptance suite that declares expectations first:
|
||||
must-work routes → `200`; `/admin/*`, `/user`, `/user/repos` → `403`.
|
||||
|
||||
## 8. What rotation does *not* fix
|
||||
|
||||
- **A cleartext channel.** If the endpoint is `http://`, the *new* token is
|
||||
exposed identically from first use. Raise TLS separately.
|
||||
- **Git history.** A secret committed and pushed cannot be fixed by any store or
|
||||
palace operation — it needs rotation *and* history surgery.
|
||||
- **Agent-authored content.** Stage-write redactors see transcripts only, never
|
||||
`add_drawer` / `checkpoint` / `diary_write` output. Never type a secret into
|
||||
the palace yourself; nothing downstream will catch it.
|
||||
- **Plaintext/encrypted drift.** Gitignored plaintext `.env` files go stale while
|
||||
`.env.age` moves on, so old values linger on disk (and in backups) long after
|
||||
rotation. They are a common source of "mystery" fingerprints in a census.
|
||||
|
||||
## 9. This fleet's secret store (verify, do not assume)
|
||||
|
||||
- All `*.env.age` live in **one** repo: `joakimp/docker-compose-repo`. `myconfigs`
|
||||
has none.
|
||||
- Every `.age` file has **one X25519 recipient** — a single key tracked in
|
||||
`myconfigs` under git-crypt. Unlocking git-crypt therefore decrypts the entire
|
||||
fleet's secrets, including hosts you have no access to. The age layer adds no
|
||||
isolation beyond git-crypt.
|
||||
- Flow: `./fetch-secrets.sh <host>` (decrypt → `.env`) → edit → `./encrypt-secrets.sh <host>`
|
||||
→ commit → push → `docker compose up -d --force-recreate`.
|
||||
- **Always pass the host argument** to `encrypt-secrets.sh`. Bare, it walks the
|
||||
whole tree and re-encrypts every `.env` it finds, re-nonced, including stale
|
||||
ones — silently rolling back other hosts' secrets.
|
||||
- After any re-encrypt, check the header still shows exactly **one X25519
|
||||
recipient**; a hand-rolled `age -r` locks the rest of the fleet out, and the
|
||||
failure only appears on another machine, later.
|
||||
@@ -428,10 +428,56 @@ An obligation you never agreed to is noise, so the sender states it:
|
||||
| `to_agent="*"` (any status) | broadcast FYI | nothing |
|
||||
| any other status (`ready`, `applied`, `blocked`, …) | a statement of fact | nothing |
|
||||
|
||||
**That table says what you *owe*. Delivery is stricter, and the difference bites:
|
||||
the mailbox is an obligation channel, not a news channel.** Mailbox candidates are
|
||||
drawn with `status="open"`, so an event carrying any **terminal** status
|
||||
(`applied`, `superseded`, `failed`, `blocked`) is never a candidate — *whoever it
|
||||
is addressed to*. A `task.reply` written to a named machine to share a finding is
|
||||
delivered to nobody, ever, and neither is any `event_ack`. It sits in the log
|
||||
until somebody reads the log.
|
||||
|
||||
So the most natural inter-machine message — *"here is something you should
|
||||
know"* — is exactly the shape that gets no delivery. Pick deliberately:
|
||||
|
||||
| You want the peer to… | Write |
|
||||
|---|---|
|
||||
| **do something**, and you need it tracked until done | directed `status="open"` ask, with a `correlation_id` |
|
||||
| **know something**, no response needed | terminal-status event **plus a drawer** — the drawer is what actually reaches them, via search |
|
||||
|
||||
What does **not** work is a terminal report plus an expectation of attention.
|
||||
Measured 2026-08-26: a detailed report addressed to `pi@<peer>` with
|
||||
`status="applied"` went unread for two and a half hours until the operator quoted
|
||||
the event id by hand, with the mailbox working correctly the whole time. Full
|
||||
mechanism in the toolkit's `docs/rfc-003-coordination-log.md` §7.12.
|
||||
|
||||
One more timing fact, because it looks like negligence and is not: a delivered
|
||||
ask is queued into the agent's **next turn** (`deliverAs: "steer"`, deliberately
|
||||
no `triggerTurn`), and the poll fires when the agent is *idle*. Between delivery
|
||||
and the next turn no inference runs, so **a human starting a turn is the
|
||||
trigger** (§7.11). An agent that "has not reacted" has usually not been running.
|
||||
|
||||
Ack with `mempalace_event_ack(event_id=…, from_agent="<you>", status=…)`. It
|
||||
**appends a new event** and never mutates the original; the correlation id is
|
||||
copied for you, and `metadata.ack_of` is set to the event you answered.
|
||||
|
||||
**Claiming, and what it does not do.** `status="claimed"` announces that you have
|
||||
picked work up. Nothing requires it — a directed open ask owes "an ack *or* a
|
||||
reply", and finishing the work is a complete answer. Do it anyway when the work is
|
||||
long or the machine is unreliable, because it is the only thing that later
|
||||
distinguishes *nobody started this* from *someone started and their container
|
||||
died mid-task*. Be clear about its limits, both of which follow from candidacy
|
||||
requiring exactly `status="open"`:
|
||||
|
||||
- **It does not notify the requester.** `claimed` is not `open`, so a claim is no
|
||||
more deliverable than a finished report is (see the delivery table above). Its
|
||||
reader is whoever pulls the log.
|
||||
- **It does not quiet your own mailbox.** The ask stays owed until a *terminal*
|
||||
event of yours joins it, so a claimed-then-silent thread keeps resurfacing —
|
||||
correctly.
|
||||
|
||||
Prefer a prompt terminal reply over a claim plus a long silence; claim *in
|
||||
addition*, when the gap between pickup and finish is where a machine might die.
|
||||
|
||||
#### What you actually owe — derive it, do not read it off `status`
|
||||
|
||||
The log is append-only and `status` is written **once**, so it is an honest
|
||||
@@ -502,6 +548,17 @@ Two consequences worth internalising:
|
||||
- **Address the stamped name you actually saw** in a `from_agent` field, e.g.
|
||||
`pi@tor-ms22`. A bare `pi` reaches nobody's mailbox once stamping is live, and
|
||||
older events in the log still carry bare names — do not copy them.
|
||||
- **The rule runs in reverse too: what you put in YOUR OWN `from_agent` decides
|
||||
where every reply to your event goes.** Nothing stops you writing a synthetic
|
||||
or borrowed identity there, and a reply is always addressed back to exactly
|
||||
that string — so if no live session ever runs as it, the reply is stored,
|
||||
searchable, and delivered to no one. Measured cost: a directed ask sent under
|
||||
a synthetic sender got two correct replies, one of them an urgent security
|
||||
finding, and both sat unread for ~2h20m because nobody's mailbox was that
|
||||
identity (RFC 003 §7.13). Authoring under a synthetic name is fine for a
|
||||
deliberate control experiment — this fleet does it on purpose — but then
|
||||
**name the real identity to reply to inside the body**, because the address
|
||||
line is not a safe place to also carry provenance.
|
||||
- **Use `status="open"` only when you truly need an answer.** It places an
|
||||
obligation on another machine.
|
||||
- **Never broadcast an ask.** `to_agent="*"` + `status="open"` obliges everyone
|
||||
@@ -513,7 +570,8 @@ Two consequences worth internalising:
|
||||
be matched to it at all.
|
||||
- **Corrections are new events, never edits.** Say explicitly what you retract
|
||||
and name the id — drawer or event — that carried the withdrawn claim.
|
||||
- **Put a retraction where the reader will look.** An event reaches a live agent;
|
||||
- **Put a retraction where the reader will look.** A *directed open ask* reaches a
|
||||
live agent's mailbox; a **terminal-status event reaches no mailbox at all**, and
|
||||
a *drawer* is what a future semantic search finds. If you filed advice as a
|
||||
drawer and later withdraw it, file the withdrawal as a drawer too — otherwise
|
||||
the next agent finds your original confident advice and no trace of the
|
||||
@@ -600,4 +658,5 @@ Entity-relationship triples with temporal validity. Query with `mempalace_kg_que
|
||||
- **Don't treat the palace as a task list.** It's for knowledge and context, not todos.
|
||||
- **Don't broadcast an ask, and don't leave one unanswered.** On a shared palace, `to_agent="*"` + `status="open"` obliges every machine and therefore none of them. And don't expect acking to tidy your mailbox: `status` is immutable, so the event keeps matching either way — what a terminal reply buys you is that the *derived* owed set (see *What you actually owe*) stops counting it. Leave asks unanswered and that set only grows, until everyone learns to stop looking. "Seen, not doing it" is a complete answer — silence is not.
|
||||
- **Don't assume you would have heard.** Nothing pushes another machine's message into your session. If you did not run the mailbox query at wake-up, a correction addressed to you by name can sit unread while you confidently rebuild the thing it warned you about.
|
||||
- **Don't author an ask under an identity nobody runs as, including your own throwaway labels.** The failure is symmetric to the one above: it is not that you missed a message, it is that nothing could ever have delivered the reply to you, because you addressed it at a name instead of an agent. If you must use a synthetic sender for a control or an experiment, say inside the body who should actually receive the reply.
|
||||
- **Don't invent provenance metadata, and don't hand-stamp it either.** An earlier version of this list told you to set `added_by="<harness>@<device>"` by hand; that instruction has been withdrawn, because RFC 001 §7.3.2 places provenance at the client/server boundary and the pi bridge now does it uniformly (see *Provenance is stamped for you* above) — but the withdrawal only holds where the bridge is live, so run the one-line check in that bullet first; on an older image hand-stamping is still the only signal a hand-filed drawer gets. DO NOT invent values for the palace's own metadata fields (`device`, `agent_kind`, `origin_device`): those are stamped by infrastructure that also records *how* each was determined, and a fabricated value is worse than none because it silently corrupts a future merge. DO pass `source_drawer_id` on `kg_add`. And never put a machine name in a diary's `agent_name` — it becomes the wing name and hides your entries from `diary_read`.
|
||||
|
||||
@@ -143,6 +143,10 @@ mine:
|
||||
| "`tor-ms22` is not in the SSH config" | `grep … \| head -20` — the entry was at **line 454**. `~/.ssh/config` here is ~500 lines. |
|
||||
| "the Docker host has no `docker`" | non-interactive SSH `PATH` lacks `/usr/local/bin` (§2, §3). It was at `/usr/local/bin/docker`. |
|
||||
| "no ControlMaster is running" | pattern `ssh ` (trailing space) cannot match a master: those processes **rename themselves** to `ssh: <controlpath> [mux]`. |
|
||||
| "the credential is not in the palace" | scanned `embedding_metadata.string_value` only. Drawer **text** lives in `embedding_fulltext_search_content.c0`; 554k metadata rows proved nothing. |
|
||||
| "this token is dead — 401" | probed it against the **wrong issuer**. A 401 from an instance that never issued the credential is not evidence about the credential. |
|
||||
| "that host is unreachable, can't test" | tried ports 443 and 80. It was on **3000**, and the env var I already held (`GITEA_EGL_HOST`) stated the scheme and port. |
|
||||
| "this repo has no `## Unreleased` convention" | read `CHANGELOG.md` **once**, minutes after a release commit had renamed that section to a version heading. 33 commits touch `## Unreleased`. A snapshot cannot show you a cycle. |
|
||||
|
||||
Habits that would have caught all three:
|
||||
|
||||
@@ -155,10 +159,55 @@ ssh -F "$HOME/.ssh-local/config" mac 'command -v docker || ls /usr/local/bin/doc
|
||||
|
||||
# match a process's ACTUAL argv, not the name you imagine
|
||||
ps -eo pid,etime,args | grep -Ei 'mux|mosh|ssh'
|
||||
|
||||
# to learn a repeating PROCESS or convention, read history, not the file. A
|
||||
# file's current content is one frame of a cycle, and the frame you happen to
|
||||
# catch may be the one where the thing you are looking for was just consumed.
|
||||
git log -S'## Unreleased' -- CHANGELOG.md # not `head -60 CHANGELOG.md`
|
||||
```
|
||||
|
||||
A positive result needs no such scepticism — it carries its own evidence. Only
|
||||
absence has to be *earned*, so spend the extra command there.
|
||||
Absence has to be *earned*, so spend the extra command there.
|
||||
|
||||
### …and a positive result only proves what you *actually asked*
|
||||
|
||||
An earlier version of this section claimed "a positive result needs no such
|
||||
scepticism — it carries its own evidence." **That is false, and believing it
|
||||
cost a later session three more wrong findings.** A positive result is evidence
|
||||
about the question your command really posed, which may not be the question you
|
||||
meant. The failure is invisible precisely *because* the command succeeded.
|
||||
|
||||
| Claim | The command succeeded — at answering something else |
|
||||
|---|---|
|
||||
| "EGL git over SSH works" | `ssh git@gitea.egl.lan` greeted me as `joakimp`. `~/.ssh/config` had `Host gitea*` → `HostName gitea.jordbo.se`, so I authenticated **to the wrong instance**. The real EGL account is `ecsjper`. |
|
||||
| "the port config regressed" | compared `ssh -G` output against `2222` — a value produced by **my own earlier `-p 2222` flag**, not by the config. I reported the user's edit as a regression it never caused. |
|
||||
| "the CI runners authenticate with this token" | pure fabrication, contradicted by my own scan output already on screen. The runners use per-runner `REGISTRATION_TOKEN`. |
|
||||
|
||||
Two habits that actually catch this class, both cheap:
|
||||
|
||||
```sh
|
||||
# 1. ask which RULE captured your hostname before trusting any ssh result.
|
||||
# ssh_config is first-obtained-value-wins PER KEYWORD, not per block: a
|
||||
# specific block only wins the keywords it declares, so a later `Host gitea*`
|
||||
# still supplies HostName unless the specific block restates it.
|
||||
ssh -G git@thehost | grep -E '^(hostname|port|user|identityfile)'
|
||||
|
||||
# 2. state the expected result BEFORE running the check, and diff against it.
|
||||
# This is the single technique that separated the one verification that went
|
||||
# right (10/10, expectations declared per probe) from five that went wrong
|
||||
# (results interpreted after the fact, each time in the direction I expected).
|
||||
probe "/repos/.../actions/runs" 200 # must work
|
||||
probe "/admin/users" 403 # must be denied
|
||||
```
|
||||
|
||||
And the meta-observation, which is the reason this subsection exists: across all
|
||||
five errors, **not one was caught by re-reading my own reasoning.** Every one was
|
||||
caught by a second measurement that disagreed — the SSH lie surfaced only because
|
||||
the greeting said `joakimp` while a token probe minutes earlier had said
|
||||
`ecsjper`; the fabrication surfaced only because the user read my own output back
|
||||
to me. So the operational rule is not "be careful". It is: **for a load-bearing
|
||||
claim, produce a second measurement by a different route, and expect it to
|
||||
disagree.** If you cannot think of a second route, you do not yet have a finding
|
||||
— you have a hypothesis.
|
||||
|
||||
**`dscp`/`scp` with accented filenames on a macOS host.** macOS stores filenames
|
||||
in Unicode **NFD** (decomposed — e.g. `ä` is `a` + combining U+0308), while the
|
||||
|
||||
@@ -245,6 +245,8 @@ run "socat" "socat -V"
|
||||
run "studio-expose helper" "test -x /usr/local/bin/studio-expose"
|
||||
run "image-baked pi-devbox-environment skill" \
|
||||
"test -f /usr/local/share/pi-devbox/skills/pi-devbox-environment/SKILL.md"
|
||||
run "image-baked credential-incident-response skill" \
|
||||
"test -f /usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md"
|
||||
run "global-AGENTS append snippet present" \
|
||||
"test -f /usr/local/share/pi-devbox/pi-global-AGENTS.append.md"
|
||||
run "pi-devbox block merged into pi-global-AGENTS.md" \
|
||||
@@ -596,9 +598,9 @@ exec_test "mempalace skill linked (fallback)" 'test -L $HOME/.agents/skills
|
||||
# bumped correctly but whose bytes came from the wrong place.
|
||||
exec_test "mempalace skill snapshot is current" 'f=$HOME/.agents/skills/mempalace/SKILL.md; grep -q "Provenance is stamped for you" "$f" && ! grep -q "Attribute what you file yourself" "$f" && echo ok'
|
||||
# Link TARGETS, not just link existence: with no skillset mounted (as here) the
|
||||
# baked tree must be what resolves, for all three vendored skills.
|
||||
# baked tree must be what resolves, for all four vendored skills.
|
||||
exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
|
||||
'for s in mempalace pi-extensions pi-devbox-environment; do
|
||||
'for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
|
||||
case "$(readlink -f $HOME/.agents/skills/$s)" in
|
||||
/usr/local/share/pi-devbox/skills/$s) ;;
|
||||
*) echo "$s resolves to $(readlink -f $HOME/.agents/skills/$s)" >&2; exit 1 ;;
|
||||
@@ -612,7 +614,7 @@ exec_test "vendored skills resolve to the baked tree (no skillset mounted)" \
|
||||
exec_test "pi-devbox-version reports skill sources (all baked, no skillset here)" \
|
||||
'out=$(pi-devbox-version)
|
||||
echo "$out" | grep -q "skills:" || { echo "no skills section" >&2; exit 1; }
|
||||
for s in mempalace pi-extensions pi-devbox-environment; do
|
||||
for s in mempalace pi-extensions pi-devbox-environment credential-incident-response; do
|
||||
echo "$out" | grep -qE "^ $s +baked$" \
|
||||
|| { echo "$s not reported as baked" >&2; exit 1; }
|
||||
done; echo ok'
|
||||
|
||||
Reference in New Issue
Block a user