joakimp/mempalace-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(pi): let an ask declare dormancy, so waiting work stops nagging

Browse Source 
A first-boot acceptance ask is planted deliberately unanswerable: it describes
work that becomes possible only when the device is next recreated, and it must
STAY owed until then, because closing it early to tidy the mailbox is exactly how
that work gets lost. deriveOwed could only see "directed, open, not answered, not
withdrawn", so such an ask was announced at every session start and every poll
for as long as it was correctly waiting -- measured at three days running on
emb-7kj4vr4g (2026-09-28 -> 2026-10-01), and across three consecutive releases
before that. The ask was right; announcing it was wrong, and the cost landed on
the human reading the window, who is the one reader that cannot filter it.

An ask may now declare the condition under which it is merely waiting:

  "dormant_unless": [
    { "kind": "json_field", "path": "/etc/pi-devbox/build-manifest.json",
      "field": "release_tag", "baseline": "v1.9.4" },
    { "kind": "file_mtime", "path": "/etc/hostname",
      "baseline": "2026-09-22T18:12:49Z" }
  ]

Dormant while EVERY condition still matches its baseline; live the moment ANY
differs -- which is the trigger those asks already stated in prose ("act when
EITHER differs"), now in a form the bridge can check. Two kinds, local files
only, no expression language, no shell, no network: a general evaluator in the
path that decides whether work is VISIBLE is a far worse trade than a clumsy
schema.

DORMANCY IS PROVEN, NEVER ASSUMED. Every unevaluable predicate announces the ask
instead of hiding it: missing file, unreadable file, unparseable baseline,
unknown kind, vanished field, relative path, more than eight conditions. The
dangerous failure here is not a spurious nag but work that disappears because a
predicate could not be evaluated -- indistinguishable from the ask being lost,
and not surfacing until a release needed it. An ask with no dormant_unless
behaves exactly as before, so this is backward compatible by construction.

Withheld from the ANNOUNCEMENT, never from the mailbox: deriveOwed now returns a
partition {owed, dormant} rather than a flat list, the wake-up injection lists
dormant asks once per session with ids, and a mid-session poll adds only a count
and only when the window is already open for something else. Dormant asks are
deliberately NOT added to the `surfaced` map, so one becomes announceable the
instant its baseline moves.

file_mtime compares WHOLE SECONDS in UTC. A filesystem mtime carries sub-second
residue (measured: /etc/hostname at .773761009) that a reported ISO baseline
never will, so comparing raw milliseconds would mark every such predicate
permanently "changed" -- silently disabling the feature while appearing to work.
The test records the residue for that reason.

scripts/test-dormancy.sh is this repo's first test: 22 cases, positive and
negative arms both, because a predicate that never fires makes the feature inert
and one that fires too eagerly hides real work. It copies the extension into a
temp tree with pi's typebox symlinked beside it (the copy is made per-run, so it
cannot drift like a vendored duplicate), owns its own fixtures rather than
reading /etc paths -- the first draft passed only on a pi-devbox container and
would have silently flipped to "not dormant" anywhere else -- and exits 2 for
INCONCLUSIVE rather than 0, since a test that skips quietly is the failure mode
it exists to catch. shellcheck clean.
This commit is contained in:
Joakim Persson
2026-10-01 23:59:36 +02:00
parent 2167a1b033
commit 975ab92943
3 changed files with 414 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
extensions/pi/README.md
+51
View File
@@ -378,6 +378,57 @@ the one failure this derivation exists to prevent.
protocol says a broadcast owes nobody a reply — which also means broadcasting an
ask demonstrably reaches no owed set, giving that documented anti-pattern teeth.
**Dormant asks — open, owed, and deliberately not announced.** Some asks are
planted *unanswerable on purpose*: a first-boot acceptance describes work that
becomes possible only when the device is next recreated, and it must stay owed
until then, because closing it early to tidy the mailbox is exactly how the work
gets lost. Derivation alone cannot tell that apart from neglected work, so such
an ask was announced at every session start and every poll for as long as it was
correctly waiting — measured at three days running on `emb-7kj4vr4g`
(2026-09-28 → 2026-10-01), and across three consecutive releases before that.
The ask was right; announcing it was wrong, and the cost landed on the human
reading the window, the one reader who cannot filter it.
An ask may therefore declare, in its own `metadata`, the condition under which it
is merely waiting:
```json
"dormant_unless": [
{ "kind": "json_field", "path": "/etc/pi-devbox/build-manifest.json",
"field": "release_tag", "baseline": "v1.9.4" },
{ "kind": "file_mtime", "path": "/etc/hostname",
"baseline": "2026-09-22T18:12:49Z" }
]
```
It is dormant while **every** condition still matches its baseline, and goes live
the moment **any** of them differs — the trigger such asks already stated in
prose ("act when EITHER differs"), now in a form the bridge can check. Two kinds,
both local-file-only: `file_mtime` (compared at whole seconds in UTC, because a
filesystem mtime carries sub-second residue that a reported ISO baseline never
will — comparing raw milliseconds would mark every such predicate permanently
"changed" and silently disable the feature) and `json_field` (dotted paths
allowed, compared as strings so a manifest holding `3` matches a baseline of
`"3"`). There is no expression language, no shell and no network: a general
evaluator in the path that decides whether work is *visible* is a far worse trade
than a clumsy schema.
Dormancy is **withheld from the announcement, never from the mailbox**: the
wake-up injection lists dormant asks once per session with their ids, and a
mid-session poll mentions only a count, and only when the window is already open
for something else. They are not added to the resurface map, so one becomes
announceable the instant its baseline moves.
**Dormancy must be proven, never assumed** — every unevaluable predicate shows
the ask as owed. A missing file, an unreadable one, a baseline that will not
parse, an unknown `kind`, a vanished field, a relative path, more than eight
conditions: each announces. The dangerous failure here is not a spurious nag but
work that disappears because a predicate could not be evaluated, which would be
indistinguishable from the ask being lost and would not surface until a release
needed it. An ask with no `dormant_unless` behaves exactly as it did before the
feature existed. `scripts/test-dormancy.sh` exercises all of it, positive and
negative arms both.
Gated on `MEMPALACE_PI_DEVICE` **and** `MEMPALACE_REMOTE_URL` (the same pair as
the stamper, since an unstamped client has no address to be reached at), and
disabled outright with `MEMPALACE_MAILBOX=0` (notifications alone with
extensions/pi/mempalace.ts
+207 -9
View File
@@ -92,6 +92,7 @@
*/
import { type ChildProcessWithoutNullStreams, spawn } from "node:child_process";
import { readFileSync, statSync } from "node:fs";
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { Type } from "typebox";
@@ -157,6 +158,165 @@ const sleep = (ms: number): Promise<void> =>
if (typeof t.unref === "function") t.unref();
});
// ── Dormancy ──────────────────────────────────────────────────────────────────
/**
* Is a standing ask DORMANT — correct to leave open, wrong to announce?
*
* The problem this solves, measured rather than imagined. A first-boot
* acceptance ask is planted deliberately unanswerable: it describes work that
* becomes possible only when the device is next recreated, and it must STAY
* owed until then, because closing it early to tidy the mailbox is exactly how
* that work gets lost. But deriveOwed could only see "directed, open, not
* answered, not withdrawn", so such an ask is announced at every session start
* and every poll for as long as it is correctly waiting — three days running on
* emb-7kj4vr4g (2026-09-28 → 2026-10-01), and three consecutive releases before
* that. The ask was right; announcing it was wrong. The cost lands on the human
* reading the window, who is the one reader that cannot filter it.
*
* So an ask may now declare, in its own metadata, the condition under which it
* is merely waiting:
*
* "dormant_unless": [
* { "kind": "json_field", "path": "/etc/pi-devbox/build-manifest.json",
* "field": "release_tag", "baseline": "v1.9.4" },
* { "kind": "file_mtime", "path": "/etc/hostname",
* "baseline": "2026-09-22T18:12:49Z" }
* ]
*
* It is dormant while EVERY condition still matches its baseline, and goes live
* the moment ANY of them differs — which is the trigger those asks already
* state in prose ("act when EITHER differs"), now in a form a machine can
* check. Dormant asks are withheld from the ANNOUNCED owed set, never from the
* mailbox: the wake-up injection still lists them once per session, so they
* stay discoverable and cannot be silently dropped.
*
* THE SAFETY RULE, and the reason every branch below reports `unchanged: false`
* on doubt: dormancy must be PROVEN, never assumed. A missing file, an
* unreadable one, a baseline that will not parse, an unknown `kind`, a field
* that has vanished — each is a reason to SHOW the ask, not to hide it. The
* dangerous failure here is not a spurious nag; it is work that vanishes
* because a predicate could not be evaluated, which would be indistinguishable
* from the ask being lost and would not surface until a release needed it.
*
* No eval, no shell, no network: the predicate language is deliberately two
* fixed checks against local files. A general expression evaluator sitting in
* the path that decides whether work is VISIBLE is a far worse trade than a
* slightly clumsy schema.
*/
export type DormancyVerdict = { dormant: boolean; reason: string };
/** Split of the derived owed-set: what to announce, and what is merely waiting. */
export type OwedSplit = { owed: LogEvent[]; dormant: LogEvent[] };
// Bounds, so a malformed or hostile event cannot turn a mailbox read into real
// work. Both are deliberately small: these predicates describe boot state, and
// anything needing more than a handful of local checks is not a dormancy rule.
const MAX_DORMANCY_CONDITIONS = 8;
const MAX_DORMANCY_JSON_BYTES = 256 * 1024;
const dormancyCondition = (c: unknown): { unchanged: boolean; reason: string } => {
if (typeof c !== "object" || c === null || Array.isArray(c))
return { unchanged: false, reason: "condition is not an object" };
const { kind, path, baseline, field } = c as Record<string, unknown>;
// Absolute paths only: a relative one would resolve against whatever cwd the
// pi process happens to hold, which is not a property of the device whose
// state the baseline describes.
if (typeof path !== "string" || !path.startsWith("/") || path.includes("\0"))
return {
unchanged: false,
reason: `path must be an absolute string, got ${JSON.stringify(path)}`,
};
if (typeof baseline !== "string" && typeof baseline !== "number")
return { unchanged: false, reason: "baseline must be a string or a number" };
if (kind === "file_mtime") {
// Compared at WHOLE SECONDS in UTC, because the two routes that produce
// these baselines disagree below that: a filesystem mtime carries
// sub-second residue (measured: /etc/hostname at .773761009) that a
// hand-written or reported ISO baseline never will. Comparing raw
// milliseconds would make every such predicate permanently "changed",
// i.e. would silently disable the feature while appearing to work.
const expected = Date.parse(String(baseline));
if (!Number.isFinite(expected))
return {
unchanged: false,
reason: `baseline is not a parseable date: ${JSON.stringify(baseline)}`,
};
let mtimeMs: number;
try {
mtimeMs = statSync(path).mtimeMs;
} catch {
return { unchanged: false, reason: `cannot stat ${path}` };
}
return Math.floor(mtimeMs / 1000) === Math.floor(expected / 1000)
? { unchanged: true, reason: `${path} mtime still at baseline` }
: { unchanged: false, reason: `${path} mtime moved from baseline` };
}
if (kind === "json_field") {
if (typeof field !== "string" || field.length === 0)
return { unchanged: false, reason: "json_field needs a non-empty field name" };
let text: string;
try {
const buf = readFileSync(path);
if (buf.byteLength > MAX_DORMANCY_JSON_BYTES)
return {
unchanged: false,
reason: `${path} exceeds the ${MAX_DORMANCY_JSON_BYTES}-byte cap`,
};
text = buf.toString("utf8");
} catch {
return { unchanged: false, reason: `cannot read ${path}` };
}
let doc: unknown;
try {
doc = JSON.parse(text);
} catch {
return { unchanged: false, reason: `${path} is not valid JSON` };
}
let cur: unknown = doc;
for (const seg of field.split(".")) {
if (
typeof cur !== "object" ||
cur === null ||
!Object.prototype.hasOwnProperty.call(cur, seg)
)
return { unchanged: false, reason: `${path} has no field ${field}` };
cur = (cur as Record<string, unknown>)[seg];
}
if (cur !== null && typeof cur === "object")
return { unchanged: false, reason: `field ${field} is not a scalar` };
// Compared as strings on purpose: the baseline arrives as JSON metadata, so
// a manifest holding 3 and a baseline saying "3" are the same fact.
return String(cur) === String(baseline)
? { unchanged: true, reason: `${field} still at baseline` }
: { unchanged: false, reason: `${field} moved from baseline` };
}
return { unchanged: false, reason: `unknown dormancy kind ${JSON.stringify(kind)}` };
};
export function evaluateDormancy(
metadata: Record<string, unknown> | null | undefined,
): DormancyVerdict {
const raw = metadata?.dormant_unless;
// Every one of these returns NOT dormant, i.e. "announce it". An ask without
// a predicate behaves exactly as it did before this feature existed.
if (raw === undefined || raw === null) return { dormant: false, reason: "no dormant_unless" };
if (!Array.isArray(raw)) return { dormant: false, reason: "dormant_unless is not an array" };
if (raw.length === 0) return { dormant: false, reason: "dormant_unless is empty" };
if (raw.length > MAX_DORMANCY_CONDITIONS)
return {
dormant: false,
reason: `too many conditions (${raw.length} > ${MAX_DORMANCY_CONDITIONS})`,
};
for (const c of raw) {
const v = dormancyCondition(c);
if (!v.unchanged) return { dormant: false, reason: v.reason };
}
return { dormant: true, reason: `all ${raw.length} condition(s) still at baseline` };
}
class StdioMcpClient implements IMcpClient {
private proc: ChildProcessWithoutNullStreams | null = null;
private nextId = 1;
@@ -1205,8 +1365,8 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
* Directed asks addressed to this device with no terminal reply from it, and
* not explicitly withdrawn by whoever sent them (see isWithdrawn).
*/
async function deriveOwed(): Promise<LogEvent[]> {
if (!mailboxEnabled || !available) return [];
async function deriveOwed(): Promise<OwedSplit> {
if (!mailboxEnabled || !available) return { owed: [], dormant: [] };
try {
const [candidatesRaw, mineRaw, inboundRaw] = await Promise.all([
// Every cursor-less event_list call in this file passes `order` explicitly.
@@ -1254,13 +1414,21 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
// You cannot owe yourself.
return c.from_agent !== mailboxAddress;
});
if (candidates.length === 0) return [];
if (candidates.length === 0) return { owed: [], dormant: [] };
const mine = parseEvents(mineRaw);
const inbound = parseEvents(inboundRaw);
return candidates.filter((c) => !isAnswered(c, mine) && !isWithdrawn(c, inbound));
const live = candidates.filter((c) => !isAnswered(c, mine) && !isWithdrawn(c, inbound));
// PARTITION, not filter. A dormant ask is still owed in the protocol
// sense — it has no terminal reply and must not be closed — so it stays in
// the return value where the mailbox can still account for it, and only
// the ANNOUNCING paths below treat the two halves differently.
const owed: LogEvent[] = [];
const dormant: LogEvent[] = [];
for (const c of live) (evaluateDormancy(c.metadata).dormant ? dormant : owed).push(c);
return { owed, dormant };
} catch {
// Fail silent and open: a mailbox read must never break a session.
return [];
return { owed: [], dormant: [] };
}
}
@@ -1356,6 +1524,15 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
})
.join("\n");
const DORMANT_NOTE =
"Each of these declares a `dormant_unless` predicate in its metadata that STILL " +
"matches this device's current state, so the work it describes is not yet " +
"possible. They are NOT closed and NOT answered — leave them open. They return to " +
"the announced owed-set by themselves the moment a baseline stops matching. If a " +
"predicate cannot be evaluated at all (missing file, unparseable baseline, unknown " +
"kind) the ask is announced as owed instead, deliberately: dormancy has to be " +
"proven, never assumed.";
const OWED_HOWTO =
"To close one, append an event on the SAME correlation_id with a terminal status " +
"(applied/superseded/failed/blocked). An ack alone does NOT clear it, and neither " +
@@ -1463,7 +1640,8 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
lastMailboxPollAt = Date.now();
void (async () => {
try {
const [owed, closed] = await Promise.all([deriveOwed(), deriveClosed()]);
const [owedSplit, closed] = await Promise.all([deriveOwed(), deriveClosed()]);
const { owed, dormant } = owedSplit;
const now = Date.now();
const unseen = (list: LogEvent[]): LogEvent[] =>
list.filter((e) => {
@@ -1491,6 +1669,13 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
`${CLOSED_NOTE}\n\n${formatOwed(newsRaw)}`,
);
}
// A dormant ask NEVER opens this window on its own — the early return
// above is what actually stops the nag. But once the window is open for
// something else, one line of count is nearly free and keeps a waiting
// ask from feeling lost between session starts.
if (dormant.length > 0) {
blocks.push(`${dormant.length} dormant ask(s), not listed here. ${DORMANT_NOTE}`);
}
pi.sendMessage(
{
customType: "mempalace-mailbox",
@@ -1555,10 +1740,11 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
sections.push(`## mempalace_diary_read\n\n(error: ${(err as Error).message})`);
}
// Tier 1 mailbox: what this device owes a reply to, derived rather than read
// off a filter. deriveOwed() swallows its own failures and returns [], so a
// broken palace costs a missing section, never a broken wake-up.
// off a filter. deriveOwed() swallows its own failures and returns an empty
// split, so a broken palace costs a missing section, never a broken wake-up.
if (mailboxEnabled) {
const [owed, closed] = await Promise.all([deriveOwed(), deriveClosed()]);
const [owedSplit, closed] = await Promise.all([deriveOwed(), deriveClosed()]);
const { owed, dormant } = owedSplit;
if (owed.length > 0) {
// Record what this injection showed, so the first mid-session poll does
// not re-announce the identical list minutes later. Without this the
@@ -1572,6 +1758,18 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
`this device. ${OWED_HOWTO}\n\n${formatOwed(owed)}`,
);
}
// Once per session, WITH ids. The wake-up injection is the one place a
// dormant ask should be visible: "what is this device still carrying?" is a
// session-start question, and answering it here is what keeps the mid-session
// silence honest rather than concealing. Deliberately NOT added to
// `surfaced`: that map exists to suppress repeats of ANNOUNCED work, and a
// dormant ask must stay announceable the instant its baseline moves.
if (dormant.length > 0) {
sections.push(
`## logstream mailbox (${dormant.length} dormant — waiting, nothing owed yet)\n\n` +
`${DORMANT_NOTE}\n\n${formatOwed(dormant)}`,
);
}
const news = closed.filter((e) => e.id && !surfaced.has(e.id));
if (news.length > 0) {
const now = Date.now();