From d5a03f4ff50e88707259d5d9679173d1347855c6 Mon Sep 17 00:00:00 2001 From: Joakim Persson Date: Tue, 13 Oct 2026 15:32:07 +0200 Subject: [PATCH] feat(pi): print and JSON mode skip the wake-up and mailbox poll (MEMPALACE_WAKEUP_PRINT=1 to keep them) In print and JSON mode, `pi -p` and `--mode json` no longer get the before_agent_start wake-up injection (status, diary and the tier-1 mailbox), and the mid-session mailbox poll is off as well. The poll has its own guard, not just `wokeUp`, because `pi -p -c` resumes and a resume sets wokeUp. The client still connects, and the mempalace_* tools stay registered. MEMPALACE_WAKEUP_PRINT=1 restores both. TUI and RPC sessions are unchanged. Why: a one-shot run is a script asking one question. The wake-up cost it 12 palace calls and a large orientation message, and it is meant for the operator of an interactive session. It can also do harm: on tor-ms22 a flash model given "Reply with exactly: PONG" read the injected diary's open items as its task, ran about 12 tool calls, and filed a wrong drawer in the shared palace. Measured on tor-ms22 (workspace copy, -ne -nt --no-session -e) with a fetch probe that re-wraps pi's own fetch. The earlier preload-only fetch wrapper had logged 0 lines in every run, because pi replaces globalThis.fetch. - pi -p: 3 palace requests (initialize, initialized, tools/list), 1.6 s. - --mode json: the same requests, 1.0 s, and no mempalace-wakeup in stdout. - --mode json with MEMPALACE_WAKEUP_PRINT=1: 15 requests, 12 of them tools/call (status 1, diary 1, event_list 10), 3.6 s, and the wake-up is in stdout. New scripts/test-wakeup-mode.sh has these cases: tui, print, json, print resumed, tui resumed (a control that shows the poll does run), print with MEMPALACE_WAKEUP_PRINT=1, and rpc. f590a3a's source fails the print, json and print-resumed cases, with 12, 12 and 5 calls. All seven scripts/test-*.sh pass. --- extensions/pi/README.md | 23 ++++++ extensions/pi/mempalace.ts | 21 ++++- scripts/test-wakeup-mode.sh | 150 ++++++++++++++++++++++++++++++++++++ 3 files changed, 192 insertions(+), 2 deletions(-) create mode 100755 scripts/test-wakeup-mode.sh diff --git a/extensions/pi/README.md b/extensions/pi/README.md index d7bc8eb..f225dbf 100644 --- a/extensions/pi/README.md +++ b/extensions/pi/README.md @@ -38,6 +38,9 @@ dependencies (~300 MB). injects the result as a `mempalace-wakeup` system message so the agent orients itself the way `~/.agents/skills/mempalace/SKILL.md` describes. Skipped on resume/fork (context is already in the thread). + Also skipped in print and JSON mode (`pi -p`, `--mode json`), together + with the mid-session mailbox poll, unless `MEMPALACE_WAKEUP_PRINT=1`. + See [Debugging](#pi--p-prints-its-answer-then-takes-minutes-to-exit). 4. **Automatic transcript feeding** (`session_shutdown`, and a debounced `agent_settled`): stages + mines this pi installation's own session transcripts into the palace with no user action needed — **as of @@ -116,6 +119,7 @@ side of the wiring. |---|---|---| | `MEMPALACE_FEED` | `1` | Set `0` to disable automatic feeding entirely. | | `MEMPALACE_FEED_PRINT` | unset | Set `1` to feed in print and JSON mode too (`pi -p`, `--mode json`). They skip the feed by default, because the next interactive session's feed files the same sessions without duplicates. See [Debugging](#pi--p-prints-its-answer-then-takes-minutes-to-exit). | +| `MEMPALACE_WAKEUP_PRINT` | unset | Set `1` to keep the wake-up injection and the mid-session mailbox poll in print and JSON mode. They are skipped there by default. The `mempalace_*` tools stay registered either way. | | `MEMPALACE_FEED_BIN` | `mempalace-pi-session` | Helper to run. | | `MEMPALACE_FEED_WING` | `wing_conversations` | Target wing — passed to both the exporter and the `mempalace_mine` call. | | `MEMPALACE_FEED_DEBOUNCE_MS` | `600000` (10 min) | Minimum gap between mid-session (`agent_settled`) feeds. Bounds crash loss to one window instead of a whole session. | @@ -608,6 +612,25 @@ skips an unchanged file, and for a grown file it purges that file's drawers before refiling. A device that only ever runs `pi -p` sets `MEMPALACE_FEED_PRINT=1`, or its transcripts are never fed. +**Print and JSON mode skip the wake-up too (also 2026-10-13).** The wake-up +injection costs 12 palace calls: `mempalace_status`, `mempalace_diary_read`, +and 10 `mempalace_event_list` for the tier-1 mailbox. It also adds a large +orientation message to a script's one question. It is meant for the operator +of an interactive session, who sees the same thing in their next one. It can +also do harm. In one tor-ms22 probe, a flash model was given +`Reply with exactly: PONG`. It read the injected diary's open items as its +task, ran about 12 tool calls, and filed a wrong drawer in the shared palace. +So in print and JSON mode the bridge now skips the injection. It also skips +the mid-session mailbox poll, through its own guard: `pi -p -c` resumes, and a +resume sets the flag the poll would otherwise use. The client still connects +and the `mempalace_*` tools stay registered, so a prompt that asks for the +palace gets it. Measured on tor-ms22 with a fetch probe that re-wraps pi's +own `fetch`: `pi -p` made 3 palace requests (`initialize`, `initialized`, +`tools/list`) and exited in 1.6 s. With `MEMPALACE_WAKEUP_PRINT=1` it made 15, +including 12 `tools/call`, and the wake-up appeared in the JSON stream. +`scripts/test-wakeup-mode.sh` pins it; `f590a3a`'s source fails its print, +json and print-resumed cases. + ## The `Type.Unsafe` gotcha Earlier versions of this extension registered every MCP tool with diff --git a/extensions/pi/mempalace.ts b/extensions/pi/mempalace.ts index 69f8f6a..cafface 100644 --- a/extensions/pi/mempalace.ts +++ b/extensions/pi/mempalace.ts @@ -20,7 +20,8 @@ * - Wake-up (auto): on first user prompt of a fresh session, inject * `mempalace_status` + `mempalace_diary_read` output as context so the * agent orients itself the way the mempalace skill describes. Skipped - * on resume/fork (palace context is already in the thread). + * on resume/fork (palace context is already in the thread), and in + * print/json mode (`pi -p`, `--mode json`) unless MEMPALACE_WAKEUP_PRINT=1. * - Feeding (auto): stage + mine this container's pi transcripts into the * palace on `session_shutdown` and on a debounced `agent_settled`. Needs * no LLM turn (pi transcripts are JSONL on disk), which is why it CAN be @@ -1200,6 +1201,19 @@ export default async function mempalaceExtension(pi: ExtensionAPI) { }; const feedsThisRun = (ctx: unknown): boolean => feedInPrintMode || !isOneShot(ctx); + // Print and JSON mode skip the palace start-up work as well, unless + // MEMPALACE_WAKEUP_PRINT=1. That is the wake-up injection (status + diary + + // tier-1 mailbox) and the mid-session mailbox poll. A one-shot run is a + // script asking one question. The wake-up costs it several palace round + // trips and a large orientation message, and it serves the operator of an + // interactive session, who sees it next time they open one. It can also do + // harm: on tor-ms22 a flash model given "Reply with exactly: PONG" read the + // injected diary's open items as its task, ran ~12 tool calls, and filed a + // wrong drawer. The mempalace_* tools stay registered, so a prompt that asks + // for the palace still gets it. + const wakeupInPrintMode = process.env.MEMPALACE_WAKEUP_PRINT === "1"; + const wakesThisRun = (ctx: unknown): boolean => wakeupInPrintMode || !isOneShot(ctx); + // Mid-session feed. A hard container kill runs no handler at all, so this is // what bounds crash loss to one debounce window instead of a whole session. // Re-mining a grown transcript purges and refiles that source_file, so @@ -1675,6 +1689,8 @@ export default async function mempalaceExtension(pi: ExtensionAPI) { pi.on("agent_settled", async (_event, ctx) => { if (!mailboxEnabled || !available) return; + // Explicit, not left to `wokeUp`: `pi -p -c` resumes, which sets wokeUp. + if (!wakesThisRun(ctx)) return; if (!wokeUp) return; // the wake-up injection has not run yet; it does look #1 if (Date.now() - lastMailboxPollAt < mailboxPollMs) return; // Claim the slot BEFORE awaiting anything, so overlapping settles cannot @@ -1759,8 +1775,9 @@ export default async function mempalaceExtension(pi: ExtensionAPI) { }); // --- Auto wake-up (mempalace skill Phase 1) --- - pi.on("before_agent_start", async (_event, _ctx) => { + pi.on("before_agent_start", async (_event, ctx) => { if (wokeUp || !available) return; + if (!wakesThisRun(ctx)) return; // one-shot run: see wakesThisRun wokeUp = true; // one-shot, even if the calls below fail const sections: string[] = []; diff --git a/scripts/test-wakeup-mode.sh b/scripts/test-wakeup-mode.sh new file mode 100755 index 0000000..2851292 --- /dev/null +++ b/scripts/test-wakeup-mode.sh @@ -0,0 +1,150 @@ +#!/usr/bin/env bash +# test-wakeup-mode.sh — print and JSON mode (`pi -p`, `--mode json`) must not +# do the palace start-up work of extensions/pi/mempalace.ts unless +# MEMPALACE_WAKEUP_PRINT=1. That work is the wake-up injection (status, diary, +# tier-1 mailbox) and the mid-session mailbox poll. TUI and RPC sessions keep +# both. +# +# WHY (2026-10-13, tor-ms22). A one-shot run is a script asking one question. +# The wake-up cost it several palace round trips and a large orientation +# message, and in one probe the model read the injected diary's open items as +# its task. Given "Reply with exactly: PONG", it ran about 12 tool calls and +# filed a wrong drawer in the shared palace. +# +# Same method as test-feed-exit.sh: the SHIPPED text is type-stripped and loaded +# under a stub pi. A local JSON-RPC server, in a separate process, logs every +# tools/call. The driver fires session_start, before_agent_start, agent_settled +# and session_shutdown with the ctx in DRIVER_CTX, and reports whether +# before_agent_start returned the wake-up message. +# tui wake-up message, status + diary + event_list calls +# print / json no message and NO palace call at all +# print, resumed (`pi -p -c`; resume sets wokeUp) the mid-session poll +# must stay off too: no event_list call +# tui, resumed control for the line above: the poll DOES run there, +# so a zero is not vacuous +# print + MEMPALACE_WAKEUP_PRINT=1 wake-up message, as before +# rpc wake-up message (long-lived, keeps it) +set -euo pipefail +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +SRC="${1:-$REPO_ROOT/extensions/pi/mempalace.ts}" +WORK="$(mktemp -d)" +SERVER_PID="" +cleanup() { [ -n "$SERVER_PID" ] && kill "$SERVER_PID" 2>/dev/null; rm -rf "$WORK"; } +trap cleanup EXIT +[ -r "$SRC" ] || { echo "FAIL: cannot read $SRC" >&2; exit 2; } + +cat >"$WORK/extract.mjs" <<'EXTRACT' +import { readFileSync, writeFileSync } from "node:fs"; +import { stripTypeScriptTypes } from "node:module"; +let src = readFileSync(process.argv[2], "utf8"); +const imp = 'import { Type } from "typebox";'; +if (!src.includes(imp)) throw new Error(`expected import line not found: ${imp}`); +src = src.replace(imp, 'const Type = { Unsafe: (s) => s, Object: () => ({}) };'); +if (!src.includes("before_agent_start")) throw new Error("extension has no before_agent_start handler"); +writeFileSync(process.argv[3], stripTypeScriptTypes(src, { mode: "strip" })); +EXTRACT +node --no-warnings "$WORK/extract.mjs" "$SRC" "$WORK/ext.mjs" || exit 2 +node --check "$WORK/ext.mjs" || { echo "FAIL: stripped extension does not parse" >&2; exit 2; } +echo "[extract] type-stripped $(basename "$SRC") ($(wc -c <"$WORK/ext.mjs") bytes)" + +cat >"$WORK/server.mjs" <<'SERVER' +import { createServer } from "node:http"; +import { appendFileSync } from "node:fs"; +const tools = ["mempalace_status", "mempalace_diary_read", "mempalace_event_list"] + .map((name) => ({ name, description: "", inputSchema: { type: "object" } })); +const server = createServer((req, res) => { + let body = ""; + req.on("data", (c) => (body += c)); + req.on("end", () => { + const msg = JSON.parse(body); + if (msg.id === undefined) { res.writeHead(202); res.end(); return; } + const reply = (result) => { + res.writeHead(200, { "content-type": "application/json", connection: "close" }); + res.end(JSON.stringify({ jsonrpc: "2.0", id: msg.id, result })); + }; + if (msg.method === "initialize") return reply({ protocolVersion: "2024-11-05", capabilities: {}, serverInfo: { name: "fake", version: "0" } }); + if (msg.method === "tools/list") return reply({ tools }); + if (msg.method === "tools/call") { + const name = msg.params?.name; + appendFileSync(process.env.CALL_LOG, `${name}\n`); + const text = name === "mempalace_event_list" ? JSON.stringify({ events: [], count: 0 }) : `${name} ok`; + return reply({ content: [{ type: "text", text }] }); + } + reply({}); + }); +}); +server.listen(0, "127.0.0.1", () => console.log(server.address().port)); +SERVER +CALL_LOG="$WORK/calls.log" node "$WORK/server.mjs" >"$WORK/port" & +SERVER_PID=$! +for _ in $(seq 50); do [ -s "$WORK/port" ] && break; sleep 0.1; done +PORT="$(cat "$WORK/port")" +[ -n "$PORT" ] || { echo "FAIL: fake palace did not start" >&2; exit 2; } + +cat >"$WORK/driver.mjs" <<'DRIVER' +const ctx = { ...JSON.parse(process.env.DRIVER_CTX || "{}"), ui: { notify() {} } }; +const handlers = {}; +const pi = new Proxy({}, { + get: (_t, k) => k === "on" ? (ev, fn) => { (handlers[ev] ||= []).push(fn); } : () => {}, +}); +const { default: ext } = await import("./ext.mjs"); +await ext(pi); +const fire = async (ev, event) => { + const out = []; + for (const fn of handlers[ev] || []) out.push(await fn({ type: ev, ...event }, ctx)); + return out; +}; +await fire("session_start", { reason: process.env.DRIVER_REASON || "startup" }); +const results = await fire("before_agent_start", { prompt: "PONG" }); +const woke = results.some((r) => r?.message?.customType === "mempalace-wakeup"); +console.log(`WAKEUP_MESSAGE=${woke ? "yes" : "no"}`); +await fire("agent_settled", {}); +await new Promise((r) => setTimeout(r, 800)); // the mailbox poll is not awaited +await fire("session_shutdown", { reason: "quit" }); +DRIVER + +failures=0 +check() { if [ "$1" = 0 ]; then echo "ok $2"; else echo "FAIL $2"; failures=$((failures + 1)); fi; } +calls() { grep -c "^$1\$" "$WORK/calls.log" || true; } + +run_case() { # $1 ctx JSON, $2 session_start reason, [$3 WAKEUP_PRINT] -> MSG RC ALL STATUS DIARY EVENTS + : >"$WORK/calls.log" + set +e + out="$(cd "$WORK" && env -u MEMPALACE_REMOTE_TOKEN -u MEMPALACE_MAILBOX \ + MEMPALACE_REMOTE_URL="http://127.0.0.1:$PORT/mcp" MEMPALACE_PI_DEVICE=testdev \ + MEMPALACE_FEED=0 DRIVER_CTX="$1" DRIVER_REASON="$2" MEMPALACE_WAKEUP_PRINT="${3:-}" \ + timeout 60 node --no-warnings driver.mjs 2>"$WORK/driver.err")" + RC=$? + set -e + MSG="$(printf '%s\n' "$out" | sed -n 's/^WAKEUP_MESSAGE=//p')" + ALL="$(wc -l <"$WORK/calls.log" | tr -d ' ')" + STATUS="$(calls mempalace_status)"; DIARY="$(calls mempalace_diary_read)"; EVENTS="$(calls mempalace_event_list)" + echo "[$4] rc=$RC message=${MSG:-?} calls=$ALL (status=$STATUS diary=$DIARY event_list=$EVENTS)" +} + +run_case '{"mode":"tui","hasUI":true}' startup "" tui +check "$([ "$RC" = 0 ] && [ "$MSG" = yes ] && [ "$STATUS" = 1 ] && [ "$DIARY" = 1 ] && [ "$EVENTS" -ge 1 ] && echo 0 || echo 1)" \ + "tui: wake-up injected, with status, diary and mailbox reads" + +for m in print json; do + run_case "{\"mode\":\"$m\",\"hasUI\":false}" startup "" "$m" + check "$([ "$RC" = 0 ] && [ "$MSG" = no ] && [ "$ALL" = 0 ] && echo 0 || echo 1)" \ + "$m: no wake-up message and no palace call at all (calls=$ALL)" +done + +run_case '{"mode":"print","hasUI":false}' resume "" "print resumed" +check "$([ "$RC" = 0 ] && [ "$MSG" = no ] && [ "$EVENTS" = 0 ] && echo 0 || echo 1)" \ + "print resumed (pi -p -c): the mid-session mailbox poll stays off (event_list=$EVENTS)" + +run_case '{"mode":"tui","hasUI":true}' resume "" "tui resumed (control)" +check "$([ "$RC" = 0 ] && [ "$MSG" = no ] && [ "$EVENTS" -ge 1 ] && echo 0 || echo 1)" \ + "control: tui resumed skips the injection but DOES poll (event_list=$EVENTS), so the zero above is not vacuous" + +run_case '{"mode":"print","hasUI":false}' startup 1 "print+WAKEUP_PRINT=1" +check "$([ "$RC" = 0 ] && [ "$MSG" = yes ] && [ "$STATUS" = 1 ] && echo 0 || echo 1)" \ + "print + MEMPALACE_WAKEUP_PRINT=1: wake-up injected, as before" + +run_case '{"mode":"rpc","hasUI":true}' startup "" rpc +check "$([ "$RC" = 0 ] && [ "$MSG" = yes ] && echo 0 || echo 1)" "rpc: wake-up injected (long-lived)" + +if [ "$failures" = 0 ]; then echo "PASS: all assertions held"; else echo "FAIL: $failures assertion(s) failed"; exit 1; fi