joakimp/mempalace-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(pi): print and JSON mode skip the wake-up and mailbox poll (MEMPALACE_WAKEUP_PRINT=1 to keep them)

Browse Source 
In print and JSON mode, `pi -p` and `--mode json` no longer get the
before_agent_start wake-up injection (status, diary and the tier-1 mailbox),
and the mid-session mailbox poll is off as well. The poll has its own guard,
not just `wokeUp`, because `pi -p -c` resumes and a resume sets wokeUp. The
client still connects, and the mempalace_* tools stay registered.
MEMPALACE_WAKEUP_PRINT=1 restores both. TUI and RPC sessions are unchanged.

Why: a one-shot run is a script asking one question. The wake-up cost it 12
palace calls and a large orientation message, and it is meant for the
operator of an interactive session. It can also do harm: on tor-ms22 a flash
model given "Reply with exactly: PONG" read the injected diary's open items
as its task, ran about 12 tool calls, and filed a wrong drawer in the shared
palace.

Measured on tor-ms22 (workspace copy, -ne -nt --no-session -e) with a fetch
probe that re-wraps pi's own fetch. The earlier preload-only fetch wrapper had
logged 0 lines in every run, because pi replaces globalThis.fetch.
- pi -p: 3 palace requests (initialize, initialized, tools/list), 1.6 s.
- --mode json: the same requests, 1.0 s, and no mempalace-wakeup in stdout.
- --mode json with MEMPALACE_WAKEUP_PRINT=1: 15 requests, 12 of them
  tools/call (status 1, diary 1, event_list 10), 3.6 s, and the wake-up is in
  stdout.

New scripts/test-wakeup-mode.sh has these cases: tui, print, json, print
resumed, tui resumed (a control that shows the poll does run), print with
MEMPALACE_WAKEUP_PRINT=1, and rpc. f590a3a's source fails the print, json and
print-resumed cases, with 12, 12 and 5 calls. All seven scripts/test-*.sh
pass.
This commit is contained in:
Joakim Persson
2026-10-13 15:32:07 +02:00
parent f590a3ac5b
commit d5a03f4ff5
3 changed files with 192 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
extensions/pi/README.md
+23
View File
@@ -38,6 +38,9 @@ dependencies (~300 MB).
injects the result as a `mempalace-wakeup` system message so the
agent orients itself the way `~/.agents/skills/mempalace/SKILL.md`
describes. Skipped on resume/fork (context is already in the thread).
Also skipped in print and JSON mode (`pi -p`, `--mode json`), together
with the mid-session mailbox poll, unless `MEMPALACE_WAKEUP_PRINT=1`.
See [Debugging](#pi--p-prints-its-answer-then-takes-minutes-to-exit).
4. **Automatic transcript feeding** (`session_shutdown`, and a debounced
`agent_settled`): stages + mines this pi installation's own session
transcripts into the palace with no user action needed — **as of
@@ -116,6 +119,7 @@ side of the wiring.
|---|---|---|
| `MEMPALACE_FEED` | `1` | Set `0` to disable automatic feeding entirely. |
| `MEMPALACE_FEED_PRINT` | unset | Set `1` to feed in print and JSON mode too (`pi -p`, `--mode json`). They skip the feed by default, because the next interactive session's feed files the same sessions without duplicates. See [Debugging](#pi--p-prints-its-answer-then-takes-minutes-to-exit). |
| `MEMPALACE_WAKEUP_PRINT` | unset | Set `1` to keep the wake-up injection and the mid-session mailbox poll in print and JSON mode. They are skipped there by default. The `mempalace_*` tools stay registered either way. |
| `MEMPALACE_FEED_BIN` | `mempalace-pi-session` | Helper to run. |
| `MEMPALACE_FEED_WING` | `wing_conversations` | Target wing — passed to both the exporter and the `mempalace_mine` call. |
| `MEMPALACE_FEED_DEBOUNCE_MS` | `600000` (10 min) | Minimum gap between mid-session (`agent_settled`) feeds. Bounds crash loss to one window instead of a whole session. |
@@ -608,6 +612,25 @@ skips an unchanged file, and for a grown file it purges that file's drawers
before refiling. A device that only ever runs `pi -p` sets
`MEMPALACE_FEED_PRINT=1`, or its transcripts are never fed.
**Print and JSON mode skip the wake-up too (also 2026-10-13).** The wake-up
injection costs 12 palace calls: `mempalace_status`, `mempalace_diary_read`,
and 10 `mempalace_event_list` for the tier-1 mailbox. It also adds a large
orientation message to a script's one question. It is meant for the operator
of an interactive session, who sees the same thing in their next one. It can
also do harm. In one tor-ms22 probe, a flash model was given
`Reply with exactly: PONG`. It read the injected diary's open items as its
task, ran about 12 tool calls, and filed a wrong drawer in the shared palace.
So in print and JSON mode the bridge now skips the injection. It also skips
the mid-session mailbox poll, through its own guard: `pi -p -c` resumes, and a
resume sets the flag the poll would otherwise use. The client still connects
and the `mempalace_*` tools stay registered, so a prompt that asks for the
palace gets it. Measured on tor-ms22 with a fetch probe that re-wraps pi's
own `fetch`: `pi -p` made 3 palace requests (`initialize`, `initialized`,
`tools/list`) and exited in 1.6 s. With `MEMPALACE_WAKEUP_PRINT=1` it made 15,
including 12 `tools/call`, and the wake-up appeared in the JSON stream.
`scripts/test-wakeup-mode.sh` pins it; `f590a3a`'s source fails its print,
json and print-resumed cases.
## The `Type.Unsafe` gotcha
Earlier versions of this extension registered every MCP tool with
extensions/pi/mempalace.ts
+19 -2
View File
@@ -20,7 +20,8 @@
* - Wake-up (auto): on first user prompt of a fresh session, inject
* `mempalace_status` + `mempalace_diary_read` output as context so the
* agent orients itself the way the mempalace skill describes. Skipped
* on resume/fork (palace context is already in the thread).
* on resume/fork (palace context is already in the thread), and in
* print/json mode (`pi -p`, `--mode json`) unless MEMPALACE_WAKEUP_PRINT=1.
* - Feeding (auto): stage + mine this container's pi transcripts into the
* palace on `session_shutdown` and on a debounced `agent_settled`. Needs
* no LLM turn (pi transcripts are JSONL on disk), which is why it CAN be
@@ -1200,6 +1201,19 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
};
const feedsThisRun = (ctx: unknown): boolean => feedInPrintMode || !isOneShot(ctx);
// Print and JSON mode skip the palace start-up work as well, unless
// MEMPALACE_WAKEUP_PRINT=1. That is the wake-up injection (status + diary +
// tier-1 mailbox) and the mid-session mailbox poll. A one-shot run is a
// script asking one question. The wake-up costs it several palace round
// trips and a large orientation message, and it serves the operator of an
// interactive session, who sees it next time they open one. It can also do
// harm: on tor-ms22 a flash model given "Reply with exactly: PONG" read the
// injected diary's open items as its task, ran ~12 tool calls, and filed a
// wrong drawer. The mempalace_* tools stay registered, so a prompt that asks
// for the palace still gets it.
const wakeupInPrintMode = process.env.MEMPALACE_WAKEUP_PRINT === "1";
const wakesThisRun = (ctx: unknown): boolean => wakeupInPrintMode || !isOneShot(ctx);
// Mid-session feed. A hard container kill runs no handler at all, so this is
// what bounds crash loss to one debounce window instead of a whole session.
// Re-mining a grown transcript purges and refiles that source_file, so
@@ -1675,6 +1689,8 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
pi.on("agent_settled", async (_event, ctx) => {
if (!mailboxEnabled || !available) return;
// Explicit, not left to `wokeUp`: `pi -p -c` resumes, which sets wokeUp.
if (!wakesThisRun(ctx)) return;
if (!wokeUp) return; // the wake-up injection has not run yet; it does look #1
if (Date.now() - lastMailboxPollAt < mailboxPollMs) return;
// Claim the slot BEFORE awaiting anything, so overlapping settles cannot
@@ -1759,8 +1775,9 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
});
// --- Auto wake-up (mempalace skill Phase 1) ---
pi.on("before_agent_start", async (_event, _ctx) => {
pi.on("before_agent_start", async (_event, ctx) => {
if (wokeUp || !available) return;
if (!wakesThisRun(ctx)) return; // one-shot run: see wakesThisRun
wokeUp = true; // one-shot, even if the calls below fail
const sections: string[] = [];
scripts/test-wakeup-mode.sh
Executable
+150
View File
@@ -0,0 +1,150 @@
#!/usr/bin/env bash
# test-wakeup-mode.sh — print and JSON mode (`pi -p`, `--mode json`) must not
# do the palace start-up work of extensions/pi/mempalace.ts unless
# MEMPALACE_WAKEUP_PRINT=1. That work is the wake-up injection (status, diary,
# tier-1 mailbox) and the mid-session mailbox poll. TUI and RPC sessions keep
# both.
#
# WHY (2026-10-13, tor-ms22). A one-shot run is a script asking one question.
# The wake-up cost it several palace round trips and a large orientation
# message, and in one probe the model read the injected diary's open items as
# its task. Given "Reply with exactly: PONG", it ran about 12 tool calls and
# filed a wrong drawer in the shared palace.
#
# Same method as test-feed-exit.sh: the SHIPPED text is type-stripped and loaded
# under a stub pi. A local JSON-RPC server, in a separate process, logs every
# tools/call. The driver fires session_start, before_agent_start, agent_settled
# and session_shutdown with the ctx in DRIVER_CTX, and reports whether
# before_agent_start returned the wake-up message.
# tui wake-up message, status + diary + event_list calls
# print / json no message and NO palace call at all
# print, resumed (`pi -p -c`; resume sets wokeUp) the mid-session poll
# must stay off too: no event_list call
# tui, resumed control for the line above: the poll DOES run there,
# so a zero is not vacuous
# print + MEMPALACE_WAKEUP_PRINT=1 wake-up message, as before
# rpc wake-up message (long-lived, keeps it)
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SRC="${1:-$REPO_ROOT/extensions/pi/mempalace.ts}"
WORK="$(mktemp -d)"
SERVER_PID=""
cleanup() { [ -n "$SERVER_PID" ] && kill "$SERVER_PID" 2>/dev/null; rm -rf "$WORK"; }
trap cleanup EXIT
[ -r "$SRC" ] || { echo "FAIL: cannot read $SRC" >&2; exit 2; }
cat >"$WORK/extract.mjs" <<'EXTRACT'
import { readFileSync, writeFileSync } from "node:fs";
import { stripTypeScriptTypes } from "node:module";
let src = readFileSync(process.argv[2], "utf8");
const imp = 'import { Type } from "typebox";';
if (!src.includes(imp)) throw new Error(`expected import line not found: ${imp}`);
src = src.replace(imp, 'const Type = { Unsafe: (s) => s, Object: () => ({}) };');
if (!src.includes("before_agent_start")) throw new Error("extension has no before_agent_start handler");
writeFileSync(process.argv[3], stripTypeScriptTypes(src, { mode: "strip" }));
EXTRACT
node --no-warnings "$WORK/extract.mjs" "$SRC" "$WORK/ext.mjs" || exit 2
node --check "$WORK/ext.mjs" || { echo "FAIL: stripped extension does not parse" >&2; exit 2; }
echo "[extract] type-stripped $(basename "$SRC") ($(wc -c <"$WORK/ext.mjs") bytes)"
cat >"$WORK/server.mjs" <<'SERVER'
import { createServer } from "node:http";
import { appendFileSync } from "node:fs";
const tools = ["mempalace_status", "mempalace_diary_read", "mempalace_event_list"]
.map((name) => ({ name, description: "", inputSchema: { type: "object" } }));
const server = createServer((req, res) => {
let body = "";
req.on("data", (c) => (body += c));
req.on("end", () => {
const msg = JSON.parse(body);
if (msg.id === undefined) { res.writeHead(202); res.end(); return; }
const reply = (result) => {
res.writeHead(200, { "content-type": "application/json", connection: "close" });
res.end(JSON.stringify({ jsonrpc: "2.0", id: msg.id, result }));
};
if (msg.method === "initialize") return reply({ protocolVersion: "2024-11-05", capabilities: {}, serverInfo: { name: "fake", version: "0" } });
if (msg.method === "tools/list") return reply({ tools });
if (msg.method === "tools/call") {
const name = msg.params?.name;
appendFileSync(process.env.CALL_LOG, `${name}\n`);
const text = name === "mempalace_event_list" ? JSON.stringify({ events: [], count: 0 }) : `${name} ok`;
return reply({ content: [{ type: "text", text }] });
}
reply({});
});
});
server.listen(0, "127.0.0.1", () => console.log(server.address().port));
SERVER
CALL_LOG="$WORK/calls.log" node "$WORK/server.mjs" >"$WORK/port" &
SERVER_PID=$!
for _ in $(seq 50); do [ -s "$WORK/port" ] && break; sleep 0.1; done
PORT="$(cat "$WORK/port")"
[ -n "$PORT" ] || { echo "FAIL: fake palace did not start" >&2; exit 2; }
cat >"$WORK/driver.mjs" <<'DRIVER'
const ctx = { ...JSON.parse(process.env.DRIVER_CTX || "{}"), ui: { notify() {} } };
const handlers = {};
const pi = new Proxy({}, {
get: (_t, k) => k === "on" ? (ev, fn) => { (handlers[ev] ||= []).push(fn); } : () => {},
});
const { default: ext } = await import("./ext.mjs");
await ext(pi);
const fire = async (ev, event) => {
const out = [];
for (const fn of handlers[ev] || []) out.push(await fn({ type: ev, ...event }, ctx));
return out;
};
await fire("session_start", { reason: process.env.DRIVER_REASON || "startup" });
const results = await fire("before_agent_start", { prompt: "PONG" });
const woke = results.some((r) => r?.message?.customType === "mempalace-wakeup");
console.log(`WAKEUP_MESSAGE=${woke ? "yes" : "no"}`);
await fire("agent_settled", {});
await new Promise((r) => setTimeout(r, 800)); // the mailbox poll is not awaited
await fire("session_shutdown", { reason: "quit" });
DRIVER
failures=0
check() { if [ "$1" = 0 ]; then echo "ok $2"; else echo "FAIL $2"; failures=$((failures + 1)); fi; }
calls() { grep -c "^$1\$" "$WORK/calls.log" || true; }
run_case() { # $1 ctx JSON, $2 session_start reason, [$3 WAKEUP_PRINT] -> MSG RC ALL STATUS DIARY EVENTS
: >"$WORK/calls.log"
set +e
out="$(cd "$WORK" && env -u MEMPALACE_REMOTE_TOKEN -u MEMPALACE_MAILBOX \
MEMPALACE_REMOTE_URL="http://127.0.0.1:$PORT/mcp" MEMPALACE_PI_DEVICE=testdev \
MEMPALACE_FEED=0 DRIVER_CTX="$1" DRIVER_REASON="$2" MEMPALACE_WAKEUP_PRINT="${3:-}" \
timeout 60 node --no-warnings driver.mjs 2>"$WORK/driver.err")"
RC=$?
set -e
MSG="$(printf '%s\n' "$out" | sed -n 's/^WAKEUP_MESSAGE=//p')"
ALL="$(wc -l <"$WORK/calls.log" | tr -d ' ')"
STATUS="$(calls mempalace_status)"; DIARY="$(calls mempalace_diary_read)"; EVENTS="$(calls mempalace_event_list)"
echo "[$4] rc=$RC message=${MSG:-?} calls=$ALL (status=$STATUS diary=$DIARY event_list=$EVENTS)"
}
run_case '{"mode":"tui","hasUI":true}' startup "" tui
check "$([ "$RC" = 0 ] && [ "$MSG" = yes ] && [ "$STATUS" = 1 ] && [ "$DIARY" = 1 ] && [ "$EVENTS" -ge 1 ] && echo 0 || echo 1)" \
"tui: wake-up injected, with status, diary and mailbox reads"
for m in print json; do
run_case "{\"mode\":\"$m\",\"hasUI\":false}" startup "" "$m"
check "$([ "$RC" = 0 ] && [ "$MSG" = no ] && [ "$ALL" = 0 ] && echo 0 || echo 1)" \
"$m: no wake-up message and no palace call at all (calls=$ALL)"
done
run_case '{"mode":"print","hasUI":false}' resume "" "print resumed"
check "$([ "$RC" = 0 ] && [ "$MSG" = no ] && [ "$EVENTS" = 0 ] && echo 0 || echo 1)" \
"print resumed (pi -p -c): the mid-session mailbox poll stays off (event_list=$EVENTS)"
run_case '{"mode":"tui","hasUI":true}' resume "" "tui resumed (control)"
check "$([ "$RC" = 0 ] && [ "$MSG" = no ] && [ "$EVENTS" -ge 1 ] && echo 0 || echo 1)" \
"control: tui resumed skips the injection but DOES poll (event_list=$EVENTS), so the zero above is not vacuous"
run_case '{"mode":"print","hasUI":false}' startup 1 "print+WAKEUP_PRINT=1"
check "$([ "$RC" = 0 ] && [ "$MSG" = yes ] && [ "$STATUS" = 1 ] && echo 0 || echo 1)" \
"print + MEMPALACE_WAKEUP_PRINT=1: wake-up injected, as before"
run_case '{"mode":"rpc","hasUI":true}' startup "" rpc
check "$([ "$RC" = 0 ] && [ "$MSG" = yes ] && echo 0 || echo 1)" "rpc: wake-up injected (long-lived)"
if [ "$failures" = 0 ]; then echo "PASS: all assertions held"; else echo "FAIL: $failures assertion(s) failed"; exit 1; fi