feat(pi): print and JSON mode skip the feed by default (MEMPALACE_FEED_PRINT=1 to keep it)

After 2f0edf5 a `pi -p` still waited for one full catch-up feed. lastFeedAt
starts at 0, so the first settled turn feeds, and session_shutdown waits for
that feed. On tor-ms22 this took 45 s (prepare 11 s + mine 31 s) for a 3 s
answer. That feed almost never files the run's own transcript, because a
one-shot run rarely passes the exporter's gate (4 messages, 1000 assistant
chars). What it files is the device's other sessions, and the next interactive
feed re-offers those without duplicates. The source_file is the staged path,
fixed per session UUID. An unchanged file is skipped. For a grown file, its
drawers are purged and then refiled.

Now, when ctx.mode is "print" or "json", neither agent_settled nor
session_shutdown feeds. Without ctx.mode, the bridge falls back to
hasUI === false. A stale ctx keeps the old behaviour. RPC mode is long-lived
and feeds as before. MEMPALACE_FEED_PRINT=1 restores feeding in print and JSON
mode, for a device that only ever runs pi -p.

Measured on tor-ms22 with the workspace copy (-ne -nt --no-session -e):
- pi -p: 2.9 s / 2.8 s
- --mode json: 3.8 s
- -p with MEMPALACE_FEED_PRINT=1: 30.8 s; the probe saw the prepare and race
  timers, so the feed ran.

scripts/test-feed-exit.sh now fires agent_settled and then session_shutdown
with a ctx, and gained cases C (print), D (json), E (print +
MEMPALACE_FEED_PRINT=1) and F (rpc). Against 2f0edf5's source, C and D fail
(mines=1, about 1 s wait). All six scripts/test-*.sh pass.
This commit is contained in:
2026-10-13 14:51:37 +02:00
parent 2f0edf51cf
commit f590a3ac5b
3 changed files with 90 additions and 17 deletions
+22 -8
View File
@@ -84,7 +84,9 @@ The bridge feeds this pi installation's own session transcripts into the
palace by itself — no scheduler, no cron, no manual invocation. It fires on palace by itself — no scheduler, no cron, no manual invocation. It fires on
`session_shutdown` (covers quit, `/new`, `/resume`, `/fork`) and on a `session_shutdown` (covers quit, `/new`, `/resume`, `/fork`) and on a
debounced `agent_settled` (covers a long session that later crashes, since a debounced `agent_settled` (covers a long session that later crashes, since a
hard kill runs no shutdown handler at all). hard kill runs no shutdown handler at all). Interactive (TUI) and RPC sessions
feed. Print and JSON mode (`pi -p`, `--mode json`) do not, unless
`MEMPALACE_FEED_PRINT=1`.
The work is split across two processes, and the reason is a hard constraint, The work is split across two processes, and the reason is a hard constraint,
not a style choice: **the palace is single-writer.** A live pi session not a style choice: **the palace is single-writer.** A live pi session
@@ -113,6 +115,7 @@ side of the wiring.
| Var | Default | Effect | | Var | Default | Effect |
|---|---|---| |---|---|---|
| `MEMPALACE_FEED` | `1` | Set `0` to disable automatic feeding entirely. | | `MEMPALACE_FEED` | `1` | Set `0` to disable automatic feeding entirely. |
| `MEMPALACE_FEED_PRINT` | unset | Set `1` to feed in print and JSON mode too (`pi -p`, `--mode json`). They skip the feed by default, because the next interactive session's feed files the same sessions without duplicates. See [Debugging](#pi--p-prints-its-answer-then-takes-minutes-to-exit). |
| `MEMPALACE_FEED_BIN` | `mempalace-pi-session` | Helper to run. | | `MEMPALACE_FEED_BIN` | `mempalace-pi-session` | Helper to run. |
| `MEMPALACE_FEED_WING` | `wing_conversations` | Target wing — passed to both the exporter and the `mempalace_mine` call. | | `MEMPALACE_FEED_WING` | `wing_conversations` | Target wing — passed to both the exporter and the `mempalace_mine` call. |
| `MEMPALACE_FEED_DEBOUNCE_MS` | `600000` (10 min) | Minimum gap between mid-session (`agent_settled`) feeds. Bounds crash loss to one window instead of a whole session. | | `MEMPALACE_FEED_DEBOUNCE_MS` | `600000` (10 min) | Minimum gap between mid-session (`agent_settled`) feeds. Bounds crash loss to one window instead of a whole session. |
@@ -586,13 +589,24 @@ Fixed 2026-10-13: the timer is cleared when the race settles.
`session_shutdown` resolving, both when the mine answers and when it times `session_shutdown` resolving, both when the mine answers and when it times
out. Before the fix the first case held node for the whole deadline. out. Before the fix the first case held node for the whole deadline.
**What is still expected.** The first settled turn of every session runs a **Print and JSON mode skip the feed (also 2026-10-13).** With only the timer
catch-up feed (`lastFeedAt` starts at 0), and `session_shutdown` waits for it. fix, a `pi -p` still waited for one full feed. The first settled turn of every
A `pi -p` therefore still exits only after one full feed: measured at 45 s on session runs a catch-up feed (`lastFeedAt` starts at 0), and `session_shutdown`
tor-ms22, of which `--prepare` took 11 s and the mine 31 s. A script that waits for it: 45 s on tor-ms22, of which `--prepare` took 11 s and the mine
wants `pi -p` to return at once, and does not need this run's transcript 31 s, for a 3 s answer. That feed almost never files the run's own transcript,
filed now, sets `MEMPALACE_FEED=0`. The next interactive session's feed picks because a one-shot run rarely passes the exporter's quality gate (4 messages,
up the transcript, if it qualifies. 1000 assistant characters). What it files is the device's *other* sessions, and
the next interactive feed re-offers those anyway. So when `ctx.mode` is
`print` or `json`, the bridge now feeds neither on `agent_settled` nor on
`session_shutdown`. Measured on tor-ms22: `pi -p` 2.9 s and `--mode json`
3.8 s, against 45 s before; `MEMPALACE_FEED_PRINT=1` gives 30.8 s (a smaller
backlog than the 45 s run). RPC mode is long-lived and feeds as before.
Nothing is filed twice when a later session re-offers a transcript. The
`source_file` is the staged path, which is fixed per pi session UUID. The mine
skips an unchanged file, and for a grown file it purges that file's drawers
before refiling. A device that only ever runs `pi -p` sets
`MEMPALACE_FEED_PRINT=1`, or its transcripts are never fed.
## The `Type.Unsafe` gotcha ## The `Type.Unsafe` gotcha
+30 -3
View File
@@ -33,6 +33,8 @@
* client also means it automatically targets whichever palace this bridge * client also means it automatically targets whichever palace this bridge
* is pointed at (local stdio or a shared remote one). * is pointed at (local stdio or a shared remote one).
* - MEMPALACE_FEED=0 disable feeding entirely * - MEMPALACE_FEED=0 disable feeding entirely
* - MEMPALACE_FEED_PRINT=1 also feed in print/json mode (`pi -p`,
* `--mode json`), which skip it by default
* - MEMPALACE_FEED_BIN helper to run (default mempalace-pi-session) * - MEMPALACE_FEED_BIN helper to run (default mempalace-pi-session)
* - MEMPALACE_FEED_WING target wing (default wing_conversations) * - MEMPALACE_FEED_WING target wing (default wing_conversations)
* - MEMPALACE_FEED_DEBOUNCE_MS min gap between mid-session feeds (default 600000) * - MEMPALACE_FEED_DEBOUNCE_MS min gap between mid-session feeds (default 600000)
@@ -1174,20 +1176,45 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
return feedInFlight; return feedInFlight;
} }
// Print and JSON mode (`pi -p`, `--mode json`) do NOT feed unless
// MEMPALACE_FEED_PRINT=1. A one-shot run's own transcript almost never meets
// the exporter's quality gate (MIN_MESSAGES=4, MIN_ASSISTANT_CHARS=1000), so
// what such a run would file is the device's OTHER sessions. That is a
// catch-up, because lastFeedAt starts at 0, and session_shutdown waits for
// it: 45 s on tor-ms22 (prepare 11 s + mine 31 s) for a 3 s answer.
// Skipping loses nothing. The source_file is the staged path, fixed per
// session UUID, and the next interactive feed re-offers every qualifying
// session: an unchanged file is skipped, a grown one is purged and refiled.
// RPC mode is long-lived and feeds as before. A pi whose ctx has no `mode`
// falls back to hasUI, which is false only in print/json mode.
const feedInPrintMode = process.env.MEMPALACE_FEED_PRINT === "1";
const isOneShot = (ctx: unknown): boolean => {
try {
const c = ctx as { mode?: string; hasUI?: boolean } | undefined;
const mode = c?.mode;
if (mode === "print" || mode === "json") return true;
return mode === undefined && c?.hasUI === false;
} catch {
return false; // a stale ctx throws on access: keep the old behaviour
}
};
const feedsThisRun = (ctx: unknown): boolean => feedInPrintMode || !isOneShot(ctx);
// Mid-session feed. A hard container kill runs no handler at all, so this is // Mid-session feed. A hard container kill runs no handler at all, so this is
// what bounds crash loss to one debounce window instead of a whole session. // what bounds crash loss to one debounce window instead of a whole session.
// Re-mining a grown transcript purges and refiles that source_file, so // Re-mining a grown transcript purges and refiles that source_file, so
// repeated ticks refresh a session's drawers rather than duplicating them. // repeated ticks refresh a session's drawers rather than duplicating them.
pi.on("agent_settled", async () => { pi.on("agent_settled", async (_event, ctx) => {
if (!feedsThisRun(ctx)) return;
if (Date.now() - lastFeedAt < feedDebounceMs) return; if (Date.now() - lastFeedAt < feedDebounceMs) return;
void feedPalace("tick"); // deliberately not awaited: never stall a turn void feedPalace("tick"); // deliberately not awaited: never stall a turn
}); });
pi.on("session_shutdown", async () => { pi.on("session_shutdown", async (_event, ctx) => {
// Feed before stopping the client: we are the palace holder, so nothing // Feed before stopping the client: we are the palace holder, so nothing
// else can mine while we live. pi awaits this handler, so the mine really // else can mine while we live. pi awaits this handler, so the mine really
// does complete; feedMineTimeoutMs keeps a wedged palace from hanging exit. // does complete; feedMineTimeoutMs keeps a wedged palace from hanging exit.
await feedPalace("shutdown"); if (feedsThisRun(ctx)) await feedPalace("shutdown");
client.stop(); client.stop();
}); });
+38 -6
View File
@@ -24,6 +24,14 @@
# handler resolves. Before the fix it lived until the 20 s deadline. # handler resolves. Before the fix it lived until the 20 s deadline.
# B. mine never answers, deadline 2 s: the deadline is still a liveness # B. mine never answers, deadline 2 s: the deadline is still a liveness
# guard; the handler resolves at about 2 s and node exits. # guard; the handler resolves at about 2 s and node exits.
# And the print-mode default (2026-10-13): `pi -p` / `--mode json` do not feed.
# C. ctx.mode "print": neither agent_settled nor session_shutdown mines.
# D. ctx.mode "json": the same.
# E. ctx.mode "print" + MEMPALACE_FEED_PRINT=1: mines, as before.
# F. ctx.mode "rpc" (long-lived): mines, as before.
# Every case fires agent_settled first and then session_shutdown, the order a
# real one-turn run sees. In the feeding cases the shutdown joins the tick's
# in-flight feed, so a run mines exactly once.
set -euo pipefail set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SRC="${1:-$REPO_ROOT/extensions/pi/mempalace.ts}" SRC="${1:-$REPO_ROOT/extensions/pi/mempalace.ts}"
@@ -93,10 +101,12 @@ PORT="$(cat "$WORK/port")"
[ -n "$PORT" ] || { echo "FAIL: fake palace did not start" >&2; exit 2; } [ -n "$PORT" ] || { echo "FAIL: fake palace did not start" >&2; exit 2; }
# ------------------------------------------------------------------ driver ---- # ------------------------------------------------------------------ driver ----
# Loads the extension under a stub pi, fires session_shutdown, prints when the # Loads the extension under a stub pi, fires agent_settled and then
# session_shutdown with the ctx given in DRIVER_CTX, prints when the shutdown
# handler resolved, then RETURNS. Anything still referenced keeps node alive. # handler resolved, then RETURNS. Anything still referenced keeps node alive.
cat >"$WORK/driver.mjs" <<'DRIVER' cat >"$WORK/driver.mjs" <<'DRIVER'
const t0 = Date.now(); const t0 = Date.now();
const ctx = JSON.parse(process.env.DRIVER_CTX || "{}");
const handlers = {}; const handlers = {};
const pi = new Proxy({}, { const pi = new Proxy({}, {
get: (_t, k) => k === "on" ? (ev, fn) => { (handlers[ev] ||= []).push(fn); } : () => {}, get: (_t, k) => k === "on" ? (ev, fn) => { (handlers[ev] ||= []).push(fn); } : () => {},
@@ -105,7 +115,8 @@ const { default: ext } = await import("./ext.mjs");
await ext(pi); await ext(pi);
if (!handlers.session_shutdown) { console.log("NO_SHUTDOWN_HANDLER"); process.exitCode = 3; } if (!handlers.session_shutdown) { console.log("NO_SHUTDOWN_HANDLER"); process.exitCode = 3; }
else { else {
for (const fn of handlers.session_shutdown) await fn({ type: "session_shutdown", reason: "quit" }, {}); for (const fn of handlers.agent_settled || []) await fn({ type: "agent_settled" }, ctx);
for (const fn of handlers.session_shutdown) await fn({ type: "session_shutdown", reason: "quit" }, ctx);
console.log(`HANDLER_RESOLVED_MS=${Date.now() - t0}`); console.log(`HANDLER_RESOLVED_MS=${Date.now() - t0}`);
} }
process.on("exit", () => console.log(`EXIT_MS=${Date.now() - t0}`)); process.on("exit", () => console.log(`EXIT_MS=${Date.now() - t0}`));
@@ -114,13 +125,14 @@ DRIVER
failures=0 failures=0
check() { if [ "$1" = 0 ]; then echo "ok $2"; else echo "FAIL $2"; failures=$((failures + 1)); fi; } check() { if [ "$1" = 0 ]; then echo "ok $2"; else echo "FAIL $2"; failures=$((failures + 1)); fi; }
run_case() { # $1 mine delay ms, $2 feed mine timeout ms -> sets HANDLER EXIT RC run_case() { # $1 mine delay ms, $2 feed mine timeout ms, $3 ctx JSON, [$4 FEED_PRINT] -> sets HANDLER EXIT RC MINES
curl -fsS "http://127.0.0.1:$PORT/delay/$1" >/dev/null curl -fsS "http://127.0.0.1:$PORT/delay/$1" >/dev/null
: >"$WORK/calls.log" : >"$WORK/calls.log"
set +e set +e
out="$(cd "$WORK" && env -u MEMPALACE_REMOTE_TOKEN -u MEMPALACE_PI_DEVICE \ out="$(cd "$WORK" && env -u MEMPALACE_REMOTE_TOKEN -u MEMPALACE_PI_DEVICE \
MEMPALACE_REMOTE_URL="http://127.0.0.1:$PORT/mcp" MEMPALACE_FEED=1 \ MEMPALACE_REMOTE_URL="http://127.0.0.1:$PORT/mcp" MEMPALACE_FEED=1 \
MEMPALACE_FEED_BIN="$WORK/fake-feed" MEMPALACE_FEED_MINE_TIMEOUT_MS="$2" \ MEMPALACE_FEED_BIN="$WORK/fake-feed" MEMPALACE_FEED_MINE_TIMEOUT_MS="$2" \
DRIVER_CTX="$3" MEMPALACE_FEED_PRINT="${4:-}" \
timeout 60 node --no-warnings driver.mjs 2>"$WORK/driver.err")" timeout 60 node --no-warnings driver.mjs 2>"$WORK/driver.err")"
RC=$? RC=$?
set -e set -e
@@ -130,7 +142,7 @@ run_case() { # $1 mine delay ms, $2 feed mine timeout ms -> sets HANDLER EXIT RC
} }
# A. normal case: the mine wins the race. # A. normal case: the mine wins the race.
run_case 1000 20000 run_case 1000 20000 '{"mode":"tui","hasUI":true}'
echo "[A] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES" echo "[A] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && echo 0 || echo 1)" "A: driver exits cleanly (rc=$RC)" check "$([ "$RC" = 0 ] && echo 0 || echo 1)" "A: driver exits cleanly (rc=$RC)"
check "$([ "$MINES" = 1 ] && echo 0 || echo 1)" "A: shutdown really mined the staged source once (not a vacuous pass)" check "$([ "$MINES" = 1 ] && echo 0 || echo 1)" "A: shutdown really mined the staged source once (not a vacuous pass)"
@@ -139,12 +151,32 @@ check "$([ -n "$EXIT" ] && [ -n "$HANDLER" ] && [ $((EXIT - HANDLER)) -lt 3000 ]
"A: node exits within 3 s of the handler resolving ($(( ${EXIT:-0} - ${HANDLER:-0} )) ms; the 20 s mine deadline must not hold it)" "A: node exits within 3 s of the handler resolving ($(( ${EXIT:-0} - ${HANDLER:-0} )) ms; the 20 s mine deadline must not hold it)"
# B. wedged palace: the deadline must still end the wait. # B. wedged palace: the deadline must still end the wait.
run_case 60000 2000 run_case 60000 2000 '{"mode":"tui","hasUI":true}'
echo "[B] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES" echo "[B] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && echo 0 || echo 1)" "B: driver exits cleanly (rc=$RC)" check "$([ "$RC" = 0 ] && echo 0 || echo 1)" "B: driver exits cleanly (rc=$RC)"
check "$([ -n "$HANDLER" ] && [ "$HANDLER" -ge 1900 ] && [ "$HANDLER" -lt 10000 ] && echo 0 || echo 1)" "B: session_shutdown gives up at the 2 s deadline (${HANDLER:-?} ms)" check "$([ -n "$HANDLER" ] && [ "$HANDLER" -ge 1900 ] && [ "$HANDLER" -lt 10000 ] && echo 0 || echo 1)" "B: session_shutdown gives up at the 2 s deadline (${HANDLER:-?} ms)"
check "$(grep -q 'feed (shutdown) failed: .*2000ms' "$WORK/driver.err" && echo 0 || echo 1)" "B: the timeout is reported on stderr" check "$(grep -Eq 'feed \((tick|shutdown)\) failed: .*2000ms' "$WORK/driver.err" && echo 0 || echo 1)" "B: the timeout is reported on stderr"
check "$([ -n "$EXIT" ] && [ -n "$HANDLER" ] && [ $((EXIT - HANDLER)) -lt 3000 ] && echo 0 || echo 1)" \ check "$([ -n "$EXIT" ] && [ -n "$HANDLER" ] && [ $((EXIT - HANDLER)) -lt 3000 ] && echo 0 || echo 1)" \
"B: node exits within 3 s of the handler resolving ($(( ${EXIT:-0} - ${HANDLER:-0} )) ms; the 60 s server delay must not hold it)" "B: node exits within 3 s of the handler resolving ($(( ${EXIT:-0} - ${HANDLER:-0} )) ms; the 60 s server delay must not hold it)"
# C/D. one-shot modes skip the feed: no mine, and node exits at once.
for m in print json; do
run_case 1000 20000 "{\"mode\":\"$m\",\"hasUI\":false}"
echo "[$m] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && [ "$MINES" = 0 ] && echo 0 || echo 1)" "$m mode: no mine at agent_settled or session_shutdown (mines=$MINES)"
check "$([ -n "$HANDLER" ] && [ "$HANDLER" -lt 900 ] && [ -n "$EXIT" ] && [ $((EXIT - HANDLER)) -lt 3000 ] && echo 0 || echo 1)" \
"$m mode: shutdown does not wait for a feed (${HANDLER:-?} ms) and node exits promptly"
done
# E. the opt-in restores feeding in print mode.
run_case 1000 20000 '{"mode":"print","hasUI":false}' 1
echo "[print+FEED_PRINT=1] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && [ "$MINES" = 1 ] && [ -n "$HANDLER" ] && [ "$HANDLER" -ge 900 ] && echo 0 || echo 1)" \
"print mode + MEMPALACE_FEED_PRINT=1: mines once and shutdown waits for it (mines=$MINES, ${HANDLER:-?} ms)"
# F. RPC mode is long-lived (hasUI true) and keeps feeding.
run_case 1000 20000 '{"mode":"rpc","hasUI":true}'
echo "[rpc] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && [ "$MINES" = 1 ] && echo 0 || echo 1)" "rpc mode: still mines (mines=$MINES)"
if [ "$failures" = 0 ]; then echo "PASS: all assertions held"; else echo "FAIL: $failures assertion(s) failed"; exit 1; fi if [ "$failures" = 0 ]; then echo "PASS: all assertions held"; else echo "FAIL: $failures assertion(s) failed"; exit 1; fi