joakimp/mempalace-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(pi): print and JSON mode skip the feed by default (MEMPALACE_FEED_PRINT=1 to keep it)

Browse Source 
After 2f0edf5 a `pi -p` still waited for one full catch-up feed. lastFeedAt
starts at 0, so the first settled turn feeds, and session_shutdown waits for
that feed. On tor-ms22 this took 45 s (prepare 11 s + mine 31 s) for a 3 s
answer. That feed almost never files the run's own transcript, because a
one-shot run rarely passes the exporter's gate (4 messages, 1000 assistant
chars). What it files is the device's other sessions, and the next interactive
feed re-offers those without duplicates. The source_file is the staged path,
fixed per session UUID. An unchanged file is skipped. For a grown file, its
drawers are purged and then refiled.

Now, when ctx.mode is "print" or "json", neither agent_settled nor
session_shutdown feeds. Without ctx.mode, the bridge falls back to
hasUI === false. A stale ctx keeps the old behaviour. RPC mode is long-lived
and feeds as before. MEMPALACE_FEED_PRINT=1 restores feeding in print and JSON
mode, for a device that only ever runs pi -p.

Measured on tor-ms22 with the workspace copy (-ne -nt --no-session -e):
- pi -p: 2.9 s / 2.8 s
- --mode json: 3.8 s
- -p with MEMPALACE_FEED_PRINT=1: 30.8 s; the probe saw the prepare and race
  timers, so the feed ran.

scripts/test-feed-exit.sh now fires agent_settled and then session_shutdown
with a ctx, and gained cases C (print), D (json), E (print +
MEMPALACE_FEED_PRINT=1) and F (rpc). Against 2f0edf5's source, C and D fail
(mines=1, about 1 s wait). All six scripts/test-*.sh pass.
This commit is contained in:
Joakim Persson
2026-10-13 14:51:37 +02:00
parent 2f0edf51cf
commit f590a3ac5b
3 changed files with 90 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
extensions/pi/README.md
+22 -8
View File
@@ -84,7 +84,9 @@ The bridge feeds this pi installation's own session transcripts into the
palace by itself — no scheduler, no cron, no manual invocation. It fires on
`session_shutdown` (covers quit, `/new`, `/resume`, `/fork`) and on a
debounced `agent_settled` (covers a long session that later crashes, since a
hard kill runs no shutdown handler at all).
hard kill runs no shutdown handler at all). Interactive (TUI) and RPC sessions
feed. Print and JSON mode (`pi -p`, `--mode json`) do not, unless
`MEMPALACE_FEED_PRINT=1`.
The work is split across two processes, and the reason is a hard constraint,
not a style choice: **the palace is single-writer.** A live pi session
@@ -113,6 +115,7 @@ side of the wiring.
| Var | Default | Effect |
|---|---|---|
| `MEMPALACE_FEED` | `1` | Set `0` to disable automatic feeding entirely. |
| `MEMPALACE_FEED_PRINT` | unset | Set `1` to feed in print and JSON mode too (`pi -p`, `--mode json`). They skip the feed by default, because the next interactive session's feed files the same sessions without duplicates. See [Debugging](#pi--p-prints-its-answer-then-takes-minutes-to-exit). |
| `MEMPALACE_FEED_BIN` | `mempalace-pi-session` | Helper to run. |
| `MEMPALACE_FEED_WING` | `wing_conversations` | Target wing — passed to both the exporter and the `mempalace_mine` call. |
| `MEMPALACE_FEED_DEBOUNCE_MS` | `600000` (10 min) | Minimum gap between mid-session (`agent_settled`) feeds. Bounds crash loss to one window instead of a whole session. |
@@ -586,13 +589,24 @@ Fixed 2026-10-13: the timer is cleared when the race settles.
`session_shutdown` resolving, both when the mine answers and when it times
out. Before the fix the first case held node for the whole deadline.
**What is still expected.** The first settled turn of every session runs a
catch-up feed (`lastFeedAt` starts at 0), and `session_shutdown` waits for it.
A `pi -p` therefore still exits only after one full feed: measured at 45 s on
tor-ms22, of which `--prepare` took 11 s and the mine 31 s. A script that
wants `pi -p` to return at once, and does not need this run's transcript
filed now, sets `MEMPALACE_FEED=0`. The next interactive session's feed picks
up the transcript, if it qualifies.
**Print and JSON mode skip the feed (also 2026-10-13).** With only the timer
fix, a `pi -p` still waited for one full feed. The first settled turn of every
session runs a catch-up feed (`lastFeedAt` starts at 0), and `session_shutdown`
waits for it: 45 s on tor-ms22, of which `--prepare` took 11 s and the mine
31 s, for a 3 s answer. That feed almost never files the run's own transcript,
because a one-shot run rarely passes the exporter's quality gate (4 messages,
1000 assistant characters). What it files is the device's *other* sessions, and
the next interactive feed re-offers those anyway. So when `ctx.mode` is
`print` or `json`, the bridge now feeds neither on `agent_settled` nor on
`session_shutdown`. Measured on tor-ms22: `pi -p` 2.9 s and `--mode json`
3.8 s, against 45 s before; `MEMPALACE_FEED_PRINT=1` gives 30.8 s (a smaller
backlog than the 45 s run). RPC mode is long-lived and feeds as before.
Nothing is filed twice when a later session re-offers a transcript. The
`source_file` is the staged path, which is fixed per pi session UUID. The mine
skips an unchanged file, and for a grown file it purges that file's drawers
before refiling. A device that only ever runs `pi -p` sets
`MEMPALACE_FEED_PRINT=1`, or its transcripts are never fed.
## The `Type.Unsafe` gotcha
extensions/pi/mempalace.ts
+30 -3
View File
@@ -33,6 +33,8 @@
* client also means it automatically targets whichever palace this bridge
* is pointed at (local stdio or a shared remote one).
* - MEMPALACE_FEED=0 disable feeding entirely
* - MEMPALACE_FEED_PRINT=1 also feed in print/json mode (`pi -p`,
* `--mode json`), which skip it by default
* - MEMPALACE_FEED_BIN helper to run (default mempalace-pi-session)
* - MEMPALACE_FEED_WING target wing (default wing_conversations)
* - MEMPALACE_FEED_DEBOUNCE_MS min gap between mid-session feeds (default 600000)
@@ -1174,20 +1176,45 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
return feedInFlight;
}
// Print and JSON mode (`pi -p`, `--mode json`) do NOT feed unless
// MEMPALACE_FEED_PRINT=1. A one-shot run's own transcript almost never meets
// the exporter's quality gate (MIN_MESSAGES=4, MIN_ASSISTANT_CHARS=1000), so
// what such a run would file is the device's OTHER sessions. That is a
// catch-up, because lastFeedAt starts at 0, and session_shutdown waits for
// it: 45 s on tor-ms22 (prepare 11 s + mine 31 s) for a 3 s answer.
// Skipping loses nothing. The source_file is the staged path, fixed per
// session UUID, and the next interactive feed re-offers every qualifying
// session: an unchanged file is skipped, a grown one is purged and refiled.
// RPC mode is long-lived and feeds as before. A pi whose ctx has no `mode`
// falls back to hasUI, which is false only in print/json mode.
const feedInPrintMode = process.env.MEMPALACE_FEED_PRINT === "1";
const isOneShot = (ctx: unknown): boolean => {
try {
const c = ctx as { mode?: string; hasUI?: boolean } | undefined;
const mode = c?.mode;
if (mode === "print" || mode === "json") return true;
return mode === undefined && c?.hasUI === false;
} catch {
return false; // a stale ctx throws on access: keep the old behaviour
}
};
const feedsThisRun = (ctx: unknown): boolean => feedInPrintMode || !isOneShot(ctx);
// Mid-session feed. A hard container kill runs no handler at all, so this is
// what bounds crash loss to one debounce window instead of a whole session.
// Re-mining a grown transcript purges and refiles that source_file, so
// repeated ticks refresh a session's drawers rather than duplicating them.
pi.on("agent_settled", async () => {
pi.on("agent_settled", async (_event, ctx) => {
if (!feedsThisRun(ctx)) return;
if (Date.now() - lastFeedAt < feedDebounceMs) return;
void feedPalace("tick"); // deliberately not awaited: never stall a turn
});
pi.on("session_shutdown", async () => {
pi.on("session_shutdown", async (_event, ctx) => {
// Feed before stopping the client: we are the palace holder, so nothing
// else can mine while we live. pi awaits this handler, so the mine really
// does complete; feedMineTimeoutMs keeps a wedged palace from hanging exit.
await feedPalace("shutdown");
if (feedsThisRun(ctx)) await feedPalace("shutdown");
client.stop();
});