joakimp/mempalace-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: joakimp/mempalace-toolkit:e2b060a94054face15cd15ac5e5314ead215986b
Branches Tags
joakimp/mempalace-toolkit:main
...
compare: joakimp/mempalace-toolkit:main
Branches Tags
joakimp/mempalace-toolkit:main

10 Commits
e2b060a940 ... main

Author SHA1 Message Date
joakimp d5a03f4ff5 feat(pi): print and JSON mode skip the wake-up and mailbox poll (MEMPALACE_WAKEUP_PRINT=1 to keep them) 
In print and JSON mode, `pi -p` and `--mode json` no longer get the
before_agent_start wake-up injection (status, diary and the tier-1 mailbox),
and the mid-session mailbox poll is off as well. The poll has its own guard,
not just `wokeUp`, because `pi -p -c` resumes and a resume sets wokeUp. The
client still connects, and the mempalace_* tools stay registered.
MEMPALACE_WAKEUP_PRINT=1 restores both. TUI and RPC sessions are unchanged.

Why: a one-shot run is a script asking one question. The wake-up cost it 12
palace calls and a large orientation message, and it is meant for the
operator of an interactive session. It can also do harm: on tor-ms22 a flash
model given "Reply with exactly: PONG" read the injected diary's open items
as its task, ran about 12 tool calls, and filed a wrong drawer in the shared
palace.

Measured on tor-ms22 (workspace copy, -ne -nt --no-session -e) with a fetch
probe that re-wraps pi's own fetch. The earlier preload-only fetch wrapper had
logged 0 lines in every run, because pi replaces globalThis.fetch.
- pi -p: 3 palace requests (initialize, initialized, tools/list), 1.6 s.
- --mode json: the same requests, 1.0 s, and no mempalace-wakeup in stdout.
- --mode json with MEMPALACE_WAKEUP_PRINT=1: 15 requests, 12 of them
  tools/call (status 1, diary 1, event_list 10), 3.6 s, and the wake-up is in
  stdout.

New scripts/test-wakeup-mode.sh has these cases: tui, print, json, print
resumed, tui resumed (a control that shows the poll does run), print with
MEMPALACE_WAKEUP_PRINT=1, and rpc. f590a3a's source fails the print, json and
print-resumed cases, with 12, 12 and 5 calls. All seven scripts/test-*.sh
pass.
 2026-10-13 15:32:07 +02:00
joakimp f590a3ac5b feat(pi): print and JSON mode skip the feed by default (MEMPALACE_FEED_PRINT=1 to keep it) 
After 2f0edf5 a `pi -p` still waited for one full catch-up feed. lastFeedAt
starts at 0, so the first settled turn feeds, and session_shutdown waits for
that feed. On tor-ms22 this took 45 s (prepare 11 s + mine 31 s) for a 3 s
answer. That feed almost never files the run's own transcript, because a
one-shot run rarely passes the exporter's gate (4 messages, 1000 assistant
chars). What it files is the device's other sessions, and the next interactive
feed re-offers those without duplicates. The source_file is the staged path,
fixed per session UUID. An unchanged file is skipped. For a grown file, its
drawers are purged and then refiled.

Now, when ctx.mode is "print" or "json", neither agent_settled nor
session_shutdown feeds. Without ctx.mode, the bridge falls back to
hasUI === false. A stale ctx keeps the old behaviour. RPC mode is long-lived
and feeds as before. MEMPALACE_FEED_PRINT=1 restores feeding in print and JSON
mode, for a device that only ever runs pi -p.

Measured on tor-ms22 with the workspace copy (-ne -nt --no-session -e):
- pi -p: 2.9 s / 2.8 s
- --mode json: 3.8 s
- -p with MEMPALACE_FEED_PRINT=1: 30.8 s; the probe saw the prepare and race
  timers, so the feed ran.

scripts/test-feed-exit.sh now fires agent_settled and then session_shutdown
with a ctx, and gained cases C (print), D (json), E (print +
MEMPALACE_FEED_PRINT=1) and F (rpc). Against 2f0edf5's source, C and D fail
(mines=1, about 1 s wait). All six scripts/test-*.sh pass.
 2026-10-13 14:51:37 +02:00
joakimp 2f0edf51cf fix(pi): clear the feed's mine-race timer, so pi -p exits when the mine returns 
feedPalace() raced mempalace_mine against a bare setTimeout(reject,
feedMineTimeoutMs) and never cleared it. When the mine won (the normal case)
the losing 300 000 ms timer stayed referenced. Interactive and RPC mode call
process.exit() and never showed it. Print mode (`pi -p`, `--mode json`) returns
and waits for an empty event loop, so every `pi -p` on a devbox printed its
answer and then idled for the rest of the five minutes.

Measured on tor-ms22 (pi 1.1.0, NODE_OPTIONS=--import probe):
- before: PONG at 4 s, exit at 315.1 s. The 300 000 ms timer from
  mempalace.ts:1144 (created +15.1 s) was the ONLY active resource from +50 s on.
- after:  PONG at 3 s, exit at 45.2 s, the moment the mine's socket closed.
- MEMPALACE_FEED=0: 3.8 s.
pi 1.0.0 behaved the same as 1.1.0, so this is not a pi regression.

The timer is now cleared in a finally once the race settles. It stays
referenced while it runs because it is the liveness guard.

scripts/test-feed-exit.sh loads the type-stripped SHIPPED extension under a
stub pi with a fake palace in a separate process and fires session_shutdown:
- A: the mine answers in 1 s with a 20 s deadline. Node must exit within 3 s
  of the handler resolving. The pre-fix source FAILS here (19 000 ms).
- B: the mine never answers, 2 s deadline. The deadline still ends the wait,
  and the failure is logged.
All five scripts/test-*.sh pass.

README Debugging section: the remaining expected wait (one catch-up feed per
session, ~45 s here) and MEMPALACE_FEED=0 for scripts that want an instant
return.
 2026-10-13 13:55:55 +02:00
joakimp a30af61883 pi bridge: RemoteMcpClient is now the only copy (pi-extensions retired mcp-loader.ts) 
pi-extensions 750b2a8 removed mcp-loader.ts in favour of pi's built-in MCP
support (builtin:mcp, ~/.pi/agent/mcp.json). The bridge's RemoteMcpClient was
vendored from that file and kept in step by an MCP-STREAMABLE-HTTP-CLIENT-SYNC
token, checked by scripts/check-mcp-client-sync.sh. With the canonical copy
gone the guard could only ever SKIP (or, on images that still bake
/opt/pi-extensions/extensions/mcp-loader.ts, compare against a dead file), so
it is removed rather than left looking like a check.

Comment and README only; no code change. The bridge still type-strips and
imports (node 24), and evaluateDormancy is exported as before. The README
records why MemPalace itself does not move to the built-in: its wake-up,
mailbox, feed and stamping need their own client, and built-in tools would be
mcp__mempalace__* with codemode exposure by default.
 2026-10-13 00:17:02 +02:00
Joakim Persson 975ab92943 feat(pi): let an ask declare dormancy, so waiting work stops nagging 
A first-boot acceptance ask is planted deliberately unanswerable: it describes
work that becomes possible only when the device is next recreated, and it must
STAY owed until then, because closing it early to tidy the mailbox is exactly how
that work gets lost. deriveOwed could only see "directed, open, not answered, not
withdrawn", so such an ask was announced at every session start and every poll
for as long as it was correctly waiting -- measured at three days running on
emb-7kj4vr4g (2026-09-28 -> 2026-10-01), and across three consecutive releases
before that. The ask was right; announcing it was wrong, and the cost landed on
the human reading the window, who is the one reader that cannot filter it.

An ask may now declare the condition under which it is merely waiting:

  "dormant_unless": [
    { "kind": "json_field", "path": "/etc/pi-devbox/build-manifest.json",
      "field": "release_tag", "baseline": "v1.9.4" },
    { "kind": "file_mtime", "path": "/etc/hostname",
      "baseline": "2026-09-22T18:12:49Z" }
  ]

Dormant while EVERY condition still matches its baseline; live the moment ANY
differs -- which is the trigger those asks already stated in prose ("act when
EITHER differs"), now in a form the bridge can check. Two kinds, local files
only, no expression language, no shell, no network: a general evaluator in the
path that decides whether work is VISIBLE is a far worse trade than a clumsy
schema.

DORMANCY IS PROVEN, NEVER ASSUMED. Every unevaluable predicate announces the ask
instead of hiding it: missing file, unreadable file, unparseable baseline,
unknown kind, vanished field, relative path, more than eight conditions. The
dangerous failure here is not a spurious nag but work that disappears because a
predicate could not be evaluated -- indistinguishable from the ask being lost,
and not surfacing until a release needed it. An ask with no dormant_unless
behaves exactly as before, so this is backward compatible by construction.

Withheld from the ANNOUNCEMENT, never from the mailbox: deriveOwed now returns a
partition {owed, dormant} rather than a flat list, the wake-up injection lists
dormant asks once per session with ids, and a mid-session poll adds only a count
and only when the window is already open for something else. Dormant asks are
deliberately NOT added to the `surfaced` map, so one becomes announceable the
instant its baseline moves.

file_mtime compares WHOLE SECONDS in UTC. A filesystem mtime carries sub-second
residue (measured: /etc/hostname at .773761009) that a reported ISO baseline
never will, so comparing raw milliseconds would mark every such predicate
permanently "changed" -- silently disabling the feature while appearing to work.
The test records the residue for that reason.

scripts/test-dormancy.sh is this repo's first test: 22 cases, positive and
negative arms both, because a predicate that never fires makes the feature inert
and one that fires too eagerly hides real work. It copies the extension into a
temp tree with pi's typebox symlinked beside it (the copy is made per-run, so it
cannot drift like a vendored duplicate), owns its own fixtures rather than
reading /etc paths -- the first draft passed only on a pi-devbox container and
would have silently flipped to "not dormant" anywhere else -- and exits 2 for
INCONCLUSIVE rather than 0, since a test that skips quietly is the failure mode
it exists to catch. shellcheck clean.
 2026-10-01 23:59:36 +02:00
joakimp 2167a1b033 fix(mailbox): pass order: "desc" on every cursor-less event_list call 
Three of the five `mempalace_event_list` calls in the pi extension relied on
the server's default ordering: deriveOwed's `status: "open"` candidate query
(limit 50) and both windows in deriveClosed (from_agent limit 100, to_agent
limit 50). deriveOwed's other two calls already say `order: "desc"` and carry
the comment explaining why: on mempalace <= 3.9.0 the default is `asc`, so a
cursor-less `limit: N` returns the OLDEST N events, and once a device passes N
authored events its newest asks and the replies that close them fall outside
the join window. deriveClosed had exactly that latent truncation.

Why now: mempalace 3.10.0 (2026-09-15) flips the cursor-less default to
newest-first ("Logstream listings default to the newest events when no cursor
is given"). That change is server-side -- the extension talks to the fleet
hub over MEMPALACE_REMOTE_URL, so it lands when the hub upgrades, not when a
client does -- and it would have silently FIXED deriveClosed on 3.10.0 while
leaving it broken against any 3.9.0 hub. A mailbox verdict that depends on
which server version answers is the wrong shape. Saying `order` explicitly
makes both derive* functions read the same window on either version.

The selection in deriveClosed (isStrictlyAfter per correlation) is
order-independent, so this changes WHICH events are in the window, not how
the winner is picked. No behaviour change on a device with < 50 inbound and
< 100 authored events; the fleet hub is at 176 events total today, so the
window was about to matter.

Verified: file compiles under pi's own loader (jiti 2.7.0 from the installed
pi-coding-agent) before and after; `order:"desc"` literal count 3 -> 7, which
is the 3 added arguments plus 1 comment mention. `node --check` and a bare
`tsc` were tried first and both fail identically on HEAD (inline `type`
imports / no @types/node) -- checker faults, not this change.
 2026-09-22 14:15:22 +02:00
joakimp 817b3a82b7 fix(feed): the mine's deadline never reached the transport; 60 s cut it off 
The 2026-09 change raised MEMPALACE_FEED_MINE_TIMEOUT_MS to 300 000 and
raced it against client.callTool("mempalace_mine"). But callTool() had no
way to carry a deadline, so every call went out under the transport's
generic per-request timeout (MEMPALACE_MCP_TIMEOUT_MS, 60 000), which fired
first on every honest 60 s+ mine. Operators saw

    feed (tick) failed: mempalace remote request 'tools/call' failed:
    timed out after 60000ms

instead of the message the change had aimed at, and the 300 s was
unreachable. On stdio it was worse than noise: that transport kills the
server child on timeout, so the mine was actually aborted at 60 s.

callTool(name, args, { timeoutMs }) now passes a per-call deadline to both
transports; feedPalace() uses it for the mine. Plain calls keep the short
default — a query taking 60 s is still wedged. The Promise.race stays as the
liveness guard for a transport with its timeout disabled (0).

scripts/test-mcp-call-timeout.sh cuts RemoteMcpClient out of the shipped
file (as test-owed-withdrawal.sh does for the mailbox predicates), drives it
against a local JSON-RPC server that delays tools/call, and asserts: plain
call rejects at the generic deadline; the override outlives it; the override
is itself a deadline. Fails on the previous commit (2 of 6), passes here.
Vendored-copy delta noted in the header; protocol untouched, sync token
unchanged (check-mcp-client-sync.sh passes against pi-extensions).
 2026-09-18 17:04:59 +02:00
joakimp dab989b068 fix(mailbox-tests): the owed-set suite's own gate refused to let it run on node 24 
scripts/test-owed-withdrawal.sh has not executed a single assertion since the
image moved to node 24. Its precondition line

    node --experimental-strip-types --check "$SRC"

exits 2 on an unmodified extensions/pi/mempalace.ts, and the script is
`set -euo pipefail` with `|| exit 2`, so all 17 assertions and both regression
guards were skipped. Measured on pi-devbox v1.9.1, node v24.21.0.

CAUSE, MEASURED AND NARROWER THAN IT LOOKS. The reported error names the inline
type-import at mempalace.ts:92, which invites the reading "that import is
unusual". It is not the import: `node --check` does not type-strip AT ALL. A file
whose entire content is `const x: number = 1;` fails identically, with or without
--experimental-strip-types, while `node --experimental-strip-types <same file>`
executes it fine. --check has never been type-aware; execution is what gained
stripping. So this gate could never validate TypeScript, on any node.

WHY IT USED TO PASS -- INFERENCE, NOT MEASUREMENT. e2b060a's message records this
suite running on 2026-09-09 with a control pass and four mutation kills, when the
image shipped node v22.23.2; v1.9.1 ships v24.21.0. No node 22 exists on the box
where this was diagnosed, so the counterfactual was not executed. Treat "22
stripped for --check and 24 stopped" as a hypothesis consistent with the record,
not as a measured cause. What IS measured is the present-tense behaviour above.

WHY THIS MATTERS MORE THAN A RED TEST. This suite exists because isWithdrawn is
the one rule in the extension that can go wrong SILENTLY -- a wrong rule does not
throw and does not log, it makes a real unanswered ask vanish from a mailbox
forever. The rule shipped in e2b060a and has been baked since v1.9.1; the thing
that makes its failure mode visible has been dark for the same period. The
mitigating half: it failed CLOSED (exit 2, loud), never vacuously green. A gate
that cannot run must not pass, and it did not.

THE FIX. Strip first, then syntax-check the emitted JS: version-stable, and it
still refuses malformed input. `mode: "strip"` blanks type syntax without moving
anything, so offsets and line numbers survive and a reported error line still
points at the right line of the original .ts (69157 B in, 69157 B out).

THE EXIT CODES ARE NOW SPLIT, AND THAT IS THE POINT. 3 = the gate itself cannot
run (no module.stripTypeScriptTypes, i.e. node < 22.13). 2 = the source does not
parse. Collapsing the two is how this defect disguised itself: it printed
"mempalace.ts does not parse" while mempalace.ts was fine, sending a reader to
inspect the wrong file. Note the stripper is itself a parser, so a genuine syntax
error surfaces as an exception from the strip call rather than from --check; that
path is caught and reported as 2, not 3. Both remain failures. Neither passes.

VERIFIED IN SIX DIRECTIONS on this image, each expectation written down first:
  unmutated source          -> rc=0, PASSED (17/17)
  malformed TypeScript      -> rc=2, "does not parse: Expression expected"
  stripper made unavailable -> rc=3, "cannot strip", and NOT "does not parse"
                               (simulated by doctoring the runtime through
                               NODE_OPTIONS, so the shipped line ran as shipped)
  M1 marker requirement removed -> FAILED: 3   (no-marker, other-thread, prose)
  M2 third-party guard removed  -> FAILED: 1
  M3 to_agent guard removed     -> FAILED: 2   (broadcast and wrong-device share it)
  M4 ordering guard removed     -> FAILED: 1
The four kill counts are the same ones e2b060a recorded, so sensitivity is
restored rather than merely asserted.

WORTH KNOWING FOR THE NEXT PERSON WHO MUTATES THIS: the extractor carries its own
guard that rejects an isWithdrawn which no longer mentions "withdraws", so the
obvious M1 (delete the marker check outright) is refused before any assertion
runs -- correctly, but it looks like a crash. Mutate to `... || true` instead,
which removes the requirement while keeping the key mentioned.

SC2016 is disabled on the node invocation with a stated reason: the single quotes
are deliberate, the payload is JavaScript and `${process.version}` must reach
node rather than the shell. Checked that the file is shellcheck-clean at default
severity, as it was before this change, and at the -S error severity the
pi-devbox gate uses.

NOT FIXED HERE. Nothing in CI runs this suite -- it is a script an operator
invokes, which is precisely why a gate that fails loudly still went unnoticed
across a node bump. The harness's own runner two hundred lines below still passes
--experimental-strip-types, deliberately: it is a no-op on 24 and required on 22,
so it keeps the suite runnable on both. And the real reason this was found at all
is unrelated to CI: isWithdrawn was being exercised end-to-end on a released
image against the live logstream for the first time (project/pi-devbox seq 140),
and the suite was reached for as corroboration.
 2026-09-14 16:27:29 +02:00
Joakim Persson e68ee2071c docs(pi-ext): document what the mine deadline does, and what the message means 
Two corrections to the operator-facing docs, both exposed by 309980b.

The env table listed the default as 30000, which is now wrong, and described the
var as capping "the mempalace_mine call". It never did: it bounds how long the
extension WAITS. The mine keeps running on the server. That exact misreading is
what made a 30s deadline look safe on a call measured at 30-60s.

Added a Debugging entry for "feed (tick) failed: mine timed out after ...ms",
because every operator on this fleet has seen it and it was documented nowhere.
It states the three things a reader needs: nothing was lost (the transcript is
staged before the mine, and mine --mode convos dedups by source_file and is
idempotent); do NOT retry harder from the client, because the palace is a single
writer and a blind retry turns one slow mine into a queue; and after 309980b the
message should not appear on a healthy fleet, so if it does it now MEANS
something -- a mine exceeding five minutes, i.e. look at palace size or another
writer holding the lock rather than raising the timeout again.
 2026-09-10 20:58:12 +02:00
Joakim Persson 309980b62c fix(pi-ext): stop the feed tick from launching overlapping mines 
"[mempalace ext] feed (tick) failed: mine timed out after 30000ms" was parked as
cosmetic on 2026-08-27. It is not cosmetic: the tight deadline was the trigger,
but the defect is a positive feedback loop that puts multiple writers on a
single-writer palace.

lastFeedAt was assigned only AFTER a successful await. The Promise.race abandons
our WAIT and cannot cancel the server's work, so on a mine that takes longer
than the deadline -- measured at 30-60s in normal operation, against a 30s
deadline -- the catch ran with lastFeedAt UNCHANGED. That left both guards in
the agent_settled handler open at once: the debounce test
(`Date.now() - lastFeedAt < feedDebounceMs`) passed because lastFeedAt was still
stale, and feedInFlight was already null because `run` had settled. Every
subsequent settled turn therefore launched another mine on top of the one still
running, each making the next slower and the next timeout likelier -- which is
why operators saw the message many times per session instead of at most once per
10-minute debounce window.

Fix, three lines:
- move `lastFeedAt = Date.now()` to before the await, so a timeout still starts
  the debounce clock. A timeout is not a "did not happen": the mine is running
  server-side and `mine --mode convos` dedups by source_file and is idempotent.
- raise MEMPALACE_FEED_MINE_TIMEOUT_MS from 30_000 to 300_000. The mine is the
  slowest thing this extension does yet carried the tightest deadline: 4x
  tighter than the prepare step before it (120_000) and 10x tighter than the
  init handshake (300_000), a fast call. All three were introduced together in
  29e660e and this one was never revisited. 300_000 matches the init timeout
  because liveness is the only job left for this deadline -- it cannot cancel
  the server's work, so it must sit far above the slowest honest completion.

Simulated both guards over 10 minutes of settled turns at 20s intervals with a
60s mine: BEFORE 16 mines launched, 15 of them overlapping an already-running
mine; AFTER 2 launched, 0 overlapping. With a mine that exceeds even the new
deadline (400s): BEFORE 16/15, AFTER still 2/0 -- the lastFeedAt move is what
actually fixes it, and it holds even when the timeout still fires. The raise
stops the spurious message; the move stops the pile-up.

NOT fixed here: the message still goes to process.stderr, which pi renders into
the TUI input field. That needs a pi-side channel or a log file, and is tracked
separately. After this change the message should be rare, and when it does
appear it means something real: a mine exceeding five minutes.
 2026-09-10 20:48:33 +02:00
8 changed files with 1217 additions and 138 deletions
Expand all files Collapse all files
extensions/pi/README.md
+173 -7
View File
@@ -38,6 +38,9 @@ dependencies (~300 MB).
injects the result as a `mempalace-wakeup` system message so the
agent orients itself the way `~/.agents/skills/mempalace/SKILL.md`
describes. Skipped on resume/fork (context is already in the thread).
Also skipped in print and JSON mode (`pi -p`, `--mode json`), together
with the mid-session mailbox poll, unless `MEMPALACE_WAKEUP_PRINT=1`.
See [Debugging](#pi--p-prints-its-answer-then-takes-minutes-to-exit).
4. **Automatic transcript feeding** (`session_shutdown`, and a debounced
`agent_settled`): stages + mines this pi installation's own session
transcripts into the palace with no user action needed — **as of
@@ -84,7 +87,9 @@ The bridge feeds this pi installation's own session transcripts into the
palace by itself — no scheduler, no cron, no manual invocation. It fires on
`session_shutdown` (covers quit, `/new`, `/resume`, `/fork`) and on a
debounced `agent_settled` (covers a long session that later crashes, since a
hard kill runs no shutdown handler at all).
hard kill runs no shutdown handler at all). Interactive (TUI) and RPC sessions
feed. Print and JSON mode (`pi -p`, `--mode json`) do not, unless
`MEMPALACE_FEED_PRINT=1`.
The work is split across two processes, and the reason is a hard constraint,
not a style choice: **the palace is single-writer.** A live pi session
@@ -113,11 +118,13 @@ side of the wiring.
| Var | Default | Effect |
|---|---|---|
| `MEMPALACE_FEED` | `1` | Set `0` to disable automatic feeding entirely. |
| `MEMPALACE_FEED_PRINT` | unset | Set `1` to feed in print and JSON mode too (`pi -p`, `--mode json`). They skip the feed by default, because the next interactive session's feed files the same sessions without duplicates. See [Debugging](#pi--p-prints-its-answer-then-takes-minutes-to-exit). |
| `MEMPALACE_WAKEUP_PRINT` | unset | Set `1` to keep the wake-up injection and the mid-session mailbox poll in print and JSON mode. They are skipped there by default. The `mempalace_*` tools stay registered either way. |
| `MEMPALACE_FEED_BIN` | `mempalace-pi-session` | Helper to run. |
| `MEMPALACE_FEED_WING` | `wing_conversations` | Target wing — passed to both the exporter and the `mempalace_mine` call. |
| `MEMPALACE_FEED_DEBOUNCE_MS` | `600000` (10 min) | Minimum gap between mid-session (`agent_settled`) feeds. Bounds crash loss to one window instead of a whole session. |
| `MEMPALACE_FEED_PREPARE_TIMEOUT_MS` | `120000` | Kills a wedged `--prepare` subprocess. |
| `MEMPALACE_FEED_MINE_TIMEOUT_MS` | `30000` | Caps the `mempalace_mine` call so a stalled palace can't hang session exit. |
| `MEMPALACE_FEED_MINE_TIMEOUT_MS` | `300000` (5 min) | Bounds how long the extension *waits* for `mempalace_mine`, so a stalled palace can't hang session exit. It does **not** cancel the mine — see [Debugging](#debugging). Raised from `30000` in 2026-09: the mine is the slowest call this extension makes (30–60s in normal operation), so the old deadline fired routinely and reported healthy behaviour as an error. |
**Remote palace:** if `$MEMPALACE_REMOTE_URL` is set (see
[Transport](#transport-local-vs-external)), `mempalace_mine`'s source path is
@@ -211,12 +218,16 @@ chosen at load time:
also pins `Host`/`Origin`, so a request arriving under the wrong hostname is
refused even when the port is open.
Implementation note: the HTTP client (`RemoteMcpClient`) is **vendored** from
Implementation note: the HTTP client (`RemoteMcpClient`) was **vendored** from
[`pi-extensions`](https://gitea.jordbo.se/joakimp/pi-extensions)'
`mcp-loader.ts`. A `MCP-STREAMABLE-HTTP-CLIENT-SYNC` token keeps the two
copies from drifting — [`scripts/check-mcp-client-sync.sh`](../../scripts/check-mcp-client-sync.sh)
fails if they diverge (it skips gracefully when the `pi-extensions` checkout
isn't present).
`mcp-loader.ts`. That loader was retired on 2026-10-12 (pi-extensions
`750b2a8`) in favour of pi's built-in MCP support, so this copy is now the
only one. The sync token and `scripts/check-mcp-client-sync.sh` that guarded
the pair were removed with it. MemPalace itself does **not** move to the
built-in: the bridge's wake-up, mailbox, feed and provenance stamping need
their own client, and built-in MCP tools would be renamed
`mcp__mempalace__*` with `codemode` exposure (not declared to the model) by
default.
## Fail-soft
@@ -378,6 +389,57 @@ the one failure this derivation exists to prevent.
protocol says a broadcast owes nobody a reply — which also means broadcasting an
ask demonstrably reaches no owed set, giving that documented anti-pattern teeth.
**Dormant asks — open, owed, and deliberately not announced.** Some asks are
planted *unanswerable on purpose*: a first-boot acceptance describes work that
becomes possible only when the device is next recreated, and it must stay owed
until then, because closing it early to tidy the mailbox is exactly how the work
gets lost. Derivation alone cannot tell that apart from neglected work, so such
an ask was announced at every session start and every poll for as long as it was
correctly waiting — measured at three days running on `emb-7kj4vr4g`
(2026-09-28 → 2026-10-01), and across three consecutive releases before that.
The ask was right; announcing it was wrong, and the cost landed on the human
reading the window, the one reader who cannot filter it.
An ask may therefore declare, in its own `metadata`, the condition under which it
is merely waiting:
```json
"dormant_unless": [
{ "kind": "json_field", "path": "/etc/pi-devbox/build-manifest.json",
"field": "release_tag", "baseline": "v1.9.4" },
{ "kind": "file_mtime", "path": "/etc/hostname",
"baseline": "2026-09-22T18:12:49Z" }
]
```
It is dormant while **every** condition still matches its baseline, and goes live
the moment **any** of them differs — the trigger such asks already stated in
prose ("act when EITHER differs"), now in a form the bridge can check. Two kinds,
both local-file-only: `file_mtime` (compared at whole seconds in UTC, because a
filesystem mtime carries sub-second residue that a reported ISO baseline never
will — comparing raw milliseconds would mark every such predicate permanently
"changed" and silently disable the feature) and `json_field` (dotted paths
allowed, compared as strings so a manifest holding `3` matches a baseline of
`"3"`). There is no expression language, no shell and no network: a general
evaluator in the path that decides whether work is *visible* is a far worse trade
than a clumsy schema.
Dormancy is **withheld from the announcement, never from the mailbox**: the
wake-up injection lists dormant asks once per session with their ids, and a
mid-session poll mentions only a count, and only when the window is already open
for something else. They are not added to the resurface map, so one becomes
announceable the instant its baseline moves.
**Dormancy must be proven, never assumed** — every unevaluable predicate shows
the ask as owed. A missing file, an unreadable one, a baseline that will not
parse, an unknown `kind`, a vanished field, a relative path, more than eight
conditions: each announces. The dangerous failure here is not a spurious nag but
work that disappears because a predicate could not be evaluated, which would be
indistinguishable from the ask being lost and would not surface until a release
needed it. An ask with no `dormant_unless` behaves exactly as it did before the
feature existed. `scripts/test-dormancy.sh` exercises all of it, positive and
negative arms both.
Gated on `MEMPALACE_PI_DEVICE` **and** `MEMPALACE_REMOTE_URL` (the same pair as
the stamper, since an unstamped client has no address to be reached at), and
disabled outright with `MEMPALACE_MAILBOX=0` (notifications alone with
@@ -423,6 +485,9 @@ one — the long-lived server is only killed when a request genuinely stalls.
- `MEMPALACE_MCP_TIMEOUT_MS` — tool-call/request timeout. Default `60000`.
Kept short on purpose: a *query* taking this long is genuinely wedged.
The one call that is not a query — the feed's `mempalace_mine` — passes its
own deadline (`MEMPALACE_FEED_MINE_TIMEOUT_MS`) down to the transport per
call, so this default does not apply to it (since 2026-09-18; see below).
- `MEMPALACE_MCP_INIT_TIMEOUT_MS` — `initialize` + `tools/list` handshake
timeout. Default `300000`. Deliberately generous: a genuine first
cold-open over virtiofs can legitimately take minutes, and killing a
@@ -465,6 +530,107 @@ the next tool call transparently respawns `mempalace-mcp` and retries.
`mempalace-mcp` manually with raw JSON-RPC on stdin to read the
server-side error — much faster than guessing.
### `feed (tick) failed: mine timed out after …ms`
**Nothing has been lost.** The deadline bounds only how long the extension
*waits*; it cannot cancel the mine, which continues on the server. The
transcript is already staged before the mine is invoked, and
`mempalace mine --mode convos` dedups by `source_file` and is idempotent, so the
work either completed after the deadline or is redone by the next tick.
**Do not "fix" it by retrying harder from the client.** The palace is a single
writer; a blind retry is what turns one slow mine into a queue of them.
Before 2026-09 this message appeared many times per session, which made it look
like a persistent failure. That was a real defect, now fixed: `lastFeedAt` was
recorded only after a *successful* wait, so a timeout left the debounce clock
stale and every following settled turn started another mine on top of the one
still running. Two changes — recording the attempt before the wait, and raising
the deadline to sit far above the slowest honest completion — mean a healthy
fleet should now never see it.
If you *do* still see it, it is now informative rather than noise: a mine
exceeded five minutes. Check palace size and whether another writer (a
host-side feeder, a scheduled `mine`) is holding the write lock, rather than
raising the timeout again.
### `feed (tick) failed: mempalace remote request 'tools/call' failed: timed out after 60000ms`
Same event, different deadline — and the same reassurance: **nothing has been
lost**, the mine continues server-side.
This is what the previous message turned into after the 2026-09 change, and
it exposed that the change was incomplete. The feed's five-minute deadline was
only *raced* against the call; `callTool()` had no way to carry it, so the
transport's generic per-request timeout (`MEMPALACE_MCP_TIMEOUT_MS`, 60 s)
fired first on every honest 60 s+ mine, and the 300 s was unreachable. On the
stdio transport this was worse than noise: a per-request timeout there kills the
server child, so the mine really was aborted at 60 s.
Fixed 2026-09-18: `callTool(name, args, { timeoutMs })` passes a per-call
deadline to both transports and the feed uses it for the mine.
`scripts/test-mcp-call-timeout.sh` pins the contract (a plain call still honours
the short default; the override is honoured and is itself a deadline). Seeing
this message on a fixed build means a mine exceeded *five* minutes — treat it as
the previous section says.
### `pi -p` prints its answer, then takes minutes to exit
Print mode (`pi -p`, `--mode json`) does not call `process.exit()` the way
interactive and RPC mode do: it returns and waits for node's event loop to
empty. Until 2026-10-13 the feed raced its mine against a bare `setTimeout`
that nothing cleared, so when the mine won (the normal case) the losing
`MEMPALACE_FEED_MINE_TIMEOUT_MS` timer kept node alive for the rest of its
five minutes. Measured on tor-ms22, a `pi -p` printed `PONG` at 4 s and exited
at 315 s, which is the timer's creation at 15 s plus 300 s. From 35 s after the
mine returned until exit, that timer was the only live resource. Interactive
sessions were never affected. The hang predates pi 1.1.0 (pi 1.0.0 behaved the
same), and `pi-task`/`fork` children were never affected because they run
`--no-extensions`.
Fixed 2026-10-13: the timer is cleared when the race settles.
`scripts/test-feed-exit.sh` pins it. Node must exit within 3 s of
`session_shutdown` resolving, both when the mine answers and when it times
out. Before the fix the first case held node for the whole deadline.
**Print and JSON mode skip the feed (also 2026-10-13).** With only the timer
fix, a `pi -p` still waited for one full feed. The first settled turn of every
session runs a catch-up feed (`lastFeedAt` starts at 0), and `session_shutdown`
waits for it: 45 s on tor-ms22, of which `--prepare` took 11 s and the mine
31 s, for a 3 s answer. That feed almost never files the run's own transcript,
because a one-shot run rarely passes the exporter's quality gate (4 messages,
1000 assistant characters). What it files is the device's *other* sessions, and
the next interactive feed re-offers those anyway. So when `ctx.mode` is
`print` or `json`, the bridge now feeds neither on `agent_settled` nor on
`session_shutdown`. Measured on tor-ms22: `pi -p` 2.9 s and `--mode json`
3.8 s, against 45 s before; `MEMPALACE_FEED_PRINT=1` gives 30.8 s (a smaller
backlog than the 45 s run). RPC mode is long-lived and feeds as before.
Nothing is filed twice when a later session re-offers a transcript. The
`source_file` is the staged path, which is fixed per pi session UUID. The mine
skips an unchanged file, and for a grown file it purges that file's drawers
before refiling. A device that only ever runs `pi -p` sets
`MEMPALACE_FEED_PRINT=1`, or its transcripts are never fed.
**Print and JSON mode skip the wake-up too (also 2026-10-13).** The wake-up
injection costs 12 palace calls: `mempalace_status`, `mempalace_diary_read`,
and 10 `mempalace_event_list` for the tier-1 mailbox. It also adds a large
orientation message to a script's one question. It is meant for the operator
of an interactive session, who sees the same thing in their next one. It can
also do harm. In one tor-ms22 probe, a flash model was given
`Reply with exactly: PONG`. It read the injected diary's open items as its
task, ran about 12 tool calls, and filed a wrong drawer in the shared palace.
So in print and JSON mode the bridge now skips the injection. It also skips
the mid-session mailbox poll, through its own guard: `pi -p -c` resumes, and a
resume sets the flag the poll would otherwise use. The client still connects
and the `mempalace_*` tools stay registered, so a prompt that asks for the
palace gets it. Measured on tor-ms22 with a fetch probe that re-wraps pi's
own `fetch`: `pi -p` made 3 palace requests (`initialize`, `initialized`,
`tools/list`) and exited in 1.6 s. With `MEMPALACE_WAKEUP_PRINT=1` it made 15,
including 12 `tools/call`, and the wake-up appeared in the JSON stream.
`scripts/test-wakeup-mode.sh` pins it; `f590a3a`'s source fails its print,
json and print-resumed cases.
## The `Type.Unsafe` gotcha
Earlier versions of this extension registered every MCP tool with
extensions/pi/mempalace.ts
+378 -51
View File
@@ -20,7 +20,8 @@
* - Wake-up (auto): on first user prompt of a fresh session, inject
* `mempalace_status` + `mempalace_diary_read` output as context so the
* agent orients itself the way the mempalace skill describes. Skipped
* on resume/fork (palace context is already in the thread).
* on resume/fork (palace context is already in the thread), and in
* print/json mode (`pi -p`, `--mode json`) unless MEMPALACE_WAKEUP_PRINT=1.
* - Feeding (auto): stage + mine this container's pi transcripts into the
* palace on `session_shutdown` and on a debounced `agent_settled`. Needs
* no LLM turn (pi transcripts are JSONL on disk), which is why it CAN be
@@ -33,6 +34,8 @@
* client also means it automatically targets whichever palace this bridge
* is pointed at (local stdio or a shared remote one).
* - MEMPALACE_FEED=0 disable feeding entirely
* - MEMPALACE_FEED_PRINT=1 also feed in print/json mode (`pi -p`,
* `--mode json`), which skip it by default
* - MEMPALACE_FEED_BIN helper to run (default mempalace-pi-session)
* - MEMPALACE_FEED_WING target wing (default wing_conversations)
* - MEMPALACE_FEED_DEBOUNCE_MS min gap between mid-session feeds (default 600000)
@@ -67,7 +70,9 @@
* child, so pi gets an error instead of hanging and later calls fail fast.
* This is a per-REQUEST timeout, not a process-lifetime one — the
* long-lived server is only killed when a request genuinely stalls.
* - MEMPALACE_MCP_TIMEOUT_MS tool-call/request timeout (default 60000)
* - MEMPALACE_MCP_TIMEOUT_MS tool-call/request timeout (default 60000);
* the feed's mine carries its own, longer
* deadline (MEMPALACE_FEED_MINE_TIMEOUT_MS)
* - MEMPALACE_MCP_INIT_TIMEOUT_MS initialize+tools/list timeout (default 300000)
* Set either to 0 to disable (legacy unbounded behavior).
*
@@ -90,6 +95,7 @@
*/
import { type ChildProcessWithoutNullStreams, spawn } from "node:child_process";
import { readFileSync, statSync } from "node:fs";
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { Type } from "typebox";
@@ -116,7 +122,13 @@ interface IMcpClient {
readonly alive: boolean;
onExit: (() => void) | null;
start(): Promise<void>;
callTool(name: string, args: Record<string, unknown>): Promise<any>;
/**
* `opts.timeoutMs` overrides the transport's generic per-request deadline
* for THIS call only. Callers that knowingly invoke a long server-side job
* (the feed's `mempalace_mine`) pass their own deadline here; everything
* else keeps the short default, which is the wedged-query guard.
*/
callTool(name: string, args: Record<string, unknown>, opts?: { timeoutMs?: number }): Promise<any>;
ensureAlive(): Promise<boolean>;
stop(): void | Promise<void>;
}
@@ -149,6 +161,165 @@ const sleep = (ms: number): Promise<void> =>
if (typeof t.unref === "function") t.unref();
});
// ── Dormancy ──────────────────────────────────────────────────────────────────
/**
* Is a standing ask DORMANT — correct to leave open, wrong to announce?
*
* The problem this solves, measured rather than imagined. A first-boot
* acceptance ask is planted deliberately unanswerable: it describes work that
* becomes possible only when the device is next recreated, and it must STAY
* owed until then, because closing it early to tidy the mailbox is exactly how
* that work gets lost. But deriveOwed could only see "directed, open, not
* answered, not withdrawn", so such an ask is announced at every session start
* and every poll for as long as it is correctly waiting — three days running on
* emb-7kj4vr4g (2026-09-28 → 2026-10-01), and three consecutive releases before
* that. The ask was right; announcing it was wrong. The cost lands on the human
* reading the window, who is the one reader that cannot filter it.
*
* So an ask may now declare, in its own metadata, the condition under which it
* is merely waiting:
*
* "dormant_unless": [
* { "kind": "json_field", "path": "/etc/pi-devbox/build-manifest.json",
* "field": "release_tag", "baseline": "v1.9.4" },
* { "kind": "file_mtime", "path": "/etc/hostname",
* "baseline": "2026-09-22T18:12:49Z" }
* ]
*
* It is dormant while EVERY condition still matches its baseline, and goes live
* the moment ANY of them differs — which is the trigger those asks already
* state in prose ("act when EITHER differs"), now in a form a machine can
* check. Dormant asks are withheld from the ANNOUNCED owed set, never from the
* mailbox: the wake-up injection still lists them once per session, so they
* stay discoverable and cannot be silently dropped.
*
* THE SAFETY RULE, and the reason every branch below reports `unchanged: false`
* on doubt: dormancy must be PROVEN, never assumed. A missing file, an
* unreadable one, a baseline that will not parse, an unknown `kind`, a field
* that has vanished — each is a reason to SHOW the ask, not to hide it. The
* dangerous failure here is not a spurious nag; it is work that vanishes
* because a predicate could not be evaluated, which would be indistinguishable
* from the ask being lost and would not surface until a release needed it.
*
* No eval, no shell, no network: the predicate language is deliberately two
* fixed checks against local files. A general expression evaluator sitting in
* the path that decides whether work is VISIBLE is a far worse trade than a
* slightly clumsy schema.
*/
export type DormancyVerdict = { dormant: boolean; reason: string };
/** Split of the derived owed-set: what to announce, and what is merely waiting. */
export type OwedSplit = { owed: LogEvent[]; dormant: LogEvent[] };
// Bounds, so a malformed or hostile event cannot turn a mailbox read into real
// work. Both are deliberately small: these predicates describe boot state, and
// anything needing more than a handful of local checks is not a dormancy rule.
const MAX_DORMANCY_CONDITIONS = 8;
const MAX_DORMANCY_JSON_BYTES = 256 * 1024;
const dormancyCondition = (c: unknown): { unchanged: boolean; reason: string } => {
if (typeof c !== "object" || c === null || Array.isArray(c))
return { unchanged: false, reason: "condition is not an object" };
const { kind, path, baseline, field } = c as Record<string, unknown>;
// Absolute paths only: a relative one would resolve against whatever cwd the
// pi process happens to hold, which is not a property of the device whose
// state the baseline describes.
if (typeof path !== "string" || !path.startsWith("/") || path.includes("\0"))
return {
unchanged: false,
reason: `path must be an absolute string, got ${JSON.stringify(path)}`,
};
if (typeof baseline !== "string" && typeof baseline !== "number")
return { unchanged: false, reason: "baseline must be a string or a number" };
if (kind === "file_mtime") {
// Compared at WHOLE SECONDS in UTC, because the two routes that produce
// these baselines disagree below that: a filesystem mtime carries
// sub-second residue (measured: /etc/hostname at .773761009) that a
// hand-written or reported ISO baseline never will. Comparing raw
// milliseconds would make every such predicate permanently "changed",
// i.e. would silently disable the feature while appearing to work.
const expected = Date.parse(String(baseline));
if (!Number.isFinite(expected))
return {
unchanged: false,
reason: `baseline is not a parseable date: ${JSON.stringify(baseline)}`,
};
let mtimeMs: number;
try {
mtimeMs = statSync(path).mtimeMs;
} catch {
return { unchanged: false, reason: `cannot stat ${path}` };
}
return Math.floor(mtimeMs / 1000) === Math.floor(expected / 1000)
? { unchanged: true, reason: `${path} mtime still at baseline` }
: { unchanged: false, reason: `${path} mtime moved from baseline` };
}
if (kind === "json_field") {
if (typeof field !== "string" || field.length === 0)
return { unchanged: false, reason: "json_field needs a non-empty field name" };
let text: string;
try {
const buf = readFileSync(path);
if (buf.byteLength > MAX_DORMANCY_JSON_BYTES)
return {
unchanged: false,
reason: `${path} exceeds the ${MAX_DORMANCY_JSON_BYTES}-byte cap`,
};
text = buf.toString("utf8");
} catch {
return { unchanged: false, reason: `cannot read ${path}` };
}
let doc: unknown;
try {
doc = JSON.parse(text);
} catch {
return { unchanged: false, reason: `${path} is not valid JSON` };
}
let cur: unknown = doc;
for (const seg of field.split(".")) {
if (
typeof cur !== "object" ||
cur === null ||
!Object.prototype.hasOwnProperty.call(cur, seg)
)
return { unchanged: false, reason: `${path} has no field ${field}` };
cur = (cur as Record<string, unknown>)[seg];
}
if (cur !== null && typeof cur === "object")
return { unchanged: false, reason: `field ${field} is not a scalar` };
// Compared as strings on purpose: the baseline arrives as JSON metadata, so
// a manifest holding 3 and a baseline saying "3" are the same fact.
return String(cur) === String(baseline)
? { unchanged: true, reason: `${field} still at baseline` }
: { unchanged: false, reason: `${field} moved from baseline` };
}
return { unchanged: false, reason: `unknown dormancy kind ${JSON.stringify(kind)}` };
};
export function evaluateDormancy(
metadata: Record<string, unknown> | null | undefined,
): DormancyVerdict {
const raw = metadata?.dormant_unless;
// Every one of these returns NOT dormant, i.e. "announce it". An ask without
// a predicate behaves exactly as it did before this feature existed.
if (raw === undefined || raw === null) return { dormant: false, reason: "no dormant_unless" };
if (!Array.isArray(raw)) return { dormant: false, reason: "dormant_unless is not an array" };
if (raw.length === 0) return { dormant: false, reason: "dormant_unless is empty" };
if (raw.length > MAX_DORMANCY_CONDITIONS)
return {
dormant: false,
reason: `too many conditions (${raw.length} > ${MAX_DORMANCY_CONDITIONS})`,
};
for (const c of raw) {
const v = dormancyCondition(c);
if (!v.unchanged) return { dormant: false, reason: v.reason };
}
return { dormant: true, reason: `all ${raw.length} condition(s) still at baseline` };
}
class StdioMcpClient implements IMcpClient {
private proc: ChildProcessWithoutNullStreams | null = null;
private nextId = 1;
@@ -372,8 +543,12 @@ class StdioMcpClient implements IMcpClient {
});
}
async callTool(name: string, args: Record<string, unknown>): Promise<any> {
return this.request("tools/call", { name, arguments: args });
async callTool(name: string, args: Record<string, unknown>, opts?: { timeoutMs?: number }): Promise<any> {
// The per-call override matters MORE here than for the HTTP client: on
// timeout this transport kills the server child, so a generic deadline
// that undercuts a long mine does not merely abandon the wait — it
// aborts the mine.
return this.request("tools/call", { name, arguments: args }, opts?.timeoutMs ?? this.requestTimeoutMs);
}
/** SIGTERM then SIGKILL grace, for stall recovery. */
@@ -404,23 +579,25 @@ class StdioMcpClient implements IMcpClient {
// ───────────────────────────────────────────────────────────────────────
// RemoteMcpClient — EXTERNAL transport (streamable-HTTP / sessionless JSON-RPC).
//
// VENDORED from pi-extensions/extensions/mcp-loader.ts (class RemoteMcpClient).
// Kept as a copy rather than a shared import because the two repos ship
// independently; the canonical implementation lives in mcp-loader.ts. Port
// protocol fixes in both directions.
// ORIGIN: vendored from pi-extensions/extensions/mcp-loader.ts (class
// RemoteMcpClient). mcp-loader.ts was RETIRED on 2026-10-12 (pi-extensions
// 750b2a8) in favour of pi's built-in MCP support, so THIS copy is now the only
// one. There is no canonical partner left to sync with, so the former
// MCP-STREAMABLE-HTTP-CLIENT-SYNC token and scripts/check-mcp-client-sync.sh
// were removed with it. The last canonical version is
// `git show a88fa99:extensions/mcp-loader.ts` in pi-extensions.
//
// MCP-STREAMABLE-HTTP-CLIENT-SYNC: v1
// This token must match the canonical block's. Bump it in BOTH files whenever
// the streamable-HTTP protocol handling changes; scripts/check-mcp-client-sync.sh
// fails when they diverge (skips gracefully if pi-extensions isn't checked out).
//
// Deltas from the canonical copy:
// Deltas from that last canonical copy:
// • protocolVersion pinned to "2024-11-05" to match mempalace-mcp's stdio
// handshake (the server echoes whatever it is sent, so cosmetic today,
// but keeps both transports consistent).
// • per-request AbortController timeout honouring MEMPALACE_MCP_TIMEOUT_MS /
// MEMPALACE_MCP_INIT_TIMEOUT_MS, mirroring StdioMcpClient's timeout ethos.
// • alive / ensureAlive / onExit to satisfy IMcpClient.
// • callTool() takes an optional per-call `{ timeoutMs }` (IMcpClient
// contract, see there) so the feed's long-running mine is not cut off by
// the generic per-request deadline. Not a protocol change; sync token
// unchanged.
//
// NOTE: mempalace-mcp --transport http is a SESSIONLESS, stateless JSON-RPC
// server (no Mcp-Session-Id, always application/json, Connection: close), so
@@ -466,8 +643,8 @@ class RemoteMcpClient implements IMcpClient {
this.healthy = true;
}
async callTool(name: string, args: Record<string, unknown>): Promise<any> {
return this.request("tools/call", { name, arguments: args });
async callTool(name: string, args: Record<string, unknown>, opts?: { timeoutMs?: number }): Promise<any> {
return this.request("tools/call", { name, arguments: args }, { timeoutMs: opts?.timeoutMs });
}
/**
@@ -855,7 +1032,22 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
const feedWing = process.env.MEMPALACE_FEED_WING ?? "wing_conversations";
const feedDebounceMs = num(process.env.MEMPALACE_FEED_DEBOUNCE_MS, 600_000);
const feedPrepareTimeoutMs = num(process.env.MEMPALACE_FEED_PREPARE_TIMEOUT_MS, 120_000);
const feedMineTimeoutMs = num(process.env.MEMPALACE_FEED_MINE_TIMEOUT_MS, 30_000);
// 300_000, raised from 30_000 on 2026-09-10. The mine is the SLOWEST thing
// this extension does — it offers every qualifying session transcript to a
// SINGLE-WRITER palace, measured at 30–60s in normal operation and growing
// with the corpus — yet it carried by far the TIGHTEST deadline: 4x tighter
// than the prepare step that precedes it (120_000) and 10x tighter than the
// init handshake (300_000), which is a fast call. All three were introduced
// together in 29e660e (2026-08-12) and this one was never revisited, so the
// deadline fired during entirely normal operation and the resulting message
// read as an error when nothing had gone wrong.
//
// Matched to the init timeout because liveness is the ONLY legitimate job
// left for this deadline: the Promise.race below abandons our WAIT, it cannot
// cancel the server's work, so the deadline buys nothing except an escape
// from a permanently hung call. It must therefore sit far above the slowest
// honest completion, not near it.
const feedMineTimeoutMs = num(process.env.MEMPALACE_FEED_MINE_TIMEOUT_MS, 300_000);
let lastFeedAt = 0; // 0 => the first settled turn also acts as a catch-up
let feedInFlight: Promise<void> | null = null;
@@ -909,25 +1101,70 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
try {
const source = await prepareFeed(reason);
if (!source) return;
await Promise.race([
client.callTool("mempalace_mine", {
source,
mode: "convos",
wing: feedWing,
// Internal call: it does not pass through the registered tool's
// execute(), so it stamps itself. These ARE this harness's own
// transcripts from this device, so the harness segment is the
// agent (not `miner`) even though the tool is `mine`.
agent: stampProvenance ? `${agentName}@${device}` : agentName,
}),
new Promise((_resolve, reject) =>
setTimeout(
() => reject(new Error(`mine timed out after ${feedMineTimeoutMs}ms`)),
feedMineTimeoutMs,
),
),
]);
// Record the attempt HERE, before awaiting — not after a successful
// wait. The race below abandons only our WAIT; the mine keeps running
// server-side, and `mine --mode convos` dedups by source_file and is
// idempotent, so a timeout is emphatically not a "did not happen".
//
// Leaving lastFeedAt stale on the timeout path defeated the debounce
// guard in the agent_settled handler below (`Date.now() - lastFeedAt <
// feedDebounceMs`): with lastFeedAt unchanged that guard passed on EVERY
// settled turn, and because `run` had already settled, feedInFlight was
// null too — so BOTH guards stood open. Each settled turn then launched
// another mine while the previous one was still running: overlapping
// writers queueing on a single-writer palace, each making the next one
// slower and the next timeout likelier. That positive feedback loop,
// not the tight deadline by itself, is why operators saw "mine timed out
// after 30000ms" many times per session rather than at most once per
// debounce window.
lastFeedAt = Date.now();
// The deadline is passed DOWN to the transport as well as raced
// here. Until 2026-09-18 it was only raced: callTool() had no way
// to carry it, so the transport's generic per-request timeout
// (MEMPALACE_MCP_TIMEOUT_MS, 60 000) fired first on every honest
// 60 s+ mine — "remote request 'tools/call' failed: timed out
// after 60000ms" — and the 300 000 below was unreachable. The
// race stays as the liveness guard for a transport whose timeout
// is disabled (0).
//
// The race's timer MUST be cleared when the race settles (finally,
// below). Until 2026-10-13 it was a bare setTimeout that nothing
// cleared, so when the mine WON — the normal case — the losing
// timer stayed referenced for the rest of feedMineTimeoutMs. In
// interactive and RPC mode pi calls process.exit() and nobody saw
// it; print mode (`pi -p`, `--mode json`) returns and waits for an
// empty event loop, so every `pi -p` on a devbox printed its answer
// and then idled until the 300 s deadline (measured on tor-ms22: the
// only live resource from the mine's return to exit). Kept ref'd on
// purpose while it runs: it is the liveness guard.
// scripts/test-feed-exit.sh pins both halves.
let raceTimer: ReturnType<typeof setTimeout> | undefined;
try {
await Promise.race([
client.callTool(
"mempalace_mine",
{
source,
mode: "convos",
wing: feedWing,
// Internal call: it does not pass through the registered tool's
// execute(), so it stamps itself. These ARE this harness's own
// transcripts from this device, so the harness segment is the
// agent (not `miner`) even though the tool is `mine`.
agent: stampProvenance ? `${agentName}@${device}` : agentName,
},
{ timeoutMs: feedMineTimeoutMs },
),
new Promise((_resolve, reject) => {
raceTimer = setTimeout(
() => reject(new Error(`mine timed out after ${feedMineTimeoutMs}ms`)),
feedMineTimeoutMs,
);
}),
]);
} finally {
clearTimeout(raceTimer);
}
} catch (err) {
process.stderr.write(
`[mempalace ext] feed (${reason}) failed: ${(err as Error).message}\n`,
@@ -940,20 +1177,58 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
return feedInFlight;
}
// Print and JSON mode (`pi -p`, `--mode json`) do NOT feed unless
// MEMPALACE_FEED_PRINT=1. A one-shot run's own transcript almost never meets
// the exporter's quality gate (MIN_MESSAGES=4, MIN_ASSISTANT_CHARS=1000), so
// what such a run would file is the device's OTHER sessions. That is a
// catch-up, because lastFeedAt starts at 0, and session_shutdown waits for
// it: 45 s on tor-ms22 (prepare 11 s + mine 31 s) for a 3 s answer.
// Skipping loses nothing. The source_file is the staged path, fixed per
// session UUID, and the next interactive feed re-offers every qualifying
// session: an unchanged file is skipped, a grown one is purged and refiled.
// RPC mode is long-lived and feeds as before. A pi whose ctx has no `mode`
// falls back to hasUI, which is false only in print/json mode.
const feedInPrintMode = process.env.MEMPALACE_FEED_PRINT === "1";
const isOneShot = (ctx: unknown): boolean => {
try {
const c = ctx as { mode?: string; hasUI?: boolean } | undefined;
const mode = c?.mode;
if (mode === "print" || mode === "json") return true;
return mode === undefined && c?.hasUI === false;
} catch {
return false; // a stale ctx throws on access: keep the old behaviour
}
};
const feedsThisRun = (ctx: unknown): boolean => feedInPrintMode || !isOneShot(ctx);
// Print and JSON mode skip the palace start-up work as well, unless
// MEMPALACE_WAKEUP_PRINT=1. That is the wake-up injection (status + diary +
// tier-1 mailbox) and the mid-session mailbox poll. A one-shot run is a
// script asking one question. The wake-up costs it several palace round
// trips and a large orientation message, and it serves the operator of an
// interactive session, who sees it next time they open one. It can also do
// harm: on tor-ms22 a flash model given "Reply with exactly: PONG" read the
// injected diary's open items as its task, ran ~12 tool calls, and filed a
// wrong drawer. The mempalace_* tools stay registered, so a prompt that asks
// for the palace still gets it.
const wakeupInPrintMode = process.env.MEMPALACE_WAKEUP_PRINT === "1";
const wakesThisRun = (ctx: unknown): boolean => wakeupInPrintMode || !isOneShot(ctx);
// Mid-session feed. A hard container kill runs no handler at all, so this is
// what bounds crash loss to one debounce window instead of a whole session.
// Re-mining a grown transcript purges and refiles that source_file, so
// repeated ticks refresh a session's drawers rather than duplicating them.
pi.on("agent_settled", async () => {
pi.on("agent_settled", async (_event, ctx) => {
if (!feedsThisRun(ctx)) return;
if (Date.now() - lastFeedAt < feedDebounceMs) return;
void feedPalace("tick"); // deliberately not awaited: never stall a turn
});
pi.on("session_shutdown", async () => {
pi.on("session_shutdown", async (_event, ctx) => {
// Feed before stopping the client: we are the palace holder, so nothing
// else can mine while we live. pi awaits this handler, so the mine really
// does complete; feedMineTimeoutMs keeps a wedged palace from hanging exit.
await feedPalace("shutdown");
if (feedsThisRun(ctx)) await feedPalace("shutdown");
client.stop();
});
@@ -1146,14 +1421,20 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
* Directed asks addressed to this device with no terminal reply from it, and
* not explicitly withdrawn by whoever sent them (see isWithdrawn).
*/
async function deriveOwed(): Promise<LogEvent[]> {
if (!mailboxEnabled || !available) return [];
async function deriveOwed(): Promise<OwedSplit> {
if (!mailboxEnabled || !available) return { owed: [], dormant: [] };
try {
const [candidatesRaw, mineRaw, inboundRaw] = await Promise.all([
// Every cursor-less event_list call in this file passes `order` explicitly.
// The server's default is not ours to lean on: mempalace <= 3.9.0 defaults
// to `asc`, 3.10.0 flips a cursor-less listing to newest-first. Both
// derive* joins want the NEWEST window (see the comment below), so say so
// and the verdict stops depending on which server version answers.
client.callTool("mempalace_event_list", {
to_agent: mailboxAddress,
status: "open",
limit: 50,
order: "desc",
}),
// `order: "desc"` is a fix, not a flourish. event_list defaults to `asc`
// (append order), so this asked for the OLDEST 100 events this device
@@ -1189,13 +1470,21 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
// You cannot owe yourself.
return c.from_agent !== mailboxAddress;
});
if (candidates.length === 0) return [];
if (candidates.length === 0) return { owed: [], dormant: [] };
const mine = parseEvents(mineRaw);
const inbound = parseEvents(inboundRaw);
return candidates.filter((c) => !isAnswered(c, mine) && !isWithdrawn(c, inbound));
const live = candidates.filter((c) => !isAnswered(c, mine) && !isWithdrawn(c, inbound));
// PARTITION, not filter. A dormant ask is still owed in the protocol
// sense — it has no terminal reply and must not be closed — so it stays in
// the return value where the mailbox can still account for it, and only
// the ANNOUNCING paths below treat the two halves differently.
const owed: LogEvent[] = [];
const dormant: LogEvent[] = [];
for (const c of live) (evaluateDormancy(c.metadata).dormant ? dormant : owed).push(c);
return { owed, dormant };
} catch {
// Fail silent and open: a mailbox read must never break a session.
return [];
return { owed: [], dormant: [] };
}
}
@@ -1218,10 +1507,15 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
async function deriveClosed(): Promise<LogEvent[]> {
if (!mailboxEnabled || !available) return [];
try {
// `order: "desc"` for the same reason as deriveOwed: without it a 3.9.0
// server hands back the OLDEST window, so past 100 authored events this
// device's newest asks (and past 50 inbound, the replies that close them)
// fall outside the join. The selection below is order-independent
// (isStrictlyAfter), so this only fixes WHICH events are in the window.
const [mineRaw, inboundRaw] = await Promise.all([
client.callTool("mempalace_event_list", { from_agent: mailboxAddress, limit: 100 }),
client.callTool("mempalace_event_list", { from_agent: mailboxAddress, limit: 100, order: "desc" }),
// NO status filter, deliberately: that omission is the entire fix.
client.callTool("mempalace_event_list", { to_agent: mailboxAddress, limit: 50 }),
client.callTool("mempalace_event_list", { to_agent: mailboxAddress, limit: 50, order: "desc" }),
]);
// Correlations this device opened as a DIRECTED ask. A broadcast owes
// nobody a reply, so it cannot be closed by one either.
@@ -1286,6 +1580,15 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
})
.join("\n");
const DORMANT_NOTE =
"Each of these declares a `dormant_unless` predicate in its metadata that STILL " +
"matches this device's current state, so the work it describes is not yet " +
"possible. They are NOT closed and NOT answered — leave them open. They return to " +
"the announced owed-set by themselves the moment a baseline stops matching. If a " +
"predicate cannot be evaluated at all (missing file, unparseable baseline, unknown " +
"kind) the ask is announced as owed instead, deliberately: dormancy has to be " +
"proven, never assumed.";
const OWED_HOWTO =
"To close one, append an event on the SAME correlation_id with a terminal status " +
"(applied/superseded/failed/blocked). An ack alone does NOT clear it, and neither " +
@@ -1386,6 +1689,8 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
pi.on("agent_settled", async (_event, ctx) => {
if (!mailboxEnabled || !available) return;
// Explicit, not left to `wokeUp`: `pi -p -c` resumes, which sets wokeUp.
if (!wakesThisRun(ctx)) return;
if (!wokeUp) return; // the wake-up injection has not run yet; it does look #1
if (Date.now() - lastMailboxPollAt < mailboxPollMs) return;
// Claim the slot BEFORE awaiting anything, so overlapping settles cannot
@@ -1393,7 +1698,8 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
lastMailboxPollAt = Date.now();
void (async () => {
try {
const [owed, closed] = await Promise.all([deriveOwed(), deriveClosed()]);
const [owedSplit, closed] = await Promise.all([deriveOwed(), deriveClosed()]);
const { owed, dormant } = owedSplit;
const now = Date.now();
const unseen = (list: LogEvent[]): LogEvent[] =>
list.filter((e) => {
@@ -1421,6 +1727,13 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
`${CLOSED_NOTE}\n\n${formatOwed(newsRaw)}`,
);
}
// A dormant ask NEVER opens this window on its own — the early return
// above is what actually stops the nag. But once the window is open for
// something else, one line of count is nearly free and keeps a waiting
// ask from feeling lost between session starts.
if (dormant.length > 0) {
blocks.push(`${dormant.length} dormant ask(s), not listed here. ${DORMANT_NOTE}`);
}
pi.sendMessage(
{
customType: "mempalace-mailbox",
@@ -1462,8 +1775,9 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
});
// --- Auto wake-up (mempalace skill Phase 1) ---
pi.on("before_agent_start", async (_event, _ctx) => {
pi.on("before_agent_start", async (_event, ctx) => {
if (wokeUp || !available) return;
if (!wakesThisRun(ctx)) return; // one-shot run: see wakesThisRun
wokeUp = true; // one-shot, even if the calls below fail
const sections: string[] = [];
@@ -1485,10 +1799,11 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
sections.push(`## mempalace_diary_read\n\n(error: ${(err as Error).message})`);
}
// Tier 1 mailbox: what this device owes a reply to, derived rather than read
// off a filter. deriveOwed() swallows its own failures and returns [], so a
// broken palace costs a missing section, never a broken wake-up.
// off a filter. deriveOwed() swallows its own failures and returns an empty
// split, so a broken palace costs a missing section, never a broken wake-up.
if (mailboxEnabled) {
const [owed, closed] = await Promise.all([deriveOwed(), deriveClosed()]);
const [owedSplit, closed] = await Promise.all([deriveOwed(), deriveClosed()]);
const { owed, dormant } = owedSplit;
if (owed.length > 0) {
// Record what this injection showed, so the first mid-session poll does
// not re-announce the identical list minutes later. Without this the
@@ -1502,6 +1817,18 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
`this device. ${OWED_HOWTO}\n\n${formatOwed(owed)}`,
);
}
// Once per session, WITH ids. The wake-up injection is the one place a
// dormant ask should be visible: "what is this device still carrying?" is a
// session-start question, and answering it here is what keeps the mid-session
// silence honest rather than concealing. Deliberately NOT added to
// `surfaced`: that map exists to suppress repeats of ANNOUNCED work, and a
// dormant ask must stay announceable the instant its baseline moves.
if (dormant.length > 0) {
sections.push(
`## logstream mailbox (${dormant.length} dormant — waiting, nothing owed yet)\n\n` +
`${DORMANT_NOTE}\n\n${formatOwed(dormant)}`,
);
}
const news = closed.filter((e) => e.id && !surfaced.has(e.id));
if (news.length > 0) {
const now = Date.now();
scripts/check-mcp-client-sync.sh
-77
View File
@@ -1,77 +0,0 @@
#!/usr/bin/env bash
# check-mcp-client-sync.sh — drift guard for the vendored streamable-HTTP MCP client.
#
# The RemoteMcpClient in extensions/pi/mempalace.ts is a VENDORED copy of the
# canonical implementation in pi-extensions/extensions/mcp-loader.ts (the two
# repos ship independently, so a shared import isn't practical for one small,
# protocol-frozen class — see the annotated header in mempalace.ts).
#
# Both blocks carry a matching sync token:
# MCP-STREAMABLE-HTTP-CLIENT-SYNC: vN
# Bump it in BOTH files whenever the streamable-HTTP protocol handling changes.
# This guard fails when the tokens diverge, so a protocol fix in one file can't
# silently skip the other.
#
# It is a LOCAL developer guard: it needs the sibling pi-extensions checkout to
# compare against. If that isn't found it SKIPS (exit 0) rather than failing —
# so it's safe to wire into CI that only checks out this repo.
#
# Point it at a specific checkout with: PI_EXTENSIONS_DIR=/path/to/pi-extensions
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
VENDORED="$REPO_ROOT/extensions/pi/mempalace.ts"
TOKEN_RE='MCP-STREAMABLE-HTTP-CLIENT-SYNC: v[0-9]+'
extract_token() { grep -oE "$TOKEN_RE" "$1" 2>/dev/null | head -1 | grep -oE 'v[0-9]+' || true; }
if [[ ! -f "$VENDORED" ]]; then
echo "ERROR: vendored file not found: $VENDORED" >&2
exit 1
fi
VENDORED_TOKEN="$(extract_token "$VENDORED")"
if [[ -z "$VENDORED_TOKEN" ]]; then
echo "FAIL: no '$TOKEN_RE' token in $VENDORED" >&2
echo " The vendored RemoteMcpClient must carry a sync token." >&2
exit 1
fi
# Locate the canonical mcp-loader.ts across common dev layouts.
CANDIDATES=(
"${PI_EXTENSIONS_DIR:-}/extensions/mcp-loader.ts"
"$REPO_ROOT/../pi-extensions/extensions/mcp-loader.ts"
"/workspace/pi-extensions/extensions/mcp-loader.ts"
"$HOME/src/pi-extensions/extensions/mcp-loader.ts"
"$HOME/src/src_local/pi-extensions/extensions/mcp-loader.ts"
"/opt/pi-extensions/extensions/mcp-loader.ts"
)
CANONICAL=""
for c in "${CANDIDATES[@]}"; do
[[ -n "$c" && -f "$c" ]] && { CANONICAL="$c"; break; }
done
if [[ -z "$CANONICAL" ]]; then
echo "SKIP: canonical pi-extensions/extensions/mcp-loader.ts not found — cannot compare."
echo " (vendored token: $VENDORED_TOKEN). Set PI_EXTENSIONS_DIR to enable the check."
exit 0
fi
CANONICAL_TOKEN="$(extract_token "$CANONICAL")"
if [[ -z "$CANONICAL_TOKEN" ]]; then
echo "FAIL: no '$TOKEN_RE' token in canonical $CANONICAL" >&2
exit 1
fi
if [[ "$VENDORED_TOKEN" != "$CANONICAL_TOKEN" ]]; then
echo "FAIL: MCP client sync token drift." >&2
echo " vendored ($VENDORED): $VENDORED_TOKEN" >&2
echo " canonical ($CANONICAL): $CANONICAL_TOKEN" >&2
echo " A streamable-HTTP protocol change landed in one file but not the other." >&2
echo " Reconcile RemoteMcpClient in both, then bump the token in BOTH to match." >&2
exit 1
fi
echo "OK: MCP client sync token matches ($VENDORED_TOKEN)."
echo " vendored: $VENDORED"
echo " canonical: $CANONICAL"
scripts/test-dormancy.sh
Executable
+156
View File
@@ -0,0 +1,156 @@
#!/usr/bin/env bash
# test-dormancy.sh — exercise the mailbox dormancy predicate (dormant_unless).
#
# Why this exists as a script and not a note: `evaluateDormancy` decides whether
# an ask is SHOWN to the agent, so a silent regression there does not look like a
# bug — it looks like an empty mailbox. The positive arms matter as much as the
# negative ones: a predicate that never fires makes the feature a no-op, and a
# predicate that fires too eagerly hides real work. Both arms run here.
#
# It cannot import extensions/pi/mempalace.ts in place, because that file imports
# `typebox`, which pi provides at RUNTIME and this repo has no node_modules for.
# So it copies the file into a temp tree with the resolved deps symlinked beside
# it. The copy is made BY this script on every run, so it cannot drift from the
# source the way a vendored duplicate would.
#
# Fixtures are created here rather than read from the host: the first draft used
# /etc/hostname and /etc/pi-devbox/build-manifest.json, which made the positive
# arms pass only on a pi-devbox container and silently flip to "not dormant"
# anywhere else — a device-dependent test that reports success by doing nothing.
#
# Exit codes, deliberately distinct:
# 0 every case behaved as specified
# 1 at least one case FAILED (a real regression)
# 2 INCONCLUSIVE — deps could not be resolved, so nothing was proven
# (never 0: a test that skips quietly is the failure mode it should catch)
set -uo pipefail
# ── Args ──────────────────────────────────────────────────────────────────────
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
sed -n '2,25p' "$0" | sed 's/^# \?//'
exit 0
fi
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SRC="$REPO_ROOT/extensions/pi/mempalace.ts"
[[ -r "$SRC" ]] || {
echo "INCONCLUSIVE: cannot read $SRC" >&2
exit 2
}
# ── Resolve pi's runtime deps (discovered, never hardcoded) ───────────────────
GLOBAL_ROOT="$(npm root -g 2>/dev/null || true)"
PI_PKG=""
for cand in \
"${GLOBAL_ROOT:+$GLOBAL_ROOT/@earendil-works/pi-coding-agent}" \
"/usr/lib/node_modules/@earendil-works/pi-coding-agent" \
"/usr/local/lib/node_modules/@earendil-works/pi-coding-agent"; do
[[ -n "$cand" && -d "$cand" ]] && {
PI_PKG="$cand"
break
}
done
[[ -n "$PI_PKG" && -d "$PI_PKG/node_modules/typebox" ]] || {
echo "INCONCLUSIVE: pi-coding-agent / typebox not resolvable (looked under 'npm root -g')." >&2
echo " Nothing was proven. Install pi, or run this where pi is installed." >&2
exit 2
}
# Node must be able to strip types from a .ts entry point (Node >= 22.6).
node -e 'process.exit(0)' 2>/dev/null || {
echo "INCONCLUSIVE: no usable node" >&2
exit 2
}
# ── Build the throwaway tree ──────────────────────────────────────────────────
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
mkdir -p "$WORK/extensions/pi" "$WORK/node_modules/@earendil-works"
cp "$SRC" "$WORK/extensions/pi/mempalace.ts"
ln -s "$PI_PKG/node_modules/typebox" "$WORK/node_modules/typebox"
ln -s "$PI_PKG" "$WORK/node_modules/@earendil-works/pi-coding-agent"
printf '{"type":"module"}\n' > "$WORK/package.json"
# Prove the copy is the source, so a PASS cannot be about a stale file.
if ! cmp -s "$SRC" "$WORK/extensions/pi/mempalace.ts"; then
echo "INCONCLUSIVE: the copy differs from the source" >&2
exit 2
fi
# ── Fixtures, owned by this test ──────────────────────────────────────────────
printf '{"release_tag":"v1.9.4","nested":{"deep":{"leaf":"found"}},"count":3}\n' > "$WORK/manifest.json"
printf 'not json at all\n' > "$WORK/notjson.txt"
: > "$WORK/stamp"
# ── The truth table ───────────────────────────────────────────────────────────
cat > "$WORK/run.mjs" <<'EOF'
import { statSync } from "node:fs";
import { evaluateDormancy } from "./extensions/pi/mempalace.ts";
const W = process.env.WORK;
const STAMP = `${W}/stamp`;
const MAN = `${W}/manifest.json`;
// Floor to whole seconds: that is the contract, and the residue below proves
// why the implementation must do the same.
const atBaseline = new Date(Math.floor(statSync(STAMP).mtimeMs / 1000) * 1000).toISOString();
const mt = (baseline, path = STAMP) => ({ kind: "file_mtime", path, baseline });
const jf = (field, baseline, path = MAN) => ({ kind: "json_field", path, field, baseline });
const cases = [
// No predicate => behave exactly as before the feature existed.
["no metadata", undefined, false],
["metadata without the key", { foo: "bar" }, false],
["dormant_unless is a string", { dormant_unless: "release != x" }, false],
["dormant_unless is empty", { dormant_unless: [] }, false],
["more than 8 conditions", { dormant_unless: Array(9).fill(mt(atBaseline)) }, false],
// POSITIVE arms. If these ever read false the feature is inert.
["file_mtime at baseline", { dormant_unless: [mt(atBaseline)] }, true],
["json_field at baseline", { dormant_unless: [jf("release_tag", "v1.9.4")] }, true],
["both at baseline", { dormant_unless: [jf("release_tag", "v1.9.4"), mt(atBaseline)] }, true],
["dotted field path", { dormant_unless: [jf("nested.deep.leaf", "found")] }, true],
["number vs string baseline", { dormant_unless: [jf("count", "3")] }, true],
// The trigger firing: ANY condition differing wakes the ask.
["file_mtime moved", { dormant_unless: [mt("2020-01-01T00:00:00Z")] }, false],
["json_field moved", { dormant_unless: [jf("release_tag", "v1.9.5")] }, false],
["one same, one moved", { dormant_unless: [mt(atBaseline), jf("release_tag", "v1.9.5")] }, false],
// FAIL-VISIBLE arms: dormancy unproven => announce.
["missing file", { dormant_unless: [mt(atBaseline, "/nonexistent/path")] }, false],
["unparseable baseline", { dormant_unless: [mt("not-a-date")] }, false],
["relative path", { dormant_unless: [{ kind: "file_mtime", path: "etc/hostname", baseline: atBaseline }] }, false],
["unknown kind", { dormant_unless: [{ kind: "uptime_lt", path: "/etc/hostname", baseline: "1d" }] }, false],
["missing json field", { dormant_unless: [jf("no_such_field", "x")] }, false],
["file is not JSON", { dormant_unless: [jf("a", "b", `${W}/notjson.txt`)] }, false],
["field is not scalar", { dormant_unless: [jf("nested", "x")] }, false],
["condition is not an object", { dormant_unless: ["release_tag"] }, false],
["baseline is an object", { dormant_unless: [{ kind: "file_mtime", path: STAMP, baseline: {} }] }, false],
];
let pass = 0;
const failed = [];
for (const [name, meta, want] of cases) {
const v = evaluateDormancy(meta);
const ok = v.dormant === want;
if (ok) pass++;
else failed.push(name);
console.log(
` ${ok ? "PASS" : "FAIL"} dormant=${String(v.dormant).padEnd(5)} want=${String(want).padEnd(5)} ` +
`${name.padEnd(28)} :: ${v.reason}`,
);
}
console.log(`\n ${pass}/${cases.length} passed`);
if (failed.length) console.log(` FAILED: ${failed.join(", ")}`);
// Recorded because it is the reason file_mtime floors to seconds: a real mtime
// carries sub-second residue that an ISO baseline does not.
console.log(` (stamp mtimeMs=${statSync(STAMP).mtimeMs}, baseline=${atBaseline})`);
process.exit(failed.length === 0 ? 0 : 1);
EOF
cd "$WORK" || exit 2
export WORK
node "$WORK/run.mjs"
exit $?
scripts/test-feed-exit.sh
Executable
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env bash
# test-feed-exit.sh — once the feed's mine has settled, the pi extension in
# extensions/pi/mempalace.ts must leave NOTHING behind that keeps node alive.
#
# WHY THIS EXISTS. `pi -p` (print mode, also `--mode json`) does not call
# process.exit(): it returns from main() and lets node exit when the event loop
# is empty. Interactive and RPC mode call process.exit() and never noticed. The
# feed raced its mine against a bare
# new Promise((_, reject) => setTimeout(reject, feedMineTimeoutMs))
# whose timer was never cleared. When the mine won the race (the normal case,
# 30-60 s), the losing 300 000 ms timer stayed referenced, so every `pi -p` on a
# devbox printed its answer and then sat idle for the rest of those 5 minutes.
# Measured on tor-ms22 2026-10-13: the 300 000 ms timer from mempalace.ts was the
# ONLY active resource from 35 s after the mine returned until exit at ~316 s.
#
# Like test-mcp-call-timeout.sh, it runs the SHIPPED text: the whole extension
# is type-stripped with node's own stripper (the `typebox` import is replaced by
# a stub) and loaded under a stub pi. A fake feed helper prints MINE_SOURCE, and
# a local JSON-RPC server (in a SEPARATE process, so it cannot hold the driver's
# event loop) answers the mine. The driver fires session_shutdown, which awaits
# the feed, and then simply returns. The harness measures how long node takes to
# exit after that.
# A. mine answers in 1 s, deadline 20 s: node must exit right after the
# handler resolves. Before the fix it lived until the 20 s deadline.
# B. mine never answers, deadline 2 s: the deadline is still a liveness
# guard; the handler resolves at about 2 s and node exits.
# And the print-mode default (2026-10-13): `pi -p` / `--mode json` do not feed.
# C. ctx.mode "print": neither agent_settled nor session_shutdown mines.
# D. ctx.mode "json": the same.
# E. ctx.mode "print" + MEMPALACE_FEED_PRINT=1: mines, as before.
# F. ctx.mode "rpc" (long-lived): mines, as before.
# Every case fires agent_settled first and then session_shutdown, the order a
# real one-turn run sees. In the feeding cases the shutdown joins the tick's
# in-flight feed, so a run mines exactly once.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SRC="${1:-$REPO_ROOT/extensions/pi/mempalace.ts}"
WORK="$(mktemp -d)"
SERVER_PID=""
cleanup() { [ -n "$SERVER_PID" ] && kill "$SERVER_PID" 2>/dev/null; rm -rf "$WORK"; }
trap cleanup EXIT
[ -r "$SRC" ] || { echo "FAIL: cannot read $SRC" >&2; exit 2; }
# ---------------------------------------------------------------- extractor ---
cat >"$WORK/extract.mjs" <<'EXTRACT'
import { readFileSync, writeFileSync } from "node:fs";
import { stripTypeScriptTypes } from "node:module";
let src = readFileSync(process.argv[2], "utf8");
const imp = 'import { Type } from "typebox";';
if (!src.includes(imp)) throw new Error(`expected import line not found: ${imp}`);
src = src.replace(imp, 'const Type = { Unsafe: (s) => s, Object: () => ({}) };');
if (!src.includes("feedPalace")) throw new Error("extension does not mention feedPalace");
writeFileSync(process.argv[3], stripTypeScriptTypes(src, { mode: "strip" }));
EXTRACT
node --no-warnings "$WORK/extract.mjs" "$SRC" "$WORK/ext.mjs" || exit 2
node --check "$WORK/ext.mjs" || { echo "FAIL: stripped extension does not parse" >&2; exit 2; }
echo "[extract] type-stripped $(basename "$SRC") ($(wc -c <"$WORK/ext.mjs") bytes)"
# ------------------------------------------------------------ fake feed bin ---
cat >"$WORK/fake-feed" <<'FEED'
#!/bin/sh
echo "MINE_SOURCE=/nonexistent/fake-stage"
FEED
chmod +x "$WORK/fake-feed"
# ----------------------------------------------------------- fake palace ------
# Sessionless JSON-RPC like `mempalace-mcp --transport http`. The mine's delay
# is set per case through a tiny control endpoint (GET /delay/<ms>), so one
# server serves both cases.
cat >"$WORK/server.mjs" <<'SERVER'
import { createServer } from "node:http";
import { appendFileSync } from "node:fs";
let mineDelay = 1000;
const server = createServer((req, res) => {
if (req.url.startsWith("/delay/")) { mineDelay = Number(req.url.slice(7)); res.end("ok"); return; }
let body = "";
req.on("data", (c) => (body += c));
req.on("end", () => {
const msg = JSON.parse(body);
if (msg.id === undefined) { res.writeHead(202); res.end(); return; }
const reply = (result) => {
if (res.destroyed) return;
res.writeHead(200, { "content-type": "application/json", connection: "close" });
res.end(JSON.stringify({ jsonrpc: "2.0", id: msg.id, result }));
};
if (msg.method === "initialize") return reply({ protocolVersion: "2024-11-05", capabilities: {}, serverInfo: { name: "fake", version: "0" } });
if (msg.method === "tools/list") return reply({ tools: [{ name: "mempalace_mine", description: "", inputSchema: { type: "object" } }] });
if (msg.method === "tools/call") {
appendFileSync(process.env.CALL_LOG, `${msg.params?.name} source=${msg.params?.arguments?.source}\n`);
return void setTimeout(() => reply({ content: [{ type: "text", text: "mined" }] }), mineDelay);
}
reply({});
});
});
server.listen(0, "127.0.0.1", () => console.log(server.address().port));
SERVER
CALL_LOG="$WORK/calls.log" node "$WORK/server.mjs" >"$WORK/port" &
SERVER_PID=$!
for _ in $(seq 50); do [ -s "$WORK/port" ] && break; sleep 0.1; done
PORT="$(cat "$WORK/port")"
[ -n "$PORT" ] || { echo "FAIL: fake palace did not start" >&2; exit 2; }
# ------------------------------------------------------------------ driver ----
# Loads the extension under a stub pi, fires agent_settled and then
# session_shutdown with the ctx given in DRIVER_CTX, prints when the shutdown
# handler resolved, then RETURNS. Anything still referenced keeps node alive.
cat >"$WORK/driver.mjs" <<'DRIVER'
const t0 = Date.now();
const ctx = JSON.parse(process.env.DRIVER_CTX || "{}");
const handlers = {};
const pi = new Proxy({}, {
get: (_t, k) => k === "on" ? (ev, fn) => { (handlers[ev] ||= []).push(fn); } : () => {},
});
const { default: ext } = await import("./ext.mjs");
await ext(pi);
if (!handlers.session_shutdown) { console.log("NO_SHUTDOWN_HANDLER"); process.exitCode = 3; }
else {
for (const fn of handlers.agent_settled || []) await fn({ type: "agent_settled" }, ctx);
for (const fn of handlers.session_shutdown) await fn({ type: "session_shutdown", reason: "quit" }, ctx);
console.log(`HANDLER_RESOLVED_MS=${Date.now() - t0}`);
}
process.on("exit", () => console.log(`EXIT_MS=${Date.now() - t0}`));
DRIVER
failures=0
check() { if [ "$1" = 0 ]; then echo "ok $2"; else echo "FAIL $2"; failures=$((failures + 1)); fi; }
run_case() { # $1 mine delay ms, $2 feed mine timeout ms, $3 ctx JSON, [$4 FEED_PRINT] -> sets HANDLER EXIT RC MINES
curl -fsS "http://127.0.0.1:$PORT/delay/$1" >/dev/null
: >"$WORK/calls.log"
set +e
out="$(cd "$WORK" && env -u MEMPALACE_REMOTE_TOKEN -u MEMPALACE_PI_DEVICE \
MEMPALACE_REMOTE_URL="http://127.0.0.1:$PORT/mcp" MEMPALACE_FEED=1 \
MEMPALACE_FEED_BIN="$WORK/fake-feed" MEMPALACE_FEED_MINE_TIMEOUT_MS="$2" \
DRIVER_CTX="$3" MEMPALACE_FEED_PRINT="${4:-}" \
timeout 60 node --no-warnings driver.mjs 2>"$WORK/driver.err")"
RC=$?
set -e
HANDLER="$(printf '%s\n' "$out" | sed -n 's/^HANDLER_RESOLVED_MS=//p')"
EXIT="$(printf '%s\n' "$out" | sed -n 's/^EXIT_MS=//p')"
MINES="$(grep -c '^mempalace_mine source=/nonexistent/fake-stage' "$WORK/calls.log" || true)"
}
# A. normal case: the mine wins the race.
run_case 1000 20000 '{"mode":"tui","hasUI":true}'
echo "[A] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && echo 0 || echo 1)" "A: driver exits cleanly (rc=$RC)"
check "$([ "$MINES" = 1 ] && echo 0 || echo 1)" "A: shutdown really mined the staged source once (not a vacuous pass)"
check "$([ -n "$HANDLER" ] && [ "$HANDLER" -ge 900 ] && [ "$HANDLER" -lt 10000 ] && echo 0 || echo 1)" "A: session_shutdown awaited the 1 s mine (${HANDLER:-?} ms)"
check "$([ -n "$EXIT" ] && [ -n "$HANDLER" ] && [ $((EXIT - HANDLER)) -lt 3000 ] && echo 0 || echo 1)" \
"A: node exits within 3 s of the handler resolving ($(( ${EXIT:-0} - ${HANDLER:-0} )) ms; the 20 s mine deadline must not hold it)"
# B. wedged palace: the deadline must still end the wait.
run_case 60000 2000 '{"mode":"tui","hasUI":true}'
echo "[B] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && echo 0 || echo 1)" "B: driver exits cleanly (rc=$RC)"
check "$([ -n "$HANDLER" ] && [ "$HANDLER" -ge 1900 ] && [ "$HANDLER" -lt 10000 ] && echo 0 || echo 1)" "B: session_shutdown gives up at the 2 s deadline (${HANDLER:-?} ms)"
check "$(grep -Eq 'feed \((tick|shutdown)\) failed: .*2000ms' "$WORK/driver.err" && echo 0 || echo 1)" "B: the timeout is reported on stderr"
check "$([ -n "$EXIT" ] && [ -n "$HANDLER" ] && [ $((EXIT - HANDLER)) -lt 3000 ] && echo 0 || echo 1)" \
"B: node exits within 3 s of the handler resolving ($(( ${EXIT:-0} - ${HANDLER:-0} )) ms; the 60 s server delay must not hold it)"
# C/D. one-shot modes skip the feed: no mine, and node exits at once.
for m in print json; do
run_case 1000 20000 "{\"mode\":\"$m\",\"hasUI\":false}"
echo "[$m] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && [ "$MINES" = 0 ] && echo 0 || echo 1)" "$m mode: no mine at agent_settled or session_shutdown (mines=$MINES)"
check "$([ -n "$HANDLER" ] && [ "$HANDLER" -lt 900 ] && [ -n "$EXIT" ] && [ $((EXIT - HANDLER)) -lt 3000 ] && echo 0 || echo 1)" \
"$m mode: shutdown does not wait for a feed (${HANDLER:-?} ms) and node exits promptly"
done
# E. the opt-in restores feeding in print mode.
run_case 1000 20000 '{"mode":"print","hasUI":false}' 1
echo "[print+FEED_PRINT=1] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && [ "$MINES" = 1 ] && [ -n "$HANDLER" ] && [ "$HANDLER" -ge 900 ] && echo 0 || echo 1)" \
"print mode + MEMPALACE_FEED_PRINT=1: mines once and shutdown waits for it (mines=$MINES, ${HANDLER:-?} ms)"
# F. RPC mode is long-lived (hasUI true) and keeps feeding.
run_case 1000 20000 '{"mode":"rpc","hasUI":true}'
echo "[rpc] rc=$RC handler=${HANDLER:-?}ms exit=${EXIT:-?}ms mines=$MINES"
check "$([ "$RC" = 0 ] && [ "$MINES" = 1 ] && echo 0 || echo 1)" "rpc mode: still mines (mines=$MINES)"
if [ "$failures" = 0 ]; then echo "PASS: all assertions held"; else echo "FAIL: $failures assertion(s) failed"; exit 1; fi
scripts/test-mcp-call-timeout.sh
Executable
+134
View File
@@ -0,0 +1,134 @@
#!/usr/bin/env bash
# test-mcp-call-timeout.sh — the per-call deadline of RemoteMcpClient in
# extensions/pi/mempalace.ts, and the override that the feed's mine relies on.
#
# WHY THIS EXISTS. The feed tick calls `mempalace_mine` through
# `client.callTool()`. Its own deadline (MEMPALACE_FEED_MINE_TIMEOUT_MS,
# 300 000) was raised in 2026-09 because the mine legitimately takes 30-60 s on
# a shared single-writer palace — but callTool() carried no way to pass that
# deadline down, so the transport's generic per-request timeout
# (MEMPALACE_MCP_TIMEOUT_MS, 60 000) fired first, and operators saw
# feed (tick) failed: mempalace remote request 'tools/call' failed: timed out after 60000ms
# instead of the message the 2026-09 change had aimed at. The outer race was
# unreachable in practice. This harness pins both halves of the contract:
# 1. a plain callTool() still honours the short per-request timeout
# (a query taking that long really is wedged — keep it short);
# 2. callTool(name, args, { timeoutMs }) honours the override, so a long
# server-side job can be given its own, longer, deadline.
# Before the fix, (2) fails: the third argument was silently ignored.
#
# Like test-owed-withdrawal.sh, it runs the SHIPPED text: the class is cut out
# of mempalace.ts by brace matching, type-stripped with node's own stripper,
# and driven against a local JSON-RPC server that delays tools/call.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SRC="${1:-$REPO_ROOT/extensions/pi/mempalace.ts}"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
[ -r "$SRC" ] || { echo "FAIL: cannot read $SRC" >&2; exit 2; }
# ---------------------------------------------------------------- extractor ---
cat >"$WORK/extract.mjs" <<'EXTRACT'
import { readFileSync, writeFileSync } from "node:fs";
import { stripTypeScriptTypes } from "node:module";
const src = readFileSync(process.argv[2], "utf8");
/** Cut a top-level `const NAME = ...;` out of the source, verbatim. */
function decl(name) {
const start = src.indexOf(`const ${name} =`);
if (start < 0) throw new Error(`declaration not found: ${name}`);
return src.slice(start, span(start));
}
/** Cut `class NAME ... { ... }` out of the source, verbatim. */
function klass(name) {
const start = src.indexOf(`class ${name} `);
if (start < 0) throw new Error(`class not found: ${name}`);
return src.slice(start, span(start, true));
}
/** End offset of the statement starting at `start` (brace-matched). */
function span(start, braceOnly = false) {
let depth = 0, inStr = null, sawBrace = false;
for (let i = start; i < src.length; i++) {
const c = src[i], prev = src[i - 1];
if (inStr) { if (c === inStr && prev !== "\\") inStr = null; continue; }
if (c === '"' || c === "'" || c === "`") { inStr = c; continue; }
if (c === "/" && src[i + 1] === "/") { i = src.indexOf("\n", i); if (i < 0) break; continue; }
if (c === "/" && src[i + 1] === "*") { i = src.indexOf("*/", i) + 1; continue; }
if (c === "{" || c === "(" || c === "[") { depth++; if (c === "{") sawBrace = true; }
else if (c === "}" || c === ")" || c === "]") { depth--; if (braceOnly && sawBrace && depth === 0) return i + 1; }
else if (!braceOnly && c === ";" && depth === 0) return i + 1;
}
throw new Error(`unterminated statement at ${start}`);
}
const parts = [decl("num"), decl("REMOTE_PROTOCOL_VERSION"), decl("REMOTE_CLIENT_INFO"), klass("RemoteMcpClient")];
for (const [i, p] of parts.entries()) if (p.length < 30) throw new Error(`extraction ${i} implausibly short: ${p}`);
if (!parts[3].includes("tools/call")) throw new Error("RemoteMcpClient does not mention tools/call");
const ts = parts.join("\n\n") + "\nexport { RemoteMcpClient };\n";
writeFileSync(process.argv[3], stripTypeScriptTypes(ts, { mode: "strip" }));
EXTRACT
node --no-warnings "$WORK/extract.mjs" "$SRC" "$WORK/client.mjs" || exit 2
node --check "$WORK/client.mjs" || { echo "FAIL: extracted client does not parse" >&2; exit 2; }
echo "[extract] pulled RemoteMcpClient from $(basename "$SRC") ($(wc -c <"$WORK/client.mjs") bytes)"
# ------------------------------------------------------------------ harness ---
cat >"$WORK/run.mjs" <<'RUN'
import { createServer } from "node:http";
import { RemoteMcpClient } from "./client.mjs";
// A sessionless JSON-RPC server like `mempalace-mcp --transport http`:
// initialize / tools/list answer at once; tools/call sleeps SLOW_MS first.
const SLOW_MS = 1500;
const server = createServer((req, res) => {
let body = "";
req.on("data", (c) => (body += c));
req.on("end", () => {
const msg = JSON.parse(body);
if (msg.id === undefined) { res.writeHead(202); res.end(); return; } // notification
const reply = (result) => {
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ jsonrpc: "2.0", id: msg.id, result }));
};
if (msg.method === "initialize") return reply({ protocolVersion: "2024-11-05", capabilities: {}, serverInfo: { name: "fake", version: "0" } });
if (msg.method === "tools/list") return reply({ tools: [{ name: "slow", description: "", inputSchema: { type: "object" } }] });
if (msg.method === "tools/call") return void setTimeout(() => reply({ content: [{ type: "text", text: "done" }] }), SLOW_MS);
reply({});
});
});
await new Promise((r) => server.listen(0, "127.0.0.1", r));
const url = `http://127.0.0.1:${server.address().port}/mcp`;
let failures = 0;
const check = (ok, label) => { console.log(`${ok ? "ok " : "FAIL"} ${label}`); if (!ok) failures++; };
// Short per-request deadline, deliberately below SLOW_MS.
process.env.MEMPALACE_MCP_TIMEOUT_MS = "400";
const client = new RemoteMcpClient(url);
await client.start();
check(client.alive === true, "start(): initialize + tools/list complete, client alive");
// 1. A plain call honours the short deadline — this is the wedged-query guard.
let err = null;
const t0 = Date.now();
try { await client.callTool("slow", {}); } catch (e) { err = e; }
check(err !== null && /timed out after 400ms/.test(err.message), `plain callTool() rejects at the per-request deadline (${err && err.message})`);
check(Date.now() - t0 < SLOW_MS, "…and rejects BEFORE the server would have answered");
check(client.alive === false, "a timeout marks the client unhealthy (documented side effect; ensureAlive() revives)");
// 2. A call with its own deadline outlives the generic one — the feed's mine.
await client.ensureAlive();
err = null;
let result = null;
try { result = await client.callTool("slow", {}, { timeoutMs: 5000 }); } catch (e) { err = e; }
check(err === null && result && result.content?.[0]?.text === "done", `callTool(name, args, { timeoutMs: 5000 }) waits past the generic deadline and gets the result (${err ? err.message : "ok"})`);
// 3. The override is itself a deadline, not "no deadline".
err = null;
try { await client.callTool("slow", {}, { timeoutMs: 200 }); } catch (e) { err = e; }
check(err !== null && /timed out after 200ms/.test(err.message), `callTool(..., { timeoutMs: 200 }) rejects at ITS deadline (${err && err.message})`);
server.close();
console.log(failures === 0 ? "PASS: all assertions held" : `FAIL: ${failures} assertion(s) failed`);
process.exit(failures === 0 ? 0 : 1);
RUN
cd "$WORK" && node --no-warnings run.mjs
scripts/test-owed-withdrawal.sh
+44 -3
View File
@@ -23,7 +23,9 @@
# withdrawal had no effect, so tor-ms22 was still being told it owed a reply 41h
# later for a release it never installed.
#
# Usage: scripts/test-owed-withdrawal.sh [source.ts] (exit 0 = all rules behave)
# Usage: scripts/test-owed-withdrawal.sh [source.ts]
# exit 0 = all rules behave 1 = a rule broke
# exit 2 = source unreadable or does not parse 3 = the gate itself cannot run
#
# The optional argument exists so the suite can be pointed at a deliberately
# MUTATED copy of the source to prove it is sensitive — a suite that has never
@@ -39,8 +41,47 @@ trap 'rm -rf "$WORK"' EXIT
[ -r "$SRC" ] || { echo "FAIL: cannot read $SRC" >&2; exit 2; }
# A gate that cannot run must not pass — the standing rule in this repo.
node --experimental-strip-types --check "$SRC" \
|| { echo "FAIL: $SRC does not parse" >&2; exit 2; }
#
# NOT `node --check`: it does not type-strip, so it rejects ANY TypeScript —
# `const x: number = 1` included, not just the inline type-import at the top of
# mempalace.ts. It passed on node 22.x and stopped passing on node 24.x
# (measured on pi-devbox v1.9.1, node v24.21.0: this gate exited 2 on an
# unmodified mempalace.ts, so all 17 assertions below refused to run). Strip
# first, then syntax-check the emitted JS. `mode: "strip"` blanks type syntax
# without moving anything, so byte offsets and line numbers survive and a
# reported error line still points at the right line of the ORIGINAL .ts.
#
# The two failure modes are reported separately and on purpose. "Cannot strip"
# is a fact about the toolchain; "does not parse" is a fact about the source.
# Collapsing them is what made this very defect present itself as
# "mempalace.ts does not parse" when mempalace.ts was fine.
#
# SC2016 is disabled deliberately: the single quotes are the point. What follows
# is JavaScript, and `${process.version}` must reach node, not be expanded by the
# shell first.
# shellcheck disable=SC2016
node --no-warnings -e '
const { readFileSync, writeFileSync } = require("node:fs");
const { stripTypeScriptTypes } = require("node:module");
if (typeof stripTypeScriptTypes !== "function") {
console.error(`FAIL: node ${process.version} cannot strip TypeScript ` +
`(module.stripTypeScriptTypes needs >= 22.13) — the gate is unavailable, ` +
`which is NOT a statement about the source`);
process.exit(3);
}
let js;
try {
js = stripTypeScriptTypes(readFileSync(process.argv[1], "utf8"), { mode: "strip" });
} catch (err) {
// The stripper is itself a parser, so a syntax error lands HERE, not in the
// --check below. This is a statement about the source: exit 2, not 3.
console.error(`FAIL: ${process.argv[1]} does not parse: ${err.message}`);
process.exit(2);
}
writeFileSync(process.argv[2], js);
' "$SRC" "$WORK/stripped.mjs" || exit $?
node --check "$WORK/stripped.mjs" \
|| { echo "FAIL: $SRC does not parse (stripped output rejected)" >&2; exit 2; }
# ---------------------------------------------------------------- extractor ---
cat >"$WORK/extract.mjs" <<'EXTRACT'
scripts/test-wakeup-mode.sh
Executable
+150
View File
@@ -0,0 +1,150 @@
#!/usr/bin/env bash
# test-wakeup-mode.sh — print and JSON mode (`pi -p`, `--mode json`) must not
# do the palace start-up work of extensions/pi/mempalace.ts unless
# MEMPALACE_WAKEUP_PRINT=1. That work is the wake-up injection (status, diary,
# tier-1 mailbox) and the mid-session mailbox poll. TUI and RPC sessions keep
# both.
#
# WHY (2026-10-13, tor-ms22). A one-shot run is a script asking one question.
# The wake-up cost it several palace round trips and a large orientation
# message, and in one probe the model read the injected diary's open items as
# its task. Given "Reply with exactly: PONG", it ran about 12 tool calls and
# filed a wrong drawer in the shared palace.
#
# Same method as test-feed-exit.sh: the SHIPPED text is type-stripped and loaded
# under a stub pi. A local JSON-RPC server, in a separate process, logs every
# tools/call. The driver fires session_start, before_agent_start, agent_settled
# and session_shutdown with the ctx in DRIVER_CTX, and reports whether
# before_agent_start returned the wake-up message.
# tui wake-up message, status + diary + event_list calls
# print / json no message and NO palace call at all
# print, resumed (`pi -p -c`; resume sets wokeUp) the mid-session poll
# must stay off too: no event_list call
# tui, resumed control for the line above: the poll DOES run there,
# so a zero is not vacuous
# print + MEMPALACE_WAKEUP_PRINT=1 wake-up message, as before
# rpc wake-up message (long-lived, keeps it)
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SRC="${1:-$REPO_ROOT/extensions/pi/mempalace.ts}"
WORK="$(mktemp -d)"
SERVER_PID=""
cleanup() { [ -n "$SERVER_PID" ] && kill "$SERVER_PID" 2>/dev/null; rm -rf "$WORK"; }
trap cleanup EXIT
[ -r "$SRC" ] || { echo "FAIL: cannot read $SRC" >&2; exit 2; }
cat >"$WORK/extract.mjs" <<'EXTRACT'
import { readFileSync, writeFileSync } from "node:fs";
import { stripTypeScriptTypes } from "node:module";
let src = readFileSync(process.argv[2], "utf8");
const imp = 'import { Type } from "typebox";';
if (!src.includes(imp)) throw new Error(`expected import line not found: ${imp}`);
src = src.replace(imp, 'const Type = { Unsafe: (s) => s, Object: () => ({}) };');
if (!src.includes("before_agent_start")) throw new Error("extension has no before_agent_start handler");
writeFileSync(process.argv[3], stripTypeScriptTypes(src, { mode: "strip" }));
EXTRACT
node --no-warnings "$WORK/extract.mjs" "$SRC" "$WORK/ext.mjs" || exit 2
node --check "$WORK/ext.mjs" || { echo "FAIL: stripped extension does not parse" >&2; exit 2; }
echo "[extract] type-stripped $(basename "$SRC") ($(wc -c <"$WORK/ext.mjs") bytes)"
cat >"$WORK/server.mjs" <<'SERVER'
import { createServer } from "node:http";
import { appendFileSync } from "node:fs";
const tools = ["mempalace_status", "mempalace_diary_read", "mempalace_event_list"]
.map((name) => ({ name, description: "", inputSchema: { type: "object" } }));
const server = createServer((req, res) => {
let body = "";
req.on("data", (c) => (body += c));
req.on("end", () => {
const msg = JSON.parse(body);
if (msg.id === undefined) { res.writeHead(202); res.end(); return; }
const reply = (result) => {
res.writeHead(200, { "content-type": "application/json", connection: "close" });
res.end(JSON.stringify({ jsonrpc: "2.0", id: msg.id, result }));
};
if (msg.method === "initialize") return reply({ protocolVersion: "2024-11-05", capabilities: {}, serverInfo: { name: "fake", version: "0" } });
if (msg.method === "tools/list") return reply({ tools });
if (msg.method === "tools/call") {
const name = msg.params?.name;
appendFileSync(process.env.CALL_LOG, `${name}\n`);
const text = name === "mempalace_event_list" ? JSON.stringify({ events: [], count: 0 }) : `${name} ok`;
return reply({ content: [{ type: "text", text }] });
}
reply({});
});
});
server.listen(0, "127.0.0.1", () => console.log(server.address().port));
SERVER
CALL_LOG="$WORK/calls.log" node "$WORK/server.mjs" >"$WORK/port" &
SERVER_PID=$!
for _ in $(seq 50); do [ -s "$WORK/port" ] && break; sleep 0.1; done
PORT="$(cat "$WORK/port")"
[ -n "$PORT" ] || { echo "FAIL: fake palace did not start" >&2; exit 2; }
cat >"$WORK/driver.mjs" <<'DRIVER'
const ctx = { ...JSON.parse(process.env.DRIVER_CTX || "{}"), ui: { notify() {} } };
const handlers = {};
const pi = new Proxy({}, {
get: (_t, k) => k === "on" ? (ev, fn) => { (handlers[ev] ||= []).push(fn); } : () => {},
});
const { default: ext } = await import("./ext.mjs");
await ext(pi);
const fire = async (ev, event) => {
const out = [];
for (const fn of handlers[ev] || []) out.push(await fn({ type: ev, ...event }, ctx));
return out;
};
await fire("session_start", { reason: process.env.DRIVER_REASON || "startup" });
const results = await fire("before_agent_start", { prompt: "PONG" });
const woke = results.some((r) => r?.message?.customType === "mempalace-wakeup");
console.log(`WAKEUP_MESSAGE=${woke ? "yes" : "no"}`);
await fire("agent_settled", {});
await new Promise((r) => setTimeout(r, 800)); // the mailbox poll is not awaited
await fire("session_shutdown", { reason: "quit" });
DRIVER
failures=0
check() { if [ "$1" = 0 ]; then echo "ok $2"; else echo "FAIL $2"; failures=$((failures + 1)); fi; }
calls() { grep -c "^$1\$" "$WORK/calls.log" || true; }
run_case() { # $1 ctx JSON, $2 session_start reason, [$3 WAKEUP_PRINT] -> MSG RC ALL STATUS DIARY EVENTS
: >"$WORK/calls.log"
set +e
out="$(cd "$WORK" && env -u MEMPALACE_REMOTE_TOKEN -u MEMPALACE_MAILBOX \
MEMPALACE_REMOTE_URL="http://127.0.0.1:$PORT/mcp" MEMPALACE_PI_DEVICE=testdev \
MEMPALACE_FEED=0 DRIVER_CTX="$1" DRIVER_REASON="$2" MEMPALACE_WAKEUP_PRINT="${3:-}" \
timeout 60 node --no-warnings driver.mjs 2>"$WORK/driver.err")"
RC=$?
set -e
MSG="$(printf '%s\n' "$out" | sed -n 's/^WAKEUP_MESSAGE=//p')"
ALL="$(wc -l <"$WORK/calls.log" | tr -d ' ')"
STATUS="$(calls mempalace_status)"; DIARY="$(calls mempalace_diary_read)"; EVENTS="$(calls mempalace_event_list)"
echo "[$4] rc=$RC message=${MSG:-?} calls=$ALL (status=$STATUS diary=$DIARY event_list=$EVENTS)"
}
run_case '{"mode":"tui","hasUI":true}' startup "" tui
check "$([ "$RC" = 0 ] && [ "$MSG" = yes ] && [ "$STATUS" = 1 ] && [ "$DIARY" = 1 ] && [ "$EVENTS" -ge 1 ] && echo 0 || echo 1)" \
"tui: wake-up injected, with status, diary and mailbox reads"
for m in print json; do
run_case "{\"mode\":\"$m\",\"hasUI\":false}" startup "" "$m"
check "$([ "$RC" = 0 ] && [ "$MSG" = no ] && [ "$ALL" = 0 ] && echo 0 || echo 1)" \
"$m: no wake-up message and no palace call at all (calls=$ALL)"
done
run_case '{"mode":"print","hasUI":false}' resume "" "print resumed"
check "$([ "$RC" = 0 ] && [ "$MSG" = no ] && [ "$EVENTS" = 0 ] && echo 0 || echo 1)" \
"print resumed (pi -p -c): the mid-session mailbox poll stays off (event_list=$EVENTS)"
run_case '{"mode":"tui","hasUI":true}' resume "" "tui resumed (control)"
check "$([ "$RC" = 0 ] && [ "$MSG" = no ] && [ "$EVENTS" -ge 1 ] && echo 0 || echo 1)" \
"control: tui resumed skips the injection but DOES poll (event_list=$EVENTS), so the zero above is not vacuous"
run_case '{"mode":"print","hasUI":false}' startup 1 "print+WAKEUP_PRINT=1"
check "$([ "$RC" = 0 ] && [ "$MSG" = yes ] && [ "$STATUS" = 1 ] && echo 0 || echo 1)" \
"print + MEMPALACE_WAKEUP_PRINT=1: wake-up injected, as before"
run_case '{"mode":"rpc","hasUI":true}' startup "" rpc
check "$([ "$RC" = 0 ] && [ "$MSG" = yes ] && echo 0 || echo 1)" "rpc: wake-up injected (long-lived)"
if [ "$failures" = 0 ]; then echo "PASS: all assertions held"; else echo "FAIL: $failures assertion(s) failed"; exit 1; fi