mempalace-toolkit/extensions/pi/README.md

pi ↔ MemPalace MCP bridge

The canonical source of ~/.pi/agent/extensions/mempalace.ts — the TypeScript extension that wires MemPalace's MCP server into the pi coding-agent harness. Installs wake-up context injection, per-tool schema passthrough, and a /mempalace-diary slash-command.

This directory only holds the bridge. Pi's own base config (keybindings, environment loader, settings template) lives in the sibling pi-toolkit repo — split out 2026-05-05 so opencode-devbox can build slim containers that include pi without dragging in mempalace's dependencies (~300 MB).

Jump to:

• What it does
• Transport: local vs external
• Automatic transcript feeding
• The Type.Unsafe gotcha
• Deploying pi with mempalace on a new machine
• Fail-soft, identity, debugging

---

What it does

1. Connects to MemPalace and does the MCP handshake (initialize + notifications/initialized + tools/list). By default it spawns mempalace-mcp as a local stdio subprocess (StdioMcpClient); if $MEMPALACE_REMOTE_URL is set it instead talks to a shared MemPalace over HTTP (RemoteMcpClient) and spawns no local process — see Transport.
2. Registers each MCP tool as a pi tool with its real inputSchema passed through via Type.Unsafe(...) (see gotcha below).
3. Wake-up auto-injection (before_agent_start, one-shot per fresh session): calls mempalace_status + mempalace_diary_read and injects the result as a mempalace-wakeup system message so the agent orients itself the way ~/.agents/skills/mempalace/SKILL.md describes. Skipped on resume/fork (context is already in the thread).
4. Automatic transcript feeding (session_shutdown, and a debounced agent_settled): stages + mines this pi installation's own session transcripts into the palace with no user action needed — as of mempalace-toolkit 29e660e (2026-08-12); see the version gate below, because "the extension is installed" does not imply "this copy can feed". Unlike the diary below, this needs no LLM turn — it's a subprocess + a tool call — so it can run on session_shutdown where the diary cannot. See Automatic transcript feeding.
5. Manual wind-down via a /mempalace-diary [topic] slash command: sends a prompt asking the LLM to call mempalace_diary_write with an AAAK-formatted entry summarizing the session. This one stays manual because it needs the LLM to compose the entry, and session_shutdown fires too late to drive another LLM turn — a constraint that applies to the diary specifically, not to feeding (see above).

Automatic transcript feeding

⚠️ Version gate — requires mempalace-toolkit ≥ 29e660e (2026-08-12), and "installed" is not the same question as "capable". Feeding was added to this extension on 2026-08-12. A copy baked into a container image built before that date has no feed path at all — its entire session_shutdown handler is client.stop() — and it fails the only way a memory system must not: silently, looking exactly like a healthy run with nothing to do.

Check the deployed artifact, never the repo. /opt/* in an image is baked at build time and can be days behind a bind-mounted clone, and ~/.pi/agent/extensions/mempalace.ts is usually a symlink into that baked copy:

grep -c MEMPALACE_FEED "$(readlink -f ~/.pi/agent/extensions/mempalace.ts)"   # 0 = cannot feed

Zero hits means this machine needs the fallback recipes in contrib/ until it is rebuilt, regardless of what the toolkit repo's HEAD looks like. Date the deployed copy with stat plus that content probe — not git log, which fails with "detected dubious ownership" inside a root-owned /opt tree. As of 2026-08-14 the whole pi-devbox fleet fails this check.

The bridge feeds this pi installation's own session transcripts into the palace by itself — no scheduler, no cron, no manual invocation. It fires on session_shutdown (covers quit, /new, /resume, /fork) and on a debounced agent_settled (covers a long session that later crashes, since a hard kill runs no shutdown handler at all).

The work is split across two processes, and the reason is a hard constraint, not a style choice: the palace is single-writer. A live pi session always holds it through this extension's own mempalace-mcp subprocess, so an unattended mempalace mine from anywhere else fails outright with palace ... is held by PID <n>. The bridge therefore:

1. Runs mempalace-pi-session --prepare --reason <trigger> --wing <wing> as a subprocess. This does every palace-free step — parse pi's JSONL, apply the quality threshold, stage the export, and (remote mode only) rsync it to the palace host — and prints one line, MINE_SOURCE=<path>, without ever touching the palace.
2. Calls the mempalace_mine MCP tool through this extension's own client on that path. Going through the client that already holds the lock is the only way to write during a live session, and it automatically targets whichever palace the bridge is pointed at — local stdio or a shared remote one.

mempalace-pi-session (in this repo's bin/) is the actual exporter and owns the quality gate, the remote transport, and every flag — see its --help for the full reference; this section only covers the extension's side of the wiring.

Env knobs (extension side):

Var	Default	Effect
MEMPALACE_FEED	1	Set 0 to disable automatic feeding entirely.
MEMPALACE_FEED_BIN	mempalace-pi-session	Helper to run.
MEMPALACE_FEED_WING	wing_conversations	Target wing — passed to both the exporter and the mempalace_mine call.
MEMPALACE_FEED_DEBOUNCE_MS	600000 (10 min)	Minimum gap between mid-session (agent_settled) feeds. Bounds crash loss to one window instead of a whole session.
MEMPALACE_FEED_PREPARE_TIMEOUT_MS	120000	Kills a wedged --prepare subprocess.
MEMPALACE_FEED_MINE_TIMEOUT_MS	30000	Caps the mempalace_mine call so a stalled palace can't hang session exit.

Remote palace: if $MEMPALACE_REMOTE_URL is set (see Transport), mempalace_mine's source path is expanded on the server, which cannot see this machine's transcripts — that's exactly why step 1 above rsyncs first in that mode. Configure the inbox with MEMPALACE_PI_SSH_TARGET (required for remote feeding — feeding is silently skipped without it), MEMPALACE_PI_SSH_CONFIG, and MEMPALACE_PI_REMOTE_PATH; see mempalace-pi-session --help.

Concurrency: overlapping triggers coalesce — a session_shutdown landing while a debounced tick is still running joins that in-flight feed instead of racing it. mempalace-pi-session itself also takes a non-blocking flock, so even two independent invocations (e.g. this extension and the container-start catch-up some devbox images run) never race each other; losing that race is harmless because the next trigger re-exports from scratch.

Transport: local vs external

The bridge speaks the same MCP protocol over two interchangeable transports, chosen at load time:

• Local (default) — spawns mempalace-mcp as a stdio subprocess; the palace lives wherever that process opens it (default ~/.mempalace). This is the hardened path with per-request timeouts and respawn/self-heal (below).

• External — set MEMPALACE_REMOTE_URL to a MemPalace HTTP endpoint (e.g. https://mempalace.jordbo.se/mcp, the live fleet primary — full path including /mcp, no trailing slash) and the bridge connects over HTTP instead, spawning no local process. Use this to share one palace across several harnesses/containers (pi + opencode + native). MEMPALACE_REMOTE_TOKEN, if set, is sent as Authorization: Bearer <token>. Use https:// for anything crossing a network — the plaintext http:// example that stood here until 2026-08-14 predated the reverse proxy.

  The two transports are either/or, decided once at load time: with the URL set, writes go only to the remote palace. There is no dual-write, no local mirror, and no local mempalace-mcp process at all.

  ⚠️ Consequence: once MEMPALACE_REMOTE_URL is set, the mempalace CLI on that machine is no longer a valid way to inspect or feed the palace the agent is using. The CLI has no remote support whatsoever — its only selector is --palace <path> — so it reads and writes the LOCAL on-disk archive. After a flip that archive is frozen, yet mempalace status / mempalace search still report a plausible drawer count and look exactly like success: a false-positive machine. Memories filed with the CLI post-flip land in the dead archive, not in the shared palace. Use the agent's own palace tools (which go over HTTP), and mine backfills on the palace host.

  Setting these for a native pi install — there is no .env to edit. Worth stating plainly, because the obvious guess is wrong: pi loads no dotenv file and has no env block in settings.json, and this extension reads process.env and nothing else (createClient() → process.env.MEMPALACE_REMOTE_URL). A native install therefore inherits whatever the shell that launches pi exports — that is the only hook. So export them from your shell rc, or from a file it explicitly sources:

  # ~/.zshrc (or ~/.bashrc) — if you keep secrets in ~/.config/pi/.env,
  # nothing sources it for you; do it yourself:
  set -a; [ -f ~/.config/pi/.env ] && . ~/.config/pi/.env; set +a
  

  Two traps. A pi launched from a GUI (Spotlight, dock, an editor's terminal that spawns a non-login shell) does not necessarily read that rc, so it can silently stay on the local palace. And exporting the variable in the shell where you edited the rc does not affect an already-running pi — the transport is chosen once at extension load. Confirm the result the same way as a container flip: ask the agent for mempalace_status and check the reported palace path is the remote host's, not your own $HOME/.mempalace/palace — see docs/phase-1-exposure-runbook.md §3.8.

  Serve such an endpoint with mempalace serve --host 172.17.0.1 --port 8765 (the pi-devbox / opencode-devbox repos ship a docker-compose.mempalace.yml for exactly this).

  The HTTP transport is authenticated as of mempalace 3.6.0 — earlier docs here said otherwise, from the v1.3.0 era. serve mints a bearer token, keeps it 0600, passes it via the environment (never argv), compares it with hmac.compare_digest, and refuses to bind a non-loopback host without one unless --allow-insecure. It also pins Host and allowlists Origin (anti-DNS-rebinding), and can terminate TLS itself.

  Two binds to avoid. 0.0.0.0 publishes the palace to the whole LAN. And 127.0.0.1 is the trap that looks safe: the Host pin is enforced only on loopback binds, so behind a tunnel every proxied request 403s — and token auto-minting is gated on the bind being non-loopback, so it starts with no authentication at all, no warning. Bind the docker0 gateway (172.17.0.1): reachable from the host and its containers, not from the LAN. See docs/phase-1-exposure-runbook.md.

  Implementation note: the HTTP client (RemoteMcpClient) is vendored from pi-extensions' mcp-loader.ts. A MCP-STREAMABLE-HTTP-CLIENT-SYNC token keeps the two copies from drifting — scripts/check-mcp-client-sync.sh fails if they diverge (it skips gracefully when the pi-extensions checkout isn't present).

Fail-soft

If mempalace-mcp can't be spawned (PATH missing, binary crashes at startup, …) the extension logs to stderr and returns early. pi keeps working without palace tools rather than refusing to start.

In remote mode the triggers differ but the outcome is identical. An unreachable server, a DNS failure, or an HTTP 401 from a wrong/expired token all end the same way: after bounded retries the extension prints mempalace-mcp unavailable after retries; continuing without palace tools and does not register the palace tools.

It is fail-closed, not fail-local: it does not quietly fall back to the local palace, so a remote outage can never scatter memories into a local copy nobody will look at again. The practical corollary, worth knowing before you debug the wrong layer: "the agent has no mempalace_* tools" is the expected symptom of a server, token, or DNS fault, not of a broken install. Diagnose it with a direct curl to MEMPALACE_REMOTE_URL — see docs/phase-1-exposure-runbook.md §3.8. The design rationale for de-registering rather than degrading is in docs/rfc-001-global-palace.md §2 and §4.1.

Identity

agent_name for diary calls comes from $MEMPALACE_AGENT_NAME, defaulting to "pi". First diary write against that identity creates wing_<name> in the palace. Set the env var if you want to run pi under a distinct identity on a given machine (e.g. pi-laptop vs pi-server).

Stall protection (per-request timeout)

Every JSON-RPC request to mempalace-mcp carries a timeout. Without it, a wedged server (classically: an OrbStack/virtiofs cold-open of a large chroma.sqlite3 or an HNSW load) leaves the awaiting promise pending forever, which freezes the pi TUI — ESC cancels the LLM stream, not a pending tool execute(). On timeout the extension rejects the request and kills the stalled child (SIGTERM→SIGKILL), so pi gets a clear error instead of hanging. This is a per-REQUEST timeout, not a process-lifetime one — the long-lived server is only killed when a request genuinely stalls.

• MEMPALACE_MCP_TIMEOUT_MS — tool-call/request timeout. Default 60000. Kept short on purpose: a query taking this long is genuinely wedged.
• MEMPALACE_MCP_INIT_TIMEOUT_MS — initialize + tools/list handshake timeout. Default 300000. Deliberately generous: a genuine first cold-open over virtiofs can legitimately take minutes, and killing a still-progressing init only to respawn and re-pay the same cold cost is strictly worse than waiting.
• Set either to 0 to disable (legacy unbounded behavior).

Self-heal (respawn instead of a permanent latch)

A stall-kill (or any crash) used to be a permanent latch: available flipped off and stayed off until you restarted pi. It is now self-healing — the next tool call transparently respawns mempalace-mcp and retries.

• Respawns use capped exponential backoff so a persistently-broken server can't hot-loop: MEMPALACE_MCP_MAX_RESPAWNS attempts (default 2; set 0 to disable self-heal and keep the old fail-fast latch), with MEMPALACE_MCP_RESPAWN_BACKOFF_MS (default 1000) doubled per attempt.
• The budget resets on any successful JSON-RPC response — proof the server is actually live — so a server that recovers regains full patience, while one that keeps dying hits the cap and stays down (then restart pi).
• Why the long init timeout and bounded respawn compose rather than overlap: once a server has opened the palace once, the OS page cache is warm, so respawn cold-opens are fast. The long init timeout prevents killing a healthy first cold-open; the respawn handles a genuinely dead server cheaply afterwards. (Note the HNSW deserialize is CPU work that isn't page-cacheable across spawns, which is exactly why we can't rely on respawn-warming alone and keep the generous init budget.)
• The initial startup is tolerant too: if the very first start() fails, the extension runs the same bounded respawn before falling back to fail-soft (pi keeps working without palace tools).

Debugging

• MEMPALACE_EXT_DEBUG=1 — surface mempalace-mcp stderr into pi's stderr. Without this, stderr is drained silently so a misbehaving server doesn't flood the TUI.
• If a tool call fails with a generic "Internal tool error", spawn mempalace-mcp manually with raw JSON-RPC on stdin to read the server-side error — much faster than guessing.

The Type.Unsafe gotcha

Earlier versions of this extension registered every MCP tool with parameters: Type.Object({}, { additionalProperties: true }), which discarded each tool's real inputSchema. The LLM then saw no parameter names and had to guess, leading to bugs like mempalace_diary_read being called with agent= instead of the required agent_name= and crashing the Python server with TypeError: missing 1 required positional argument.

The fix (≈ lines 160-170) is to wrap the incoming JSON Schema with Type.Unsafe<...>(tool.inputSchema). TypeBox schemas are plain JSON Schema at runtime plus a Symbol marker, so wrapping an externally-sourced schema with Unsafe is sufficient — no conversion to a full TypeBox tree is needed, and the LLM now sees every tool's real parameter names.

If you ever need to re-loosen the schema for debugging, fall back to the Type.Object({}, { additionalProperties: true }) default only for that specific tool, not globally.

---

Deploying pi with mempalace on a new machine

This is the "pi + memory" recipe. For pi without mempalace, see pi-toolkit's README.

0. Prerequisites

• Shell: zsh + oh-my-zsh recommended (both toolkits install loaders into ~/.oh-my-zsh/custom/; bash works too, installers print the manual source snippet).
• git, node ≥ 20, uv, tmux ≥ 3.2, pi installed upstream.
• AWS credentials reachable via AWS_PROFILE — only if using amazon-bedrock as pi's provider.

1. Dotfiles (if you keep one)

Brings ~/.config/pi/.env (AWS creds, git-crypt encrypted), tmux CSI-u extended keys, and other machine state:

git clone <your-dotfiles> ~/src/dotfiles
cd ~/src/dotfiles
git-crypt unlock <key>
./provision.sh --profile <profile>    # or your equivalent tool

2. Install pi upstream

brew install pi-coding-agent       # macOS
# or see https://github.com/earendil-works/pi for Linux
pi --help                          # creates ~/.pi/agent/

3. Install pi-toolkit (base pi config)

git clone ssh://git@gitea.jordbo.se:2222/joakimp/pi-toolkit.git ~/pi-toolkit
cd ~/pi-toolkit && ./install.sh

Symlinks keybindings.json, copies pi-env.zsh into ~/.oh-my-zsh/custom/, and prints the settings.json bootstrap command.

4. Bootstrap pi settings

cp ~/pi-toolkit/settings.example.json ~/.pi/agent/settings.json
$EDITOR ~/.pi/agent/settings.json   # eu./us./anthropic: prefix

5. Install mempalace CLI + this toolkit

uv tool install mempalace
git clone ssh://git@gitea.jordbo.se:2222/joakimp/mempalace-toolkit.git ~/mempalace-toolkit
cd ~/mempalace-toolkit && ./install.sh

Detects pi, symlinks mempalace.ts into ~/.pi/agent/extensions/. Also detects pi-toolkit artifacts and prints a green check (or a warning telling you to install pi-toolkit first if you skipped step 3).

6. Register mempalace MCP with opencode (if applicable)

Skip if this box is pi-only. Otherwise:

• Install opencode-toolkit so ~/.config/opencode/.env is sourced into every shell (GitHub / Gitea / other MCP server tokens).
• Register the mempalace MCP server in ~/.config/opencode/opencode.json — see root README § Registering mempalace with opencode.

7. First run

exec zsh
pi   # should start with defaults; wake-up injection shows palace status

If the wake-up doesn't print, run MEMPALACE_EXT_DEBUG=1 pi to surface mempalace-mcp stderr.

Verification checklist

# MCP bridge in place
ls -la ~/.pi/agent/extensions/mempalace.ts    # → this repo

# pi-toolkit artifacts also in place
ls -la ~/.pi/agent/keybindings.json           # → pi-toolkit
ls -la ~/.oh-my-zsh/custom/pi-env.zsh         # cp from pi-toolkit

# Env loaded
zsh -ic 'echo $AWS_PROFILE $AWS_REGION'

# Palace reachable
mempalace status

Uninstall

cd ~/mempalace-toolkit && ./install.sh --uninstall --yes   # bridge only
cd ~/pi-toolkit        && ./install.sh --uninstall --yes   # pi base config
# Leaves pi itself, mempalace CLI, and ~/.config/pi/.env alone.

---

File layout

mempalace-toolkit/
└── extensions/
    └── pi/
        ├── README.md        ← this file
        └── mempalace.ts     ← symlinked into ~/.pi/agent/extensions/

Pi base config (keybindings, env loader, settings template) lives in pi-toolkit. install.sh detects pi via ~/.pi/agent/extensions/ and runs a check_pi_toolkit probe that warns if pi-toolkit's artifacts are missing.