feeders: stage beside the palace, not in ~/.cache; document Phase 1 exposure

Staging default moves out of ~/.cache to <palace-root>/pi-stage (pi) and
<palace-root>/opencode-stage (opencode), resolved with mempalace's own
palace-path precedence ($MEMPALACE_PALACE_PATH -> $MEMPAL_PALACE_PATH ->
~/.mempalace/config.json -> ~/.mempalace/palace), then dirname.

Why: the convos miner keys dedup on the *staged* path, so a wiped stage plus a
sync scoped to include it prunes the drawers mined from those sources --
deleting memories, not a cache. Under ~/.cache that state was reachable by
anything treating a cache as disposable. Staging inside the palace makes the
coupling structural: the stage cannot be wiped without touching the palace
itself. Overrides ($MEMPALACE_PI_STAGE / $MEMPALACE_SESSION_STAGE, --stage) are
unchanged. Note the old default had never been created on any host, so this
closed a latent hazard, not a live one.

Measured, and the docs now claim only this much: sync prunes only within the
scope it is given -- wing-only, 1299 scanned / 1299 out of scope / 0 removed;
scoped at the palace root, 651 kept / 648 out of scope. The previous blanket
"sync prunes every drawer" wording overstated it, which is a liability: the next
reader disproves the overstatement and discards the real constraint with it.

Also in this change:
- cron log dir ~/.cache/mempalace-session -> ~/.cache/mempalace-logs. The stage
  left that namespace, so the old name now read as "the stage".
- AGENTS.md: the convos miner *does* check mtime (verified against upstream
  convo_miner.py); the previous "no mtime check" claim was wrong.
- smoke-test assertions use `mktemp -d` for --sessions-dir. One pointed at /tmp,
  which still held earlier synthetic transcripts, so a --dry-run exported a fake
  session into the real stage: --dry-run skips the mine, not the export.

docs/phase-1-exposure-runbook.md -- the newt/DNS/auth step that RFC 001 and the
synlig runbook leave open (runbook section 4, items 2 and 5). Port 8765 at /mcp,
newt targets 172.17.0.1, and the authentication is the single shared bearer
token (RFC 6.2, decided 2026-08-09) rather than per-device proxy users. The
latter cannot work today: mempalace validates exactly one token, and Pangolin's
SSO/PIN/password are browser-shaped while every client here is a headless
JSON-RPC POST -- enabling that protection breaks the clients it protects. The
per-device axis that *does* exist is the feeder's SSH key + per-device inbox.

New finding recorded there: a loopback bind does not merely 403 behind a tunnel
(already known, runbook 2.4) -- it also silently starts the server with no token
at all, because auto-minting is gated on the bind being non-loopback.

extensions/pi/README.md: the HTTP transport IS authenticated as of mempalace
3.6.0; the "sessionless and unauthenticated" note dated from the v1.3.0 era.
Closes the RFC section 8 Phase-0 hygiene item.
This commit is contained in:
Joakim Persson
2026-08-12 17:04:01 +02:00
parent 3626946013
commit 29e660e18f
15 changed files with 1019 additions and 66 deletions
+9 -1
View File
@@ -152,7 +152,15 @@ For `mempalace-docs`, test on a small repo (e.g. this one) first:
- The companion skill lives at `~/.agents/skills/opencode-mempalace-bridge/SKILL.md` and is a **symlink into this repo**. Editing that file edits `SKILL.md` here. To propagate to Claude Code / Kiro, run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils). - The companion skill lives at `~/.agents/skills/opencode-mempalace-bridge/SKILL.md` and is a **symlink into this repo**. Editing that file edits `SKILL.md` here. To propagate to Claude Code / Kiro, run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils).
- The opencode DB path defaults to `~/.local/share/opencode/opencode.db`. Override via `$OPENCODE_DB` or `--db`. - The opencode DB path defaults to `~/.local/share/opencode/opencode.db`. Override via `$OPENCODE_DB` or `--db`.
- The mempalace miner **skips symlinks** (as of v3.3.3 — `miner.py` line ~828). That's why the wrappers use `cp -p` / explicit file writes for staging, not symlinks. - The mempalace miner **skips symlinks** (as of v3.3.3 — `miner.py` line ~828). That's why the wrappers use `cp -p` / explicit file writes for staging, not symlinks.
- The convos miner dedups on `source_file` path only (no mtime check). Staging filenames must be stable per session; deleting a staged JSONL forces a re-mine. - The convos miner dedups on `source_file` path **and** `mtime`
(`file_already_mined(..., check_mtime=True)` in upstream `convo_miner.py`).
A changed/grown transcript is detected, purged, and refiled — it is not
silently skipped. What must still be stable across runs is the *staged
path itself*: if a wrapper's staging dir is wiped, dedup has nothing to
compare against and `mempalace sync` will treat the vanished sources as
deleted and prune the drawers mined from them. (An earlier version of this
file claimed "no mtime check" — that was wrong; verified against
`convo_miner.py` 2026-08.)
- The docs miner dedups on `source_file` path + `mtime`. That's why staging uses `cp -p` (preserves mtime). - The docs miner dedups on `source_file` path + `mtime`. That's why staging uses `cp -p` (preserves mtime).
## Colocated skill pattern ## Colocated skill pattern
+84 -6
View File
@@ -31,6 +31,13 @@ So on a machine using opencode + the "docs-first palace hygiene" policy, three g
The two wrappers in `bin/` close gaps **1** and **2**. Gap **3** is upstream work (see §6). The two wrappers in `bin/` close gaps **1** and **2**. Gap **3** is upstream work (see §6).
(Pi is a different story on gap 3: `bin/mempalace-pi-session` closes pi's
version of gap 2 the same way `mempalace-session` closes opencode's, but pi's
gap 3 is already closed **without** any upstream dependency — its bridge
extension self-triggers the feed on `session_shutdown` and a debounced
`agent_settled`. See §3's `mempalace-pi-session` subsection and
[`extensions/pi/README.md`](extensions/pi/README.md).)
--- ---
## 2. The architecture ## 2. The architecture
@@ -53,7 +60,7 @@ The two wrappers in `bin/` close gaps **1** and **2**. Gap **3** is upstream wor
│ │ │ cache dir │ │ │ │ cache dir │
└─────┬──────────┘ └────┬──────────────┘ └─────┬──────────┘ └────┬──────────────┘
│ │ │ │
│ ~/.cache/mempalace-docs/<wing>/ │ ~/.cache/mempalace-session/<wing>/ │ ~/.cache/mempalace-docs/<wing>/ │ <palace-root>/opencode-stage/<wing>/
│ │ │ │
┌─────▼──────────┐ ┌────▼──────────────┐ ┌─────▼──────────┐ ┌────▼──────────────┐
│ mempalace mine │ │ mempalace mine │ │ mempalace mine │ │ mempalace mine │
@@ -120,7 +127,7 @@ What it drops: source code (`.py`, `.ts`, `.go`, `.rs`, …), lockfiles, `.git`,
- `step-start` / `step-finish` → dropped as noise. - `step-start` / `step-finish` → dropped as noise.
- `reasoning` → kept, prefixed with `[reasoning]`. - `reasoning` → kept, prefixed with `[reasoning]`.
4. Serialize as Claude Code JSONL (`{"type": "user"|"assistant", "message": {"content": [...]}}`) — the one convos format the miner already understands. 4. Serialize as Claude Code JSONL (`{"type": "user"|"assistant", "message": {"content": [...]}}`) — the one convos format the miner already understands.
5. Stage at `~/.cache/mempalace-session/<wing>/<slug>_<id>.jsonl` with `mtime` = `session.time_updated` (deterministic, stable under dedup). 5. Stage at `<palace-root>/opencode-stage/<wing>/<slug>_<id>.jsonl` with `mtime` = `session.time_updated` (deterministic, stable under dedup).
**Filters:** **Filters:**
@@ -130,6 +137,73 @@ What it drops: source code (`.py`, `.ts`, `.go`, `.rs`, …), lockfiles, `.git`,
**Then:** invokes `mempalace mine --mode convos` against the cache dir. A post-mine `mempalace repair` is **opt-in** via `--repair` — it is intentionally *not* the default because the in-place HNSW rebuild has corrupted live palaces on past runs. Never pass `--repair` from an unattended schedule. **Then:** invokes `mempalace mine --mode convos` against the cache dir. A post-mine `mempalace repair` is **opt-in** via `--repair` — it is intentionally *not* the default because the in-place HNSW rebuild has corrupted live palaces on past runs. Never pass `--repair` from an unattended schedule.
### `bin/mempalace-pi-session` — pi coding-agent → palace bridge
**Input:** pi's own JSONL session transcripts under `~/.pi/agent/sessions/**/*.jsonl`
(no SQLite export step needed — pi already writes files, unlike opencode).
**Output:** palace drawers in `wing_conversations` (or `--wing` override), same
Claude Code JSONL staging shape as `mempalace-session` above.
The transform pipeline is the same shape as `mempalace-session`'s (synthetic
`[session: title | cwd | date | source: pi]` header, per-message dispatch,
`toolCall``tool_use`, `toolResult``tool_result`, `mtime` copied onto the
staged file for dedup stability) — see the script's own header comment for
the exhaustive per-role mapping. What's architecturally different from the
opencode wrapper is why this one has a **two-phase mode** and a **remote
transport**, neither of which `mempalace-session` needs:
**Two phases, because the palace is single-writer.** `mempalace` enforces
this with a per-palace `flock` (`palace.py`); the CLI's own error is explicit:
`palace ... is held by PID <n> (mempalace-mcp); wait for it to finish`. Unlike
opencode's session mine (which always runs *between* agent sessions, when
nothing else holds the palace), pi's own bridge extension
(`extensions/pi/mempalace.ts`) holds the palace open via its `mempalace-mcp`
subprocess for the *entire* live session — and that extension is also what
triggers the feed, on `session_shutdown` and a debounced `agent_settled`. An
unattended `mempalace mine` invoked from anywhere else during that window
would simply fail. So the wrapper splits:
- `--prepare` — export, threshold, stage (+ `rsync` in remote mode). Never
opens the palace. Prints `MINE_SOURCE=<path>`.
- (default, no `--prepare`) — the above, then mines it. If that mine hits
contention ("is held by"), it's treated as **success, not failure** (exit
0 with an informational message) — the holder's own extension will mine
what got staged. This inverts the CLI's own convention (`MineAlreadyRunning`
→ exit 1) deliberately, because at the wrapper layer the contention has a
benign interpretation the raw CLI can't know about.
The pi extension calls `--prepare` as a subprocess, then feeds `MINE_SOURCE`
to `mempalace_mine` through its own already-open MCP client — the only
process that can write during a live session, because it *is* the lock
holder. This is the mechanism behind [`extensions/pi/README.md` § Automatic
transcript feeding](extensions/pi/README.md#automatic-transcript-feeding).
**Remote transport, because there is no remote-palace CLI.** `mempalace`'s
`--backend` flag selects a *vector store* (chroma/qdrant/pgvector/milvus),
not a remote palace — the only remote surface is the HTTP MCP server
(`mempalace-mcp --transport http`, see [RFC-001](docs/rfc-001-global-palace.md)).
And `mempalace_mine`'s `source` path is expanded **in that server process**,
so a remote server has no way to see this machine's staged exports. `--mode
remote` (auto-selected when `$MEMPALACE_REMOTE_URL` is set) therefore
`rsync`s the stage to a per-device inbox on the palace host, then asks the
server to mine its own local copy of that inbox over the same HTTP
`tools/call` transport the extension uses. Requires
`$MEMPALACE_PI_SSH_TARGET`; see `--help` for the rest
(`MEMPALACE_PI_SSH_CONFIG`, `MEMPALACE_PI_REMOTE_PATH`, `MEMPALACE_PI_DEVICE`).
**Filters:** two gates, both required — stricter than `mempalace-session`'s
single filter because pi's transcripts have a failure mode opencode's don't:
- `--min-messages N` (default 4) — same idea as opencode's filter, raised
because pi's tool loops inflate turn counts fast.
- `--min-assistant-chars N` (default 1000) — counts assistant *text* only,
tool results excluded. Needed because pi expands skill/context text into
the user prompt: an abandoned session can have a huge "user" turn and
almost no assistant output (observed case: 13,380 injected-context user
chars answered with 38 assistant chars), so a message-count-only filter
would have filed 22 pure-noise drawers from that one session. Real
sessions on the same corpus measured 15,900100,000 assistant chars.
--- ---
## 4. Setup recipe (new machine) ## 4. Setup recipe (new machine)
@@ -168,7 +242,11 @@ The devbox uses two named Docker volumes so these persist across container recre
- `devbox-palace``~/.mempalace/palace` (the palace itself) - `devbox-palace``~/.mempalace/palace` (the palace itself)
- `devbox-data``~/.local/share/opencode` (opencode's SQLite DB) - `devbox-data``~/.local/share/opencode` (opencode's SQLite DB)
Code at `/workspace/mempalace-toolkit` is a bind mount from the host — survives container recreate and syncs via gitea. Staging directories (`~/.cache/mempalace-{docs,session}/`) are ephemeral but cheap to rebuild. Code at `/workspace/mempalace-toolkit` is a bind mount from the host — survives container recreate and syncs via gitea. The docs staging dir (`~/.cache/mempalace-docs/`) is ephemeral and cheap to rebuild.
The **conversation** staging dirs are not: `<palace-root>/opencode-stage/` and
`<palace-root>/pi-stage/` hold the exact paths the palace's `source_file` dedup keys on, so they
live beside the palace deliberately and share its lifetime. Wiping one forces a full re-mine at
best, and lets a scoped `mempalace sync` prune every drawer mined from it at worst.
**After container recreate**, just re-run `./install.sh` (idempotent) to relink `bin/` into the fresh `~/.local/bin/`. **After container recreate**, just re-run `./install.sh` (idempotent) to relink `bin/` into the fresh `~/.local/bin/`.
@@ -304,7 +382,7 @@ Quick-start (cron):
```bash ```bash
sed "s|USER|$USER|g" contrib/cron/mempalace-session.cron \ sed "s|USER|$USER|g" contrib/cron/mempalace-session.cron \
| (crontab -l 2>/dev/null; cat) | crontab - | (crontab -l 2>/dev/null; cat) | crontab -
mkdir -p ~/.cache/mempalace-session mkdir -p ~/.cache/mempalace-logs
``` ```
#### Verification #### Verification
@@ -323,7 +401,7 @@ A healthy run produces one of:
- **Incremental run**: zero to a few dozen new drawers (whatever grew since last run). - **Incremental run**: zero to a few dozen new drawers (whatever grew since last run).
- **Rerun with no new activity**: zero new drawers, only the repair step runs. - **Rerun with no new activity**: zero new drawers, only the repair step runs.
A run that files far more drawers than expected may indicate a staging-dir wipe (forcing a full re-mine) — check `~/.cache/mempalace-session/<wing>/` modification times. A run that files far more drawers than expected may indicate a staging-dir wipe (forcing a full re-mine) — check `<palace-root>/opencode-stage/<wing>/` modification times.
### Cost profile (reference) ### Cost profile (reference)
@@ -355,7 +433,7 @@ These gaps should ideally close upstream, making the wrappers thinner or obsolet
3. **Opencode harness in `hooks_cli.py`** — mempalace's hooks CLI only knows `claude-code` + `codex` today. Adding `opencode` would let the auto-save diary path work on opencode too. Pairs with #2 above. 3. **Opencode harness in `hooks_cli.py`** — mempalace's hooks CLI only knows `claude-code` + `codex` today. Adding `opencode` would let the auto-save diary path work on opencode too. Pairs with #2 above.
4. **SQLite mode for `mempalace mine --mode convos`** — if upstream ever adds direct SQLite ingest for opencode, `mempalace-session` loses its reason to exist (the export-to-JSONL dance goes away). 4. **SQLite mode for `mempalace mine --mode convos`** — if upstream ever adds direct SQLite ingest for opencode, `mempalace-session` loses its reason to exist (the export-to-JSONL dance goes away).
When #1 merges, retire `mempalace-docs` to a thin shim. When #2 + #3 land together, `mempalace-session` becomes a manual-only fallback (cron / backfill) while hooks handle live saves. When #1 merges, retire `mempalace-docs` to a thin shim. When #2 + #3 land together, `mempalace-session` becomes a manual-only fallback (cron / backfill) while hooks handle live saves. (`mempalace-pi-session` has no equivalent entry here: pi's bridge extension already self-triggers the feed with no upstream dependency — see §3.)
Separately tracked in [`docs/rfc-001-global-palace.md`](docs/rfc-001-global-palace.md): moving from one palace *per machine* to a **single primary palace with per-machine local fallback** (`mempalace-edge`). That RFC also records upstream items of its own — server-side `origin_device` provenance stamped from a per-device credential, per-wing ACLs, a `mempalace_kg_supersede` tool-classification fix, and a guard against running `mempalace sync` on a shared palace. Separately tracked in [`docs/rfc-001-global-palace.md`](docs/rfc-001-global-palace.md): moving from one palace *per machine* to a **single primary palace with per-machine local fallback** (`mempalace-edge`). That RFC also records upstream items of its own — server-side `origin_device` provenance stamped from a per-device credential, per-wing ACLs, a `mempalace_kg_supersede` tool-classification fix, and a guard against running `mempalace sync` on a shared palace.
+113 -3
View File
@@ -5,6 +5,7 @@ Producer-side tooling for [MemPalace](https://github.com/MemPalace/mempalace)
**What this repo contains:** **What this repo contains:**
- `bin/mempalace-session` — exports [opencode](https://github.com/anomalyco/opencode) session history from its local SQLite DB to Claude Code JSONL, then mines it via `mempalace mine --mode convos`. - `bin/mempalace-session` — exports [opencode](https://github.com/anomalyco/opencode) session history from its local SQLite DB to Claude Code JSONL, then mines it via `mempalace mine --mode convos`.
- `bin/mempalace-pi-session` — the same idea for the [pi coding-agent](https://github.com/earendil-works/pi): exports its native JSONL session transcripts and mines them the same way. Unlike `mempalace-session`, this one is normally invoked *for you* — the pi bridge extension (below) runs it automatically on `session_shutdown` and a debounced `agent_settled`, so most pi machines never need the `contrib/` scheduling templates at all. See [`mempalace-pi-session`](#mempalace-pi-session) below and [`extensions/pi/README.md` § Automatic transcript feeding](extensions/pi/README.md#automatic-transcript-feeding).
- `bin/mempalace-docs` — mines project directories into MemPalace while excluding source code, keeping the palace signal-dense. - `bin/mempalace-docs` — mines project directories into MemPalace while excluding source code, keeping the palace signal-dense.
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — **canonical spec**: architecture diagram, component details, setup recipe, operational notes, upstream-retirement roadmap. - [`ARCHITECTURE.md`](ARCHITECTURE.md) — **canonical spec**: architecture diagram, component details, setup recipe, operational notes, upstream-retirement roadmap.
- [`SKILL.md`](SKILL.md) — the companion agent skill, symlinked into `~/.agents/skills/opencode-mempalace-bridge/` on install. - [`SKILL.md`](SKILL.md) — the companion agent skill, symlinked into `~/.agents/skills/opencode-mempalace-bridge/` on install.
@@ -323,7 +324,17 @@ A machine running only one of these has half a memory. Full treatment with pract
### Keeping it fresh (automation) ### Keeping it fresh (automation)
Manual invocation is fine while you're actively driving the machine, but long-running devboxes benefit from a weekly automated mine. [`contrib/`](contrib/) ships ready-to-install templates: **pi:** nothing to set up. The pi bridge extension
(`extensions/pi/mempalace.ts`) feeds the palace by itself — see
[`extensions/pi/README.md` § Automatic transcript feeding](extensions/pi/README.md#automatic-transcript-feeding).
The templates below aren't needed unless you're running
`mempalace-pi-session` somewhere without that extension (e.g. a bare pi
install, or a host-side catch-up job).
**opencode**, and pi installs without the bridge: manual invocation is fine
while you're actively driving the machine, but long-running devboxes benefit
from a weekly automated mine. [`contrib/`](contrib/) ships ready-to-install
templates:
- **systemd user timer** (recommended on Linux): survives reboots, catches missed runs, logs to `journalctl`. - **systemd user timer** (recommended on Linux): survives reboots, catches missed runs, logs to `journalctl`.
- **launchd user agent** (recommended on macOS): native-equivalent — logs to `~/Library/Logs/`, single-instance guarantees, `ProcessType=Background` throttling. - **launchd user agent** (recommended on macOS): native-equivalent — logs to `~/Library/Logs/`, single-instance guarantees, `ProcessType=Background` throttling.
@@ -421,12 +432,12 @@ mempalace-session --help
- Tool outputs → `tool_result` blocks in a follow-up human message, folded back into the assistant turn by the mempalace normalizer. - Tool outputs → `tool_result` blocks in a follow-up human message, folded back into the assistant turn by the mempalace normalizer.
- `step-start` / `step-finish` parts are dropped as noise. `reasoning` parts are kept with a `[reasoning]` prefix. - `step-start` / `step-finish` parts are dropped as noise. `reasoning` parts are kept with a `[reasoning]` prefix.
**Dedup:** staging at `~/.cache/mempalace-session/<wing>/` with deterministic per-session filenames (`<slug>_<id>.jsonl`). The convos miner keys on `source_file`, so re-runs skip unchanged sessions. To force re-mining a session, delete its JSONL from the staging dir. **Dedup:** staging at `<palace-root>/opencode-stage/<wing>/` (override: `$MEMPALACE_SESSION_STAGE`) with deterministic per-session filenames (`<slug>_<id>.jsonl`). The convos miner keys on `source_file`, so re-runs skip unchanged sessions. To force re-mining a session, delete its JSONL from the staging dir.
**`--dry-run` is dedup-aware.** Each session is tagged `[NEW]` (would be filed) or `[SKIP]` (already in the palace), and the summary breaks down the count: **`--dry-run` is dedup-aware.** Each session is tagged `[NEW]` (would be filed) or `[SKIP]` (already in the palace), and the summary breaks down the count:
``` ```
Exported 62 session(s) to ~/.cache/mempalace-session/wing_conversations Exported 62 session(s) to /home/you/.mempalace/opencode-stage/wing_conversations
0 new → will be filed on mine 0 new → will be filed on mine
62 already filed → will be skipped (dedup by source_file) 62 already filed → will be skipped (dedup by source_file)
@@ -441,6 +452,105 @@ If the palace is unreachable (fresh install, moved, permission-denied) the wrapp
--- ---
## `mempalace-pi-session`
Pi coding-agent → MemPalace session bridge. Reads pi's own JSONL session
transcripts under `~/.pi/agent/sessions/`, converts each qualifying session
to Claude Code JSONL, and files via `mempalace mine --mode convos`. On most
machines you never run this by hand — the pi bridge extension
(`extensions/pi/mempalace.ts`) invokes it automatically; see
[`extensions/pi/README.md` § Automatic transcript feeding](extensions/pi/README.md#automatic-transcript-feeding).
Manual invocation is for a bare pi install without that extension, a
host-side catch-up job, or just poking at the export by hand.
```bash
mempalace-pi-session # export + mine everything qualifying
mempalace-pi-session --wing my_convos # custom wing (default: wing_conversations)
mempalace-pi-session --session <uuid-prefix> # one session only
mempalace-pi-session --since 2026-04-01 # only sessions updated on/after date
mempalace-pi-session --min-messages 6 # stricter turn-count filter (default: 4)
mempalace-pi-session --min-assistant-chars 2000 # stricter "did anything happen" filter (default: 1000)
mempalace-pi-session --dry-run # export + list, skip mine
mempalace-pi-session --prepare # export + stage only; print MINE_SOURCE=<path>, never opens the palace
mempalace-pi-session --mode remote # ship the stage to a remote palace host instead of mining locally
mempalace-pi-session --help
```
**Why `--prepare` exists:** the palace is single-writer. If a pi session for
this same palace is currently open, its own bridge extension already holds
the palace via `mempalace-mcp`, so an unattended `mempalace mine` from
anywhere else fails with `palace ... is held by PID <n>`. `--prepare` does
only the palace-free half (export, threshold, staging, and the `rsync` in
remote mode) and hands the mine off to whoever already holds the lock —
which is exactly what the pi extension does with its own MCP client. Run
without `--prepare`, the tool does the mine itself, and treats that
contention as success (exit 0, informational message) rather than failure —
the holder will mine what got staged.
**What gets exported per session:**
- Synthetic header injected as the first user turn
(`[session: <title> | <cwd> | <date> | source: pi]`) so the palace can find
sessions by topic, not just by ID, and so pi/opencode/other results stay
distinguishable in search.
- User/assistant messages extracted from pi's JSONL `message` entries.
- Assistant `toolCall` blocks → Claude Code `tool_use` blocks.
- `toolResult` role messages → `tool_result` blocks, folded back into the
assistant turn by the mempalace normalizer.
- `bashExecution`, `custom` (display-only), `branchSummary`,
`compactionSummary` → rendered as text annotations.
- `thinking` blocks and image content → dropped (noise / not text).
**Filter — two gates, both required** (this is stricter than
`mempalace-session`'s single message-count filter, and deliberately so —
see below):
1. `--min-messages` user+assistant turns (default **4**).
2. `--min-assistant-chars` characters of assistant *text*, tool results
excluded (default **1000**).
The second gate exists because message count alone isn't enough for pi: pi
expands skill/context text into the user prompt, so an abandoned session can
have a huge "user" turn and almost nothing on the assistant side — e.g. a
real observed case with a 13,380-char injected-context user turn answered
"Ready. What would you like to work on?" (38 chars). Total size said
substantial; assistant size correctly said nothing happened. Measured real
sessions on the same machine ran 15,900100,000 assistant chars, so the
1000-char default sits with wide margin on both sides.
**Dedup:** staging under `$MEMPALACE_PI_STAGE/<wing>/` (default
`<palace-root>/pi-stage/<wing>/` — beside the palace, so the stage cannot be
wiped independently of the dedup keys pointing at it) with deterministic per-session-UUID
filenames, and the export preserves the source session's `mtime` on the
staged file. The convos miner is mtime-aware (see the Gotchas in
[`AGENTS.md`](AGENTS.md) — an older version of this doc claimed otherwise),
so re-runs on an unchanged session are a no-op, and re-feeding a **grown**
session (a live one being fed mid-conversation) purges and refiles that
session's drawers instead of duplicating them.
**Staging must persist.** Dedup keys on the *staged* path, not the original
transcript, so if the stage is wiped, a `mempalace sync` scoped to include it
sees those source files as gone and prunes the drawers mined from them —
deleting the memories, not just the cache. This is why the stage now defaults to
`<palace-root>/pi-stage`: it inherits whatever persistence the palace has, so
the files and the dedup keys that reference them cannot be separated by
wiping something that merely looks disposable. Override with
`$MEMPALACE_PI_STAGE` only if you have somewhere *more* durable than the palace.
**Remote palace:** if `$MEMPALACE_REMOTE_URL` is set, there is no
remote-palace CLI to mine into directly — `mempalace_mine` expands its
source path in the *server* process, which cannot see this machine's staged
exports. `--mode remote` (or `--mode auto`, which detects
`$MEMPALACE_REMOTE_URL`) instead `rsync`s the stage to a per-device inbox on
the palace host and asks the server to mine its own local copy. Requires
`$MEMPALACE_PI_SSH_TARGET` (`user@host:path`); see `MEMPALACE_PI_SSH_CONFIG`,
`MEMPALACE_PI_REMOTE_PATH`, and `MEMPALACE_PI_DEVICE` in `--help` for the
rest. Deploying that primary — newt, DNS, and why the auth is a shared bearer
token rather than per-device proxy users — is
[`docs/phase-1-exposure-runbook.md`](docs/phase-1-exposure-runbook.md).
---
## Companion agent skill ## Companion agent skill
Installing this repo symlinks `SKILL.md` into `~/.agents/skills/opencode-mempalace-bridge/SKILL.md`, where it's auto-discovered by opencode (and by Claude Code / Kiro if you run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils)). Installing this repo symlinks `SKILL.md` into `~/.agents/skills/opencode-mempalace-bridge/SKILL.md`, where it's auto-discovered by opencode (and by Claude Code / Kiro if you run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils)).
+18 -2
View File
@@ -20,6 +20,13 @@ The `mempalace` skill covers *using* the palace (search, diary, KG). This skill
Both follow the same **stage-to-cache-then-mine** idiom — they curate input into `~/.cache/…/<wing>/`, then delegate to `mempalace mine`. Both follow the same **stage-to-cache-then-mine** idiom — they curate input into `~/.cache/…/<wing>/`, then delegate to `mempalace mine`.
**Pi is out of scope for this skill.** The pi coding-agent has its own
wrapper, `mempalace-pi-session`, but — unlike opencode — pi's bridge
extension (`extensions/pi/mempalace.ts`) invokes it automatically on session
shutdown and a debounced mid-session tick, so a pi machine with that
extension installed needs none of this skill's manual/scheduled recipe. See
`extensions/pi/README.md` § Automatic transcript feeding in the same repo.
## When to Load This Skill ## When to Load This Skill
- User asks "how does the palace get fed?" or mentions setting up mempalace on a new machine. - User asks "how does the palace get fed?" or mentions setting up mempalace on a new machine.
@@ -86,7 +93,14 @@ A docs-heavy repo should produce ~510 drawers per file. >15 drawers/file on a
### Dedup is free — re-running is safe ### Dedup is free — re-running is safe
- `mempalace-docs`: dedup keyed on `source_file` path + `mtime`. Unchanged files skipped. - `mempalace-docs`: dedup keyed on `source_file` path + `mtime`. Unchanged files skipped.
- `mempalace-session`: dedup keyed on `source_file` path alone (no mtime check for convos). Staging filenames are deterministic per session (`<slug>_<id>.jsonl`), so re-runs skip already-filed sessions. - `mempalace-session` / `mempalace-pi-session`: the convos miner also dedups
on `source_file` path + `mtime` (`file_already_mined(..., check_mtime=True)`
in upstream `convo_miner.py` — a prior version of this line said "no mtime
check", which was wrong). Staging filenames are deterministic per session,
so a re-run on unchanged content is a no-op, and a grown/changed session is
detected, purged, and refiled rather than duplicated. What must stay
stable is the staging *path itself* — wiping the staging dir makes dedup
(and `mempalace sync`) treat those sources as gone.
Second run immediately after first → 0 new drawers, only the post-mine `repair` step runs (~5 min on 5k drawers). Second run immediately after first → 0 new drawers, only the post-mine `repair` step runs (~5 min on 5k drawers).
@@ -110,8 +124,10 @@ mempalace-session --session ses_abc123 # one specific session
### Force re-mine ### Force re-mine
```bash ```bash
rm -rf ~/.cache/mempalace-session/<wing>/ # nukes staging dir rm -rf <palace-root>/opencode-stage/<wing>/ # nukes staging dir
mempalace-session # stages + mines fresh mempalace-session # stages + mines fresh
# Do NOT run `mempalace sync` between those two commands: with the stage gone,
# a scoped sync prunes the drawers mined from it instead of refiling them.
``` ```
Staging is ephemeral by design; the palace is the source of truth. Staging is ephemeral by design; the palace is the source of truth.
+308 -28
View File
@@ -10,22 +10,88 @@
# Strategy (mirrors mempalace-session): # Strategy (mirrors mempalace-session):
# 1. Walk ~/.pi/agent/sessions/**/*.jsonl and export each qualifying session # 1. Walk ~/.pi/agent/sessions/**/*.jsonl and export each qualifying session
# to a Claude Code JSONL file (format the mempalace normalizer speaks). # to a Claude Code JSONL file (format the mempalace normalizer speaks).
# 2. Stage exports under ~/.cache/mempalace-pi-session/<wing>/. # 2. Stage exports under $MEMPALACE_PI_STAGE/<wing> (default
# <palace-root>/pi-stage/<wing> — alongside the palace it feeds).
# 3. Run `mempalace mine --mode convos` against the staging dir. # 3. Run `mempalace mine --mode convos` against the staging dir.
# #
# TWO PHASES (--prepare), because the palace is single-writer
# mempalace refuses a CLI mine while another process holds the palace:
# "palace ... is held by PID <n> (mempalace-mcp); wait for it to finish"
# A live pi session ALWAYS has a holder — the mempalace extension's own
# mempalace-mcp. So an unattended CLI mine only works when no session is
# live (e.g. a container-start catch-up); during a session the mine must be
# performed by the process that already holds the palace. Hence:
#
# --prepare export + stage (+ rsync in remote mode) and print
# MINE_SOURCE=<path>, without ever opening the palace.
# (default) the above, then mine it ourselves. Contention is treated as
# success-with-nothing-to-do, not failure: the holder's own
# extension will mine what we staged.
#
# The pi mempalace extension drives exactly this: it runs --prepare on
# session_shutdown and on a debounced agent_settled, then calls
# mempalace_mine on MINE_SOURCE through its existing MCP client.
#
# TRANSPORTS (--mode, default auto)
# local Mine into the local palace with the mempalace CLI.
# remote $MEMPALACE_REMOTE_URL is set, so the palace lives on another host.
# There is no remote-palace CLI — only the HTTP MCP server — and
# mempalace_mine expands its source path in the SERVER process, so
# the server cannot see this machine's staged exports. We therefore
# rsync the stage into a per-device inbox on the palace host and ask
# the server to mine its own local path. Requires
# MEMPALACE_PI_SSH_TARGET (where to rsync) and
# MEMPALACE_PI_REMOTE_PATH (what that inbox is called server-side).
#
# Labelling: every exported transcript begins with a synthetic header # Labelling: every exported transcript begins with a synthetic header
# [session: <title> | <cwd> | <YYYY-MM-DD> | source: pi] # [session: <title> | <cwd> | <YYYY-MM-DD> | source: pi]
# so post-mine search results are self-identifying (pi vs opencode vs other). # so post-mine search results are self-identifying (pi vs opencode vs other).
# #
# Dedup: mempalace convos mode keys on source_file (absolute staging path). # Dedup: mempalace convos mode keys on source_file (absolute staging path).
# Staging paths are deterministic per pi session UUID, so re-runs are # Staging paths are deterministic per pi session UUID, and the export copies
# idempotent until session content actually changes. # the source session's mtime onto the staged file, so re-runs are idempotent
# until session content actually changes. A GROWN session is purged and
# refiled for that source_file by the miner, so re-feeding a live session
# refreshes its drawers instead of duplicating them.
# #
# Session filter: sessions with fewer than --min-messages *user+assistant* # Staging location: source_file dedup keys on the staged path, so if the stage
# messages (default 3) are skipped to avoid filing single-prompt throwaways. # is wiped the palace is left with drawers whose source files look deleted.
# `mempalace sync` prunes exactly those — but only within the scope it is
# given. Measured on this layout: scoped at the palace root the staged sources
# are in scope (kept 651), while a wing-only sync reports them out_of_scope and
# leaves them alone. So the data loss is conditional on how sync is invoked,
# which is far too thin a margin to rely on.
#
# The stage therefore defaults NEXT TO THE PALACE (<palace-root>/pi-stage,
# resolved the way mempalace itself resolves the palace: $MEMPALACE_PALACE_PATH
# → $MEMPAL_PALACE_PATH → ~/.mempalace/config.json → ~/.mempalace/palace).
#
# That makes the invariant structural rather than documented: the stage and the
# dedup keys that reference it share one lifetime, so the dangerous state —
# palace survives, stage does not — can no longer be reached by wiping
# something that merely looks disposable. A cache dir (the obvious choice, and
# the old default) is exactly wrong here: it persists just long enough to look
# correct, then takes the memories with it. Override with MEMPALACE_PI_STAGE
# only if the target is at least as durable as the palace.
#
# In remote mode the local stage is only a shipping buffer — dedup lives on the
# server, keyed by the server-side inbox path — so its durability is moot there.
#
# Session filter: two gates, both required.
# 1. --min-messages <N> user+assistant turns (default 4). Tool loops inflate
# assistant turns fast in pi, so a real working session clears this
# easily; a single abandoned prompt does not.
# 2. --min-assistant-chars <N> characters of assistant *text* (default 1000),
# excluding tool results. Assistant volume, not total volume: pi expands
# skills/context into the user prompt, so an abandoned session can carry a
# 13k-char "user" message answered with "Ready. What would you like to
# work on?" — total size says substantial, assistant size correctly says
# nothing happened.
# #
# Usage: # Usage:
# mempalace-pi-session # mempalace-pi-session
# mempalace-pi-session --prepare
# mempalace-pi-session --mode remote
# mempalace-pi-session --wing <name> # mempalace-pi-session --wing <name>
# mempalace-pi-session --session <uuid-prefix> # mempalace-pi-session --session <uuid-prefix>
# mempalace-pi-session --since 2026-04-01 # mempalace-pi-session --since 2026-04-01
@@ -34,13 +100,16 @@
# mempalace-pi-session --help # mempalace-pi-session --help
# #
# Exit codes: # Exit codes:
# 0 success # 0 success (including "nothing qualified", "another run holds the lock",
# and "palace held by a live session")
# 1 usage / argument error # 1 usage / argument error
# 2 pi sessions dir missing # 2 pi sessions dir missing
# 3 mempalace CLI not installed # 3 mempalace CLI not installed / rsync missing in remote mode
# 4 mine failed # 4 mine failed
# 5 remote transport failed (rsync or HTTP tools/call)
# #
# Dependencies: bash, python3 (stdlib only), mempalace (v3.3.3+) # Dependencies: bash, python3 (stdlib only), mempalace (v3.3.3+);
# rsync + ssh in remote mode.
set -euo pipefail set -euo pipefail
@@ -49,11 +118,50 @@ AGENT="${USER:-mempalace}"
WING="wing_conversations" WING="wing_conversations"
SESSION_ID="" SESSION_ID=""
SINCE="" SINCE=""
MIN_MESSAGES=3 MIN_MESSAGES=4
MIN_ASSISTANT_CHARS=1000
DRY_RUN=0 DRY_RUN=0
DO_REPAIR=0 DO_REPAIR=0
PREPARE_ONLY=0
MODE="auto"
REASON=""
PI_SESSIONS_DIR="${PI_SESSIONS_DIR:-$HOME/.pi/agent/sessions}" PI_SESSIONS_DIR="${PI_SESSIONS_DIR:-$HOME/.pi/agent/sessions}"
# Resolve the palace ROOT (the dir holding palace/, knowledge_graph.sqlite3,
# config.json) using mempalace's own precedence, so the stage lands next to
# whichever palace this host actually feeds. Mirrors config.py:palace_path()
# (env → config.json → default) and takes the parent. Only evaluated when
# MEMPALACE_PI_STAGE is unset, so the common path costs nothing.
palace_root() {
python3 - <<'PY' 2>/dev/null || echo "$HOME/.mempalace"
import json, os
p = os.environ.get("MEMPALACE_PALACE_PATH") or os.environ.get("MEMPAL_PALACE_PATH")
if p:
p = os.path.abspath(os.path.expanduser(p))
else:
cfg = os.path.expanduser("~/.mempalace/config.json")
p = None
if os.path.exists(cfg):
try:
with open(cfg) as fh:
v = json.load(fh).get("palace_path")
p = os.path.expanduser(v) if v else None
except Exception:
p = None
p = p or os.path.expanduser("~/.mempalace/palace")
print(os.path.dirname(p.rstrip("/")))
PY
}
STAGE_ROOT="${MEMPALACE_PI_STAGE:-$(palace_root)/pi-stage}"
# Remote transport (see TRANSPORTS in the header)
REMOTE_URL="${MEMPALACE_REMOTE_URL:-}"
REMOTE_TOKEN="${MEMPALACE_REMOTE_TOKEN:-}"
SSH_TARGET="${MEMPALACE_PI_SSH_TARGET:-}"
SSH_CONFIG="${MEMPALACE_PI_SSH_CONFIG:-}"
REMOTE_PATH="${MEMPALACE_PI_REMOTE_PATH:-/data/feed}"
DEVICE="${MEMPALACE_PI_DEVICE:-$(hostname)}"
# ── Usage ──────────────────────────────────────────────────────────── # ── Usage ────────────────────────────────────────────────────────────
usage() { usage() {
cat <<'EOF' cat <<'EOF'
@@ -67,10 +175,28 @@ Options:
--session <prefix> Export one session only (match on UUID prefix) --session <prefix> Export one session only (match on UUID prefix)
--since <YYYY-MM-DD> Only sessions last modified on/after this date --since <YYYY-MM-DD> Only sessions last modified on/after this date
--min-messages <N> Skip sessions with fewer than N user+assistant --min-messages <N> Skip sessions with fewer than N user+assistant
turns (default: 3) turns (default: 4)
--min-assistant-chars <N>
Skip sessions with fewer than N characters of
assistant text, tool results excluded (default: 1000).
Catches abandoned sessions whose bulk is injected
skill/context text in the user prompt.
--agent <name> Agent name recorded on drawers (default: $USER) --agent <name> Agent name recorded on drawers (default: $USER)
--sessions-dir <path> Path to pi sessions dir (default: $PI_SESSIONS_DIR --sessions-dir <path> Path to pi sessions dir (default: $PI_SESSIONS_DIR
or ~/.pi/agent/sessions) or ~/.pi/agent/sessions)
--stage <path> Staging root (default: $MEMPALACE_PI_STAGE, else
<palace-root>/pi-stage — next to the palace, so the
stage cannot be wiped independently of the dedup keys
that point at it). Exports go in <root>/<wing>.
See "Staging location" in the header before moving it.
--mode <m> auto|local|remote (default: auto — remote when
$MEMPALACE_REMOTE_URL is set)
--prepare Export + stage (+ rsync in remote mode), print
MINE_SOURCE=<path>, and stop without opening the
palace. For callers that will do the mine themselves
through a live MCP connection.
--reason <label> Label this run in its output (e.g. shutdown, tick,
container-start). Useful when triggers log to a file.
--dry-run Export + list; do not mine into palace. Each session --dry-run Export + list; do not mine into palace. Each session
is tagged [NEW] or [SKIP] based on whether its is tagged [NEW] or [SKIP] based on whether its
source_file is already in the palace. source_file is already in the palace.
@@ -102,14 +228,27 @@ Transcript shape per session:
Dedup: Dedup:
- source_file = absolute staging path (deterministic per pi session UUID) - source_file = absolute staging path (deterministic per pi session UUID)
- Re-runs skip unchanged sessions. To force re-mining, delete the staging - Re-runs skip unchanged sessions; a GROWN session (mtime changed) has its
dir: rm -rf ~/.cache/mempalace-pi-session/<wing>/ old drawers purged and is refiled, so re-feeding a live session refreshes
rather than duplicates.
- To force re-mining, delete the staging dir:
rm -rf <palace-root>/pi-stage/<wing>/
That forces a refile — but do NOT run `mempalace sync` while the stage is
missing, or the drawers mined from it get pruned instead.
Rationale: Rationale:
Pi's extension ecosystem could, in principle, stream sessions into the Two complementary paths feed the palace from pi, and they cover different
palace as they happen, but that requires a resident MCP connection during failure modes:
every pi session. This wrapper is the batch, harness-agnostic alternative: - The pi mempalace bridge extension (extensions/pi/mempalace.ts) drives
it reads the durable on-disk JSONL and mines it on a schedule. this script with --prepare on session_shutdown and on a debounced
agent_settled, then mines through its own live MCP connection. That is
the primary path: it needs no scheduling and it is the only way to write
while a session holds the palace.
- Running this script directly is the batch/recovery path: a
container-start or host-level catch-up that picks up transcripts nothing
mined at the time — notably after a SIGKILL, where no pi handler runs at
all. It reads the durable on-disk JSONL, so it does not care whether the
session that produced it exited cleanly.
EOF EOF
} }
@@ -121,6 +260,11 @@ while [[ $# -gt 0 ]]; do
--session) SESSION_ID="${2:-}"; shift 2 ;; --session) SESSION_ID="${2:-}"; shift 2 ;;
--since) SINCE="${2:-}"; shift 2 ;; --since) SINCE="${2:-}"; shift 2 ;;
--min-messages) MIN_MESSAGES="${2:-}"; shift 2 ;; --min-messages) MIN_MESSAGES="${2:-}"; shift 2 ;;
--min-assistant-chars) MIN_ASSISTANT_CHARS="${2:-}"; shift 2 ;;
--stage) STAGE_ROOT="${2:-}"; shift 2 ;;
--mode) MODE="${2:-}"; shift 2 ;;
--prepare) PREPARE_ONLY=1; shift ;;
--reason) REASON="${2:-}"; shift 2 ;;
--agent) AGENT="${2:-}"; shift 2 ;; --agent) AGENT="${2:-}"; shift 2 ;;
--sessions-dir) PI_SESSIONS_DIR="${2:-}"; shift 2 ;; --sessions-dir) PI_SESSIONS_DIR="${2:-}"; shift 2 ;;
--dry-run) DRY_RUN=1; shift ;; --dry-run) DRY_RUN=1; shift ;;
@@ -138,32 +282,67 @@ if [[ ! -d "$PI_SESSIONS_DIR" ]]; then
echo " override with --sessions-dir <path> or PI_SESSIONS_DIR env var" >&2 echo " override with --sessions-dir <path> or PI_SESSIONS_DIR env var" >&2
exit 2 exit 2
fi fi
if ! command -v mempalace >/dev/null 2>&1; then case "$MODE" in
auto) if [[ -n "$REMOTE_URL" ]]; then MODE="remote"; else MODE="local"; fi ;;
local|remote) ;;
*) echo "error: --mode must be auto|local|remote" >&2; exit 1 ;;
esac
# The mempalace CLI is only needed when WE do the mine. --prepare never opens
# the palace, and remote mode talks to the server over HTTP.
if [[ $PREPARE_ONLY -eq 0 && "$MODE" == "local" ]] && ! command -v mempalace >/dev/null 2>&1; then
echo "error: mempalace CLI not found in PATH" >&2 echo "error: mempalace CLI not found in PATH" >&2
exit 3 exit 3
fi fi
if ! [[ "$MIN_MESSAGES" =~ ^[0-9]+$ ]]; then if [[ "$MODE" == "remote" ]]; then
echo "error: --min-messages must be an integer" >&2 command -v rsync >/dev/null 2>&1 || { echo "error: rsync not found (needed for --mode remote)" >&2; exit 3; }
exit 1 if [[ -z "$SSH_TARGET" ]]; then
echo "error: MEMPALACE_PI_SSH_TARGET unset (needed for --mode remote)" >&2
exit 1
fi
# The devbox generates a dedicated LAN-jump key/config; prefer it if present.
if [[ -z "$SSH_CONFIG" && -f "$HOME/.ssh-local/config" ]]; then
SSH_CONFIG="$HOME/.ssh-local/config"
fi
fi fi
for _n in MIN_MESSAGES MIN_ASSISTANT_CHARS; do
if ! [[ "${!_n}" =~ ^[0-9]+$ ]]; then
_flag="--$(printf '%s' "${_n,,}" | tr '_' '-')"
echo "error: $_flag must be an integer" >&2
exit 1
fi
done
# ── Staging dir ────────────────────────────────────────────────────── # ── Staging dir ──────────────────────────────────────────────────────
CACHE_ROOT="${XDG_CACHE_HOME:-$HOME/.cache}/mempalace-pi-session" # Deterministic per-wing path so source_file dedup works across re-runs. See
STAGE="$CACHE_ROOT/$WING" # "Staging location" in the header for why this should not be disposable.
STAGE="${STAGE_ROOT%/}/$WING"
mkdir -p "$STAGE" mkdir -p "$STAGE"
[[ -n "$REASON" ]] && echo "mempalace-pi-session [$REASON] mode=$MODE stage=$STAGE"
# ── Single-writer guard ──────────────────────────────────────────────
# Non-blocking: overlapping triggers (a session_shutdown landing on top of a
# debounced mid-session run) must not queue or race. Losing a run is harmless
# — the next one re-exports from scratch.
exec 9>"${STAGE_ROOT%/}/.lock"
if command -v flock >/dev/null 2>&1 && ! flock -n 9; then
echo "another mempalace-pi-session run holds the lock; skipping"
exit 0
fi
# ── Export sessions (Python heredoc) ──────────────────────────────── # ── Export sessions (Python heredoc) ────────────────────────────────
# Parses pi JSONL files and writes Claude Code JSONL per session into $STAGE. # Parses pi JSONL files and writes Claude Code JSONL per session into $STAGE.
# Also classifies each export as NEW/ALREADY FILED (by source_file lookup) # Also classifies each export as NEW/ALREADY FILED (by source_file lookup)
# so --dry-run reports the real mine-set size. Classification is advisory; # so --dry-run reports the real mine-set size. Classification is advisory;
# `mempalace mine --mode convos` is still the authoritative dedup. # `mempalace mine --mode convos` is still the authoritative dedup.
export_count=$(python3 - "$PI_SESSIONS_DIR" "$STAGE" "$SESSION_ID" "$SINCE" "$MIN_MESSAGES" <<'PY' export_count=$(python3 - "$PI_SESSIONS_DIR" "$STAGE" "$SESSION_ID" "$SINCE" "$MIN_MESSAGES" "$MIN_ASSISTANT_CHARS" <<'PY'
import json, os, sqlite3, sys import json, os, sqlite3, sys
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
sessions_dir, stage, session_filter, since, min_messages = sys.argv[1:6] sessions_dir, stage, session_filter, since, min_messages, min_assistant_chars = sys.argv[1:7]
min_messages = int(min_messages) min_messages = int(min_messages)
min_assistant_chars = int(min_assistant_chars)
stage = Path(stage) stage = Path(stage)
sessions_dir = Path(sessions_dir) sessions_dir = Path(sessions_dir)
@@ -178,7 +357,15 @@ if since:
# ── Load palace's already-filed source_files (best-effort, read-only) ── # ── Load palace's already-filed source_files (best-effort, read-only) ──
already_filed = set() already_filed = set()
palace_path = os.environ.get("MEMPALACE_PATH", os.path.expanduser("~/.mempalace/palace")) # Mirror mempalace's own resolution order (config.py): MEMPALACE_PALACE_PATH,
# then the legacy MEMPAL_PALACE_PATH, then the default. NOT "MEMPALACE_PATH" —
# that name is not a mempalace concept, and reading it silently degraded this
# NEW/SKIP preview to "everything is new" wherever some other tool had set it.
palace_path = (
os.environ.get("MEMPALACE_PALACE_PATH")
or os.environ.get("MEMPAL_PALACE_PATH")
or os.path.expanduser("~/.mempalace/palace")
)
chroma_db = Path(palace_path) / "chroma.sqlite3" chroma_db = Path(palace_path) / "chroma.sqlite3"
if chroma_db.is_file(): if chroma_db.is_file():
try: try:
@@ -282,6 +469,7 @@ if session_filter:
exported = 0 exported = 0
skipped_short = 0 skipped_short = 0
skipped_quiet = 0
skipped_malformed = 0 skipped_malformed = 0
skipped_already_filed = 0 skipped_already_filed = 0
@@ -318,6 +506,7 @@ for path in paths:
continue continue
title = derive_title(entries, fallback=session_uuid[:8]) title = derive_title(entries, fallback=session_uuid[:8])
assistant_chars = 0
out_lines = [] out_lines = []
out_lines.append({ out_lines.append({
"type": "user", "type": "user",
@@ -337,6 +526,7 @@ for path in paths:
out_lines.append({"type": "user", "message": {"content": text}}) out_lines.append({"type": "user", "message": {"content": text}})
elif role == "assistant": elif role == "assistant":
text, tool_uses, _ = extract_text(msg.get("content")) text, tool_uses, _ = extract_text(msg.get("content"))
assistant_chars += len(text.strip())
blocks = [] blocks = []
if text.strip(): if text.strip():
blocks.append({"type": "text", "text": text}) blocks.append({"type": "text", "text": text})
@@ -406,6 +596,17 @@ for path in paths:
skipped_short += 1 skipped_short += 1
continue continue
# Assistant *text* volume, tool results excluded: the signal that the
# session actually did something, independent of how much injected
# skill/context text inflated the user side.
if assistant_chars < min_assistant_chars:
skipped_quiet += 1
print(
f" [QUIET] {path.name} ({turn_count} turns, {assistant_chars} assistant chars)",
file=sys.stderr,
)
continue
out_path = stage / f"pi_{session_uuid}.jsonl" out_path = stage / f"pi_{session_uuid}.jsonl"
with out_path.open("w", encoding="utf-8") as f: with out_path.open("w", encoding="utf-8") as f:
for obj in out_lines: for obj in out_lines:
@@ -428,6 +629,8 @@ print(f"EXPORTED {exported}")
print(f"ALREADY_FILED {skipped_already_filed}") print(f"ALREADY_FILED {skipped_already_filed}")
if skipped_short: if skipped_short:
print(f"SKIPPED_SHORT {skipped_short}", file=sys.stderr) print(f"SKIPPED_SHORT {skipped_short}", file=sys.stderr)
if skipped_quiet:
print(f"SKIPPED_QUIET {skipped_quiet}", file=sys.stderr)
if skipped_malformed: if skipped_malformed:
print(f"SKIPPED_MALFORMED {skipped_malformed}", file=sys.stderr) print(f"SKIPPED_MALFORMED {skipped_malformed}", file=sys.stderr)
PY PY
@@ -461,12 +664,89 @@ if [[ $DRY_RUN -eq 1 ]]; then
exit 0 exit 0
fi fi
# ── Ship to the palace host (remote mode only) ───────────────────────
# mempalace_mine expands its source path in the SERVER process, so in remote
# mode the exports have to physically exist over there. rsync --update is the
# idempotent half; the mine is the other half.
MINE_SOURCE="$STAGE"
if [[ "$MODE" == "remote" ]]; then
ssh_cmd="ssh"
[[ -n "$SSH_CONFIG" ]] && ssh_cmd="ssh -F $SSH_CONFIG"
echo ""
echo "Shipping stage to ${SSH_TARGET%/}/$DEVICE/ ..."
if ! rsync -a --update --no-owner --no-group \
-e "$ssh_cmd" \
--include='*.jsonl' --exclude='*' \
"$STAGE/" "${SSH_TARGET%/}/$DEVICE/"; then
echo "error: rsync to ${SSH_TARGET%/}/$DEVICE/ failed" >&2
exit 5
fi
MINE_SOURCE="${REMOTE_PATH%/}/$DEVICE"
fi
# ── Phase boundary ───────────────────────────────────────────────────
# --prepare hands the source path to the caller (the pi mempalace extension),
# which mines it through the MCP client that already holds the palace.
if [[ $PREPARE_ONLY -eq 1 ]]; then
echo ""
printf 'MINE_SOURCE=%s\n' "$MINE_SOURCE"
exit 0
fi
# ── Run the mine ───────────────────────────────────────────────────── # ── Run the mine ─────────────────────────────────────────────────────
echo "" echo ""
echo "Mining into wing '$WING'..." echo "Mining into wing '$WING'..."
if ! mempalace mine "$STAGE" --mode convos --wing "$WING" --agent "$AGENT"; then if [[ "$MODE" == "remote" ]]; then
echo "error: mempalace mine failed" >&2 if ! python3 - "$REMOTE_URL" "$REMOTE_TOKEN" "$MINE_SOURCE" "$WING" "$AGENT" <<'PY'
exit 4 import json, sys, urllib.error, urllib.request
url, token, source, wing, agent = sys.argv[1:6]
payload = json.dumps({
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "mempalace_mine",
"arguments": {"source": source, "mode": "convos", "wing": wing, "agent": agent},
},
}).encode()
headers = {"Content-Type": "application/json", "Accept": "application/json"}
if token:
headers["Authorization"] = f"Bearer {token}"
try:
with urllib.request.urlopen(
urllib.request.Request(url, data=payload, headers=headers), timeout=900
) as resp:
body = resp.read().decode("utf-8", "replace")
except urllib.error.URLError as exc:
print(f"error: remote mine transport failed: {exc}", file=sys.stderr)
sys.exit(1)
print(body[:4000])
sys.exit(1 if '"error"' in body else 0)
PY
then
echo "error: remote mine failed" >&2
exit 5
fi
else
# Capture output so palace-level contention can be told apart from a real
# failure. A live pi session holds the palace through its own mempalace-mcp,
# and that session's extension mines what we just staged — so contention
# means "already handled", not "broken".
set +e
mine_out="$(mempalace mine "$MINE_SOURCE" --mode convos --wing "$WING" --agent "$AGENT" 2>&1)"
mine_rc=$?
set -e
printf '%s\n' "$mine_out"
if [[ $mine_rc -ne 0 ]]; then
if printf '%s' "$mine_out" | grep -q "is held by"; then
echo ""
echo "palace is held by a live session; it will mine the staged exports itself"
exit 0
fi
echo "error: mempalace mine failed" >&2
exit 4
fi
fi fi
# ── Repair index ───────────────────────────────────────────────────── # ── Repair index ─────────────────────────────────────────────────────
+53 -7
View File
@@ -10,13 +10,26 @@
# Strategy: # Strategy:
# 1. Read opencode.db and export each qualifying session to a Claude Code # 1. Read opencode.db and export each qualifying session to a Claude Code
# JSONL file (format the mempalace normalizer already understands). # JSONL file (format the mempalace normalizer already understands).
# 2. Stage exports under ~/.cache/mempalace-session/<wing>/. # 2. Stage exports under <palace-root>/opencode-stage/<wing>/ (override with
# MEMPALACE_SESSION_STAGE).
# 3. Run `mempalace mine --mode convos` against the staging dir. # 3. Run `mempalace mine --mode convos` against the staging dir.
# #
# Dedup: mempalace convos mode keys on source_file (absolute staging path). # Dedup: mempalace convos mode keys on source_file (absolute staging path).
# The staging path is deterministic (per-wing under XDG_CACHE_HOME) so re-runs # The staging path is deterministic (per-wing under the palace root) so re-runs
# are idempotent as long as session content hasn't changed. # are idempotent as long as session content hasn't changed.
# #
# Staging location: because dedup keys on the staged path, wiping the stage
# leaves the palace holding drawers whose source files look deleted, and
# `mempalace sync` prunes exactly those when they fall inside the scope it is
# given. The stage therefore lives next to the palace it feeds (resolved via
# $MEMPALACE_PALACE_PATH → $MEMPAL_PALACE_PATH → ~/.mempalace/config.json →
# ~/.mempalace/palace), so stage and dedup keys share one lifetime and the
# dangerous state — palace survives, stage does not — cannot be reached by
# wiping something that merely looks disposable. It used to default under
# ~/.cache, which is disposable on exactly the hosts where this runs
# unattended. Override with MEMPALACE_SESSION_STAGE only if the target is at
# least as durable as the palace.
#
# Session filter: sessions with fewer than --min-messages messages (default 3) # Session filter: sessions with fewer than --min-messages messages (default 3)
# are skipped to avoid filing throwaway /exit'd sessions. # are skipped to avoid filing throwaway /exit'd sessions.
# #
@@ -87,7 +100,7 @@ Idempotency:
What gets mined: What gets mined:
- Each qualifying session → one Claude Code JSONL file - Each qualifying session → one Claude Code JSONL file
- Staged under ~/.cache/mempalace-session/<wing>/ - Staged under <palace-root>/opencode-stage/<wing>/
- Filed via `mempalace mine --mode convos` - Filed via `mempalace mine --mode convos`
Transcript shape per session: Transcript shape per session:
@@ -103,7 +116,9 @@ Transcript shape per session:
Dedup: Dedup:
- source_file = absolute staging path (deterministic per session ID) - source_file = absolute staging path (deterministic per session ID)
- Re-runs skip unchanged sessions. To force re-mining, delete the staging - Re-runs skip unchanged sessions. To force re-mining, delete the staging
dir: rm -rf ~/.cache/mempalace-session/<wing>/ dir: rm -rf <palace-root>/opencode-stage/<wing>/
(that only forces a refile; do NOT run `mempalace sync` while the stage is
missing, or the drawers mined from it get pruned instead)
Rationale: Rationale:
Opencode lacks a session-stopping hook (upstream PRs #16598, #16769 still Opencode lacks a session-stopping hook (upstream PRs #16598, #16769 still
@@ -147,8 +162,31 @@ if ! [[ "$MIN_MESSAGES" =~ ^[0-9]+$ ]]; then
fi fi
# ── Staging dir ────────────────────────────────────────────────────── # ── Staging dir ──────────────────────────────────────────────────────
# Deterministic per-wing path so source_file dedup works across re-runs. # Deterministic per-wing path so source_file dedup works across re-runs, and
CACHE_ROOT="${XDG_CACHE_HOME:-$HOME/.cache}/mempalace-session" # anchored to the palace root so the stage cannot be wiped independently of the
# dedup keys that reference it (see "Staging location" in the header).
# Mirrors mempalace config.py:palace_path() precedence, then takes the parent.
palace_root() {
python3 - <<'PY' 2>/dev/null || echo "$HOME/.mempalace"
import json, os
p = os.environ.get("MEMPALACE_PALACE_PATH") or os.environ.get("MEMPAL_PALACE_PATH")
if p:
p = os.path.abspath(os.path.expanduser(p))
else:
cfg = os.path.expanduser("~/.mempalace/config.json")
p = None
if os.path.exists(cfg):
try:
with open(cfg) as fh:
v = json.load(fh).get("palace_path")
p = os.path.expanduser(v) if v else None
except Exception:
p = None
p = p or os.path.expanduser("~/.mempalace/palace")
print(os.path.dirname(p.rstrip("/")))
PY
}
CACHE_ROOT="${MEMPALACE_SESSION_STAGE:-$(palace_root)/opencode-stage}"
STAGE="$CACHE_ROOT/$WING" STAGE="$CACHE_ROOT/$WING"
mkdir -p "$STAGE" mkdir -p "$STAGE"
@@ -185,7 +223,15 @@ if since:
# isn't reachable (first install, moved, permission-denied), we fall through # isn't reachable (first install, moved, permission-denied), we fall through
# to "everything is new" — the mine step will do the real dedup anyway. # to "everything is new" — the mine step will do the real dedup anyway.
already_filed = set() already_filed = set()
palace_path = os.environ.get("MEMPALACE_PATH", os.path.expanduser("~/.mempalace/palace")) # Mirror mempalace's own resolution order (config.py): MEMPALACE_PALACE_PATH,
# then the legacy MEMPAL_PALACE_PATH, then the default. NOT "MEMPALACE_PATH" —
# that name is not a mempalace concept, and reading it silently degraded this
# NEW/SKIP preview to "everything is new" wherever some other tool had set it.
palace_path = (
os.environ.get("MEMPALACE_PALACE_PATH")
or os.environ.get("MEMPAL_PALACE_PATH")
or os.path.expanduser("~/.mempalace/palace")
)
chroma_db = Path(palace_path) / "chroma.sqlite3" chroma_db = Path(palace_path) / "chroma.sqlite3"
if chroma_db.is_file(): if chroma_db.is_file():
try: try:
+20 -5
View File
@@ -2,6 +2,16 @@
Manual invocation of the session-mining wrappers is fine on a machine you actively drive. For long-running devboxes, a weekly automated mine keeps the palace fresh without thinking about it. This directory ships ready-to-use templates for two common scheduling mechanisms, for each wrapper. Manual invocation of the session-mining wrappers is fine on a machine you actively drive. For long-running devboxes, a weekly automated mine keeps the palace fresh without thinking about it. This directory ships ready-to-use templates for two common scheduling mechanisms, for each wrapper.
> **pi machines: check whether you need this at all.** If the pi bridge
> extension (`extensions/pi/mempalace.ts`) is installed, it already feeds the
> palace by itself on `session_shutdown` and a debounced `agent_settled`
> see [`extensions/pi/README.md` § Automatic transcript feeding](../extensions/pi/README.md#automatic-transcript-feeding).
> The templates below were written when scheduling was the *only* path for
> both harnesses; that's still true for **opencode** (no such extension
> exists), but for pi they're now a fallback — useful for a bare pi install
> without the bridge, a host-level catch-up job, or belt-and-braces coverage
> of a hard container kill (the extension's triggers don't fire on `SIGKILL`).
> **Before using either**: confirm the toolkit is installed and the wrapper works — > **Before using either**: confirm the toolkit is installed and the wrapper works —
> `mempalace-session --dry-run` (and/or `mempalace-pi-session --dry-run`) should list qualifying sessions. If that errors, fix the install before scheduling. > `mempalace-session --dry-run` (and/or `mempalace-pi-session --dry-run`) should list qualifying sessions. If that errors, fix the install before scheduling.
@@ -171,17 +181,22 @@ rm /tmp/mempalace-session.cron
crontab -l | grep mempalace crontab -l | grep mempalace
``` ```
Ensure `~/.cache/mempalace-session/` exists so the log file can be written: Ensure `~/.cache/mempalace-logs/` exists so the log file can be written:
> This is the **log** directory only. The staging dir — the transcripts the
> palace keys its `source_file` dedup on — lives beside the palace
> (`<palace-root>/opencode-stage/`), not in `~/.cache`, precisely so it cannot
> be cleaned away while the palace survives. Logs here are disposable.
```bash ```bash
mkdir -p ~/.cache/mempalace-session mkdir -p ~/.cache/mempalace-logs
``` ```
**Verify a run is happening:** **Verify a run is happening:**
```bash ```bash
# Tail the log the cron entry writes to # Tail the log the cron entry writes to
tail -f ~/.cache/mempalace-session/cron.log tail -f ~/.cache/mempalace-logs/cron.log
# Or force a run manually to prove the command is well-formed # Or force a run manually to prove the command is well-formed
mempalace-session mempalace-session
@@ -282,11 +297,11 @@ cat contrib/cron/mempalace-session-devbox.cron
(crontab -l 2>/dev/null; cat contrib/cron/mempalace-session-devbox.cron) | crontab - (crontab -l 2>/dev/null; cat contrib/cron/mempalace-session-devbox.cron) | crontab -
# Ensure the log directory exists # Ensure the log directory exists
mkdir -p ~/.cache/mempalace-session mkdir -p ~/.cache/mempalace-logs
# Verify # Verify
crontab -l | grep mempalace-session-devbox crontab -l | grep mempalace-session-devbox
tail -f ~/.cache/mempalace-session/cron-devbox.log tail -f ~/.cache/mempalace-logs/cron-devbox.log
``` ```
**Uninstall:** **Uninstall:**
+1 -1
View File
@@ -35,4 +35,4 @@ CONTAINER_USER=developer
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# m h dom mon dow command # m h dom mon dow command
0 3 * * 1 /bin/sh -c 'docker ps --filter "name=^/${CONTAINER}$" --filter "status=running" -q | grep -q . && docker exec -u "${CONTAINER_USER}" "${CONTAINER}" mempalace-session >> "$HOME/.cache/mempalace-session/cron-devbox.log" 2>&1' 0 3 * * 1 /bin/sh -c 'docker ps --filter "name=^/${CONTAINER}$" --filter "status=running" -q | grep -q . && docker exec -u "${CONTAINER_USER}" "${CONTAINER}" mempalace-session >> "$HOME/.cache/mempalace-logs/cron-devbox.log" 2>&1'
+1 -1
View File
@@ -16,4 +16,4 @@
PATH=/home/USER/.local/bin:/usr/local/bin:/usr/bin:/bin PATH=/home/USER/.local/bin:/usr/local/bin:/usr/bin:/bin
# m h dom mon dow command # m h dom mon dow command
0 3 * * 1 mempalace-session >> /home/USER/.cache/mempalace-session/cron.log 2>&1 0 3 * * 1 mempalace-session >> /home/USER/.cache/mempalace-logs/cron.log 2>&1
+1 -1
View File
@@ -8,7 +8,7 @@ ConditionPathExists=!%t/mempalace-session.lock
[Service] [Service]
Type=oneshot Type=oneshot
# The wrapper writes to ~/.cache/mempalace-session/ and the palace. # The wrapper writes to ~/.cache/mempalace-logs/ and the palace.
# Keep stdout/stderr in the journal — inspect with: # Keep stdout/stderr in the journal — inspect with:
# journalctl --user -u mempalace-session --since today # journalctl --user -u mempalace-session --since today
ExecStart=%h/.local/bin/mempalace-session ExecStart=%h/.local/bin/mempalace-session
+185
View File
@@ -0,0 +1,185 @@
# Phase 1 exposure — newt on synlig, DNS, and the client auth model
Companion to [`rfc-001-global-palace.md`](./rfc-001-global-palace.md) (design + decisions) and
[`synlig-primary-runbook.md`](./synlig-primary-runbook.md) (what is already installed on the primary).
This doc covers only the step the other two leave open: **making the primary reachable** — runbook §4
items 2 and 5.
**Status 2026-08-12 — Pangolin updated on nyvaken (done, yours). newt not yet installed on synlig.
Nothing exposed. No client `.env` flipped.**
Read this before touching Pangolin: three of the four questions this step raises were **already decided**
in RFC §6.2 on 2026-08-09, and re-deciding them differently is how the fleet ends up in two states.
---
## 1. The four questions, answered
| Question | Answer | Where it was decided |
| --- | --- | --- |
| Which port? | **8765**, path **`/mcp`** (liveness: `/healthz`) | `cli.py:2141` default; runbook §2.4 |
| What does newt target? | **`172.17.0.1:8765`** (docker0), **never** `127.0.0.1` | RFC §6.2 Transport; runbook §2.4 |
| Open, or authenticated? | **Authenticated. The primary is never an open public resource.** | RFC §6.2 Network posture |
| Per-device credentials? | **No — Phase 1 ships the single shared bearer token.** Per-device tokens are Phase 4. | RFC §6.2 Authentication |
### 1.1 Why not per-device users at the proxy
The instinct — "create a Pangolin user per container, put the credentials in each `.env`, keep the
usernames distinct" — is the right *goal* (revocation, attribution) reached through the wrong *layer*,
twice over:
1. **mempalace validates exactly one token.** `hmac.compare_digest(provided, f"Bearer {srv.auth_token}")`
(`mcp_server.py:5292-5295`) — there is no user table and no second credential. Per-device HTTP identity
is not a configuration you can express today; it is Phase 4 work (a server-side
`token → {device_id, scopes}` registry). RFC §6.2 chose the shared token for Phase 1 deliberately:
*"iterate more feature rich but more complex solutions over time."*
2. **Pangolin's HTTP auth is browser-shaped; the clients are not.** SSO login, resource PIN and resource
password all assume something that can follow a redirect, render a form and hold a session cookie.
Every MemPalace client here is a headless JSON-RPC `POST` with an `Authorization` header — pi's
extension, opencode's `type:remote` MCP entry, and `mempalace-pi-session --mode remote`'s
`urllib.request.urlopen`. Point those at a user-authenticated resource and they receive a login page
where JSON should be. Enabling that protection breaks precisely the clients it is meant to protect.
So: **Pangolin terminates TLS and nothing more** (RFC §6.2 Transport, decided 2026-08-09). The bearer token
is the authentication. This is not "unprotected" — an unauthenticated request to `/mcp` gets a 401 from
mempalace itself, verified A4/A5 in runbook §2.4.
**Consequence to accept consciously** (RFC §6.2, §7.3.2): until Phase 4 the primary **cannot tell devices
apart**. `origin_device` is client-asserted and advisory — nothing load-bearing may depend on it, and
revoking one laptop means rotating the token everywhere.
### 1.2 The one place per-device identity *does* exist today
Remote mode is not only HTTP. `mempalace_mine` expands its source path in the **server** process, so a
client's staged transcripts must physically exist on the primary. The feeder therefore ships them over
SSH into a **per-device inbox** before asking the server to mine its own copy:
```sh
rsync -a --update -e "$ssh_cmd" "$STAGE/" "${SSH_TARGET%/}/$DEVICE/" # bin/mempalace-pi-session:677-680
```
That SSH key **is** per-device identity, and it is individually revocable (one line out of
`authorized_keys`) years before Phase 4 lands. It costs nothing extra, because the mining path needs SSH
regardless.
Two implications people miss:
- **`mempalace.jordbo.se` alone does not enable mining.** HTTPS covers the read/write tool surface
(`search`, `add_drawer`, `diary_write`, `kg_*`) — genuinely useful on its own, and the reason to do this
at all. But `--mode remote` also needs `MEMPALACE_PI_SSH_TARGET` reachable. Budget for both paths.
- **`DEVICE` defaults to `$(hostname)`** (`bin/mempalace-pi-session:163`). In a container that is the
container hostname: either random per recreate (inboxes proliferate; each recreate re-mines into a fresh
empty inbox) or identical across sibling devboxes (two containers writing one inbox). **Set
`MEMPALACE_PI_DEVICE` explicitly per container.** It is a label, not a secret, so put it somewhere
reviewable — a committed compose file — where duplicates are visible. That, not username hygiene in
`.env`, is the discipline this design actually asks of you.
---
## 2. The bind trap, in full
RFC §6.2 and runbook §2.4 already say **do not bind loopback behind the tunnel**, because
`enforce_host_pin = _http_is_loopback(host)` (`mcp_server.py:5367`) makes a loopback bind reject the
proxy's forwarded `Host:` with a **403** that reads exactly like a Pangolin misconfiguration.
**Additional finding, 2026-08-12 — the same reflex also silently removes authentication.** Token
resolution in `cmd_serve` (`cli.py:1447-1450`) is:
```python
loopback = _server_is_loopback(host)
if not token and not loopback and not args.allow_insecure:
token, token_created = _load_or_create_server_token(palace_path)
```
Auto-minting is gated on the bind being **non-loopback**. A loopback bind therefore starts with **no token
at all** — no error, no warning, `--allow-insecure` not required — because the server has concluded it is
only reachable locally, while the tunnel is serving it to the internet. Bind loopback behind newt and you
get a 403 wall *and*, the moment anything relaxes the Host pin, an unauthenticated palace.
Both failure modes have the same cure, already implemented in
`contrib/systemd/mempalace-serve.service`: **bind `172.17.0.1`**. Non-loopback, so the Host pin relaxes and
the token is mandatory; docker0-only, so newt reaches it and the LAN does not.
> Belt and braces: set `MEMPALACE_MCP_HTTP_TOKEN` explicitly in the unit rather than relying on
> auto-minting. Then no future bind change can quietly drop authentication.
---
## 3. Steps
Ordered so nothing is reachable before it is authenticated.
### 3.1 Start the primary (runbook §4.3 — one `sudo`, unit already staged)
```sh
sudo loginctl enable-linger ecsjper
cd ~/.config/systemd/user && mv mempalace-serve.service.staged mempalace-serve.service
systemctl --user daemon-reload && systemctl --user enable --now mempalace-serve
curl -s 172.17.0.1:8765/healthz # expect ok
curl -s 127.0.0.1:8765/healthz # expect 403 — correct, not a bug (§2)
ss -ltnp | grep 8765 # expect 172.17.0.1:8765 only
```
### 3.2 Collect the shared token
```sh
cat ~/.mempalace/server/f5d849287f6d73f0141b29d7/token
```
Directory name is `sha256(realpath(palace))[:24]` — it changes if the palace path ever moves. Store via the
`.env.age` flow, 0600 (RFC §6.2).
### 3.3 newt on synlig
synlig runs Docker (Gitea Actions runner + digikam) but **no tunnel client** — runbook §4.2. Pangolin on
nyvaken cannot dial in; synlig must dial out. Add a `newt` container with the credentials Pangolin issues
for a new site.
Because newt runs in Docker on this box, the docker0 bind is already correct for it: from inside the
container the primary is `172.17.0.1:8765`. Verify from *inside* newt's network namespace, not from the
host, before touching DNS.
> synlig has 7.8 GiB shared with a CI runner (runbook §1). newt is small, but do not colocate anything
> else here casually.
### 3.4 DNS at the web hotel
One CNAME: `mempalace`**the same target your existing Pangolin resources use** (nyvaken's public
hostname). RFC §6.2 costed this as *"one DNS record per service on the web hotel is the whole setup cost."*
⚠️ Not verified from here: nyvaken's public FQDN, and whether your web hotel permits a CNAME at that label
(some require an A record, or forbid CNAME where other records exist). Confirm before assuming a 5-minute job.
### 3.5 Pangolin resource
- Target: newt site → `172.17.0.1:8765`, path `/mcp` (plus `/healthz` if you want the external probe).
- **Auth: none at the Pangolin layer** (§1.1). TLS termination only.
- Do **not** attach an `Origin`-injecting proxy or browser client: a *present* non-loopback `Origin` is a
hard 403 with no override (runbook §2.4 B3).
### 3.6 Verify end-to-end before flipping any client
```sh
curl -s https://mempalace.jordbo.se/healthz # ok
curl -s -o /dev/null -w '%{http_code}\n' -X POST \
https://mempalace.jordbo.se/mcp # 401 — the token is doing its job
curl -s -X POST https://mempalace.jordbo.se/mcp \
-H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | head -c 300 # 36 tools
```
The 401 check matters as much as the 200: it is the only evidence that the thing you just published to the
internet is not open. Then, and only then, Phase 1 client flip — **one machine first** (RFC §8), and
remember opencode containers need the §4.1 sidecar merge before their `.env` takes effect.
---
## 4. Still open
- **Per-device tokens** — Phase 4. Until then `origin_device` is advisory (§1.1).
- **§7.6 diary dedup** must be settled *before* the first §4.4 join; replay duplicates every entry.
- **§7.2**: never run `mempalace sync` against the shared palace. Doubly true now that the pi/opencode
feeders stage *inside* the palace root, which puts staged sources in scope for a sync of the palace dir.
- **nyvaken's public FQDN and the web hotel's CNAME rules** — unverified (§3.4).
+11 -1
View File
@@ -100,6 +100,16 @@ Two of the three pieces already exist. This is not greenfield.
> `pi-devbox/.env.example:21` still advertises `mempalace-mcp --transport http --host 0.0.0.0 --port > `pi-devbox/.env.example:21` still advertises `mempalace-mcp --transport http --host 0.0.0.0 --port
> 8765` as the way to serve a shared palace — replace with `mempalace serve --token … --tls-cert …`, > 8765` as the way to serve a shared palace — replace with `mempalace serve --token … --tls-cert …`,
> and change the example URL from `http://mempalace.lan:8765/mcp` to `https://`. > and change the example URL from `http://mempalace.lan:8765/mcp` to `https://`.
>
> **✅ Done 2026-08-12** — `pi-devbox/.env.example` and
> `mempalace-toolkit/extensions/pi/README.md` both now recommend `mempalace serve`, bind docker0 rather
> than `0.0.0.0`/loopback, use an `https://` example URL, and state that the transport *is*
> authenticated. `docker-compose.mempalace.yml` audited too, and it was worse than stale — it was
> **broken on 3.6.0 in both directions**: `--host 0.0.0.0` with no token in the environment makes the
> server refuse to start (crash-looping under `restart: unless-stopped`), and once a token *is* supplied
> the healthcheck's unauthenticated `tools/list` POST 401s, marking a healthy server unhealthy forever.
> Fixed: token now required via `${MEMPALACE_REMOTE_TOKEN:?}` (fails fast at `up`), healthcheck switched
> to the token-free `/healthz`.
### Two things that sound like the feature and are not ### Two things that sound like the feature and are not
@@ -682,7 +692,7 @@ join replays (§4.4).
| Phase | Effort | Deliverable | | Phase | Effort | Deliverable |
| --- | --- | --- | | --- | --- | --- |
| **0 — hygiene** | hours | §7 runbook: converge the KG/entities store paths **on synlig before first `serve`** (§7.1 — **done 2026-08-10**, runbook §2.3), ban `sync` on shared palaces (§7.2), fix stale "unauthenticated" docs (incl. `pi-devbox/.env.example:21`). Added 2026-08-09: settle the **diary dedup** approach and file its upstream ask (§7.6), and **dry-run the join from one palace only** (§4.4). **No provenance work here** — it is not backfill-critical (§7.3.3) and belongs to the stamper, not the agent | | **0 — hygiene** | hours | §7 runbook: converge the KG/entities store paths **on synlig before first `serve`** (§7.1 — **done 2026-08-10**, runbook §2.3), ban `sync` on shared palaces (§7.2), ~~fix stale "unauthenticated" docs (incl. `pi-devbox/.env.example:21`)~~**done 2026-08-12**, and `docker-compose.mempalace.yml` turned out to be outright broken on 3.6.0, now fixed. Added 2026-08-09: settle the **diary dedup** approach and file its upstream ask (§7.6), and **dry-run the join from one palace only** (§4.4). **No provenance work here** — it is not backfill-critical (§7.3.3) and belongs to the stamper, not the agent |
| **1 — primary up** | hours, **no code** | `mempalace serve --token --tls-cert` on a private-net host (reuse `docker-compose.mempalace.yml` — keep it a **separate standalone project**, R4 — and mind port 8765 vs pi-studio; tor-ms22 already moved to 8766). Repoint pi clients via `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN`. **opencode clients can be repointed in the same breath** — remote MCP is supported and `generate-config.py` already emits it (§9.1, §9.6), subject to the sidecar caveat in §4.1. **Shared memory today, no offline.** **Decided 2026-08-09: primary = synlig, TLS at Pangolin, single shared token (§8.1)** — mind the loopback Host-pin trap in §6.2. | | **1 — primary up** | hours, **no code** | `mempalace serve --token --tls-cert` on a private-net host (reuse `docker-compose.mempalace.yml` — keep it a **separate standalone project**, R4 — and mind port 8765 vs pi-studio; tor-ms22 already moved to 8766). Repoint pi clients via `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN`. **opencode clients can be repointed in the same breath** — remote MCP is supported and `generate-config.py` already emits it (§9.1, §9.6), subject to the sidecar caveat in §4.1. **Shared memory today, no offline.** **Decided 2026-08-09: primary = synlig, TLS at Pangolin, single shared token (§8.1)** — mind the loopback Host-pin trap in §6.2. |
| **1.5 — opencode env propagation** | hours | Make the `mcp.mempalace` subtree env-authoritative in `generate-config.py`, gated by a generated-value fingerprint (§4.1). Independent of the rest of this RFC. Without it, adopting *or reverting* the opt-in on an existing opencode container needs a manual sidecar merge or a `docker volume rm` — which also blocks R6 reversibility | | **1.5 — opencode env propagation** | hours | Make the `mcp.mempalace` subtree env-authoritative in `generate-config.py`, gated by a generated-value fingerprint (§4.1). Independent of the rest of this RFC. Without it, adopting *or reverting* the opt-in on an existing opencode container needs a manual sidecar merge or a `docker volume rm` — which also blocks R6 reversibility |
| **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. **Not** "fixes opencode" — opencode's *transport* is already fine after Phase 1; what edge adds there is offline/local-first, since `generate-config.py`'s switch is remote **or** local with no failover. **Ships with the §1.2 opt-in wiring (compose + `.env.example` + a third branch in the existing `generate-config.py`) and must pass the R1 acceptance test.** | | **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. **Not** "fixes opencode" — opencode's *transport* is already fine after Phase 1; what edge adds there is offline/local-first, since `generate-config.py`'s switch is remote **or** local with no failover. **Ships with the §1.2 opt-in wiring (compose + `.env.example` + a third branch in the existing `generate-config.py`) and must pass the R1 acceptance test.** |
+7 -1
View File
@@ -128,7 +128,13 @@ standing up a network-reachable service while you were asleep was not mine to de
## 4. Tomorrow, in order ## 4. Tomorrow, in order
1. **Pangolin update on nyvaken** (yours). > **2026-08-12: items 12 and 5 now have their own runbook —**
> [`phase-1-exposure-runbook.md`](./phase-1-exposure-runbook.md). Pangolin on nyvaken is updated (done).
> newt is still missing on synlig. That doc also records why per-device Pangolin users are the wrong layer,
> and an additional loopback finding: a loopback bind does not merely 403, it also silently starts the
> server with **no token at all** (auto-minting is gated on the bind being non-loopback).
1. **Pangolin update on nyvaken** (yours). ✅ done 2026-08-12.
2. **⚠️ synlig has no tunnel client.** `docker ps` shows only the Gitea runner and digikam — no `newt`. 2. **⚠️ synlig has no tunnel client.** `docker ps` shows only the Gitea runner and digikam — no `newt`.
Pangolin on nyvaken cannot reach synlig by itself; synlig needs a newt container (or an equivalent Pangolin on nyvaken cannot reach synlig by itself; synlig needs a newt container (or an equivalent
outbound tunnel) added. Easy to miss because Pangolin will look healthy on its own side. If newt runs outbound tunnel) added. Easy to miss because Pangolin will look healthy on its own side. If newt runs
+88 -9
View File
@@ -16,6 +16,7 @@ dependencies (~300 MB).
**Jump to:** **Jump to:**
- [What it does](#what-it-does) - [What it does](#what-it-does)
- [Transport: local vs external](#transport-local-vs-external) - [Transport: local vs external](#transport-local-vs-external)
- [Automatic transcript feeding](#automatic-transcript-feeding)
- [The `Type.Unsafe` gotcha](#the-typeunsafe-gotcha) - [The `Type.Unsafe` gotcha](#the-typeunsafe-gotcha)
- [Deploying pi with mempalace on a new machine](#deploying-pi-with-mempalace-on-a-new-machine) - [Deploying pi with mempalace on a new machine](#deploying-pi-with-mempalace-on-a-new-machine)
- [Fail-soft, identity, debugging](#fail-soft) - [Fail-soft, identity, debugging](#fail-soft)
@@ -37,11 +38,75 @@ dependencies (~300 MB).
injects the result as a `mempalace-wakeup` system message so the injects the result as a `mempalace-wakeup` system message so the
agent orients itself the way `~/.agents/skills/mempalace/SKILL.md` agent orients itself the way `~/.agents/skills/mempalace/SKILL.md`
describes. Skipped on resume/fork (context is already in the thread). describes. Skipped on resume/fork (context is already in the thread).
4. **Manual wind-down** via a `/mempalace-diary [topic]` slash command: 4. **Automatic transcript feeding** (`session_shutdown`, and a debounced
`agent_settled`): stages + mines this pi installation's own session
transcripts into the palace with no user action needed. Unlike the diary
below, this needs no LLM turn — it's a subprocess + a tool call — so it
*can* run on `session_shutdown` where the diary cannot. See
[Automatic transcript feeding](#automatic-transcript-feeding).
5. **Manual wind-down** via a `/mempalace-diary [topic]` slash command:
sends a prompt asking the LLM to call `mempalace_diary_write` with sends a prompt asking the LLM to call `mempalace_diary_write` with
an AAAK-formatted entry summarizing the session. Not fully auto an AAAK-formatted entry summarizing the session. This one stays manual
because pi sessions are typically short/tactical and because it needs the LLM to compose the entry, and `session_shutdown`
`session_shutdown` fires too late to drive another LLM turn. fires too late to drive another LLM turn — a constraint that applies to
the diary specifically, not to feeding (see above).
## Automatic transcript feeding
The bridge feeds this pi installation's own session transcripts into the
palace by itself — no scheduler, no cron, no manual invocation. It fires on
`session_shutdown` (covers quit, `/new`, `/resume`, `/fork`) and on a
debounced `agent_settled` (covers a long session that later crashes, since a
hard kill runs no shutdown handler at all).
The work is split across two processes, and the reason is a hard constraint,
not a style choice: **the palace is single-writer.** A live pi session
always holds it through this extension's own `mempalace-mcp` subprocess, so
an unattended `mempalace mine` from anywhere else fails outright with
`palace ... is held by PID <n>`. The bridge therefore:
1. Runs `mempalace-pi-session --prepare --reason <trigger> --wing <wing>` as a
subprocess. This does every palace-free step — parse pi's JSONL, apply
the quality threshold, stage the export, and (remote mode only) `rsync`
it to the palace host — and prints one line, `MINE_SOURCE=<path>`,
without ever touching the palace.
2. Calls the `mempalace_mine` MCP tool **through this extension's own
client** on that path. Going through the client that already holds the
lock is the only way to write during a live session, and it automatically
targets whichever palace the bridge is pointed at — local stdio or a
shared remote one.
`mempalace-pi-session` (in this repo's `bin/`) is the actual exporter and
owns the quality gate, the remote transport, and every flag — see its
`--help` for the full reference; this section only covers the extension's
side of the wiring.
**Env knobs (extension side):**
| Var | Default | Effect |
|---|---|---|
| `MEMPALACE_FEED` | `1` | Set `0` to disable automatic feeding entirely. |
| `MEMPALACE_FEED_BIN` | `mempalace-pi-session` | Helper to run. |
| `MEMPALACE_FEED_WING` | `wing_conversations` | Target wing — passed to both the exporter and the `mempalace_mine` call. |
| `MEMPALACE_FEED_DEBOUNCE_MS` | `600000` (10 min) | Minimum gap between mid-session (`agent_settled`) feeds. Bounds crash loss to one window instead of a whole session. |
| `MEMPALACE_FEED_PREPARE_TIMEOUT_MS` | `120000` | Kills a wedged `--prepare` subprocess. |
| `MEMPALACE_FEED_MINE_TIMEOUT_MS` | `30000` | Caps the `mempalace_mine` call so a stalled palace can't hang session exit. |
**Remote palace:** if `$MEMPALACE_REMOTE_URL` is set (see
[Transport](#transport-local-vs-external)), `mempalace_mine`'s source path is
expanded on the *server*, which cannot see this machine's transcripts —
that's exactly why step 1 above rsyncs first in that mode. Configure the
inbox with `MEMPALACE_PI_SSH_TARGET` (required for remote feeding — feeding
is silently skipped without it), `MEMPALACE_PI_SSH_CONFIG`, and
`MEMPALACE_PI_REMOTE_PATH`; see `mempalace-pi-session --help`.
**Concurrency:** overlapping triggers coalesce — a `session_shutdown` landing
while a debounced tick is still running joins that in-flight feed instead of
racing it. `mempalace-pi-session` itself also takes a non-blocking `flock`,
so even two independent invocations (e.g. this extension and the
container-start catch-up some devbox images run) never race each other;
losing that race is harmless because the next trigger re-exports from
scratch.
## Transport: local vs external ## Transport: local vs external
@@ -57,11 +122,25 @@ chosen at load time:
harnesses/containers (pi + opencode + native). `MEMPALACE_REMOTE_TOKEN`, if harnesses/containers (pi + opencode + native). `MEMPALACE_REMOTE_TOKEN`, if
set, is sent as `Authorization: Bearer <token>`. set, is sent as `Authorization: Bearer <token>`.
Serve such an endpoint with `mempalace-mcp --transport http --host 0.0.0.0 Serve such an endpoint with `mempalace serve --host 172.17.0.1 --port 8765`
--port 8765` (the `pi-devbox` / `opencode-devbox` repos ship a (the `pi-devbox` / `opencode-devbox` repos ship a
`docker-compose.mempalace.yml` for exactly this). Note: that HTTP transport is `docker-compose.mempalace.yml` for exactly this).
currently sessionless and **unauthenticated** — keep it on a trusted network
or behind a reverse proxy that enforces the bearer token. **The HTTP transport is authenticated as of mempalace 3.6.0**earlier docs
here said otherwise, from the v1.3.0 era. `serve` mints a bearer token, keeps
it 0600, passes it via the environment (never argv), compares it with
`hmac.compare_digest`, and **refuses to bind a non-loopback host without one**
unless `--allow-insecure`. It also pins `Host` and allowlists `Origin`
(anti-DNS-rebinding), and can terminate TLS itself.
Two binds to avoid. `0.0.0.0` publishes the palace to the whole LAN. And
`127.0.0.1` is the trap that looks safe: the Host pin is enforced *only* on
loopback binds, so behind a tunnel every proxied request 403s — and
token auto-minting is gated on the bind being non-loopback, so it starts with
**no authentication at all**, no warning. Bind the docker0 gateway
(`172.17.0.1`): reachable from the host and its containers, not from the LAN.
See
[`docs/phase-1-exposure-runbook.md`](../../docs/phase-1-exposure-runbook.md).
Implementation note: the HTTP client (`RemoteMcpClient`) is **vendored** from Implementation note: the HTTP client (`RemoteMcpClient`) is **vendored** from
[`pi-extensions`](https://gitea.jordbo.se/joakimp/pi-extensions)' [`pi-extensions`](https://gitea.jordbo.se/joakimp/pi-extensions)'
+120
View File
@@ -21,6 +21,21 @@
* `mempalace_status` + `mempalace_diary_read` output as context so the * `mempalace_status` + `mempalace_diary_read` output as context so the
* agent orients itself the way the mempalace skill describes. Skipped * agent orients itself the way the mempalace skill describes. Skipped
* on resume/fork (palace context is already in the thread). * on resume/fork (palace context is already in the thread).
* - Feeding (auto): stage + mine this container's pi transcripts into the
* palace on `session_shutdown` and on a debounced `agent_settled`. Needs
* no LLM turn (pi transcripts are JSONL on disk), which is why it CAN be
* automatic where the diary cannot. The file-side work is delegated to
* `mempalace-pi-session --prepare` (export + threshold + staging, plus the
* rsync to the palace host when the palace is remote); the mine itself
* must run through THIS client, because the palace is single-writer and
* this process is the holder a CLI `mempalace mine` during a live
* session dies with "palace ... is held by PID <ours>". Going through the
* client also means it automatically targets whichever palace this bridge
* is pointed at (local stdio or a shared remote one).
* - MEMPALACE_FEED=0 disable feeding entirely
* - MEMPALACE_FEED_BIN helper to run (default mempalace-pi-session)
* - MEMPALACE_FEED_WING target wing (default wing_conversations)
* - MEMPALACE_FEED_DEBOUNCE_MS min gap between mid-session feeds (default 600000)
* - Wind-down (manual): `/mempalace-diary` command prompts the LLM to * - Wind-down (manual): `/mempalace-diary` command prompts the LLM to
* write an AAAK-formatted diary entry. Not fully auto because pi * write an AAAK-formatted diary entry. Not fully auto because pi
* sessions are typically short/tactical and session_shutdown is too * sessions are typically short/tactical and session_shutdown is too
@@ -724,7 +739,112 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
}); });
} }
// --- Automatic transcript feeding ---
//
// Split deliberately: `mempalace-pi-session --prepare` does the palace-free
// file work (export + quality threshold + staging, plus the rsync to the
// palace host in remote mode) and prints the path to mine; we then mine it
// through this client. See the header note on single-writer contention.
const feedEnabled = (process.env.MEMPALACE_FEED ?? "1") !== "0";
const feedBin = process.env.MEMPALACE_FEED_BIN || "mempalace-pi-session";
const feedWing = process.env.MEMPALACE_FEED_WING ?? "wing_conversations";
const feedDebounceMs = num(process.env.MEMPALACE_FEED_DEBOUNCE_MS, 600_000);
const feedPrepareTimeoutMs = num(process.env.MEMPALACE_FEED_PREPARE_TIMEOUT_MS, 120_000);
const feedMineTimeoutMs = num(process.env.MEMPALACE_FEED_MINE_TIMEOUT_MS, 30_000);
let lastFeedAt = 0; // 0 => the first settled turn also acts as a catch-up
let feedInFlight: Promise<void> | null = null;
/** Run `mempalace-feed --prepare`; resolve the path to mine, or null. */
function prepareFeed(reason: string): Promise<string | null> {
return new Promise((resolve) => {
// A missing helper surfaces as an async 'error' event (ENOENT), not a
// throw, so the handler below is the fail-soft path.
const child = spawn(feedBin, ["--prepare", "--reason", reason, "--wing", feedWing], {
stdio: ["ignore", "pipe", "pipe"],
});
let out = "";
let settled = false;
const finish = (value: string | null) => {
if (settled) return;
settled = true;
clearTimeout(timer);
resolve(value);
};
const timer = setTimeout(() => {
try {
child.kill("SIGKILL");
} catch {
/* already gone */
}
finish(null);
}, feedPrepareTimeoutMs);
child.stdout.on("data", (chunk) => {
out += String(chunk);
});
child.stderr.on("data", () => {
/* the script keeps its own log */
});
child.on("error", () => finish(null));
child.on("exit", (code) => {
if (code !== 0) return finish(null);
const match = out.match(/^MINE_SOURCE=(.+)$/m);
finish(match ? match[1].trim() : null);
});
});
}
/**
* Stage + mine this container's transcripts. Never throws, and coalesces:
* an overlapping trigger joins the in-flight run instead of racing it.
*/
function feedPalace(reason: string): Promise<void> {
if (!feedEnabled || !available) return Promise.resolve();
if (feedInFlight) return feedInFlight;
const run = (async () => {
try {
const source = await prepareFeed(reason);
if (!source) return;
await Promise.race([
client.callTool("mempalace_mine", {
source,
mode: "convos",
wing: feedWing,
agent: agentName,
}),
new Promise((_resolve, reject) =>
setTimeout(
() => reject(new Error(`mine timed out after ${feedMineTimeoutMs}ms`)),
feedMineTimeoutMs,
),
),
]);
lastFeedAt = Date.now();
} catch (err) {
process.stderr.write(
`[mempalace ext] feed (${reason}) failed: ${(err as Error).message}\n`,
);
}
})();
feedInFlight = run.finally(() => {
feedInFlight = null;
});
return feedInFlight;
}
// Mid-session feed. A hard container kill runs no handler at all, so this is
// what bounds crash loss to one debounce window instead of a whole session.
// Re-mining a grown transcript purges and refiles that source_file, so
// repeated ticks refresh a session's drawers rather than duplicating them.
pi.on("agent_settled", async () => {
if (Date.now() - lastFeedAt < feedDebounceMs) return;
void feedPalace("tick"); // deliberately not awaited: never stall a turn
});
pi.on("session_shutdown", async () => { pi.on("session_shutdown", async () => {
// Feed before stopping the client: we are the palace holder, so nothing
// else can mine while we live. pi awaits this handler, so the mine really
// does complete; feedMineTimeoutMs keeps a wedged palace from hanging exit.
await feedPalace("shutdown");
client.stop(); client.stop();
}); });