diff --git a/docs/rfc-001-global-palace.md b/docs/rfc-001-global-palace.md index 0d518cb..e3ae989 100644 --- a/docs/rfc-001-global-palace.md +++ b/docs/rfc-001-global-palace.md @@ -5,7 +5,7 @@ | **Status** | Draft — design agreed, not implemented | | **Created** | 2026-08-08 | | **Applies to** | mempalace 3.6.0, mempalace-toolkit @ `96699f2`, pi-devbox ≥ v1.3.0 | -| **Decision** | Phases 0–2 + 4 in scope. **Phase 3 (full pull replication) explicitly deferred** — "a laptop that can reach its own stuff plus whatever it can reach" is good enough. | +| **Decision** | Phases 0–2 + 4 in scope. **Phase 3 (full pull replication) explicitly deferred** — "a laptop that can reach its own stuff plus whatever it can reach" is good enough. **Centralization is strictly opt-in — solitary devbox operation remains the default and must not change (§1.1).** | **Read this first if you are asked to "centralize MemPalace" / "sync palaces between machines".** Most of the hard-won facts below are non-obvious and two of them are actively destructive if you guess wrong @@ -28,6 +28,56 @@ One palace per machine per harness. Today: a pi-devbox container on `EMB-7KJ4VR4 primary is unreachable (writes buffer locally, reads degrade to local), and reconciles when it returns. Access is authenticated per device, with a sane authorization policy. +### 1.1 Hard requirement: solitary-first, centralization strictly opt-in + +**This is a requirement, not a preference, and it constrains every choice below.** + +Centralization solves a problem specific to *this* usage pattern: several machines and containers +(pi-devbox, opencode-devbox, native hosts) doing development and other work against a **common set of +artifacts**. For most users of the published `joakimp/pi-devbox` and `joakimp/opencode-devbox` images +it is **useless** — one machine, one palace, done. Evidence that this is already the norm: all three +sampled `opencode-devbox` deployments (`docker-compose-repo/{synlig,nyvaken,devbox-affection}/`) +contain **zero** mempalace references. + +| ID | Requirement | +| --- | --- | +| **R1** | **Solitary operation stays the default and stays unchanged.** With no `MEMPALACE_*` variables set, a devbox must behave exactly as it does today: harness → local `mempalace-mcp` over stdio, palace at `~/.mempalace`. No extra process, no outbox, no network calls, no new failure mode, no measurable startup cost. | +| **R2** | **Opt-in lives in `docker-compose.yml` + `.env`** — the mechanism users already know. No opt-in via image rebuild, no baked-in defaults, no `latest-central` variant. | +| **R3** | **Credentials only in `.env`** (`chmod 600`, gitignored). Never in `docker-compose.yml`, never in the image, never on a command line (visible in `ps`), never logged. Compose passes them through with a `${VAR:-}` empty default so solitary users never define them. | +| **R4** | **No new required services.** The primary stays in the separate standalone `docker-compose.mempalace.yml` project; it is never merged into the main compose file. A solitary `docker compose up` starts exactly what it starts today. | +| **R5** | **Degrade, never fail, when mempalace is absent.** opencode-devbox may not ship mempalace at all, so any wiring must be additive and skipped when the binary is missing — the probe-and-warn idiom `install.sh` already uses (`warn` + `return 0`, never halt). | +| **R6** | **Reversible.** Commenting the `.env` lines out returns the container to pure solitary operation, with the local palace intact and readable. | + +**Acceptance test for R1** (must pass before Phase 2 ships): bring up a devbox with no `MEMPALACE_*` +variables; assert the palace tool list, drawer counts and diary writes are identical to the previous +image, and that no edge/outbox process exists (`pgrep -f mempalace-edge` → empty). + +### 1.2 The opt-in surface + +The existing convention is already the right one — **extend it, do not invent a new one.** pi-devbox's +`.env.example` lines 12–23 already ship a commented `MEMPALACE_REMOTE_URL` / `MEMPALACE_REMOTE_TOKEN` +pair under the heading "MemPalace memory (local by default)", and `docker-compose.yml:79-83` already +ships the `devbox-palace` volume commented out. So the opt-in ladder becomes three states derived +from two variables — **both existing behaviours are preserved, the third is new**: + +| `MEMPALACE_REMOTE_URL` | `MEMPALACE_EDGE` | Behaviour | Status | +| --- | --- | --- | --- | +| unset | — | local stdio `mempalace-mcp`, palace at `~/.mempalace` | **default, today, unchanged (R1)** | +| set | unset/`0` | direct remote HTTP; no local palace, no offline | today (`96699f2`), unchanged | +| set | `1` | `mempalace-edge`: local-first writes + outbox → primary + merged reads | **new (Phase 2)** | + +> **Volume coupling reverses under edge mode — document it prominently.** Today `.env.example` correctly +> says that with `MEMPALACE_REMOTE_URL` set "*the devbox-palace volume is then irrelevant*", because the +> remote owns all state. Under **edge** mode that flips: the local palace holds the outbox and all +> local-first writes, so an un-persisted palace means **losing un-flushed writes on container +> recreate**. Opting into `MEMPALACE_EDGE=1` therefore *requires* uncommenting +> `devbox-palace:/home/developer/.mempalace`. + +Corollary for §4.1: **`mempalace-edge` must not be in the path unless opted in.** Registering it +unconditionally (letting it decide by env at runtime) is tempting — one static config for every +harness — but it violates R1 by inserting a process and a failure mode into every solitary user's +setup. Selection must happen at registration time. See open question §9.6. + --- ## 2. Verified starting point (as of 2026-08-08) @@ -45,7 +95,10 @@ Two of the three pieces already exist. This is not greenfield. > **Stale docs warning.** `docker-compose.mempalace.yml`, `.env.example` and CHANGELOG v1.3.0 all say the > HTTP transport is unauthenticated and should be fronted by a reverse proxy. That was true for > `mempalace-mcp --transport http` in the v1.3.0 era. **mempalace 3.6.0's `serve` has token + TLS -> built in** (upstream #1877). Fix those comments during Phase 1. +> built in** (upstream #1877). Fix those comments during Phase 1. Specifically: +> `pi-devbox/.env.example:21` still advertises `mempalace-mcp --transport http --host 0.0.0.0 --port +> 8765` as the way to serve a shared palace — replace with `mempalace serve --token … --tls-cert …`, +> and change the example URL from `http://mempalace.lan:8765/mcp` to `https://`. ### Two things that sound like the feature and are not @@ -127,6 +180,16 @@ Why this shape wins: - The remote transport is **sessionless JSON-RPC** today (per `96699f2`'s own note), so reconnect after an outage is cheap — there is no session to re-establish. +**But per R1, that one-line change is conditional, not baked in.** The edge binary is present in the +image (it costs nothing unused) and inserted into the path only when `MEMPALACE_EDGE=1`: + +- **pi**: `createClient()` (`extensions/pi/mempalace.ts:629`) already branches on env at startup — add a + third branch. Zero change to the default path. +- **opencode**: the MCP server entry is static JSON, so the entrypoint must template it — write + `command: ["mempalace-edge"]` only when opted in, otherwise leave today's + `["mempalace-mcp"]` untouched. If mempalace is absent entirely (as in all three sampled + opencode-devbox deployments), write nothing and warn (R5). + ### 4.2 Routing policy `service.py:29,54,72` already ships the exact three-way split we need — `READ_TOOLS`, @@ -309,9 +372,9 @@ across models silently degrades recall. | Phase | Effort | Deliverable | | --- | --- | --- | -| **0 — hygiene** | hours | §7 runbook: move KG/hallways/entities, add provenance metadata, ban `sync` on shared palaces, per-device tokens, fix stale "unauthenticated" docs | -| **1 — primary up** | hours, **no code** | `mempalace serve --token --tls-cert` on a private-net host (reuse `docker-compose.mempalace.yml`, mind port 8765 vs pi-studio — tor-ms22 already moved to 8766). Repoint pi clients via `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN`. **Shared memory today, no offline.** | -| **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. Fixes opencode as a side effect. | +| **0 — hygiene** | hours | §7 runbook: move KG/hallways/entities, add provenance metadata, ban `sync` on shared palaces, per-device tokens, fix stale "unauthenticated" docs (incl. `pi-devbox/.env.example:21`) | +| **1 — primary up** | hours, **no code** | `mempalace serve --token --tls-cert` on a private-net host (reuse `docker-compose.mempalace.yml` — keep it a **separate standalone project**, R4 — and mind port 8765 vs pi-studio; tor-ms22 already moved to 8766). Repoint pi clients via `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN`. **Shared memory today, no offline.** | +| **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. Fixes opencode as a side effect. **Ships with the §1.2 opt-in wiring (compose + `.env.example` + entrypoint templating) and must pass the R1 acceptance test.** | | **3 — pull replication** | ~1 week | **DEFERRED (2026-08-08).** Server-side op-log with monotonic seq → each edge a full offline replica. Only needed if a laptop must hold *everything* offline. Accepted trade-off: offline recall = own writes + last-reachable state. | | **4 — authz** | days | Per-wing ACL, per-device scopes, audit log, token rotation | @@ -343,6 +406,13 @@ for online clients. 5. **Does `_HTTP_REQUEST_LOCK` stay held for the duration of an MCP-triggered `mine`?** Strongly suggested by the code shape; if yes, a remote mine makes the primary unusable for its duration (another argument for §5). +6. **How does opencode-devbox learn the opt-in?** pi's transport choice is code (`createClient()`), so + it reads `.env` for free. opencode's MCP registration is **static JSON** in + `~/.config/opencode/opencode.json`, which per R1/R2 must be templated at container start rather + than baked. Unknown: whether opencode-devbox's entrypoint has a config-templating step to hook, or + whether the file is user-owned and bind-mounted (in which case the opt-in is a documented manual + edit — acceptable, but say so). Also unknown whether the published `opencode-devbox` image ships + mempalace at all; if not, edge mode there is host-side only until it does. --- @@ -370,6 +440,8 @@ Re-verify without re-reading the package. Paths relative to | Migrate as export primitive | `migrate.py:extract_drawers_from_sqlite`, `:326` | | pi remote transport + fail-soft | `mempalace-toolkit/extensions/pi/mempalace.ts:11-12`, `87`, `108`, `390`, `629-641`, `665-673` (commit `96699f2`) | | Server compose + port history | `pi-devbox/docker-compose.mempalace.yml`; `pi-devbox/CHANGELOG.md` v1.3.0; `docker-compose-repo/tor-ms22/pi-devbox/` (`df2c2ae`) | +| Existing opt-in convention (§1.2) | `pi-devbox/.env.example:12-23` ("MemPalace memory (local by default)", commented `MEMPALACE_REMOTE_URL`/`_TOKEN`); `pi-devbox/docker-compose.yml:79-83` (`devbox-palace` volume commented out) | +| opencode-devbox does not wire mempalace | `docker-compose-repo/{synlig,nyvaken,devbox-affection}/opencode-devbox/docker-compose.yml` — zero mempalace references in any | ## 11. See also