mempalace-toolkit/docs/rfc-001-global-palace.md

# RFC 001 — Global palace with local fallback (`mempalace-edge`)

| | |
| --- | --- |
| **Status** | Draft — design agreed, not implemented |
| **Created** | 2026-08-08 |
| **Applies to** | mempalace 3.6.0, mempalace-toolkit @ `96699f2`, pi-devbox ≥ v1.3.0 |
| **Decision** | Phases 0–2 + 4 in scope. **Phase 3 (full pull replication) explicitly deferred** — "a laptop that can reach its own stuff plus whatever it can reach" is good enough. **Centralization is strictly opt-in — solitary devbox operation remains the default and must not change (§1.1).** |
| **Recon update** | **2026-08-09 — §9 Q1 and Q6 are RESOLVED, both in the permissive direction** (opencode supports remote MCP; opencode-devbox already templates the mempalace entry). Neither is a blocker. See those entries for evidence; §2, §4.1 and R5 were corrected accordingly. |

**Read this first if you are asked to "centralize MemPalace" / "sync palaces between machines".** Most of the
hard-won facts below are non-obvious and two of them are actively destructive if you guess wrong
(§7.1, §7.2). The evidence index in §10 lets you re-verify any claim without re-reading 45k lines.

---

## 1. The problem

One palace per machine per harness. Today: a pi-devbox container on `EMB-7KJ4VR4G`, another on
`tor-ms22`, opencode-devbox containers, `MBP-M1-2020`, plus native hosts — each with a private
`~/.mempalace`. Consequences:

1. **Memory is sharded by accident of where you happened to be working.** A decision recorded on the
   laptop is invisible to the agent on the desktop.
2. **Container recreates are amnesia events** unless the palace happens to be host-bind-mounted.
3. **The KG is the worst hit** — `kg_query`/`kg_timeline` answers depend on which machine you ask.

**Target state:** one primary palace holds the fleet's memory. Every client keeps working when the
primary is unreachable (writes buffer locally, reads degrade to local), and reconciles when it
returns. Access is authenticated per device, with a sane authorization policy.

### 1.1 Hard requirement: solitary-first, centralization strictly opt-in

**This is a requirement, not a preference, and it constrains every choice below.**

Centralization solves a problem specific to *this* usage pattern: several machines and containers
(pi-devbox, opencode-devbox, native hosts) doing development and other work against a **common set of
artifacts**. For most users of the published `joakimp/pi-devbox` and `joakimp/opencode-devbox` images
it is **useless** — one machine, one palace, done. Evidence that this is already the norm: all three
sampled `opencode-devbox` deployments (`docker-compose-repo/{synlig,nyvaken,devbox-affection}/`)
contain **zero** mempalace references.

| ID | Requirement |
| --- | --- |
| **R1** | **Solitary operation stays the default and stays unchanged.** With no `MEMPALACE_*` variables set, a devbox must behave exactly as it does today: harness → local `mempalace-mcp` over stdio, palace at `~/.mempalace`. No extra process, no outbox, no network calls, no new failure mode, no measurable startup cost. |
| **R2** | **Opt-in lives in `docker-compose.yml` + `.env`** — the mechanism users already know. No opt-in via image rebuild, no baked-in defaults, no `latest-central` variant. |
| **R3** | **Credentials only in `.env`** (`chmod 600`, gitignored). Never in `docker-compose.yml`, never in the image, never on a command line (visible in `ps`), never logged. Compose passes them through with a `${VAR:-}` empty default so solitary users never define them. |
| **R4** | **No new required services.** The primary stays in the separate standalone `docker-compose.mempalace.yml` project; it is never merged into the main compose file. A solitary `docker compose up` starts exactly what it starts today. |
| **R5** | **Degrade, never fail, when mempalace is absent.** The published `opencode-devbox` image *does* ship mempalace by default (`Dockerfile.base:380,402` — `ARG INSTALL_MEMPALACE=true`, `MEMPALACE_VERSION=3.6.0`, installed via `uv tool install`), but it is a build arg precisely so it can be omitted to save ~300 MB — and a bind-mounted host config or a non-devbox client may have no mempalace either. So any wiring must stay additive and skip when the binary is missing — the probe-and-warn idiom `install.sh` already uses (`warn` + `return 0`, never halt). |
| **R6** | **Reversible.** Commenting the `.env` lines out returns the container to pure solitary operation, with the local palace intact and readable. |

**Acceptance test for R1** (must pass before Phase 2 ships): bring up a devbox with no `MEMPALACE_*`
variables; assert the palace tool list, drawer counts and diary writes are identical to the previous
image, and that no edge/outbox process exists (`pgrep -f mempalace-edge` → empty).

### 1.2 The opt-in surface

The existing convention is already the right one — **extend it, do not invent a new one.** pi-devbox's
`.env.example` lines 12–23 already ship a commented `MEMPALACE_REMOTE_URL` / `MEMPALACE_REMOTE_TOKEN`
pair under the heading "MemPalace memory (local by default)", and `docker-compose.yml:79-83` already
ships the `devbox-palace` volume commented out. So the opt-in ladder becomes three states derived
from two variables — **both existing behaviours are preserved, the third is new**:

| `MEMPALACE_REMOTE_URL` | `MEMPALACE_EDGE` | Behaviour | Status |
| --- | --- | --- | --- |
| unset | — | local stdio `mempalace-mcp`, palace at `~/.mempalace` | **default, today, unchanged (R1)** |
| set | unset/`0` | direct remote HTTP; no local palace, no offline | today (`96699f2`), unchanged |
| set | `1` | `mempalace-edge`: local-first writes + outbox → primary + merged reads | **new (Phase 2)** |

> **Volume coupling reverses under edge mode — document it prominently.** Today `.env.example` correctly
> says that with `MEMPALACE_REMOTE_URL` set "*the devbox-palace volume is then irrelevant*", because the
> remote owns all state. Under **edge** mode that flips: the local palace holds the outbox and all
> local-first writes, so an un-persisted palace means **losing un-flushed writes on container
> recreate**. Opting into `MEMPALACE_EDGE=1` therefore *requires* uncommenting
> `devbox-palace:/home/developer/.mempalace`.

Corollary for §4.1: **`mempalace-edge` must not be in the path unless opted in.** Registering it
unconditionally (letting it decide by env at runtime) is tempting — one static config for every
harness — but it violates R1 by inserting a process and a failure mode into every solitary user's
setup. Selection must happen at registration time. See open question §9.6.

---

## 2. Verified starting point (as of 2026-08-08)

Two of the three pieces already exist. This is not greenfield.

| Piece | State | Evidence |
| --- | --- | --- |
| **Primary server** | ✅ **Exists.** `mempalace serve --host --port --token --tls-cert --tls-key --read-only --allow-insecure`. Bearer token compared with `hmac.compare_digest`, **mandatory** on non-loopback binds (unless `--allow-insecure`), TLS 1.2+ resolved *before* bind, Host-header pinning + `Origin` allowlist (anti-DNS-rebinding), 16 MiB body cap, token-free `/healthz`. | `cli.py:cmd_serve` (~1448); `mcp_server.py:5205-5215`, `5284-5289` |
| **Remote client** | ⚠️ **Exists for pi only, and it is either/or.** `createClient()` picks stdio *or* HTTP once at process start. | `extensions/pi/mempalace.ts:629-641` |
| **Fallback + resync** | ❌ **Absent everywhere.** On remote failure the pi bridge re-handshakes the same URL, then **de-registers all palace tools** and runs blind. | `extensions/pi/mempalace.ts:665-673` |
| **opencode client** | ✅ **Remote is supported and already wired.** opencode's published schema (`https://opencode.ai/config.json`, `$defs.McpRemoteConfig`) makes `{"type":"remote","url","headers","oauth"}` a first-class sibling of `McpLocalConfig`, `headers` being a free string→string map (so bearer is a convention, not a constraint). `opencode-devbox` already emits exactly that entry when `MEMPALACE_REMOTE_URL` is set. The all-`type:local` configs in `myconfigs` are a *deployment* fact, not a capability limit. | `generate-config.py:107-118`; hook at `entrypoint-user.sh:117`; schema `$defs.McpRemoteConfig` |
| **Server compose** | ✅ Exists: `pi-devbox/docker-compose.mempalace.yml` (canonical) + a tor-ms22 derivative (`docker-compose-repo/tor-ms22/pi-devbox/`, `df2c2ae`, port 8766, uid 1000, binds the real `~/.mempalace`). Not enabled. | pi-devbox CHANGELOG v1.3.0 (2026-07-02) |

> **Stale docs warning.** `docker-compose.mempalace.yml`, `.env.example` and CHANGELOG v1.3.0 all say the
> HTTP transport is unauthenticated and should be fronted by a reverse proxy. That was true for
> `mempalace-mcp --transport http` in the v1.3.0 era. **mempalace 3.6.0's `serve` has token + TLS
> built in** (upstream #1877). Fix those comments during Phase 1. Specifically:
> `pi-devbox/.env.example:21` still advertises `mempalace-mcp --transport http --host 0.0.0.0 --port
> 8765` as the way to serve a shared palace — replace with `mempalace serve --token … --tls-cert …`,
> and change the example URL from `http://mempalace.lan:8765/mcp` to `https://`.

### Two things that sound like the feature and are not

- **`sync.py` is not replication.** It is gitignore-aware drawer *deletion*: "*Removes drawers whose
  source files are now gitignored, deleted, or moved out of the project*" (`sync.py:1-12`). See §7.2 —
  running it against a shared palace is a fleet-wide memory wipe.
- **`wal/write_log.jsonl` is not a replayable WAL.** `_WAL_REDACT_KEYS` strips
  `content`/`content_preview`/`document`/`entry`/`entry_preview`/`query`/`text` and replaces them with
  `"[REDACTED N chars]"`; all 12 references are writers, there is **no reader** and no `replay`
  function (`wal.py:74`). Reconstructing memory from it is information-theoretically impossible.

---

## 3. Why we sync operations, not databases

Row-level / file-level replication of a palace is a trap in this codebase. Four findings, each
independently disqualifying:

1. **Vectors are not portable.** Embeddings are computed **client-side** — `backends/pgvector.py:6-8`:
   "*Embeddings are still produced locally by MemPalace through the core embedding wrapper before
   vectors are written to Postgres.*" Cross-architecture ONNX determinism (arm64 macOS vs x86-64
   Linux, different execution providers) is not guaranteed. → **Ship text + metadata, always re-embed
   on receipt.** Cheap and safe: the raw text is always stored as the chroma document.
2. **KG rows cannot be copied.** `triples` has **no** `UNIQUE(subject,predicate,object,valid_from)` —
   only `id` is unique, and `make_triple_id` embeds `datetime.now()`. A row copy therefore duplicates
   every fact. But `add_triple()` guards at the application level (`SELECT id … WHERE subject=? AND
   predicate=? AND object=? AND valid_to IS NULL` → returns the existing id), so **replaying `kg_add`
   is idempotent for open facts** (`knowledge_graph.py:163-178`, `305-313`).
3. **Only one write path has deterministic IDs.**

   | Write path | ID recipe | Same content on 2 hosts → same ID? |
   | --- | --- | --- |
   | `add_drawer` / `checkpoint` | `drawer_{wing}_{room}_{sha256(wing\|room\|content)[:24]}` | **YES** — content-addressed, merges for free |
   | project/format miner | `…sha256(source_file\|chunk_index)` | **NO** — absolute path (`/Users/joakim/x` vs `/workspace/x`) |
   | convo miner | `…sha256(source_file\|extract_mode\|chunk_index)` | **NO** — same reason |
   | diary | `diary_{wing}_{YYYYmmdd_HHMMSSffffff}_{sha256(entry)[:12]}` | **NO** (µs timestamp) — but the 12-hex suffix is a usable content dedup key |
   | KG triple | `t_{s}_{p}_{o}_{sha256(valid_from\|recorded_at)[:12]}` | **NO** (`recorded_at = now()`) |

   (`ids.py:56,71` — `ID_RECIPE = "v3"`, length-prefixed delimited hashing.)
4. **Batch dedup will not save a naive merge.** `dedup.py` groups by `source_file` and only compares
   *within* a group (cosine < 0.15) → cross-host duplicates with differing paths are never compared.
   The mechanism that *does* work is write-time: `tool_check_duplicate(content, threshold=0.9)` queries
   the whole collection, and `mempalace_checkpoint` already runs it per item.

**Conclusion:** the MCP tool surface is already a small, coarse-grained, mostly-idempotent operation
vocabulary. Log the *intent*, replay the *intent*. That sidesteps chroma internals, chunking,
embedder drift and vector portability in one move.

---

## 4. Design: `mempalace-edge`, a local MCP proxy

### 4.1 Shape

Do **not** put fallback logic in `extensions/pi/mempalace.ts` — opencode would need it again, and it
may not support remote MCP at all (§9.1). Instead: a sidecar that *is* an MCP server.

```
pi        (mempalace.ts, stdio) ─┐
opencode  (mcp: type=local)     ─┼──►  mempalace-edge  ──HTTPS MCP──►   PRIMARY
mempalace CLI (local palace)    ─┘        │   (stdio MCP server)       mempalace serve
                                          ├─ child: mempalace-mcp       --token --tls-cert
                                          │  (local palace, always writable)
                                          └─ outbox.sqlite (durable op queue)
```

**`mempalace-edge` needs zero mempalace internals** — it is an MCP-to-MCP proxy. It speaks stdio down
to a child `mempalace-mcp` and HTTPS up to the primary, reusing the `RemoteMcpClient` already written
in `96699f2` (`extensions/pi/mempalace.ts:390`, vendored from `pi-extensions/mcp-loader.ts` — mind the
`MCP-STREAMABLE-HTTP-CLIENT-SYNC: v1` drift token if you copy it again).

Why this shape wins:

- **One implementation for every harness.** opencode's change is one line:
  `"command": ["mempalace-edge"]`. pi's is one line. The CLI is untouched.
- **Tools never vanish** from the tool list, so wake-up context injection keeps working offline —
  unlike today's fail-soft de-registration.
- **Survives mempalace upgrades**: coupled to the tool schema, not to chroma/HNSW.
- The remote transport is **sessionless JSON-RPC** today (per `96699f2`'s own note), so reconnect
  after an outage is cheap — there is no session to re-establish.

**But per R1, that one-line change is conditional, not baked in.** The edge binary is present in the
image (it costs nothing unused) and inserted into the path only when `MEMPALACE_EDGE=1`:

- **pi**: `createClient()` (`extensions/pi/mempalace.ts:629`) already branches on env at startup — add a
  third branch. Zero change to the default path.
- **opencode**: the MCP server entry is static JSON — but **the templating step already exists and
  already implements the first two rungs of the §1.2 ladder.**
  `rootfs/usr/local/lib/opencode-devbox/generate-config.py` (run unconditionally from
  `entrypoint-user.sh:117`) registers `mempalace` as `{"type":"remote", url, headers:{Authorization:
  Bearer …}}` when `MEMPALACE_REMOTE_URL` is set, else `{"type":"local","command":["mempalace-mcp"]}`
  when the binary is on PATH, else nothing — R5-compliant already. Its own comment states it uses the
  "*same env contract as the mempalace.ts pi extension … so one shared MemPalace can serve pi +
  opencode + native*", i.e. the two images are deliberately kept in step. **Phase 2's opencode work is
  therefore a third branch in an existing script, not a new mechanism.**

  > ⚠️ **But the opt-in does not propagate to an existing container.** `generate-config.py` *never*
  > overwrites an existing config, and `~/.config/opencode` is the named volume
  > `devbox-opencode-config` (`docker-compose.yml:64,153`) — so a config generated during solitary use
  > survives recreate, and later setting `MEMPALACE_REMOTE_URL`/`MEMPALACE_EDGE` only produces a
  > non-loaded `opencode.jsonc.proposed` sidecar for manual merge (`generate-config.py:203-244`).
  > Flipping the `.env` alone is a no-op there. Phase 1/2 docs must say: merge the sidecar, or
  > `docker volume rm` the config volume, to adopt the change — and the same applies in reverse for R6
  > (reversibility). This asymmetry with pi — whose `createClient()` re-reads env every start — is the
  > single biggest behavioural difference between the two harnesses under this RFC.

### 4.2 Routing policy

`service.py:29,54,72` already ships the exact three-way split we need — `READ_TOOLS`,
`WRITE_TOOLS`, `MAINTENANCE_TOOLS` (`{mine, sync, reconnect}`) — plus `classify_tool()`.

| Class | Primary up | Primary down |
| --- | --- | --- |
| **read** (`search`, `kg_query`, `diary_read`, `list_*`, `traverse`…) | query **both**, merge ranked lists, dedupe by drawer id | local only, response flagged `degraded: true` |
| **write** (`add_drawer`, `checkpoint`, `diary_write`, `kg_add`…) | apply local **and** enqueue → primary | apply local, enqueue, keep working |
| **maintenance** (`mine`, `sync`, `reconnect`) | **never proxied** — local only (§5, §7.2) | local only |
| **destructive** (`delete_drawer`, `delete_by_source`, `delete_tunnel`…) | admin-scoped only; tombstone, never hard-delete remotely | local only |

> ⚠️ **`classify_tool()` must be treated as fail-closed.** `mempalace_kg_supersede` is a real MCP tool
> (3 references in `mcp_server.py`) but is **absent from `WRITE_TOOLS`** → `classify_tool()` returns
> `"unknown"` for it. An edge proxy that routed "unknown" as read would silently drop supersede
> operations. Maintain our own tool→class table, default unknown ⇒ **write**, and file the upstream fix.

**The merged-read trick is what makes Phase 3 optional.** Both palaces return ranked results with
distances; if the embedder identity matches (§7.4) the distances are comparable, so a union +
re-sort + dedupe-by-id gives a correct combined result set. You get fleet-wide recall *without*
replicating anything. Offline you simply see less.

KG reads are the exception: unioning triples is unsafe because `invalidate`/`supersede` are
order-dependent. Use **primary-first, local-fallback** (no union) for `kg_*` reads, and accept that an
offline KG answer is incomplete.

### 4.3 Outbox

Model it on `daemon.py`, which already implements a durable local job queue and is the closest
existing prior art (token auth via `ensure_token`, 0600, per-palace state dir, `MAX_ATTEMPTS = 3`,
`recover_running`, `JOB_RETENTION_DAYS = 7`). Its key primitive is worth copying verbatim:

```sql
CREATE UNIQUE INDEX IF NOT EXISTS idx_jobs_dedupe_active
  ON jobs(dedupe_key) WHERE state IN ('queued', 'running');
```

A TOCTOU-safe, cross-process "at most one active job per key". Our outbox rows:

| Column | Purpose |
| --- | --- |
| `op_id` | `sha256(origin_device, local_seq, tool_name, canonical_payload)` — stable across retries, the primary's idempotency key |
| `local_seq` | Per-device monotonic counter → **causal order** for `kg_invalidate`/`supersede` replay |
| `origin_device`, `origin_label` | Provenance, stamped by edge (interim) or the primary (authoritative) — never by the agent (§7.3) |
| `tool_name`, `payload_json` | The MCP call to replay |
| `state`, `attempts`, `created_at`, `flushed_at`, `error_json` | Retry/audit |

Flush = drain in `local_seq` order, stop on first hard failure (preserve ordering), exponential
backoff, resume on reconnect. The primary de-dupes on `op_id`; `add_drawer`'s own pre-write probe and
`add_triple`'s open-fact guard make replay safe even if `op_id` tracking is lost.

`write_routing.py` is the upstream-shaped seam for this if we ever want it in core: its
`WriteRoutingPolicy{DIRECT,PREFER,REQUIRE}` → `WriteRoutingTarget{DIRECT,DAEMON,BLOCKED}` enums would
gain a `REMOTE` target. The module says it "*changes no caller defaults by itself*" — i.e. it exists
precisely to be extended.

---

## 5. Not everything should be global

Per-wing replication policy, declared in edge config:

| Policy | Wings | Rationale |
| --- | --- | --- |
| `replicated` | diaries, `wing_pi`, checkpoints, KG, hand-authored notes | Small, curated, content-addressed → **already merge-safe** |
| `local` | mined code/docs (e.g. the 13,597-drawer `workspace` wing on this host) | Derived data, re-mineable from git, and carries exactly the path-dependent IDs that break merging (§3.3) |

This shrinks the hard problem to the layer that is already safe, and it is why **`mine` must never be
remote**: the primary serializes *every* request behind one lock (§7.5), so a bulk remote mine would
stall every other agent in the fleet.

---

## 6. Security model

The transport is largely solved; **the gap is authorization, not cryptography.**

### 6.1 Threats, in priority order

1. **Memory poisoning / persistent cross-machine prompt injection — the underrated one.** Palace
   content is injected into agent context at wake-up. A shared palace means one careless or
   compromised container can plant instructions that *every other agent in the fleet reads as trusted
   memory*. Mitigation: server-stamped provenance on every synced record (§7.3.2), don't auto-inject wake-up content authored
   by devices outside a trusted set, keep an admin-only wing for anything instruction-shaped.
2. **Aggregation raises exfiltration impact.** One primary holds every machine's diaries — which
   already quote internal hostnames, paths and token prefixes. Mitigation: per-wing ACL; don't put
   secrets in memory (the WAL redaction list exists for a reason); private-network-only exposure.
3. **Accidental mass deletion** by a client (`sync`, `delete_by_source`) — see §7.2.
4. **Availability**: the server is single-writer by design; one long operation blocks everyone.

### 6.2 Policy

| Control | Decision |
| --- | --- |
| **Network posture** | Primary **never** internet-exposed. Bind loopback in the container; publish only onto the private overlay / existing tunnel (Pangolin/newt). |
| **Transport** | TLS via `serve --tls-cert/--tls-key`, or terminate TLS **+ mTLS** in the reverse proxy (cheaper than patching Python's TLS surface). |
| **Authentication** | **Per-device bearer tokens**, not the one shared token. Server-side registry `token → {device_id, scopes}`. Store in the existing `.env.age` flow, 0600 on disk. Enables revoking one laptop and rotating without a fleet outage. |
| **Authorization** | Per-device read/write **wing globs**. Server-side refusal of `mine`/`sync`/`delete_*` except for an admin device. `--read-only` gives a free observer tier. |
| **Provenance/audit** | `origin_device` (+ optional `origin_label`) + `op_id` on every record, stamped **server-side from the authenticated credential** — a client-asserted origin is a hint, not a fact (§7.3.2); server-side append-only op log. |
| **Recovery** | Deletes as tombstones; server-side backups with `backups.py` retention (`MEMPALACE_MAX_BACKUPS`). |
| **DoS** | Keep the 16 MiB cap; add rate limiting; no remote `mine`. |

---

## 7. Landmines — Phase 0 runbook

Do these **before** any cutover. 7.1 and 7.2 cause silent data loss.

### 7.1 `MEMPALACE_PALACE_PATH` ≠ `--palace` (silent empty KG)

```python
# mcp_server.py:708-711  (_palace_flag_given = bool(_args.palace), line 325)
def _resolve_kg_path() -> str:
    if _palace_flag_given:
        return os.path.join(_config.palace_path, "knowledge_graph.sqlite3")
    return DEFAULT_KG_PATH        # knowledge_graph.py:49 → ~/.mempalace/knowledge_graph.sqlite3
```

`cmd_serve` **always** passes `--palace`. So the moment you start the server, the KG becomes
`<palace>/knowledge_graph.sqlite3` — a *different file* from the live `~/.mempalace/knowledge_graph.sqlite3`.
The vector store looks fine and the knowledge graph is silently empty.

Four stores, **three** location rules:

| Store | Location rule |
| --- | --- |
| drawers | backend-abstracted (`BaseCollection`) |
| `knowledge_graph.sqlite3` | HOME-anchored **unless `--palace` flag** (`knowledge_graph.py:49`) |
| `hallways.json` | derived from `palace_path`, with legacy HOME fallback (`hallways.py:73,83`) |
| `known_entities.json` | HOME-anchored (`miner.py:701`) |

**Action:** `mv` all three files into the palace directory before first `serve`, and verify
`kg_stats` is non-empty afterwards. Corollary: **a shared pgvector/qdrant backend shares drawers and
nothing else** — it would leave the KG as fragmented as it is today. That is why the shared-backend
option is *not* the answer.

### 7.2 Never run `mempalace sync` against a shared palace

It classifies drawers whose source files are absent **on the running host** as orphans and deletes
them; `_auto_detect_project_roots` guesses roots from drawer metadata, so the blast radius is
data-dependent rather than obvious. From a laptop that lacks the repos, it is a fleet-wide wipe.
**Action:** edge blocks `mempalace_sync` from ever reaching the primary; document it; consider an
upstream `--refuse-shared` guard.

### 7.3 Provenance belongs to the sync boundary — not to the agent, and not to a solitary container

Today every drawer carries exactly `{wing, room, source_file, added_by, filed_at, id_recipe}`
(+`chunk_index`, `parent_drawer_id`); diaries add `{hall, topic, type, agent, date}`. `added_by`/`agent`
is the *agent* name (`pi`, `mcp`, `checkpoint`) — **never the machine**. So in a merged store you cannot
tell which host wrote a record, cannot audit, and cannot compute per-device high-water marks.

**Earlier drafts of this section said "stamp it everywhere, now, because it cannot be backfilled." That
was wrong on both counts.** See §7.3.3.

#### 7.3.1 What can actually be stamped (mechanics)

The metadata schema is **fixed** — there is no free-form field — and `tools/call` **whitelists arguments
to declared schema properties** (`mcp_server.py:4777`, *"Prevents callers from spoofing internal params
like added_by/source_file"*), so an extra `origin_host=…` is **silently dropped, not rejected**.

| Surface | Provenance slot | Notes |
| --- | --- | --- |
| `add_drawer`, `checkpoint` | **`added_by`** | The only one. Free-form (`strip_lone_surrogates` only, *not* `sanitize_name`, so `/` and `@` are legal). |
| `diary_write` | **none usable** | ⚠️ **Never** put a device in `agent_name`: `:3504` does `wing = f"wing_{agent_name}"` → a separate wing per host, and `diary_read` filters `{"agent": agent_name}` (`:3636`) → `diary_read("pi")` then **misses** those entries. |
| `kg_add` | **none** | Only `source_file`/`source_closet`/`source_drawer_id`. Origin is inferable only via `source_drawer_id` → that drawer's `added_by`. |

`added_by` is also **write-only today**: absent from `tool_search` results, surfacing only via
`get_drawer` → `_drawer_payload` metadata. → Upstream asks: **surface `added_by` in search results**, and
**give `kg_add` a provenance field**.

#### 7.3.2 Who should stamp it — a ladder of trust

Provenance answers "which device asserted this?" Only something that can *verify* the answer should
write it. Ranked by trustworthiness:

| Stamper | Knows the device? | Verifiable? | Uniform? | Verdict |
| --- | --- | --- | --- | --- |
| **Agent (via skill)** | No — must shell out to read env | No | No — per-call boilerplate, forgettable, improvisable | ❌ **Worst possible place.** Rejected. |
| **Client / `mempalace-edge`** | Yes, from host-supplied `.env` | No — self-asserted | Yes — one line in a proxy | ⚠️ Acceptable **interim** |
| **Primary, from the authenticated credential** | Yes | **Yes** — bound to the token | Yes, for every synced record | ✅ **Correct home** |

The decisive point: **a client-asserted origin is a hint, not a fact.** The primary is the only party
that can bind a write to an identity it verified. And under the §4 design *every* write reaches the
primary through an authenticated channel — including offline ones, at outbox-flush time — so the server
can stamp the complete set without any client cooperation. **Provenance is a property of the sync
channel, not of the record's author.**

Blocker for the ✅ row, verified in 3.6.0: `serve` takes a **single shared bearer token**
(`srv.auth_token`, `hmac.compare_digest`, `mcp_server.py:5291-5293`) and the package contains **zero**
occurrences of any device/origin concept. Server-side stamping therefore *requires* the per-device
credentials of §6 — i.e. **Phase 4**, not Phase 1. Hence the phasing:

- **Phase 2 (edge):** edge may stamp `added_by` from its configured env — self-asserted, **advisory
  only**, never load-bearing for authorization or destructive scoping.
- **Phase 4 (authz):** per-device tokens land; the primary stamps authoritatively and the client-supplied
  value becomes redundant (and must be treated as untrusted input, not merely ignored).

#### 7.3.3 Solitary containers should stamp nothing — and lose nothing by it

A pi-devbox or opencode-devbox running solitarily is a **single-origin store by definition**. Origin is
therefore a property of the *whole palace*, not of each record — so it can be assigned **wholesale at
the moment the store stops being solitary**: one `--origin-device` flag on the import/first-sync path
attributes every record from that palace to that device.

That dissolves the "impossible to backfill" argument. Per-record stamping is only necessary once records
from *multiple* origins are interleaved in one store, which is exactly and only the primary. So:

- **Solitary devboxes: no stamping, no config, no skill instruction, no behaviour change.** This is
  strictly more compliant with **R1** than the earlier draft, which quietly asked every solitary user to
  carry metadata for a feature they had opted out of.
- **Migration is unaffected**: bulk attribution at import is *more* reliable than per-record stamping,
  because it cannot be partially applied.
- **Multi-harness on one host stays solved** by `added_by` = agent name (`pi` vs `opencode`) — that is
  what the field is for, and it needs no device component.
- **A palace on a shared host bind-mount** (as tor-ms22's compose does) is still single-*device* under
  the "host owns the palace" model, so it too imports as one origin.

The one case bulk attribution cannot fix: a palace that was *already* merged from several devices without
stamps. Preventing that is precisely why the **primary** must stamp from day one of Phase 4 — it is the
only store where interleaving occurs.

#### 7.3.4 Identity fields, when a stamper does exist

For the edge (interim) and the primary (authoritative) — never for agents:

| Field | Source | Rule |
| --- | --- | --- |
| `origin_device` | `uuidgen` **once**, stored in the **host's `.env`**; later, issued with the device's token | Opaque, stable, collision-free by construction. Compared, never parsed. |
| `origin_label` | hostname, same `.env` | Human readability **only**. Never identity, uniqueness or scoping. Free to change. Optional — a hostname can leak an asset tag or username (`EMB-7KJ4VR4G`, `HOST_SSH_USER=ECSJPER`). |

Why not the obvious sources: **a container cannot discover its host's identity.** `hostname` returns the
*container ID* (`f3bf2a103473`) which changes on **every recreate**; there is no `/etc/machine-id`; and a
baked one would be *worse* — identical for every container from the same image, a guaranteed collision.
**Hostnames are also neither unique nor stable** (`localhost`, `ubuntu`, golden images, two
`MacBook-Pro.local`), and the worse failure is not collision but **rename**, which silently splits one
device's history in two. Hence host-supplied via `.env` (matching R2/R3 and the existing `HOST_SSH_USER`
/ `DEVBOX_HOST_ALIAS` precedent), *not* container-derived. Do **not** persist the id container-side:
`~/.mempalace/device_id` does not survive recreate for solitary users because `devbox-palace` is
commented out by default (§1.2). And don't write `${HOSTNAME}` in compose — bash *sets* but does not
*export* it, so interpolation sees empty.

Encoding, always three segments so arity is unambiguous:

```
added_by = "<agent>/<label>/<device>"     e.g.  pi/tor-ms22/7f3a9c2e1b04
                                                pi/-/7f3a9c2e1b04   (label withheld)
```

**The `/` is the discriminator.** A value with no `/` (`mcp`, `pi`, `checkpoint`) means *origin unknown* —
correct for the ~13.6k drawers already filed, and for every solitary palace forever.

#### 7.3.5 Fail-closed rule for any stamper

If `origin_device` is unset, blank or unreadable: **write no `added_by` override at all** and let the
default (`mcp` / `checkpoint` / agent name) stand. Never synthesize one. The tempting substitutes are all
actively harmful, because each is *confidently* wrong where absence is honestly unknown:

| Substitute | Damage |
| --- | --- |
| container hostname / ID | A **new fake device per recreate** — thousands of singleton identities, each looking legitimate |
| `unknown`, `localhost`, `devbox`, `docker`, `$USER` | **Collides across every machine** — indistinguishable from one real shared device |
| a guess from context | Unfalsifiable later |

If `origin_label` alone is missing, use the literal `-`. If `origin_device` is missing, do not stamp even
when the label is present — a hostname alone is exactly the colliding, mutable identity rejected above.

> **Note for future agents reading this RFC out of the palace:** these notes are themselves a poisoning
> vector. Do **not** start hand-stamping `added_by` because you read this section. Provenance is client
> and server infrastructure; an agent's contribution to it is to leave the field alone. The mempalace
> skill carries a one-line guard to that effect.

### 7.4 Embedder identity is client-local and currently toothless

`check_embedder_identity()` raises `DimensionMismatchError` / `EmbedderIdentityMismatchError`, but the
sidecar on this host reads `{"mempalace_drawers": {"model_name": "minilm", "dimension": 0}}` — and
`dimension: 0` means *unknown* and is **skipped**, so only the model *name* is compared. The sidecar
lives in the client's palace dir (`_sidecar.py`), so with a remote backend the check is per-client and
never centrally enforced. **Action:** the primary must reject writes from a client whose embedder
identity does not match, and merged reads (§4.2) must be disabled on mismatch — comparing distances
across models silently degrades recall.

### 7.5 Concurrency expectations

- `_HTTP_REQUEST_LOCK = threading.Lock()` wraps **every** dispatch (`mcp_server.py:5115`, held at
  `5346`/`5513`), commented: "*HTTP gives us a safer transport, not concurrent Chroma/HNSW mutation.*"
  The primary is a **single-writer, one-request-at-a-time** service. Size for a handful of agents.
- `mine_palace_lock` uses `fcntl.flock(LOCK_EX|LOCK_NB)` and exists because parallel HNSW inserts
  "*can corrupt the HNSW graph*" — but the lock **file** is HOME-derived (`~/.mempalace/locks/
  mine_palace_{sha256(realpath(palace))[:16]}.lock`) while the key is palace-derived. Two containers
  with different `~/.mempalace` mounts but the same palace path compute the same key on *different
  files* → **no mutual exclusion**.
- **Therefore: reject "put the palace on a NAS/SMB share and point everyone at it."** flock over
  NFS/SMB is unreliable, and the lock wouldn't be shared anyway. One process owning the files and
  serving HTTP is the supported model — which is exactly why `serve` exists.

---

## 8. Phasing

| Phase | Effort | Deliverable |
| --- | --- | --- |
| **0 — hygiene** | hours | §7 runbook: move KG/hallways/entities, ban `sync` on shared palaces, fix stale "unauthenticated" docs (incl. `pi-devbox/.env.example:21`). **No provenance work here** — it is not backfill-critical (§7.3.3) and belongs to the stamper, not the agent |
| **1 — primary up** | hours, **no code** | `mempalace serve --token --tls-cert` on a private-net host (reuse `docker-compose.mempalace.yml` — keep it a **separate standalone project**, R4 — and mind port 8765 vs pi-studio; tor-ms22 already moved to 8766). Repoint pi clients via `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN`. **opencode clients can be repointed in the same breath** — remote MCP is supported and `generate-config.py` already emits it (§9.1, §9.6), subject to the sidecar caveat in §4.1. **Shared memory today, no offline.** |
| **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. **Not** "fixes opencode" — opencode's *transport* is already fine after Phase 1; what edge adds there is offline/local-first, since `generate-config.py`'s switch is remote **or** local with no failover. **Ships with the §1.2 opt-in wiring (compose + `.env.example` + a third branch in the existing `generate-config.py`) and must pass the R1 acceptance test.** |
| **3 — pull replication** | ~1 week | **DEFERRED (2026-08-08).** Server-side op-log with monotonic seq → each edge a full offline replica. Only needed if a laptop must hold *everything* offline. Accepted trade-off: offline recall = own writes + last-reachable state. |
| **4 — authz** | days | Per-wing ACL, per-device scopes, audit log, token rotation |

Phase 1 is worth doing on its own — it is pure configuration and immediately ends KG fragmentation
for online clients.

---

## 9. Open questions

1. ~~**Does opencode's MCP config support a remote/HTTP transport at all?**~~ **RESOLVED 2026-08-09:
   YES.** opencode's published JSON Schema (`https://opencode.ai/config.json`) defines
   `$defs.McpRemoteConfig` = `{type:"remote" (enum), url (required), headers?: {string:string},
   oauth?, enabled?, timeout?}` as a sibling of `McpLocalConfig` inside
   `Config.properties.mcp.additionalProperties.anyOf`. So `headers` carries any bearer/API-key scheme,
   and set `oauth: false` to stop opencode probing the URL for OAuth. The earlier "every sampled entry
   is `type:local`" observation was **deployment evidence being read as schema evidence** — the schema
   settles it in one fetch. **Consequence: the edge proxy is *not* the only option for opencode.** A
   plain Phase 1 `mempalace serve --token` can be consumed directly, so nothing in the phasing depends
   on this question any more. What opencode still lacks is *local-first writes and offline fallback* —
   `generate-config.py`'s switch is remote **or** local per container, with no failover — and that is
   the real Phase 2 justification for it.
2. **Upstream or local?** `mempalace-edge` needs no core changes, so it belongs in this repo
   (`bin/` + `extensions/`). But `origin_device` stamping from the authenticated token, per-wing ACL, the `kg_supersede`
   classification fix, and a `sync --refuse-shared` guard all want to go **upstream**
   (`github.com/MemPalace/mempalace`).
3. **Chunked drawers under merge.** Oversized content splits into `{drawer_id}_chunk_NNNNNN` with
   `parent_drawer_id`, and the add-time idempotency probe checks only the *last* chunk id — a
   partially transferred chunk set may look "already present" and stay truncated. Unverified; test
   before trusting bulk replay.
4. **`migrate.py` as a bootstrap tool.** `extract_drawers_from_sqlite()` reads `{id, document,
   metadata}` straight out of chroma's SQLite (bypassing the chromadb API) and re-`add`s them with ids
   and metadata preserved and **embeddings recomputed** (`migrate.py:326`) — the right shape for a
   one-shot "seed the primary from an existing palace". Unverified: `col.add` behaviour on **id
   collision into a non-empty target**. Test before relying on it. (`exporter.py` is markdown-only,
   lossy, and has **zero callers** — not a transfer format. `backups.py` is retention pruning only.)
5. **Does `_HTTP_REQUEST_LOCK` stay held for the duration of an MCP-triggered `mine`?** Strongly
   suggested by the code shape; if yes, a remote mine makes the primary unusable for its duration
   (another argument for §5).
6. ~~**How does opencode-devbox learn the opt-in?**~~ **RESOLVED 2026-08-09: it already does.** The
   image ships `rootfs/usr/local/lib/opencode-devbox/generate-config.py`, invoked from
   `entrypoint-user.sh:117` on every start, which auto-registers the `mempalace` MCP server —
   remote+bearer when `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN` are set, local stdio when
   `mempalace-mcp` is on PATH, nothing otherwise — documented in that repo's `.env.example:38-49` and
   `.env.shared.example:30-35`. It also ships mempalace by default (see R5). So the config *is*
   templated at container start, not baked, and Phase 2 only adds a third branch. **Two residual
   constraints replace the original unknown, and both are documented in §4.1:** (a) the script never
   overwrites an existing config and `~/.config/opencode` is a *named volume*, so an env flip yields
   only an `opencode.jsonc.proposed` sidecar — adopting or reverting the opt-in needs a manual merge
   or a volume removal; (b) it no-ops entirely unless `OPENCODE_PROVIDER` is set. Prefer extending
   this script over inventing a parallel mechanism — treat it as the reference implementation, and
   check it before designing any pi-devbox-side mechanism with an opencode counterpart.

---

## 10. Evidence index

Re-verify without re-reading the package. Paths relative to
`/opt/uv-tools/mempalace/lib/python3.13/site-packages/mempalace/` (mempalace 3.6.0) unless noted.

| Claim | Where |
| --- | --- |
| Server auth/TLS/read-only | `cli.py:1448-1503`; `mcp_server.py:282-320`, `5205-5215`, `5284-5289` |
| Single global request lock | `mcp_server.py:5115`, `5346`, `5513` |
| KG path follows `--palace` flag only | `mcp_server.py:325`, `708-711`; `knowledge_graph.py:49` |
| KG DDL, no `(s,p,o)` uniqueness, open-fact guard | `knowledge_graph.py:163-178`, `305-313` |
| ID recipes | `ids.py:25`, `56`, `71`; diary at `mcp_server.py:3511-3514` |
| Tool classification (+ `kg_supersede` gap) | `service.py:29`, `54`, `72-84` |
| Write-routing seam | `write_routing.py:1-60` |
| Outbox prior art (queue DDL, token, retries) | `daemon.py` (`_init_db`, `ensure_token`, `MAX_ATTEMPTS`) |
| `sync` is destructive pruning | `sync.py:1-12` |
| WAL redacts + has no reader | `wal.py:74`, `_WAL_REDACT_KEYS` |
| Client-side embeddings | `backends/pgvector.py:6-8`; `embedding.py:6-11`; `_sidecar.py` |
| Dedup scoped by `source_file` | `dedup.py:get_source_groups` |
| Store location rules | `hallways.py:73,83`; `miner.py:701` |
| Mine lock (HOME-derived file) | `palace.py:1090`, `1123-1128` |
| Migrate as export primitive | `migrate.py:extract_drawers_from_sqlite`, `:326` |
| pi remote transport + fail-soft | `mempalace-toolkit/extensions/pi/mempalace.ts:11-12`, `87`, `108`, `390`, `629-641`, `665-673` (commit `96699f2`) |
| Server compose + port history | `pi-devbox/docker-compose.mempalace.yml`; `pi-devbox/CHANGELOG.md` v1.3.0; `docker-compose-repo/tor-ms22/pi-devbox/` (`df2c2ae`) |
| Existing opt-in convention (§1.2) | `pi-devbox/.env.example:12-23` ("MemPalace memory (local by default)", commented `MEMPALACE_REMOTE_URL`/`_TOKEN`); `pi-devbox/docker-compose.yml:79-83` (`devbox-palace` volume commented out) |
| opencode-devbox does not wire mempalace | `docker-compose-repo/{synlig,nyvaken,devbox-affection}/opencode-devbox/docker-compose.yml` — zero mempalace references in any |

## 11. See also

- [`ARCHITECTURE.md`](../ARCHITECTURE.md) — producer side (how the palace gets fed); §6 upstream roadmap
- [`extensions/pi/README.md`](../extensions/pi/README.md) — pi bridge internals
- [`SKILL.md`](../SKILL.md) — consumer-side protocol (search before answering, diary before exit)