joakimp/mempalace-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e68ee2071ca3ad396e164201181afcac65091992
mempalace-toolkit/docs/rfc-001-global-palace.md
T
pi 553d86570c provenance: stamp device+harness at the edge, not in the agent's head 
RFC 001 §7.3.2 ranks "agent stamps provenance via a skill instruction" as the
❌ worst possible place — per-call boilerplate, forgettable, improvisable. It
was right, and we had shipped exactly that: the mempalace skill told the agent
to pass added_by="<harness>@<device>" by hand. Measured on the shared palace,
199 rows had reached it unresolvable, 10 of them filed by the very agent that
wrote the instruction, in a drawer about host provenance. The trigger was a
cross-host misattribution: a session on tor-ms22 read its own diary, could not
tell that the entries were written on EMB-7KJ4VR4G, and reported another
machine's verification as this one's.

Move the same convention into the ⚠️ edge row, where it is uniform and
unforgettable (§7.3.5):

* extensions/pi/mempalace.ts defaults the writer field on every tool that has
  one — added_by (add_drawer, checkpoint), agent (mine), from_agent
  (event_append), created_by (artifact_put) — from $MEMPALACE_PI_DEVICE. An
  explicit value always wins, so filing for another device stays possible. The
  allowlist is per tool, never blanket: 3.8.0's dispatcher hard-rejects
  undeclared args with -32602, so injecting added_by into diary_write or kg_add
  (which have no such property) would break the call outright.
* mine gets miner@<device> when the caller invokes it, but <harness>@<device>
  for the bridge's own transcript feed — bulk extraction is not agent-authored
  memory, and that keeps the pi/opencode/miner taxonomy honest.
* diary_write has no metadata slot at all, and the device must never go in
  agent_name (wing = f"wing_{agent_name}" would splinter the diary per host).
  So the entry TEXT carries an AAAK field, HOST:<device>|SESSION:… — which is
  also the only channel a READER sees: search projects a fixed key set and
  diary_read returns content, so no metadata fix, not even a
  server-authoritative one, would have prevented the misattribution.
* The wake-up block now states the device and warns that diary_read interleaves
  every machine's diary.
* R1: doubly gated on MEMPALACE_PI_DEVICE and MEMPALACE_REMOTE_URL, so a
  solitary devbox stamps nothing and behaves exactly as before — which is also
  the correct semantics per §7.3.3.

Version the reconciler that was living only on synlig (bin/ + contrib/systemd/),
add --dry-run, and teach it two new rules: diary_host_marker reads the HOST:
field, and sibling_chunk propagates a resolved origin across a drawer's chunks
(a text marker lands in chunk 0 only, so a 5-chunk diary entry would otherwise
stamp 1 and leave 4 blank).

--dry-run against the real palace before deploying earned its keep twice, and
scripts/test-device-stamp.sh pins both findings with the strings it found:
HOST: was ALREADY in use with a composite grammar
(HOST:emb-7kj4vr4g.f1d3c3f89e3e.v1.8.3.pi0.84.2) and for bare container ids, so
an unvalidated rule invented devices like "f1d3c3f89e3e.pi0.84.2"; and HOST:
also carries a different SENSE elsewhere (HOST:exec.via.ssh-controlmaster->…,
meaning where I was executing). Validating against the known-device set both
refuses those and recovers the composite entries correctly. A marker convention
inherits every prior meaning of its own name.

Deployed and verified on synlig: device 14,217 → 14,317, integrity ok,
idempotent on immediate re-run, no invented device values.

RFC updates: §7.3.1 corrected (the arg whitelist is a hard -32602 in 3.8.0, not
a silent drop; get_drawer DOES return metadata, search structurally cannot;
triples and logstream live in separate databases the stamper cannot reach),
§7.3.5 added (what is deployed, including the divergence from §7.3.4's opaque
origin_device — tor-ms22 vs tor-ms22-native is that cost already visible), and
Phase 4 now carries per-device tokens motivated FIRST by revocation, with the
finding that tokens are the cheap half: core holds one scalar auth_token and has
zero device concept, so authoritative stamping needs a component we own.
2026-08-25 22:26:46 +02:00

1094 lines
90 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# RFC 001 — Global palace with local fallback (`mempalace-edge`)
| | |
| --- | --- |
| **Status** | **Phases 0–1 implemented and verified — 2026-08-10 / 08-12 / 08-14.** Primary live + seeded, first client flipped. Phases 1.5, 2 and 4 outstanding; Phase 3 deferred by decision. |
| **Created** | 2026-08-08 |
| **Applies to** | mempalace 3.6.0, mempalace-toolkit @ `96699f2`, pi-devbox ≥ v1.3.0 |
| **Decision** | Phases 0–2 + 4 in scope. **Phase 3 (full pull replication) explicitly deferred** — "a laptop that can reach its own stuff plus whatever it can reach" is good enough. **Centralization is strictly opt-in — solitary devbox operation remains the default and must not change (§1.1).** |
| **Recon update** | **2026-08-09 — §9 Q1 and Q6 are RESOLVED, both in the permissive direction** (opencode supports remote MCP; opencode-devbox already templates the mempalace entry). Neither is a blocker. See those entries for evidence; §2, §4.1 and R5 were corrected accordingly. |
| **Rollout update** | **2026-08-14 — Phase 1 is live.** Primary serving at `https://mempalace.jordbo.se/mcp` since 2026-08-12 (synlig, `mempalace-serve.service` under `systemctl --user`, palace `/home/ecsjper/.mempalace/palace`); **seeded 2026-08-14 15:07** from EMB-7KJ4VR4G's palace — which is itself a carry-over copy from the
operator's previous work computer **EMB-X1JY06WJ** (lineage and evidence in the §4.4 Deviation note) —
14,777 drawers / 9 wings / 16,337 embeddings / KG 46 entities, 34 triples (14,803 drawers by 17:00); **first client flipped and verified end-to-end** the same afternoon. ⚠️ Two things not to misread: the seed was a **file-level copy of one palace**, *not* the §4.4 MCP replay (see the Deviation note in §4.4), and it therefore **sidestepped §7.6 rather than resolving it — §7.6 remains a hard blocker for the *second* joiner (§8 Phase 0).** §2's and §4.1's predictions about the pi client were confirmed in production; §4.4's count-based verification advice was **wrong** and has been corrected. |
**Read this first if you are asked to "centralize MemPalace" / "sync palaces between machines".** Most of the
hard-won facts below are non-obvious and two of them are actively destructive if you guess wrong
(§7.1, §7.2). The evidence index in §10 lets you re-verify any claim without re-reading 45k lines.
---
## 1. The problem
One palace per machine per harness. Today: a pi-devbox container on `EMB-7KJ4VR4G`, another on
`tor-ms22`, opencode-devbox containers, `MBP-M1-2020`, plus native hosts — each with a private
`~/.mempalace`. Consequences:
1. **Memory is sharded by accident of where you happened to be working.** A decision recorded on the
laptop is invisible to the agent on the desktop.
2. **Container recreates are amnesia events** unless the palace happens to be host-bind-mounted.
3. **The KG is the worst hit** — `kg_query`/`kg_timeline` answers depend on which machine you ask.
**Target state:** one primary palace holds the fleet's memory. Every client keeps working when the
primary is unreachable (writes buffer locally, reads degrade to local), and reconciles when it
returns. Access is authenticated per device, with a sane authorization policy.
### 1.1 Hard requirement: solitary-first, centralization strictly opt-in
**This is a requirement, not a preference, and it constrains every choice below.**
Centralization solves a problem specific to *this* usage pattern: several machines and containers
(pi-devbox, opencode-devbox, native hosts) doing development and other work against a **common set of
artifacts**. For most users of the published `joakimp/pi-devbox` and `joakimp/opencode-devbox` images
it is **useless** — one machine, one palace, done. Evidence that this is already the norm: all three
sampled `opencode-devbox` deployments (`docker-compose-repo/{synlig,nyvaken,devbox-affection}/`)
contain **zero** mempalace references.
| ID | Requirement |
| --- | --- |
| **R1** | **Solitary operation stays the default and stays unchanged.** With no `MEMPALACE_*` variables set, a devbox must behave exactly as it does today: harness → local `mempalace-mcp` over stdio, palace at `~/.mempalace`. No extra process, no outbox, no network calls, no new failure mode, no measurable startup cost. |
| **R2** | **Opt-in lives in `docker-compose.yml` + `.env`** — the mechanism users already know. No opt-in via image rebuild, no baked-in defaults, no `latest-central` variant. |
| **R3** | **Credentials only in `.env`** (`chmod 600`, gitignored). Never in `docker-compose.yml`, never in the image, never on a command line (visible in `ps`), never logged. Compose passes them through with a `${VAR:-}` empty default so solitary users never define them. |
| **R4** | **No new required services.** The primary stays in the separate standalone `docker-compose.mempalace.yml` project; it is never merged into the main compose file. A solitary `docker compose up` starts exactly what it starts today. |
| **R5** | **Degrade, never fail, when mempalace is absent.** The published `opencode-devbox` image *does* ship mempalace by default (`Dockerfile.base:380,402` — `ARG INSTALL_MEMPALACE=true`, `MEMPALACE_VERSION=3.6.0`, installed via `uv tool install`), but it is a build arg precisely so it can be omitted to save ~300 MB — and a bind-mounted host config or a non-devbox client may have no mempalace either. So any wiring must stay additive and skip when the binary is missing — the probe-and-warn idiom `install.sh` already uses (`warn` + `return 0`, never halt). |
| **R6** | **Reversible.** Commenting the `.env` lines out returns the container to pure solitary operation, with the local palace intact and readable. |
**Acceptance test for R1** (must pass before Phase 2 ships): bring up a devbox with no `MEMPALACE_*`
variables; assert the palace tool list, drawer counts and diary writes are identical to the previous
image, and that no edge/outbox process exists (`pgrep -f mempalace-edge` → empty).
### 1.2 The opt-in surface
The existing convention is already the right one — **extend it, do not invent a new one.** pi-devbox's
`.env.example` lines 12–23 already ship a commented `MEMPALACE_REMOTE_URL` / `MEMPALACE_REMOTE_TOKEN`
pair under the heading "MemPalace memory (local by default)", and `docker-compose.yml:79-83` already
ships the `devbox-palace` volume commented out. So the opt-in ladder becomes three states derived
from two variables — **both existing behaviours are preserved, the third is new**:
| `MEMPALACE_REMOTE_URL` | `MEMPALACE_EDGE` | Behaviour | Status |
| --- | --- | --- | --- |
| unset | — | local stdio `mempalace-mcp`, palace at `~/.mempalace` | **default, today, unchanged (R1)** |
| set | unset/`0` | direct remote HTTP; no local palace, no offline | today (`96699f2`), unchanged |
| set | `1` | `mempalace-edge`: local-first writes + outbox → primary + merged reads | **new (Phase 2)** |
> **Volume coupling reverses under edge mode — document it prominently.** Today `.env.example` correctly
> says that with `MEMPALACE_REMOTE_URL` set "*the devbox-palace volume is then irrelevant*", because the
> remote owns all state. Under **edge** mode that flips: the local palace holds the outbox and all
> local-first writes, so an un-persisted palace means **losing un-flushed writes on container
> recreate**. Opting into `MEMPALACE_EDGE=1` therefore *requires* uncommenting
> `devbox-palace:/home/developer/.mempalace`.
Corollary for §4.1: **`mempalace-edge` must not be in the path unless opted in.** Registering it
unconditionally (letting it decide by env at runtime) is tempting — one static config for every
harness — but it violates R1 by inserting a process and a failure mode into every solitary user's
setup. Selection must happen at registration time. See open question §9.6.
---
## 2. Verified starting point (as of 2026-08-08)
Two of the three pieces already exist. This is not greenfield.
| Piece | State | Evidence |
| --- | --- | --- |
| **Primary server** | ✅ **Exists.** `mempalace serve --host --port --token --tls-cert --tls-key --read-only --allow-insecure`. Bearer token compared with `hmac.compare_digest`, **mandatory** on non-loopback binds (unless `--allow-insecure`), TLS 1.2+ resolved *before* bind, Host-header pinning + `Origin` allowlist (anti-DNS-rebinding), 16 MiB body cap, token-free `/healthz`. | `cli.py:cmd_serve` (~1448); `mcp_server.py:5205-5215`, `5284-5289` |
| **Remote client** | ⚠️ **Exists for pi only, and it is either/or.** `createClient()` picks stdio *or* HTTP once at process start. ✅ **Confirmed in production 2026-08-14** on the first flipped client: the branch is a pure transport swap, and in remote mode **no local `mempalace-mcp` child is spawned at all** (zero mempalace processes in the flipped container). So **writes go only to the primary — no dual-write, no local mirror.** | `extensions/pi/mempalace.ts:629-641` |
| **Fallback + resync** | ❌ **Absent everywhere.** On remote failure the pi bridge re-handshakes the same URL, then **de-registers all palace tools** and runs blind. ✅ **Confirmed 2026-08-14 — and the precise word is fail-*closed*, not fail-local.** After bounded retries it prints `mempalace-mcp unavailable after retries; continuing without palace tools` and registers nothing; it never silently falls back to the local palace, so a write cannot land in the wrong store. Operational corollary worth stating once: **"the palace tools vanished" is the expected symptom of a server / token / DNS fault**, not of a broken client. | `extensions/pi/mempalace.ts:665-673` |
| **`mempalace` CLI** | ❌ **No remote support whatsoever.** Its only palace selector is `--palace <path>`; otherwise it resolves `palace_path` from the local `config.json`. **Verified 2026-08-14: on a flipped client the CLI still reads and writes the now-dead local archive — and cheerfully reports ~14,777 drawers while doing so.** It is a false-positive machine: never verify a flip with `mempalace status`/`search`, and never file memories with the CLI post-flip (they land in the archive, not the primary). Corollary: a **transcript backfill must be mined on the primary host**, where the CLI's local palace *is* the primary. | `cli.py` argument surface; verified on EMB-7KJ4VR4G |
| **opencode client** | ✅ **Remote is supported and already wired.** opencode's published schema (`https://opencode.ai/config.json`, `$defs.McpRemoteConfig`) makes `{"type":"remote","url","headers","oauth"}` a first-class sibling of `McpLocalConfig`, `headers` being a free string→string map (so bearer is a convention, not a constraint). `opencode-devbox` already emits exactly that entry when `MEMPALACE_REMOTE_URL` is set. The all-`type:local` configs in `myconfigs` are a *deployment* fact, not a capability limit. | `generate-config.py:107-118`; hook at `entrypoint-user.sh:117`; schema `$defs.McpRemoteConfig` |
| **Server compose** | ✅ Exists: `pi-devbox/docker-compose.mempalace.yml` (canonical) + a tor-ms22 derivative (`docker-compose-repo/tor-ms22/pi-devbox/`, `df2c2ae`, port 8766, uid 1000, binds the real `~/.mempalace`). Not enabled. | pi-devbox CHANGELOG v1.3.0 (2026-07-02) |
> **Stale docs warning.** `docker-compose.mempalace.yml`, `.env.example` and CHANGELOG v1.3.0 all say the
> HTTP transport is unauthenticated and should be fronted by a reverse proxy. That was true for
> `mempalace-mcp --transport http` in the v1.3.0 era. **mempalace 3.6.0's `serve` has token + TLS
> built in** (upstream #1877). Fix those comments during Phase 1. Specifically:
> `pi-devbox/.env.example:21` still advertises `mempalace-mcp --transport http --host 0.0.0.0 --port
> 8765` as the way to serve a shared palace — replace with `mempalace serve --token … --tls-cert …`,
> and change the example URL from `http://mempalace.lan:8765/mcp` to `https://`.
>
> **✅ Done 2026-08-12** — `pi-devbox/.env.example` and
> `mempalace-toolkit/extensions/pi/README.md` both now recommend `mempalace serve`, bind docker0 rather
> than `0.0.0.0`/loopback, and state that the transport *is*
> authenticated. `docker-compose.mempalace.yml` audited too, and it was worse than stale — it was
> **broken on 3.6.0 in both directions**: `--host 0.0.0.0` with no token in the environment makes the
> server refuse to start (crash-looping under `restart: unless-stopped`), and once a token *is* supplied
> the healthcheck's unauthenticated `tools/list` POST 401s, marking a healthy server unhealthy forever.
> Fixed: token now required via `${MEMPALACE_REMOTE_TOKEN:?}` (fails fast at `up`), healthcheck switched
> to the token-free `/healthz`.
>
> ⚠️ **Corrected 2026-08-14 — one sub-claim above was wrong for two days.** The `https://` example URL
> landed in `pi-devbox/.env.example` only; `extensions/pi/README.md` still carried
> `http://mempalace.lan:8765/mcp` until it was fixed on **2026-08-14**. Everything else in this block
> checked out. Worth naming as a pattern, because it is cheap to repeat: **a doc's own ✅ is not evidence
> the work landed — verify it per sub-claim and per file.**
### Two things that sound like the feature and are not
- **`sync.py` is not replication.** It is gitignore-aware drawer *deletion*: "*Removes drawers whose
source files are now gitignored, deleted, or moved out of the project*" (`sync.py:1-12`). See §7.2 —
running it against a shared palace is a fleet-wide memory wipe.
- **`wal/write_log.jsonl` is not a replayable WAL.** `_WAL_REDACT_KEYS` strips
`content`/`content_preview`/`document`/`entry`/`entry_preview`/`query`/`text` and replaces them with
`"[REDACTED N chars]"`; all 12 references are writers, there is **no reader** and no `replay`
function (`wal.py:74`). Reconstructing memory from it is information-theoretically impossible.
---
## 3. Why we sync operations, not databases
Row-level / file-level replication of a palace is a trap in this codebase. Four findings, each
independently disqualifying:
1. **Vectors are not portable.** Embeddings are computed **client-side** — `backends/pgvector.py:6-8`:
"*Embeddings are still produced locally by MemPalace through the core embedding wrapper before
vectors are written to Postgres.*" Cross-architecture ONNX determinism (arm64 macOS vs x86-64
Linux, different execution providers) is not guaranteed. → **Ship text + metadata, always re-embed
on receipt.** Cheap and safe: the raw text is always stored as the chroma document.
⚠️ **Not absolute — see the 2026-08-14 deviation in §4.4.** A whole-palace file-level copy arm64 macOS →
x86-64 Linux *did* preserve working search and an intact HNSW index. Re-embedding on receipt remains the
rule for **merging** operations into an existing store; it is not a prohibition on cloning one palace
wholesale, provided the embedder sidecar travels with it (§7.4).
2. **KG rows cannot be copied.** `triples` has **no** `UNIQUE(subject,predicate,object,valid_from)` —
only `id` is unique, and `make_triple_id` embeds `datetime.now()`. A row copy therefore duplicates
every fact. But `add_triple()` guards at the application level (`SELECT id … WHERE subject=? AND
predicate=? AND object=? AND valid_to IS NULL` → returns the existing id), so **replaying `kg_add`
is idempotent for open facts** (`knowledge_graph.py:163-178`, `305-313`).
⚠️ **Scoped to open facts only** (verified 2026-08-09). The guard's `WHERE … valid_to IS NULL` means an
already-**closed** historical fact — one written with `valid_to`, or closed later by
`kg_invalidate`/`kg_supersede` — has no guard at all, so replaying it inserts a duplicate row every
time. A bootstrap that replays full KG *history* rather than just currently-open facts must dedupe
closed facts client-side on `(s,p,o,valid_from,valid_to)` (§4.4).
3. **Only one write path has deterministic IDs.**
| Write path | ID recipe | Same content on 2 hosts → same ID? |
| --- | --- | --- |
| `add_drawer` / `checkpoint` | `drawer_{wing}_{room}_{sha256(wing\|room\|content)[:24]}` | **YES** — content-addressed, merges for free |
| project/format miner | `…sha256(source_file\|chunk_index)` | **NO** — absolute path (`/Users/joakim/x` vs `/workspace/x`) |
| convo miner | `…sha256(source_file\|extract_mode\|chunk_index)` | **NO** — same reason |
| diary | `diary_{wing}_{YYYYmmdd_HHMMSSffffff}_{sha256(entry)[:12]}` | **NO** (µs timestamp) — and the write is a bare `col.add` with **no pre-write probe at all** (`mcp_server.py:3546`), unlike `add_drawer`. The 12-hex suffix is the only usable content dedup key, and it must be applied client-side (§7.6) |
| KG triple | `t_{s}_{p}_{o}_{sha256(valid_from\|recorded_at)[:12]}` | **NO** (`recorded_at = now()`) |
(`ids.py:56,71` — `ID_RECIPE = "v3"`, length-prefixed delimited hashing.)
4. **Batch dedup will not save a naive merge.** `dedup.py` groups by `source_file` and only compares
*within* a group (cosine < 0.15) → cross-host duplicates with differing paths are never compared.
The mechanism that *does* work is write-time: `tool_check_duplicate(content, threshold=0.9)` queries
the whole collection, and `mempalace_checkpoint` already runs it per item.
**Conclusion:** the MCP tool surface is already a small, coarse-grained, mostly-idempotent operation
vocabulary. Log the *intent*, replay the *intent*. That sidesteps chroma internals, chunking,
embedder drift and vector portability in one move.
**One exception, and it is not cosmetic:** `diary_write` has no idempotency guard whatsoever, so for
diaries "replay the intent" *duplicates* rather than merges (§7.6). Every other write path either
content-addresses or guards. §4.4 carries the per-record-type dedup keys a joining client must
therefore bring with it.
---
## 4. Design: `mempalace-edge`, a local MCP proxy
### 4.1 Shape
Do **not** put fallback logic in `extensions/pi/mempalace.ts` — opencode would need it again — not
because opencode lacks remote MCP (it has it, §9.1), but because *offline-first writes* are a client
concern that every harness would otherwise reimplement. Instead: a sidecar that *is* an MCP server.
```
pi (mempalace.ts, stdio) ─┐
opencode (mcp: type=local) ─┼──► mempalace-edge ──HTTPS MCP──► PRIMARY
mempalace CLI (local palace) ─┘ │ (stdio MCP server) mempalace serve
├─ child: mempalace-mcp --token --tls-cert
│ (local palace, always writable)
└─ outbox.sqlite (durable op queue)
```
**`mempalace-edge` needs zero mempalace internals** — it is an MCP-to-MCP proxy. It speaks stdio down
to a child `mempalace-mcp` and HTTPS up to the primary, reusing the `RemoteMcpClient` already written
in `96699f2` (`extensions/pi/mempalace.ts:390`, vendored from `pi-extensions/mcp-loader.ts` — mind the
`MCP-STREAMABLE-HTTP-CLIENT-SYNC: v1` drift token if you copy it again).
Why this shape wins:
- **One implementation for every harness.** opencode's change is one line:
`"command": ["mempalace-edge"]`. pi's is one line. The CLI is untouched.
- **Tools never vanish** from the tool list, so wake-up context injection keeps working offline —
unlike today's fail-soft de-registration.
- **Survives mempalace upgrades**: coupled to the tool schema, not to chroma/HNSW.
- The remote transport is **sessionless JSON-RPC** today (per `96699f2`'s own note), so reconnect
after an outage is cheap — there is no session to re-establish.
**But per R1, that one-line change is conditional, not baked in.** The edge binary is present in the
image (it costs nothing unused) and inserted into the path only when `MEMPALACE_EDGE=1`:
- **pi**: `createClient()` (`extensions/pi/mempalace.ts:629`) already branches on env at startup — add a
third branch. Zero change to the default path.
- **opencode**: the MCP server entry is static JSON — but **the templating step already exists and
already implements the first two rungs of the §1.2 ladder.**
`rootfs/usr/local/lib/opencode-devbox/generate-config.py` (run unconditionally from
`entrypoint-user.sh:117`) registers `mempalace` as `{"type":"remote", url, headers:{Authorization:
Bearer …}}` when `MEMPALACE_REMOTE_URL` is set, else `{"type":"local","command":["mempalace-mcp"]}`
when the binary is on PATH, else nothing — R5-compliant already. Its own comment states it uses the
"*same env contract as the mempalace.ts pi extension … so one shared MemPalace can serve pi +
opencode + native*", i.e. the two images are deliberately kept in step. **Phase 2's opencode work is
therefore a third branch in an existing script, not a new mechanism.**
> ⚠️ **But the opt-in does not propagate to an existing container.** `generate-config.py` *never*
> overwrites an existing config, and `~/.config/opencode` is the named volume
> `devbox-opencode-config` (`docker-compose.yml:64,153`) — so a config generated during solitary use
> survives recreate, and later setting `MEMPALACE_REMOTE_URL`/`MEMPALACE_EDGE` only produces a
> non-loaded `opencode.jsonc.proposed` sidecar for manual merge (`generate-config.py:203-244`).
> Flipping the `.env` alone is a no-op there. Phase 1/2 docs must say: merge the sidecar, or
> `docker volume rm` the config volume, to adopt the change — and the same applies in reverse for R6
> (reversibility). This asymmetry with pi — whose `createClient()` re-reads env every start — is the
> single biggest behavioural difference between the two harnesses under this RFC.
> ✅ **Fix shape (decided 2026-08-09): make the `mcp.mempalace` subtree env-authoritative.** Four
> verified inputs:
> 1. **pi is immune for a *structural* reason worth naming:** its MemPalace wiring is **code, not
> config** — `createClient()` reads env at every startup, with no generated file in the path. pi's
> own `settings.json` *does* sit on a preserved volume and *does* go stale, and pi-devbox already
> solved that properly at `pi-devbox/entrypoint-user.sh:131-162`: `jq -s '.[0] * .[1]'` deep-merge
> with **template first, live second** so the user's values always win and only *missing* keys are
> filled; arrays treated as leaves (a deliberately removed model is not re-added); rewrite only when
> the merge changes something; timestamped `.bak` first; `PI_SETTINGS_MERGE=0` to disable; invalid
> JSON on either side → skip, never clobber. **That is the pattern to port.**
> 2. **The precedence must be inverted for this one subtree.** pi's "live wins" is right for *adding*
> new keys and wrong for a *changed* env value — and a changed `MEMPALACE_REMOTE_URL` is the whole
> problem. So `mcp.mempalace` needs env-wins, which is only safe with a **fingerprint**: store a
> hash of what was last auto-generated and refresh only while the live value still matches it;
> otherwise fall back to the `.proposed` sidecar for that key. `write_proposed`
> (`generate-config.py:203-244`) already diffs rendered config against the live file, so this is an
> extension of existing logic at narrower granularity, not new machinery.
> 3. **No higher-precedence layer exists to hide in.** There is no `OPENCODE_CONFIG*` env override in
> opencode's published schema (or anywhere in `opencode-devbox`/`myconfigs`), and MCP registration
> is global rather than project-scoped, so the real file must be written.
> 4. **The fingerprint is the one genuinely new mechanism** — neither repo has a managed-marker
> convention for JSON; the only prior art is the markdown `<!-- pi-devbox:managed-block -->` idiom.
>
> This is a **self-contained opencode-devbox change, independent of the rest of this RFC**, and it is
> what turns Phase 1 into "flip `.env`, restart" for *both* harnesses — hence Phase 1.5 in §8.
### 4.2 Routing policy
`service.py:29,54,72` already ships the exact three-way split we need — `READ_TOOLS`,
`WRITE_TOOLS`, `MAINTENANCE_TOOLS` (`{mine, sync, reconnect}`) — plus `classify_tool()`.
| Class | Primary up | Primary down |
| --- | --- | --- |
| **read** (`search`, `kg_query`, `diary_read`, `list_*`, `traverse`…) | query **both**, merge ranked lists, dedupe by drawer id | local only, response flagged `degraded: true` |
| **write** (`add_drawer`, `checkpoint`, `diary_write`, `kg_add`…) | apply local **and** enqueue → primary | apply local, enqueue, keep working |
| **maintenance** (`mine`, `sync`, `reconnect`) | **never proxied** — local only (§5, §7.2) | local only |
| **destructive** (`delete_drawer`, `delete_by_source`, `delete_tunnel`…) | admin-scoped only; tombstone, never hard-delete remotely | local only |
> ⚠️ **`classify_tool()` must be treated as fail-closed.** `mempalace_kg_supersede` is a real MCP tool
> (3 references in `mcp_server.py`) but is **absent from `WRITE_TOOLS`** → `classify_tool()` returns
> `"unknown"` for it. An edge proxy that routed "unknown" as read would silently drop supersede
> operations. Maintain our own tool→class table, default unknown ⇒ **write**, and file the upstream fix.
**The merged-read trick is what makes Phase 3 optional.** Both palaces return ranked results with
distances; if the embedder identity matches (§7.4) the distances are comparable, so a union +
re-sort + dedupe-by-id gives a correct combined result set. You get fleet-wide recall *without*
replicating anything. Offline you simply see less.
KG reads are the exception: unioning triples is unsafe because `invalidate`/`supersede` are
order-dependent. Use **primary-first, local-fallback** (no union) for `kg_*` reads, and accept that an
offline KG answer is incomplete.
### 4.3 Outbox
Model it on `daemon.py`, which already implements a durable local job queue and is the closest
existing prior art (token auth via `ensure_token`, 0600, per-palace state dir, `MAX_ATTEMPTS = 3`,
`recover_running`, `JOB_RETENTION_DAYS = 7`). Its key primitive is worth copying verbatim:
```sql
CREATE UNIQUE INDEX IF NOT EXISTS idx_jobs_dedupe_active
ON jobs(dedupe_key) WHERE state IN ('queued', 'running');
```
A TOCTOU-safe, cross-process "at most one active job per key". Our outbox rows:
| Column | Purpose |
| --- | --- |
| `op_id` | `sha256(origin_device, local_seq, tool_name, canonical_payload)` — stable across retries, the primary's idempotency key |
| `local_seq` | Per-device monotonic counter → **causal order** for `kg_invalidate`/`supersede` replay |
| `origin_device`, `origin_label` | Provenance, stamped by edge (interim) or the primary (authoritative) — never by the agent (§7.3) |
| `tool_name`, `payload_json` | The MCP call to replay |
| `state`, `attempts`, `created_at`, `flushed_at`, `error_json` | Retry/audit |
Flush = drain in `local_seq` order, stop on first hard failure (preserve ordering), exponential
backoff, resume on reconnect. The primary de-dupes on `op_id`; `add_drawer`'s own pre-write probe and
`add_triple`'s open-fact guard make replay safe even if `op_id` tracking is lost.
`write_routing.py` is the upstream-shaped seam for this if we ever want it in core: its
`WriteRoutingPolicy{DIRECT,PREFER,REQUIRE}` → `WriteRoutingTarget{DIRECT,DAEMON,BLOCKED}` enums would
gain a `REMOTE` target. The module says it "*changes no caller defaults by itself*" — i.e. it exists
precisely to be extended.
### 4.4 Joining an existing primary (bootstrap)
**Framing correction (2026-08-09).** This is *not* "seed the primary from one chosen palace". Every
container joins the same way, at any time, repeatedly — so a join is **idempotent replay of local
history**, and the only real question per record type is *what dedupes it*.
> ⚠️ **Deviation (2026-08-14) — the first join did not follow this section.** It was precisely the thing
> the framing correction says this is *not*: a **file-level copy of one chosen palace**, arm64 macOS →
> x86-64 Linux, bypassing MCP replay entirely. It worked — search verified live on the primary, HNSW
> index intact, KG intact (46 entities / 34 triples). Recorded here so the next operator neither repeats
> it blind nor believes it is forbidden. **Four conditions made it safe, and all four must hold:**
>
> 1. **Same mempalace version (3.6.0) at both ends**, so the on-disk chroma/HNSW layout matched.
> 2. **`mempalace_embedder.json` travelled with the palace**, so embedder identity matched. This is the
> load-bearing one: a mismatched embedding model does **not** raise — it silently returns garbage
> search results, and §7.4 explains why the guard cannot catch it (only the model *name* is compared,
> and `dimension: 0` is skipped as unknown).
> 3. **A single source palace, so no merge semantics were exercised at all.** This is exactly what makes
> the method inapplicable to the second joiner.
> 4. **Python's `sqlite3` online-backup API** (`src.backup(dst)`) for the two sqlite DBs — WAL-safe with
> a live writer, and the only option available because **neither host has the `sqlite3` CLI**. The
> rest of `palace/` was `rsync`ed with `--exclude 'chroma.sqlite3*'`.
>
> **This sidesteps the merge problem; it does not solve it.** The second machine to join still needs the
> replay path and the dedupe keys tabulated below, and still needs §7.6 settled. Three traps found while
> doing it: **never `rsync --delete` into `~/.mempalace`** — the server's bearer token lives *inside* that
> tree at `~/.mempalace/server/<hash>/token` (§10) and there is no second copy; **palace paths cannot
> move**, because the palace directory name is a sha256 prefix of its own path; and compare sizes with
> **`stat -c%s`, not `du`** (APFS and ext4 disagree on block accounting, so `du` shows a spurious delta).
> Quiesce the writer first — the source palace's own MCP server was live throughout, which is how a write
> landed on the far side of the snapshot boundary and produced the false alarm described below.
>
> **Lineage — this was the *second* file-level copy, not the first (established 2026-08-15).** The
> "chosen palace" on EMB-7KJ4VR4G was itself carried over from **EMB-X1JY06WJ**, the operator's previous
> work computer, around 2026-07-06 when that machine was replaced. Evidence in the archive: **76 drawers
> tagged `source_machine=EMB-X1JY06WJ`**, and they are the oldest in it — earliest `filed_at`
> 2026-05-04, two months before the EMB-7KJ4VR4G stack existed — with no other `source_machine` value
> present; the remaining ~16k were filed locally afterwards (15,189 in July, 1,073 in August). So the
> primary's full provenance is **EMB-X1JY06WJ → EMB-7KJ4VR4G → synlig, by two successive whole-palace
> file copies and zero merges.**
>
> Two consequences worth stating, because this is easy to read as reassuring. First, it **strengthens**
> condition 3 rather than weakening it: the method now has two successes behind it, both of them
> single-source clones, and still not one exercise of merge semantics — so §7.6 is *less* tested than a
> "we've done this twice" reading would suggest. Second, the primary's history contains records from a
> machine that no longer exists, so `source_machine` is the only thing distinguishing them; do not treat
> it as noise when pruning or re-mining.
| Record type | Dedupe on replay | Client work needed |
| --- | --- | --- |
| `add_drawer` / `checkpoint` drawers | **Server-side**: content-addressed id + pre-write `col.get` probe → `{"reason":"already_exists"}`, no write (`mcp_server.py:2593-2600`) | **None.** Just replay |
| KG **open** facts | **Server-side**: `add_triple` guard on `(s,p,o) WHERE valid_to IS NULL` | **None.** Just replay |
| KG **closed** facts | **None** — the guard is scoped to open facts (§3.2) | Dedupe on `(s,p,o,valid_from,valid_to)` before sending |
| **Diary entries** | **None whatsoever** (§7.6) | Skip any local entry whose `sha256(entry)[:12]` suffix already exists remotely |
| Mined drawers | id is path-dependent → same content from two hosts = two rows | Out of scope: mined wings are `local` (§5) |
Existence checks available today, with **no new server code**:
| Tool | Kind | Fit |
| --- | --- | --- |
| `get_drawer(id)` | exact id, clean not-found (`mcp_server.py:3103-3114`) | The right check wherever ids are deterministic |
| `list_drawers(wing, room, since, before)` | metadata page | Bulk "what does this wing already hold" — the cheap way to collect existing diary id suffixes. Note `since`/`before` filter in **Python**, not in the backend `where` (chroma 1.5.7 rejects string `$gte`/`$lt`) |
| `check_duplicate(content, threshold)` | semantic, **whole collection, no wing/room scoping** | One embed + one HNSW query *per call* → the cost driver at thousands-of-records scale. Reserve it for diaries, where nothing cheaper works |
| `kg_query` | fact lookup | Redundant for open facts (the guard covers them); useful for closed ones |
**Two containers on one host are the *easy* case, not the hard one.** They share a bind-mounted palace,
so it is *one* palace joining once, and content-addressing makes even a concurrent double-join harmless
for everything except diaries. Belt and braces:
- **Keep join state in the shared palace, not in the container** — e.g. `<palace>/edge/bootstrap.json`
holding `{target_url: {joined_at, high_water_local_seq}}`. Both containers then see "this palace has
already joined", a recreate does not repeat the work, and it is the same durable-state-next-to-the-data
pattern as the outbox (§4.3). It must **not** live in a container-only path: `~/.mempalace` is not
preserved by default for solitary users (§1.2), which is exactly why the file belongs to the palace
directory.
- **First join is a dry run.** Bootstrap one palace, verify it, *then* let the rest join. Ordering matters
only because of diaries and closed facts; everything else is order-free.
> ⚠️ **Corrected 2026-08-14. This bullet used to say "verify counts (`status`, `kg_stats`, per-wing
> `list_drawers`) against expectations". Do not verify a join by counts.** Two independent reasons,
> both learned on the first real seed:
>
> 1. **`mempalace status` counts chunk rows, not logical drawers.** Three drawers plus one 2-chunk diary
> presented as **+9**. A count delta cannot even tell you how many *records* moved.
> 2. **Chunk counts and chunk-id sets legitimately differ between two palaces** whenever a drawer was
> updated on either side: an update preserves `drawer_id`, **re-chunks to the new length, and deletes
> the surplus chunk rows** (§9.3). A `…_chunk_000007` present on one side and absent on the other is
> therefore the ordinary signature of an **edit**, not of loss.
>
> On 2026-08-14 that second mechanism produced a confident "the seed lost a chunk" conclusion that was
> filed as a finding before being retracted. Counts **hid** the difference (14,777 vs 14,778 looked like
> one lost row); an id-set diff **over-reported** it. The two methods fail in *opposite* directions, so
> agreeing with either one alone proves nothing.
>
> **Verify by content.** For a sample of drawers, `get_drawer(<parent id>)` on *both* palaces and compare
> the **reassembled `content`**. An id-set diff is a fine first pass to *find* candidates, but it must be
> adjudicated by a content comparison before anyone concludes loss. Two cheap decisive checks once a
> difference is real: **is one side a prefix of the other** (that, and only that, is truncation), and
> **does the final chunk's length equal `len(content) - chunk_size * (n_chunks - 1)`** (800 in 3.6.0)? If
> both sides satisfy the arithmetic for their own content, both are complete and you are looking at two
> revisions, not damage.
The genuinely hard case is **two different palaces holding overlapping mined content** — the same repo
mined on a laptop and a workstation under different absolute paths. §5 excludes it by keeping mined wings
`local`; that exclusion is load-bearing, not tidiness.
---
## 5. Not everything should be global
Per-wing replication policy, declared in edge config:
| Policy | Wings | Rationale |
| --- | --- | --- |
| `replicated` | diaries, `wing_pi`, checkpoints, KG, hand-authored notes | Small, curated, content-addressed → **already merge-safe** |
| `local` | mined code/docs (e.g. the 13,597-drawer `workspace` wing on this host) | Derived data, re-mineable from git, and carries exactly the path-dependent IDs that break merging (§3.3) |
This shrinks the hard problem to the layer that is already safe, and it is why **`mine` must never be
remote**: the primary serializes *every* request behind one lock (§7.5), so a bulk remote mine would
stall every other agent in the fleet.
**Decided 2026-08-09: diaries are `replicated`.** Cross-machine continuity is the entire point, and
diaries are the highest-value content in the palace to share ("*given their value, I say go with (a)*").
Accepted consequence, stated plainly because it follows from the primary being **synlig, a work VM**
(§8.1): personal diaries will live on employer infrastructure. They already quote internal hostnames,
paths, moods and the occasional token prefix (§6.1 threat 2) — so "don't put secrets in memory" stops
being advice and becomes a precondition.
**The work/personal boundary is a property of the wing, not of the device.** Rejected design
(2026-08-09): splitting the fleet into a work primary and a personal primary along machine lines. The
reasoning is decisive — pi-devbox/opencode-devbox are *simultaneously* work and home projects, and
personal machines get used for work-adjacent work, so **the device where the work happened cannot
classify the project**. That is precisely the axis this table already encodes, which is why
`replicated`/`local` per wing is the right knob and a second primary is not needed to express it.
**Shape the config so multiple stores stay possible without paying for them now.** Phase 1 keeps
`MEMPALACE_REMOTE_URL` a **scalar** (one primary). Per-wing *targets* — this same policy column, plus a
destination — belong to the edge config in Phase 2+, so nothing in the `.env` contract has to be
un-designed later.
---
## 6. Security model
The transport is largely solved; **the gap is authorization, not cryptography.**
### 6.1 Threats, in priority order
1. **Memory poisoning / persistent cross-machine prompt injection — the underrated one.** Palace
content is injected into agent context at wake-up. A shared palace means one careless or
compromised container can plant instructions that *every other agent in the fleet reads as trusted
memory*. Mitigation: server-stamped provenance on every synced record (§7.3.2), don't auto-inject wake-up content authored
by devices outside a trusted set, keep an admin-only wing for anything instruction-shaped.
2. **Aggregation raises exfiltration impact.** One primary holds every machine's diaries — which
already quote internal hostnames, paths and token prefixes. Mitigation: per-wing ACL; don't put
secrets in memory (the WAL redaction list exists for a reason); private-network-only exposure.
3. **Accidental mass deletion** by a client (`sync`, `delete_by_source`) — see §7.2.
4. **Availability**: the server is single-writer by design; one long operation blocks everyone.
### 6.2 Policy
| Control | Decision |
| --- | --- |
| **Network posture** | Primary **never** internet-exposed. Publish only onto the private overlay / existing tunnel (Pangolin/newt). ⚠️ **The reflexive "bind loopback in the container" is the failure mode here — see Transport.** |
| **Transport** | **Decided 2026-08-09: terminate TLS in the existing Pangolin/newt tunnel**, not in `serve` (cheaper than patching Python's TLS surface; one DNS record per service on the web hotel is the whole setup cost). ✅ **Host/Origin policy verified by experiment on synlig 2026-08-10, 11/11 as predicted** (runbook §2.4) — the summary: **do not bind loopback behind the tunnel.** `enforce_host_pin = _http_is_loopback(host)` (`mcp_server.py:5367`), so a loopback bind + a proxy forwarding `Host: palace.example.com` → **403**, while a non-loopback bind → **200** (pin deliberately relaxed, "*may sit behind a proxy that rewrites Host*", `:5362-5365`); tokenless non-loopback binds require `--allow-insecure` (`cli.py:1450`). **Prefer binding the docker0 gateway (e.g. `172.17.0.1`) over `0.0.0.0`:** non-loopback, so the pin relaxes, yet reachable only from the host and its containers — so a newt container on the box reaches it and the LAN cannot. **The `Origin` check is never relaxed:** absent `Origin` is fine (every non-browser MCP client, incl. pi and opencode), a *present* non-loopback `Origin` is 403 with no override — so keep browser-based clients and `Origin`-injecting proxies out of the path. `/healthz` is Host/Origin-gated but token-free, so it works as the tunnel's liveness probe. |
| **Authentication** | Target: **per-device bearer tokens** with a server-side `token → {device_id, scopes}` registry — revoke one laptop, rotate without a fleet outage. **Decided 2026-08-09: Phase 1 ships the single shared token** ("*iterate more feature rich but more complex solutions over time*"), so per-device lands with Phase 4. Store in the existing `.env.age` flow, 0600 on disk. **Consequence: until then the primary cannot tell devices apart, so `origin_device` stays client-asserted and advisory — nothing load-bearing may depend on it (§7.3.2).** |
| **Authorization** | Per-device read/write **wing globs**. Server-side refusal of `mine`/`sync`/`delete_*` except for an admin device. `--read-only` gives a free observer tier. |
| **Provenance/audit** | `origin_device` (+ optional `origin_label`) + `op_id` on every record, stamped **server-side from the authenticated credential** — a client-asserted origin is a hint, not a fact (§7.3.2); server-side append-only op log. |
| **Recovery** | Deletes as tombstones; server-side backups with `backups.py` retention (`MEMPALACE_MAX_BACKUPS`). |
| **DoS** | Keep the 16 MiB cap; add rate limiting; no remote `mine`. |
---
## 7. Landmines — Phase 0 runbook
Do these **before** any cutover. 7.1 and 7.2 cause silent data loss; 7.6 causes silent *duplication*
the first time a palace joins (§4.4).
### 7.1 `MEMPALACE_PALACE_PATH` ≠ `--palace` (silent empty KG)
```python
# mcp_server.py:708-711 (_palace_flag_given = bool(_args.palace), line 325)
def _resolve_kg_path() -> str:
if _palace_flag_given:
return os.path.join(_config.palace_path, "knowledge_graph.sqlite3")
return DEFAULT_KG_PATH # knowledge_graph.py:49 → ~/.mempalace/knowledge_graph.sqlite3
```
`cmd_serve` **always** passes `--palace`. So the moment you start the server, the KG becomes
`<palace>/knowledge_graph.sqlite3` — a *different file* from the live `~/.mempalace/knowledge_graph.sqlite3`.
The vector store looks fine and the knowledge graph is silently empty.
Four stores, **three** location rules:
| Store | Location rule |
| --- | --- |
| drawers | backend-abstracted (`BaseCollection`) |
| `knowledge_graph.sqlite3` | HOME-anchored **unless `--palace` flag** (`knowledge_graph.py:49`) |
| `hallways.json` | derived from `palace_path`, with legacy HOME fallback (`hallways.py:73,83`) |
| `known_entities.json` | HOME-anchored (`miner.py:701`) |
**Action:** `mv` all three files into the palace directory before first `serve`, and verify
`kg_stats` is non-empty afterwards. Corollary: **a shared pgvector/qdrant backend shares drawers and
nothing else** — it would leave the KG as fragmented as it is today. That is why the shared-backend
option is *not* the answer.
⚠️ **Understated above, corrected 2026-08-10 while provisioning synlig: this is not a migration hazard,
it is the stock-default behaviour, and it is permanent.** `DEFAULT_PALACE_PATH` is
`~/.mempalace/palace` (`config.py:221`) while `DEFAULT_KG_PATH` is
`~/.mempalace/knowledge_graph.sqlite3` (`knowledge_graph.py:49`) — **those differ out of the box**, with no
custom path involved. So on *every* host, forever: `serve` (always passes `--palace`) uses the KG inside
the palace, while any CLI command run without `--palace` uses the HOME one. A one-time `mv` does not fix
that; it just relocates which of the two files is populated.
**Better action — converge the two resolution rules onto one inode** (and the only option on a greenfield
primary, where there is nothing to move):
```sh
ln -sfn palace/knowledge_graph.sqlite3 ~/.mempalace/knowledge_graph.sqlite3
ln -sfn palace/known_entities.json ~/.mempalace/known_entities.json
ln -sfn palace/hallways.json ~/.mempalace/hallways.json # added 2026-08-14, see note
```
> **Updated 2026-08-14 — `hallways.json` is symlinked too now.** This block previously said it was
> *deliberately* not symlinked, on the narrow grounds that it is already palace-derived and its HOME path
> is a warning-only legacy probe that never auto-migrates (`hallways.py:73-95`). Both facts still hold —
> but the goal changed from "symlink only what the code demands" to **"all real state lives under
> `palace/`, so one copy of `palace/` is a complete copy"**, which is what let the 2026-08-14 file-level
> seed (§4.4) treat the palace as a single self-contained unit. With all three links in place every
> parent-level path resolves, and 3.6.0's *three different* resolution rules — palace-relative for the
> served KG, HOME for the CLI KG, `dirname(palace_path)` for hallways, hardcoded HOME for
> `known_entities.json` — converge on one set of files. Revert by deleting the symlink if it ever causes
> trouble.
Verified on synlig 2026-08-10, because the WAL behaviour was the load-bearing assumption: a **dangling**
symlink is created on first `sqlite3.connect`; `-wal`/`-shm` land next to the **target** (inside the palace
dir, so the palace stays a self-contained backup/bind-mount unit) and *not* beside the symlink; a write
through one path reads back through the other; **same inode**. Full record in
[`synlig-primary-runbook.md`](./synlig-primary-runbook.md) §2.3.
### 7.2 Never run `mempalace sync` against a shared palace
It classifies drawers whose source files are absent **on the running host** as orphans and deletes
them; `_auto_detect_project_roots` guesses roots from drawer metadata, so the blast radius is
data-dependent rather than obvious. From a laptop that lacks the repos, it is a fleet-wide wipe.
**Action:** edge blocks `mempalace_sync` from ever reaching the primary; document it; consider an
upstream `--refuse-shared` guard.
> **Measured on the live primary, 2026-08-15 — the threat model is not where this section puts it.**
> An unscoped `mempalace_sync` dry-run against central (14,829 drawers) reports
> `out_of_scope: 14391`, `no_source: 438`, **`missing: 0`, `kept: 0` — so zero drawers are currently
> deletable.** The mechanism matters: a source root that is *entirely* absent yields **`out_of_scope`,
> not `missing`**, and only `missing`/`gitignored` drawers are removed. A wipe therefore needs the root to
> **exist** while the files under it do not — not a host that simply lacks the repos.
>
> Two corrections follow. **(a) The "laptop" scenario is currently blocked twice over**, and neither
> guard was designed for this: the CLI has no remote support at all, so `mempalace sync` on a client
> physically cannot reach central; and the MCP `mempalace_sync` tool executes **server-side on synlig**,
> where clients' `/workspace/...` roots do not exist — hence `out_of_scope`. Do not mistake this for
> safety by design; it is safety by coincidence of layout, and it decays.
> **(b) The real hazard arrives with the transcript feeder's remote mode.** That mode rsyncs staged
> transcripts into per-device inboxes **on synlig**, so those source files *will* exist on the palace
> host — making them in-scope for the first time. Any later stage rotation, cleanup, or per-device inbox
> removal then reclassifies that device's conversation drawers as `missing`, i.e. **prunable, and
> prunable by a sync triggered from a different device.** `bin/mempalace-pi-session` already documents the
> local form of this ("do NOT run `mempalace sync` while the stage is missing, or the drawers mined from
> it get pruned"); shared-palace multi-device turns it cross-device. **Settle stage retention on synlig,
> and the sync guard, as part of enabling the feeder fleet-wide — not after.**
### 7.3 Provenance belongs to the sync boundary — not to the agent, and not to a solitary container
Today every drawer carries exactly `{wing, room, source_file, added_by, filed_at, id_recipe}`
(+`chunk_index`, `parent_drawer_id`); diaries add `{hall, topic, type, agent, date}`. `added_by`/`agent`
is the *agent* name (`pi`, `mcp`, `checkpoint`) — **never the machine**. So in a merged store you cannot
tell which host wrote a record, cannot audit, and cannot compute per-device high-water marks.
**Verified 2026-08-09 (`mcp_server.py:2578-2585` drawers, `3527-3536` diaries): those lists are
exhaustive — there is no session, PID or conversation field either.** Two consequences, because both are
natural questions:
- **Two concurrent pi sessions** (the tmux pattern pi's own docs suggest) writing to one palace are
**indistinguishable**. Nothing records which session produced which record.
- **pi vs opencode is only *accidentally* distinguishable.** `added_by` is a free-form optional string
(default `"mcp"`; `checkpoint` resolves explicit arg → diary `agent_name` → `"checkpoint"`), and
`extensions/pi/mempalace.ts` **never sets it** for `add_drawer`/`checkpoint` — it sets identity only for
diaries (`agent_name` from `$MEMPALACE_AGENT_NAME`, default `pi`, `:758`). `kg_add` has no attribution
field at all. So the *only* real harness attribution today is the diary wing, and for drawers the value
is whatever string an LLM happened to pass. **✅ Fixed 2026-08-25: the bridge now defaults the writer
field on every write path that has one, and marks diary entries in-text — §7.3.5.**
**Design consequence: device + agent, never session.** Provenance has exactly three consumers — poisoning
triage ("which box planted this?"), per-device high-water marks, and revocation — and none of them needs
session granularity; adding it would put a field on every record with no reader. Under §6's per-device
tokens, **provenance granularity equals token granularity**: one token per host makes two containers on
that host a single origin, while a token per container separates them. §7.3.4's `<agent>/<label>/<device>`
encoding keeps the two axes orthogonal, so pi-vs-opencode stays recoverable — but only if the **edge**
populates the agent segment, never the LLM.
**Earlier drafts of this section said "stamp it everywhere, now, because it cannot be backfilled." That
was wrong on both counts.** See §7.3.3.
#### 7.3.1 What can actually be stamped (mechanics)
The metadata schema is **fixed** — there is no free-form field — and `tools/call` **whitelists arguments
to declared schema properties** (*"Prevents callers from spoofing internal params like
added_by/source_file"*), so an extra `origin_host=…` never reaches storage.
> **Corrected 2026-08-25 against 3.8.0:** that whitelist is no longer a silent drop. `mcp_server.py:6440`
> now returns a hard JSON-RPC error — `-32602 Unknown parameter '<k>' for tool <name>` — for any argument
> outside the tool's declared properties. Practical consequence for any stamper: **injection must be
> allowlisted per tool, never blanket.** Adding `added_by` to a `diary_write` or `kg_add` call no longer
> gets quietly ignored; it fails the whole call.
| Surface | Provenance slot | Notes |
| --- | --- | --- |
| `add_drawer`, `checkpoint` | **`added_by`** | The only one. Free-form (`strip_lone_surrogates` only, *not* `sanitize_name`, so `/` and `@` are legal). `checkpoint` resolves explicit arg → `diary.agent_name` → literal `"checkpoint"`, so **omitting it yields a permanently unattributable drawer** — no `@`, no source path, no rule can recover it later. |
| `mine` | **`agent`** | Reaches `added_by` on every filed drawer (`miner.py:1432`, `convo_miner.py:129,248`). Defaults to `"mempalace"`. |
| `event_append`, `artifact_put` | **`from_agent` / `created_by`, plus a free-form `metadata` dict stored verbatim** | The best-provisioned surfaces in the whole API — and they also carry their own `origin_replica` column. If a future record type needs structured provenance, this is the shape to copy. |
| `diary_write` | **none in metadata — the entry TEXT is the only channel** | ⚠️ **Never** put a device in `agent_name`: `wing = f"wing_{agent_name}"` → a separate wing per host, and `diary_read` filters `{"agent": agent_name}` → `diary_read("pi")` then **misses** those entries. `sanitize_name` does *not* block `@`, so this fails silently rather than loudly. Since 2026-08-25 the edge instead prefixes the entry with an AAAK field, `HOST:<device>|SESSION:…` (§7.3.5). |
| `kg_add` | **none** | Only `source_file`/`source_closet`/`source_drawer_id`. Origin is inferable only via `source_drawer_id` → that drawer's `added_by`, so **always pass `source_drawer_id`**. |
**Metadata is write-only to *readers*, and that asymmetry decides where provenance must live.** `added_by`
and any stamped `device` **are** returned by `get_drawer` — `_response_safe_meta`/`_safe_meta`
(`mcp_server.py:1688-1707`) is a `None`-coercion guard, *not* a key filter, so an earlier claim that the
read path strips provenance was wrong. But `search` results are assembled from a **fixed key list**
(`searcher.py:1822-1839`: drawer_id, text, wing, room, source_file, source_path, created_at, authored_at,
similarity, distance, effective_distance, closet_boost, matched_via) and `diary_read` returns content —
so **no amount of correct metadata is visible to an agent doing a search or reading its own diary.**
That is not a cosmetic gap: it is exactly how the 2026-08-25 cross-host misattribution happened
(drawer_pi-devbox_landmines_fa0002a4). Metadata serves the three consumers below; **reader-visible
attribution needs the record's own text.** → Upstream asks: **surface `added_by`/`device` in search
results**, **give `kg_add` a provenance field**, and **give `diary_write` an `added_by` that does not feed
the wing name**.
**Triples and coordination events live in different databases, which bounds any stamper's reach.**
`knowledge_graph.sqlite3` (`triples`: id, subject, predicate, object, valid_from, valid_to, confidence,
source_closet, source_file, source_drawer_id, adapter_name, extracted_at) and `logstream.sqlite3`
(`events`, `artifacts`) sit beside `chroma.sqlite3` in the palace directory. A stamper written against
Chroma's `embedding_metadata` — which is what `bin/mempalace-device-stamp` is — **cannot reach them at
all.** As of 2026-08-25 that is 156 triples and 11 events/3 artifacts, i.e. small enough to leave, but it
must be stated rather than assumed: *the palace is three stores, and provenance coverage is per store.*
#### 7.3.2 Who should stamp it — a ladder of trust
Provenance answers "which device asserted this?" Only something that can *verify* the answer should
write it. Ranked by trustworthiness:
| Stamper | Knows the device? | Verifiable? | Uniform? | Verdict |
| --- | --- | --- | --- | --- |
| **Agent (via skill)** | No — must shell out to read env | No | No — per-call boilerplate, forgettable, improvisable | ❌ **Worst possible place.** Rejected. |
| **Client / `mempalace-edge`** | Yes, from host-supplied `.env` | No — self-asserted | Yes — one line in a proxy | ⚠️ Acceptable **interim** — **implemented 2026-08-25, §7.3.5** |
| **Primary, from the authenticated credential** | Yes | **Yes** — bound to the token | Yes, for every synced record | ✅ **Correct home** |
> **The ❌ row was not hypothetical.** Between 2026-08-23 and 08-25 the mempalace skill did instruct the
> agent to pass `added_by="<harness>@<device>"` by hand, and it behaved exactly as this table predicts:
> hand-filed drawers came out as `checkpoint`/`mcp`/`pi` whenever the instruction was not recalled, and
> the agent that *wrote the instruction* then filed its own provenance drawer without it. Measured on
> 2026-08-25: 199 rows the stamper had already seen and could not resolve, of which 10 were that drawer.
> Moving the same one-line convention into the bridge (§7.3.5) removed the failure mode without changing
> the values written — which is the point of the ladder: **the value was never the problem, the writer
> was.**
The decisive point: **a client-asserted origin is a hint, not a fact.** The primary is the only party
that can bind a write to an identity it verified. And under the §4 design *every* write reaches the
primary through an authenticated channel — including offline ones, at outbox-flush time — so the server
can stamp the complete set without any client cooperation. **Provenance is a property of the sync
channel, not of the record's author.**
Blocker for the ✅ row, verified in 3.6.0 and **re-verified in 3.8.0 on 2026-08-25**: `serve` takes a
**single shared bearer token** — one scalar string, `auth_token = os.environ.get("MEMPALACE_MCP_HTTP_TOKEN")`
compared with one `hmac.compare_digest` (`mcp_server.py:7137-7140`, `:7685`) — and the package still
contains **zero** occurrences of any device/origin concept. Two consequences worth stating plainly,
because "just issue per-device tokens now" sounds like a shortcut and is not:
- **Per-device tokens are not a config change.** There is no `token → {device, scopes}` registry to
populate; the code holds one string. Multiple tokens require a code change.
- **Tokens are the cheap half.** Even with N tokens, *nothing would stamp*: an origin field would have to
be added to six write paths (`add_drawer`, `checkpoint`, `diary_write`, `kg_add`, `event_append`,
`artifact_put`) across **three** separate databases (§7.3.1). And `mempalace` is **upstream MIT**
(PyPI `mempalace`, github.com/MemPalace/mempalace) consumed via `uv tool install` — so this is an
upstream PR or a carried fork, on a package that moved 3.7.1→3.8.0 inside one week.
- **The tunnel cannot substitute.** Pangolin *terminates TLS and nothing more* (§6.2), and "a Pangolin
user per container with credentials in each `.env`" was considered and rejected in the exposure runbook
§1.2 — its HTTP auth is browser-shaped (SSO, resource PIN), not client-shaped.
So the realistic ✅ implementation is **a small authenticating rewriter we own in front of the palace**:
it holds `token → {device, label}`, and rewrites the JSON-RPC `params` to set `added_by` from the
*verified* token. That is `mempalace-edge` relocated from the client to the primary. It must track tool
schemas to stay safe (§7.3.1: `-32602` on undeclared args; `diary_write` has no slot), which is precisely
why it is Phase 4 work and not a quick win. Hence the phasing:
- **Phase 2 (edge):** edge may stamp `added_by` from its configured env — self-asserted, **advisory
only**, never load-bearing for authorization or destructive scoping. **Implemented 2026-08-25 (§7.3.5).**
- **Phase 4 (authz):** per-device tokens land; the primary stamps authoritatively and the client-supplied
value becomes redundant (and must be treated as untrusted input, not merely ignored). The upgrade is
**lossless because every stamp carries a `*_source` key** — an authoritative pass simply overwrites with
`device_source='token'`, so nothing done in Phase 2 has to be undone.
#### 7.3.3 Solitary containers should stamp nothing — and lose nothing by it
A pi-devbox or opencode-devbox running solitarily is a **single-origin store by definition**. Origin is
therefore a property of the *whole palace*, not of each record — so it can be assigned **wholesale at
the moment the store stops being solitary**: one `--origin-device` flag on the import/first-sync path
attributes every record from that palace to that device.
That dissolves the "impossible to backfill" argument. Per-record stamping is only necessary once records
from *multiple* origins are interleaved in one store, which is exactly and only the primary. So:
- **Solitary devboxes: no stamping, no config, no skill instruction, no behaviour change.** This is
strictly more compliant with **R1** than the earlier draft, which quietly asked every solitary user to
carry metadata for a feature they had opted out of.
- **Migration is unaffected**: bulk attribution at import is *more* reliable than per-record stamping,
because it cannot be partially applied.
- **Multi-harness on one host is solved *in principle*** by `added_by` = agent name (`pi` vs `opencode`) —
that is what the field is for, and it needs no device component. ⚠️ **Corrected 2026-08-09: nothing
actually sets it.** The pi extension leaves `added_by` at its default for `add_drawer`/`checkpoint`, so
in practice the value is whatever an LLM passed. The field is the right home; the client-side write that
populates it is missing, and it belongs to the edge (the ⚠️ row of §7.3.2) — not to a skill instruction.
**✅ Closed 2026-08-25: the edge write now exists (§7.3.5), and it carries the device as well as the
harness — `<harness>@<device>` — because in a shared palace the harness axis alone cannot answer "which
box planted this?".**
- **A palace on a shared host bind-mount** (as tor-ms22's compose does) is still single-*device* under
the "host owns the palace" model, so it too imports as one origin.
The one case bulk attribution cannot fix: a palace that was *already* merged from several devices without
stamps. Preventing that is precisely why the **primary** must stamp from day one of Phase 4 — it is the
only store where interleaving occurs.
#### 7.3.4 Identity fields, when a stamper does exist
For the edge (interim) and the primary (authoritative) — never for agents:
| Field | Source | Rule |
| --- | --- | --- |
| `origin_device` | `uuidgen` **once**, stored in the **host's `.env`**; later, issued with the device's token | Opaque, stable, collision-free by construction. Compared, never parsed. |
| `origin_label` | hostname, same `.env` | Human readability **only**. Never identity, uniqueness or scoping. Free to change. Optional — a hostname can leak an asset tag or username (`EMB-7KJ4VR4G`, `HOST_SSH_USER=ECSJPER`). |
Why not the obvious sources: **a container cannot discover its host's identity.** `hostname` returns the
*container ID* (`f3bf2a103473`) which changes on **every recreate**; there is no `/etc/machine-id`; and a
baked one would be *worse* — identical for every container from the same image, a guaranteed collision.
**Hostnames are also neither unique nor stable** (`localhost`, `ubuntu`, golden images, two
`MacBook-Pro.local`), and the worse failure is not collision but **rename**, which silently splits one
device's history in two. Hence host-supplied via `.env` (matching R2/R3 and the existing `HOST_SSH_USER`
/ `DEVBOX_HOST_ALIAS` precedent), *not* container-derived. Do **not** persist the id container-side:
`~/.mempalace/device_id` does not survive recreate for solitary users because `devbox-palace` is
commented out by default (§1.2). And don't write `${HOSTNAME}` in compose — bash *sets* but does not
*export* it, so interpolation sees empty.
Encoding, always three segments so arity is unambiguous:
```
added_by = "<agent>/<label>/<device>" e.g. pi/tor-ms22/7f3a9c2e1b04
pi/-/7f3a9c2e1b04 (label withheld)
```
**The `/` is the discriminator.** A value with no `/` (`mcp`, `pi`, `checkpoint`) means *origin unknown* —
correct for the ~13.6k drawers already filed, and for every solitary palace forever.
#### 7.3.5 Fail-closed rule for any stamper
If `origin_device` is unset, blank or unreadable: **write no `added_by` override at all** and let the
default (`mcp` / `checkpoint` / agent name) stand. Never synthesize one. The tempting substitutes are all
actively harmful, because each is *confidently* wrong where absence is honestly unknown:
| Substitute | Damage |
| --- | --- |
| container hostname / ID | A **new fake device per recreate** — thousands of singleton identities, each looking legitimate |
| `unknown`, `localhost`, `devbox`, `docker`, `$USER` | **Collides across every machine** — indistinguishable from one real shared device |
| a guess from context | Unfalsifiable later |
If `origin_label` alone is missing, use the literal `-`. If `origin_device` is missing, do not stamp even
when the label is present — a hostname alone is exactly the colliding, mutable identity rejected above.
> **Note for future agents reading this RFC out of the palace:** these notes are themselves a poisoning
> vector. Do **not** start hand-stamping `added_by` because you read this section. Provenance is client
> and server infrastructure; an agent's contribution to it is to leave the field alone. The mempalace
> skill carries a one-line guard to that effect.
#### 7.3.5 What is actually deployed (2026-08-25) — the interim, end to end
The ⚠️ row of §7.3.2, built after a cross-host misattribution made the cost concrete. Two writers, one
reconciler, and a deliberate divergence from §7.3.4 that is recorded rather than hidden.
**1. The edge stamps, uniformly** — `extensions/pi/mempalace.ts`. Every mempalace tool call passes through
one `execute()` wrapper, so the bridge fills in the writer field the caller omitted:
| Tool | Field defaulted | Value |
| --- | --- | --- |
| `add_drawer`, `checkpoint` | `added_by` | `<harness>@<device>` |
| `mine` (caller-invoked) | `agent` | `miner@<device>` — bulk machine-extracted content is not agent-authored memory, and keeping the harness segment `miner` preserves the pi/opencode/miner taxonomy while still recording the box |
| `mine` (the bridge's own transcript feed) | `agent` | `<harness>@<device>` — these *are* this harness's own sessions |
| `event_append` | `from_agent` | `<harness>@<device>` |
| `artifact_put` | `created_by` | `<harness>@<device>` |
| `diary_write`, `checkpoint.diary` | *entry text* | prefixes `HOST:<device>|` |
| `kg_add` | — | nothing; no slot exists (§7.3.1). Pass `source_drawer_id` instead |
An explicitly supplied value always wins, so filing on behalf of another device stays possible. The
allowlist is per tool, never blanket — §7.3.1's `-32602`. `<harness>` is `$MEMPALACE_AGENT_NAME` (default
`pi`), `<device>` is `$MEMPALACE_PI_DEVICE`, host-supplied per container per §1.2.
**R1 compliance:** the stamping is doubly gated on `MEMPALACE_PI_DEVICE` **and** `MEMPALACE_REMOTE_URL`.
A solitary devbox sets neither, so it stamps nothing and its behaviour is unchanged — which is also the
correct semantics per §7.3.3: a solitary store is single-origin, and origin belongs to the whole palace.
**2. The diary marker is in the TEXT, and that is the point.** `diary_write` has no metadata slot, but the
deeper reason is §7.3.1's read asymmetry: `search` projects a fixed key set and `diary_read` returns
content, so **metadata is invisible to the agent who will later read the entry.** A metadata-only fix,
even a server-authoritative one, would not have prevented the misattribution it was built for. The marker
is an AAAK field (`HOST:tor-ms22|SESSION:2026-08-25|…`), so it is machine-parseable *and* the first thing
a reader sees. The wake-up block now also states the device and warns that `diary_read` interleaves every
machine's diary.
**3. The primary reconciles** — `bin/mempalace-device-stamp` + `contrib/systemd/mempalace-device-stamp.{service,timer}`,
hourly on synlig. It fills `device` and `agent_kind` where absent, each paired with a `*_source` key
recording *how* it was determined, so an inference is never mistaken for a fact. It must stay on a timer,
not run once: **live re-mining replaces metadata rows and silently drops earlier stamps.** Its remaining
job after the edge exists is historic rows, re-mined rows, and diary entries. Coverage on 2026-08-25:
`agent_kind` 28,817/30,661 (94%), `device` 14,157 (46%) — the deficit is dominated by ~16k `/workspace`
project mines that are *deliberately* unattributed, because `/workspace/myconfigs/<host>/…` names the
machine a config **belongs to**, not the machine that mined it, and `/workspace` looks identical on every
devbox. Conflating subject with source would be worse than a blank.
**Two traps found by dry-running the reconciler before deploying it** (`--dry-run` exists for this reason;
the palace is shared, so a wrong rule invents device names that then have to be un-invented across 30k
rows, and `scripts/test-device-stamp.sh` pins both):
- **`HOST:` was already in use, with a different grammar** — older entries carry a composite fingerprint,
`HOST:emb-7kj4vr4g.f1d3c3f89e3e.v1.8.3.pi0.84.2` (device.container.image.pi), and some carry a bare
container id, `HOST:2efe2b06f480`. Taking the match whole would have invented devices like
`f1d3c3f89e3e.pi0.84.2`. The rule therefore validates against the known-device set (the feed inboxes)
and falls back to the first dotted segment — which *recovers* the composite entries correctly and
refuses container ids, exactly as §7.3.4 requires (containers are not devices).
- **`HOST:` also carries a different SENSE** — e.g. `HOST:exec.via.ssh-controlmaster->alpserv-2(…)`,
meaning "the box I was executing on", not "the box that wrote this". Validation rejects it, so the two
senses cannot be conflated. **A marker convention inherits every prior meaning of its own name.**
One further mechanism: chunks. A drawer's chunks are one write call, hence one origin — but a text marker
lands only in the chunk that contains it, so a 5-chunk diary entry would stamp 1 and leave 4 blank. The
reconciler propagates a resolved value across `parent_drawer_id` siblings (`device_source='sibling_chunk'`)
when they agree, and never overrides a row that resolved on its own evidence.
**Divergence from §7.3.4, stated openly.** §7.3 refers to an `<agent>/<label>/<device>` encoding that
§7.3.4 never actually defines — §7.3.4 specifies two *fields* (`origin_device`, an opaque uuid; and
`origin_label`, a hostname for readability only, explicitly "never identity"). What is deployed is
**`<harness>@<label>`**: two segments, hostname-derived, no opaque id. Why, and what it costs:
- There is nowhere to put a second field. `added_by` is one free-form string (§7.3.1), so a uuid could
only be carried *instead of* the readable label, in the one slot that also has to carry the harness.
- A uuid would have to be provisioned and kept in each host's `.env`; the label already exists there as
`MEMPALACE_PI_DEVICE`, is already the feed inbox name, and is therefore already the join key for the
`inbox_path` rule that resolves 13,350 rows.
- **The cost is real and already visible.** §7.3.4 warns that the worse failure of hostname identity is
not collision but *rename*, silently splitting one device's history. The palace holds `tor-ms22` (4,680
rows) **and** `tor-ms22-native` (3,826) — one physical machine, two device values, because the label
encoded an access mode. Under §7.3.4's design those would share an `origin_device` and differ only in
`origin_label`, which is exactly the property we gave up.
So: `<harness>@<label>` is the honest Phase-2 shape given one string to write into, and the `*_source`
keys are what make the Phase-4 upgrade to verified identity lossless. §7.3 should be read as promising
**orthogonality of the harness and device axes**, which `@` delivers — not the three-segment encoding it
appears to cite. → Fix §7.3's forward reference, or define the three-segment form in §7.3.4, when Phase 4
settles which identity the primary issues.
### 7.4 Embedder identity is client-local and currently toothless
`check_embedder_identity()` raises `DimensionMismatchError` / `EmbedderIdentityMismatchError`, but the
sidecar on this host reads `{"mempalace_drawers": {"model_name": "minilm", "dimension": 0}}` — and
`dimension: 0` means *unknown* and is **skipped**, so only the model *name* is compared. The sidecar
lives in the client's palace dir (`_sidecar.py`), so with a remote backend the check is per-client and
never centrally enforced. **Action:** the primary must reject writes from a client whose embedder
identity does not match, and merged reads (§4.2) must be disabled on mismatch — comparing distances
across models silently degrades recall.
### 7.5 Concurrency expectations
- `_HTTP_REQUEST_LOCK = threading.Lock()` wraps **every** dispatch (`mcp_server.py:5115`, held at
`5346`/`5513`), commented: "*HTTP gives us a safer transport, not concurrent Chroma/HNSW mutation.*"
The primary is a **single-writer, one-request-at-a-time** service. Size for a handful of agents.
- `mine_palace_lock` uses `fcntl.flock(LOCK_EX|LOCK_NB)` and exists because parallel HNSW inserts
"*can corrupt the HNSW graph*" — but the lock **file** is HOME-derived (`~/.mempalace/locks/
mine_palace_{sha256(realpath(palace))[:16]}.lock`) while the key is palace-derived. Two containers
with different `~/.mempalace` mounts but the same palace path compute the same key on *different
files* → **no mutual exclusion**.
- **Therefore: reject "put the palace on a NAS/SMB share and point everyone at it."** flock over
NFS/SMB is unreliable, and the lock wouldn't be shared anyway. One process owning the files and
serving HTTP is the supported model — which is exactly why `serve` exists.
### 7.6 `diary_write` has no idempotency guard — replay duplicates every entry
```python
# mcp_server.py:3511-3513, then 3546
entry_id = f"diary_{wing}_{now:%Y%m%d_%H%M%S%f}_{hashlib.sha256(entry.encode()).hexdigest()[:12]}"
...
col.add(ids=[entry_id], ...) # not upsert — and no col.get probe anywhere in tool_diary_write
```
Compare `add_drawer` 900 lines earlier, which probes `col.get(ids=idempotency_probe_ids)` and returns
`{"reason": "already_exists"}` without writing (`:2593-2600`). The contrast *is* the finding: **the µs
timestamp makes every diary id unique by construction, and nothing checks the content.** So §3's
conclusion — log the intent, replay the intent — is **false for diaries**; for them, replay is
duplication.
This matters more than it sounds, because diaries are `replicated` (§5) and are precisely the content a
join replays (§4.4).
> ⚠️ **Still open as of 2026-08-14 — and the first join did not test it.** The seed was a file-level copy
> (§4.4 Deviation), which replays no diaries and therefore *cannot* duplicate them. **§7.6 was sidestepped
> by method choice, not resolved.** Neither action below has been built. The moment a *second* palace
> joins — by any replay-based route — this becomes live again, and it is the single most likely thing for a
> future operator to get wrong, because the first join appears to have proved the path safe. It did not:
> it avoided the path.
**Actions:**
1. **Client-side, now:** before replay, list the target's diary drawers for the wing, extract the 12-hex
`sha256(entry)[:12]` suffix from each existing id, and skip any local entry whose suffix already
appears. The suffix is already content-addressed — no new key needs inventing.
2. **Upstream ask:** make `diary_write` probe-and-skip on that suffix the way `add_drawer` does on its
content hash. A small, self-contained patch in the sibling function's own idiom, and it closes the gap
for *every* future writer instead of for one migration script.
3. **Do not "fix" this by making the agent dedupe.** An agent cannot know what the primary holds without a
call, and per-call boilerplate is exactly the failure mode §7.3.2 rejects.
---
## 8. Phasing
| Phase | Effort | Deliverable |
| --- | --- | --- |
| **0 — hygiene** | hours | §7 runbook: converge the KG/entities store paths **on synlig before first `serve`** (§7.1 — **done 2026-08-10**, runbook §2.3), ban `sync` on shared palaces (§7.2), ~~fix stale "unauthenticated" docs (incl. `pi-devbox/.env.example:21`)~~ — **done 2026-08-12**, and `docker-compose.mempalace.yml` turned out to be outright broken on 3.6.0, now fixed. Added 2026-08-09: settle the **diary dedup** approach and file its upstream ask (§7.6), and **dry-run the join from one palace only** (§4.4). ✅ **Both done 2026-08-14, with one asterisk that matters:** the join was dry-run *and* executed from one palace only — but **§7.6 was sidestepped, not settled** (a file-level copy replays no diaries, so it cannot duplicate them). The §7.6 client-side dedupe and its upstream ask are **still unbuilt and are hard blockers for the second joiner.** **No provenance work here** — it is not backfill-critical (§7.3.3) and belongs to the stamper, not the agent |
| **1 — primary up** ✅ **done 2026-08-14** | hours, **no code** | `mempalace serve --token --tls-cert` on a private-net host (reuse `docker-compose.mempalace.yml` — keep it a **separate standalone project**, R4 — and mind port 8765 vs pi-studio; tor-ms22 already moved to 8766). Repoint pi clients via `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN`. **opencode clients can be repointed in the same breath** — remote MCP is supported and `generate-config.py` already emits it (§9.1, §9.6), subject to the sidecar caveat in §4.1. **Shared memory today, no offline.** **Decided 2026-08-09: primary = synlig, TLS at Pangolin, single shared token (§8.1)** — mind the loopback Host-pin trap in §6.2. |
| **1.5 — opencode env propagation** | hours | Make the `mcp.mempalace` subtree env-authoritative in `generate-config.py`, gated by a generated-value fingerprint (§4.1). Independent of the rest of this RFC. Without it, adopting *or reverting* the opt-in on an existing opencode container needs a manual sidecar merge or a `docker volume rm` — which also blocks R6 reversibility |
| **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. **Not** "fixes opencode" — opencode's *transport* is already fine after Phase 1; what edge adds there is offline/local-first, since `generate-config.py`'s switch is remote **or** local with no failover. **Ships with the §1.2 opt-in wiring (compose + `.env.example` + a third branch in the existing `generate-config.py`) and must pass the R1 acceptance test.** |
| **3 — pull replication** | ~1 week | **DEFERRED (2026-08-08).** Server-side op-log with monotonic seq → each edge a full offline replica. Only needed if a laptop must hold *everything* offline. Accepted trade-off: offline recall = own writes + last-reachable state. |
| **4 — authz** | days–week | Per-wing ACL, per-device scopes, audit log, token rotation. **Two things fold in here, decided 2026-08-25 (§7.3.2):** (a) **per-device bearer tokens, motivated first by revocation** — today one shared token covers 4+ devices and every container on them, so a single compromised or decommissioned laptop cannot be cut off without rotating the whole fleet, and that is a standing risk *independent* of provenance; (b) **authoritative server-side provenance rides along free once (a) exists** — but only via a component **we** own, because upstream core holds one scalar `auth_token` and has zero device concept, so the shape is a small authenticating rewriter in front of the palace that maps `token → {device, label}` and sets `added_by` from the verified credential. Do **not** sequence (b) before (a): tokens are the cheap half, and stamping without a verified identity is what Phase 2 already does more simply. Prerequisite understanding: §7.3.1 (`-32602` on undeclared args; three separate databases; `diary_write` has no slot) and §7.3.5 (what the interim already covers, and the `*_source` key that makes the upgrade lossless) |
Phase 1 is worth doing on its own — it is pure configuration and immediately ends KG fragmentation
for online clients.
### 8.1 Deployment decisions (2026-08-09)
| Decision | Value | Note |
| --- | --- | --- |
| **Primary host** | **synlig** — a VM in Xerces (work OpenStack cloud) | Chosen for always-on + good connectivity, *not* for work/personal reasons. Consequence: replicated content, diaries included, lives on employer infrastructure (§5, §6.1 threat 2) |
| **TLS / ingress** | **Pangolin/newt**, one DNS record per service on the web hotel | Not `serve --tls-cert`. See the bind-address/Host-pin coupling in §6.2 |
| **Authentication** | **Single shared token** for now | Per-device deferred to Phase 4; `origin_device` stays advisory until then. **Re-verified 2026-08-25 on 3.8.0:** still one scalar `auth_token` (`mcp_server.py:7685`), so per-device is a code change, not configuration — and the shared token is a **revocation gap**, now recorded as the primary motivation for Phase 4 (§8) rather than a provenance detail |
| **Diaries** | **`replicated`** | Highest-value cross-machine content; accepted placement consequence (§5) |
| **Work/personal split** | **per-wing, not per-device** | Rejects two primaries split along machine lines — the device cannot classify the project (§5) |
| **Multi-store** | Possible later, not now | `MEMPALACE_REMOTE_URL` stays a scalar in Phase 1; per-wing targets are an edge-config concern (Phase 2+) |
---
## 9. Open questions
1. ~~**Does opencode's MCP config support a remote/HTTP transport at all?**~~ **RESOLVED 2026-08-09:
YES.** opencode's published JSON Schema (`https://opencode.ai/config.json`) defines
`$defs.McpRemoteConfig` = `{type:"remote" (enum), url (required), headers?: {string:string},
oauth?, enabled?, timeout?}` as a sibling of `McpLocalConfig` inside
`Config.properties.mcp.additionalProperties.anyOf`. So `headers` carries any bearer/API-key scheme,
and set `oauth: false` to stop opencode probing the URL for OAuth. The earlier "every sampled entry
is `type:local`" observation was **deployment evidence being read as schema evidence** — the schema
settles it in one fetch. **Consequence: the edge proxy is *not* the only option for opencode.** A
plain Phase 1 `mempalace serve --token` can be consumed directly, so nothing in the phasing depends
on this question any more. What opencode still lacks is *local-first writes and offline fallback* —
`generate-config.py`'s switch is remote **or** local per container, with no failover — and that is
the real Phase 2 justification for it.
2. **Upstream or local?** `mempalace-edge` needs no core changes, so it belongs in this repo
(`bin/` + `extensions/`). But `origin_device` stamping from the authenticated token, per-wing ACL, the `kg_supersede`
classification fix, and a `sync --refuse-shared` guard all want to go **upstream**
(`github.com/MemPalace/mempalace`).
3. **Chunked drawers under merge.** ~~Oversized content splits into `{drawer_id}_chunk_NNNNNN` with
`parent_drawer_id`, and the add-time idempotency probe checks only the *last* chunk id — a
partially transferred chunk set may look "already present" and stay truncated.~~ **Largely resolved
2026-08-09: the last-chunk-only probe is deliberate, not an oversight.** The code's own comment
(`mcp_server.py:2586-2592`) states that the last chunk's presence implies every earlier one landed
*because the batched `upsert` is all-or-nothing*, and it additionally probes `drawer_id` alongside so a
re-call with identical oversized content cannot duplicate a legacy pre-#1539 single-row write. The
normal path is therefore safe by construction. What remains untested is the abnormal one: a chunk set
left partial by a crash or kill *mid-upsert*, which would then look present and stay truncated. Worth a
single fault-injection test before trusting bulk replay of oversized drawers. **Still untested as of
2026-08-14** — the first join did *not* exercise it.
⚠️ **But 2026-08-14 surfaced a benign mechanism that mimics it, and it will generate false alarms during
any join verification (§4.4).** Updating a drawer preserves `drawer_id`, **re-chunks the new content, and
deletes the surplus chunk rows**. So two palaces holding two *revisions* of one drawer legitimately
differ in chunk count and chunk-id set, with zero data loss — and the shorter side returns a clean
"not found" for the chunk ids it no longer needs, which looks exactly like the truncation this entry
warns about. **The absence of a derived chunk id is not evidence of loss.** Distinguish them by prefix
test and final-chunk arithmetic per §4.4, not by id-set membership.
4. **`migrate.py` as a bootstrap tool.** `extract_drawers_from_sqlite()` reads `{id, document,
metadata}` straight out of chroma's SQLite (bypassing the chromadb API) and re-`add`s them with ids
and metadata preserved and **embeddings recomputed** (`migrate.py:326`) — the right shape for a
one-shot "seed the primary from an existing palace". Unverified: `col.add` behaviour on **id
collision into a non-empty target**. Test before relying on it. (`exporter.py` is markdown-only,
lossy, and has **zero callers** — not a transfer format. `backups.py` is retention pruning only.)
**Reframed 2026-08-09:** §4.4 makes **MCP-level replay** the primary join path — it reuses the
server's own idempotency guards instead of trusting chroma-level `add` semantics — so `migrate.py` is
now a *fallback* for bulk transfer rather than the plan, and its untested collision behaviour only
matters if we reach for it.
5. **Does `_HTTP_REQUEST_LOCK` stay held for the duration of an MCP-triggered `mine`?** Strongly
suggested by the code shape; if yes, a remote mine makes the primary unusable for its duration
(another argument for §5).
6. ~~**How does opencode-devbox learn the opt-in?**~~ **RESOLVED 2026-08-09: it already does.** The
image ships `rootfs/usr/local/lib/opencode-devbox/generate-config.py`, invoked from
`entrypoint-user.sh:117` on every start, which auto-registers the `mempalace` MCP server —
remote+bearer when `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN` are set, local stdio when
`mempalace-mcp` is on PATH, nothing otherwise — documented in that repo's `.env.example:38-49` and
`.env.shared.example:30-35`. It also ships mempalace by default (see R5). So the config *is*
templated at container start, not baked, and Phase 2 only adds a third branch. **Two residual
constraints replace the original unknown, and both are documented in §4.1:** (a) the script never
overwrites an existing config and `~/.config/opencode` is a *named volume*, so an env flip yields
only an `opencode.jsonc.proposed` sidecar — adopting or reverting the opt-in needs a manual merge
or a volume removal; (b) it no-ops entirely unless `OPENCODE_PROVIDER` is set. Prefer extending
this script over inventing a parallel mechanism — treat it as the reference implementation, and
check it before designing any pi-devbox-side mechanism with an opencode counterpart.
7. **Is a work VM the right long-term home for personal memory?** Accepted with eyes open for Phase 1
(§8.1): synlig wins on availability and connectivity, and the alternative — splitting by machine — was
rejected for sound reasons (§5). But the placement question survives the routing question: diaries are
`replicated` (decided), diaries are personal, and synlig is employer infrastructure. Revisit when
per-wing *targets* exist (Phase 2+), at which point a second personal primary becomes an additive
config change rather than a re-architecture. Until then, treat §6.1 threat 2 as live.
---
## 10. Evidence index
Re-verify without re-reading the package. Paths relative to
`/opt/uv-tools/mempalace/lib/python3.13/site-packages/mempalace/` (mempalace 3.6.0) unless noted.
| Claim | Where |
| --- | --- |
| Server auth/TLS/read-only | `cli.py:1448-1503`; `mcp_server.py:282-320`, `5205-5215`, `5284-5289` |
| Single global request lock | `mcp_server.py:5115`, `5346`, `5513` |
| KG path follows `--palace` flag only | `mcp_server.py:325`, `708-711`; `knowledge_graph.py:49` |
| KG DDL, no `(s,p,o)` uniqueness, open-fact guard | `knowledge_graph.py:163-178`, `305-313` |
| ID recipes | `ids.py:25`, `56`, `71`; diary at `mcp_server.py:3511-3514` |
| **Diary write has no idempotency probe** | `mcp_server.py:3546` (`col.add`, no `col.get`) vs `add_drawer`'s probe at `2593-2600` |
| Chunk probe is deliberate (all-or-nothing upsert) | `mcp_server.py:2586-2592` |
| Metadata is exhaustive — no session/PID/conversation field | `mcp_server.py:2578-2585` (drawers), `3527-3536` (diaries) |
| pi extension sets identity for **diaries only** | `extensions/pi/mempalace.ts:758` (`agent_name`); no `added_by` set anywhere in the file |
| Anti-rebinding Host/Origin checks | `mcp_server.py:5160-5194`, `5276-5290`, `5362-5367`; `cli.py:1450`. **Verified empirically** — runbook §2.4 |
| Stock defaults already split the KG | `config.py:221` (`~/.mempalace/palace`) vs `knowledge_graph.py:49` (`~/.mempalace/knowledge_graph.sqlite3`) |
| Server token path (stable across restarts) | `cli.py:_server_token_path` — `~/.mempalace/server/sha256(realpath(palace))[:24]/token` |
| pi-devbox non-destructive settings merge (pattern to port) | `pi-devbox/entrypoint-user.sh:131-162` (`jq -s '.[0] * .[1]'`, `.bak`, `PI_SETTINGS_MERGE=0`) |
| No `OPENCODE_CONFIG*` env layer upstream | absent from `https://opencode.ai/config.json`; zero hits in `opencode-devbox`, `myconfigs` |
| Tool classification (+ `kg_supersede` gap) | `service.py:29`, `54`, `72-84` |
| Write-routing seam | `write_routing.py:1-60` |
| Outbox prior art (queue DDL, token, retries) | `daemon.py` (`_init_db`, `ensure_token`, `MAX_ATTEMPTS`) |
| `sync` is destructive pruning | `sync.py:1-12` |
| WAL redacts + has no reader | `wal.py:74`, `_WAL_REDACT_KEYS` |
| Client-side embeddings | `backends/pgvector.py:6-8`; `embedding.py:6-11`; `_sidecar.py` |
| Dedup scoped by `source_file` | `dedup.py:get_source_groups` |
| Store location rules | `hallways.py:73,83`; `miner.py:701` |
| Mine lock (HOME-derived file) | `palace.py:1090`, `1123-1128` |
| Migrate as export primitive | `migrate.py:extract_drawers_from_sqlite`, `:326` |
| pi remote transport + fail-soft | `mempalace-toolkit/extensions/pi/mempalace.ts:11-12`, `87`, `108`, `390`, `629-641`, `665-673` (commit `96699f2`) |
| Server compose + port history | `pi-devbox/docker-compose.mempalace.yml`; `pi-devbox/CHANGELOG.md` v1.3.0; `docker-compose-repo/tor-ms22/pi-devbox/` (`df2c2ae`) |
| Existing opt-in convention (§1.2) | `pi-devbox/.env.example:12-23` ("MemPalace memory (local by default)", commented `MEMPALACE_REMOTE_URL`/`_TOKEN`); `pi-devbox/docker-compose.yml:79-83` (`devbox-palace` volume commented out) |
| opencode-devbox does not wire mempalace | `docker-compose-repo/{synlig,nyvaken,devbox-affection}/opencode-devbox/docker-compose.yml` — zero mempalace references in any |
## 11. See also
- [`ARCHITECTURE.md`](../ARCHITECTURE.md) — producer side (how the palace gets fed); §6 upstream roadmap
- [`extensions/pi/README.md`](../extensions/pi/README.md) — pi bridge internals
- [`SKILL.md`](../SKILL.md) — consumer-side protocol (search before answering, diary before exit)
Reference in New Issue View Git Blame Copy Permalink