Staging default moves out of ~/.cache to <palace-root>/pi-stage (pi) and <palace-root>/opencode-stage (opencode), resolved with mempalace's own palace-path precedence ($MEMPALACE_PALACE_PATH -> $MEMPAL_PALACE_PATH -> ~/.mempalace/config.json -> ~/.mempalace/palace), then dirname. Why: the convos miner keys dedup on the *staged* path, so a wiped stage plus a sync scoped to include it prunes the drawers mined from those sources -- deleting memories, not a cache. Under ~/.cache that state was reachable by anything treating a cache as disposable. Staging inside the palace makes the coupling structural: the stage cannot be wiped without touching the palace itself. Overrides ($MEMPALACE_PI_STAGE / $MEMPALACE_SESSION_STAGE, --stage) are unchanged. Note the old default had never been created on any host, so this closed a latent hazard, not a live one. Measured, and the docs now claim only this much: sync prunes only within the scope it is given -- wing-only, 1299 scanned / 1299 out of scope / 0 removed; scoped at the palace root, 651 kept / 648 out of scope. The previous blanket "sync prunes every drawer" wording overstated it, which is a liability: the next reader disproves the overstatement and discards the real constraint with it. Also in this change: - cron log dir ~/.cache/mempalace-session -> ~/.cache/mempalace-logs. The stage left that namespace, so the old name now read as "the stage". - AGENTS.md: the convos miner *does* check mtime (verified against upstream convo_miner.py); the previous "no mtime check" claim was wrong. - smoke-test assertions use `mktemp -d` for --sessions-dir. One pointed at /tmp, which still held earlier synthetic transcripts, so a --dry-run exported a fake session into the real stage: --dry-run skips the mine, not the export. docs/phase-1-exposure-runbook.md -- the newt/DNS/auth step that RFC 001 and the synlig runbook leave open (runbook section 4, items 2 and 5). Port 8765 at /mcp, newt targets 172.17.0.1, and the authentication is the single shared bearer token (RFC 6.2, decided 2026-08-09) rather than per-device proxy users. The latter cannot work today: mempalace validates exactly one token, and Pangolin's SSO/PIN/password are browser-shaped while every client here is a headless JSON-RPC POST -- enabling that protection breaks the clients it protects. The per-device axis that *does* exist is the feeder's SSH key + per-device inbox. New finding recorded there: a loopback bind does not merely 403 behind a tunnel (already known, runbook 2.4) -- it also silently starts the server with no token at all, because auto-minting is gated on the bind being non-loopback. extensions/pi/README.md: the HTTP transport IS authenticated as of mempalace 3.6.0; the "sessionless and unauthenticated" note dated from the v1.3.0 era. Closes the RFC section 8 Phase-0 hygiene item.
63 KiB
RFC 001 — Global palace with local fallback (mempalace-edge)
| Status | Draft — design agreed, not implemented |
| Created | 2026-08-08 |
| Applies to | mempalace 3.6.0, mempalace-toolkit @ 96699f2, pi-devbox ≥ v1.3.0 |
| Decision | Phases 0–2 + 4 in scope. Phase 3 (full pull replication) explicitly deferred — "a laptop that can reach its own stuff plus whatever it can reach" is good enough. Centralization is strictly opt-in — solitary devbox operation remains the default and must not change (§1.1). |
| Recon update | 2026-08-09 — §9 Q1 and Q6 are RESOLVED, both in the permissive direction (opencode supports remote MCP; opencode-devbox already templates the mempalace entry). Neither is a blocker. See those entries for evidence; §2, §4.1 and R5 were corrected accordingly. |
Read this first if you are asked to "centralize MemPalace" / "sync palaces between machines". Most of the hard-won facts below are non-obvious and two of them are actively destructive if you guess wrong (§7.1, §7.2). The evidence index in §10 lets you re-verify any claim without re-reading 45k lines.
1. The problem
One palace per machine per harness. Today: a pi-devbox container on EMB-7KJ4VR4G, another on
tor-ms22, opencode-devbox containers, MBP-M1-2020, plus native hosts — each with a private
~/.mempalace. Consequences:
- Memory is sharded by accident of where you happened to be working. A decision recorded on the laptop is invisible to the agent on the desktop.
- Container recreates are amnesia events unless the palace happens to be host-bind-mounted.
- The KG is the worst hit —
kg_query/kg_timelineanswers depend on which machine you ask.
Target state: one primary palace holds the fleet's memory. Every client keeps working when the primary is unreachable (writes buffer locally, reads degrade to local), and reconciles when it returns. Access is authenticated per device, with a sane authorization policy.
1.1 Hard requirement: solitary-first, centralization strictly opt-in
This is a requirement, not a preference, and it constrains every choice below.
Centralization solves a problem specific to this usage pattern: several machines and containers
(pi-devbox, opencode-devbox, native hosts) doing development and other work against a common set of
artifacts. For most users of the published joakimp/pi-devbox and joakimp/opencode-devbox images
it is useless — one machine, one palace, done. Evidence that this is already the norm: all three
sampled opencode-devbox deployments (docker-compose-repo/{synlig,nyvaken,devbox-affection}/)
contain zero mempalace references.
| ID | Requirement |
|---|---|
| R1 | Solitary operation stays the default and stays unchanged. With no MEMPALACE_* variables set, a devbox must behave exactly as it does today: harness → local mempalace-mcp over stdio, palace at ~/.mempalace. No extra process, no outbox, no network calls, no new failure mode, no measurable startup cost. |
| R2 | Opt-in lives in docker-compose.yml + .env — the mechanism users already know. No opt-in via image rebuild, no baked-in defaults, no latest-central variant. |
| R3 | Credentials only in .env (chmod 600, gitignored). Never in docker-compose.yml, never in the image, never on a command line (visible in ps), never logged. Compose passes them through with a ${VAR:-} empty default so solitary users never define them. |
| R4 | No new required services. The primary stays in the separate standalone docker-compose.mempalace.yml project; it is never merged into the main compose file. A solitary docker compose up starts exactly what it starts today. |
| R5 | Degrade, never fail, when mempalace is absent. The published opencode-devbox image does ship mempalace by default (Dockerfile.base:380,402 — ARG INSTALL_MEMPALACE=true, MEMPALACE_VERSION=3.6.0, installed via uv tool install), but it is a build arg precisely so it can be omitted to save ~300 MB — and a bind-mounted host config or a non-devbox client may have no mempalace either. So any wiring must stay additive and skip when the binary is missing — the probe-and-warn idiom install.sh already uses (warn + return 0, never halt). |
| R6 | Reversible. Commenting the .env lines out returns the container to pure solitary operation, with the local palace intact and readable. |
Acceptance test for R1 (must pass before Phase 2 ships): bring up a devbox with no MEMPALACE_*
variables; assert the palace tool list, drawer counts and diary writes are identical to the previous
image, and that no edge/outbox process exists (pgrep -f mempalace-edge → empty).
1.2 The opt-in surface
The existing convention is already the right one — extend it, do not invent a new one. pi-devbox's
.env.example lines 12–23 already ship a commented MEMPALACE_REMOTE_URL / MEMPALACE_REMOTE_TOKEN
pair under the heading "MemPalace memory (local by default)", and docker-compose.yml:79-83 already
ships the devbox-palace volume commented out. So the opt-in ladder becomes three states derived
from two variables — both existing behaviours are preserved, the third is new:
MEMPALACE_REMOTE_URL |
MEMPALACE_EDGE |
Behaviour | Status |
|---|---|---|---|
| unset | — | local stdio mempalace-mcp, palace at ~/.mempalace |
default, today, unchanged (R1) |
| set | unset/0 |
direct remote HTTP; no local palace, no offline | today (96699f2), unchanged |
| set | 1 |
mempalace-edge: local-first writes + outbox → primary + merged reads |
new (Phase 2) |
Volume coupling reverses under edge mode — document it prominently. Today
.env.examplecorrectly says that withMEMPALACE_REMOTE_URLset "the devbox-palace volume is then irrelevant", because the remote owns all state. Under edge mode that flips: the local palace holds the outbox and all local-first writes, so an un-persisted palace means losing un-flushed writes on container recreate. Opting intoMEMPALACE_EDGE=1therefore requires uncommentingdevbox-palace:/home/developer/.mempalace.
Corollary for §4.1: mempalace-edge must not be in the path unless opted in. Registering it
unconditionally (letting it decide by env at runtime) is tempting — one static config for every
harness — but it violates R1 by inserting a process and a failure mode into every solitary user's
setup. Selection must happen at registration time. See open question §9.6.
2. Verified starting point (as of 2026-08-08)
Two of the three pieces already exist. This is not greenfield.
| Piece | State | Evidence |
|---|---|---|
| Primary server | ✅ Exists. mempalace serve --host --port --token --tls-cert --tls-key --read-only --allow-insecure. Bearer token compared with hmac.compare_digest, mandatory on non-loopback binds (unless --allow-insecure), TLS 1.2+ resolved before bind, Host-header pinning + Origin allowlist (anti-DNS-rebinding), 16 MiB body cap, token-free /healthz. |
cli.py:cmd_serve (~1448); mcp_server.py:5205-5215, 5284-5289 |
| Remote client | ⚠️ Exists for pi only, and it is either/or. createClient() picks stdio or HTTP once at process start. |
extensions/pi/mempalace.ts:629-641 |
| Fallback + resync | ❌ Absent everywhere. On remote failure the pi bridge re-handshakes the same URL, then de-registers all palace tools and runs blind. | extensions/pi/mempalace.ts:665-673 |
| opencode client | ✅ Remote is supported and already wired. opencode's published schema (https://opencode.ai/config.json, $defs.McpRemoteConfig) makes {"type":"remote","url","headers","oauth"} a first-class sibling of McpLocalConfig, headers being a free string→string map (so bearer is a convention, not a constraint). opencode-devbox already emits exactly that entry when MEMPALACE_REMOTE_URL is set. The all-type:local configs in myconfigs are a deployment fact, not a capability limit. |
generate-config.py:107-118; hook at entrypoint-user.sh:117; schema $defs.McpRemoteConfig |
| Server compose | ✅ Exists: pi-devbox/docker-compose.mempalace.yml (canonical) + a tor-ms22 derivative (docker-compose-repo/tor-ms22/pi-devbox/, df2c2ae, port 8766, uid 1000, binds the real ~/.mempalace). Not enabled. |
pi-devbox CHANGELOG v1.3.0 (2026-07-02) |
Stale docs warning.
docker-compose.mempalace.yml,.env.exampleand CHANGELOG v1.3.0 all say the HTTP transport is unauthenticated and should be fronted by a reverse proxy. That was true formempalace-mcp --transport httpin the v1.3.0 era. mempalace 3.6.0'sservehas token + TLS built in (upstream #1877). Fix those comments during Phase 1. Specifically:pi-devbox/.env.example:21still advertisesmempalace-mcp --transport http --host 0.0.0.0 --port 8765as the way to serve a shared palace — replace withmempalace serve --token … --tls-cert …, and change the example URL fromhttp://mempalace.lan:8765/mcptohttps://.✅ Done 2026-08-12 —
pi-devbox/.env.exampleandmempalace-toolkit/extensions/pi/README.mdboth now recommendmempalace serve, bind docker0 rather than0.0.0.0/loopback, use anhttps://example URL, and state that the transport is authenticated.docker-compose.mempalace.ymlaudited too, and it was worse than stale — it was broken on 3.6.0 in both directions:--host 0.0.0.0with no token in the environment makes the server refuse to start (crash-looping underrestart: unless-stopped), and once a token is supplied the healthcheck's unauthenticatedtools/listPOST 401s, marking a healthy server unhealthy forever. Fixed: token now required via${MEMPALACE_REMOTE_TOKEN:?}(fails fast atup), healthcheck switched to the token-free/healthz.
Two things that sound like the feature and are not
sync.pyis not replication. It is gitignore-aware drawer deletion: "Removes drawers whose source files are now gitignored, deleted, or moved out of the project" (sync.py:1-12). See §7.2 — running it against a shared palace is a fleet-wide memory wipe.wal/write_log.jsonlis not a replayable WAL._WAL_REDACT_KEYSstripscontent/content_preview/document/entry/entry_preview/query/textand replaces them with"[REDACTED N chars]"; all 12 references are writers, there is no reader and noreplayfunction (wal.py:74). Reconstructing memory from it is information-theoretically impossible.
3. Why we sync operations, not databases
Row-level / file-level replication of a palace is a trap in this codebase. Four findings, each independently disqualifying:
-
Vectors are not portable. Embeddings are computed client-side —
backends/pgvector.py:6-8: "Embeddings are still produced locally by MemPalace through the core embedding wrapper before vectors are written to Postgres." Cross-architecture ONNX determinism (arm64 macOS vs x86-64 Linux, different execution providers) is not guaranteed. → Ship text + metadata, always re-embed on receipt. Cheap and safe: the raw text is always stored as the chroma document. -
KG rows cannot be copied.
tripleshas noUNIQUE(subject,predicate,object,valid_from)— onlyidis unique, andmake_triple_idembedsdatetime.now(). A row copy therefore duplicates every fact. Butadd_triple()guards at the application level (SELECT id … WHERE subject=? AND predicate=? AND object=? AND valid_to IS NULL→ returns the existing id), so replayingkg_addis idempotent for open facts (knowledge_graph.py:163-178,305-313). ⚠️ Scoped to open facts only (verified 2026-08-09). The guard'sWHERE … valid_to IS NULLmeans an already-closed historical fact — one written withvalid_to, or closed later bykg_invalidate/kg_supersede— has no guard at all, so replaying it inserts a duplicate row every time. A bootstrap that replays full KG history rather than just currently-open facts must dedupe closed facts client-side on(s,p,o,valid_from,valid_to)(§4.4). -
Only one write path has deterministic IDs.
Write path ID recipe Same content on 2 hosts → same ID? add_drawer/checkpointdrawer_{wing}_{room}_{sha256(wing|room|content)[:24]}YES — content-addressed, merges for free project/format miner …sha256(source_file|chunk_index)NO — absolute path ( /Users/joakim/xvs/workspace/x)convo miner …sha256(source_file|extract_mode|chunk_index)NO — same reason diary diary_{wing}_{YYYYmmdd_HHMMSSffffff}_{sha256(entry)[:12]}NO (µs timestamp) — and the write is a bare col.addwith no pre-write probe at all (mcp_server.py:3546), unlikeadd_drawer. The 12-hex suffix is the only usable content dedup key, and it must be applied client-side (§7.6)KG triple t_{s}_{p}_{o}_{sha256(valid_from|recorded_at)[:12]}NO ( recorded_at = now())(
ids.py:56,71—ID_RECIPE = "v3", length-prefixed delimited hashing.) -
Batch dedup will not save a naive merge.
dedup.pygroups bysource_fileand only compares within a group (cosine < 0.15) → cross-host duplicates with differing paths are never compared. The mechanism that does work is write-time:tool_check_duplicate(content, threshold=0.9)queries the whole collection, andmempalace_checkpointalready runs it per item.
Conclusion: the MCP tool surface is already a small, coarse-grained, mostly-idempotent operation vocabulary. Log the intent, replay the intent. That sidesteps chroma internals, chunking, embedder drift and vector portability in one move.
One exception, and it is not cosmetic: diary_write has no idempotency guard whatsoever, so for
diaries "replay the intent" duplicates rather than merges (§7.6). Every other write path either
content-addresses or guards. §4.4 carries the per-record-type dedup keys a joining client must
therefore bring with it.
4. Design: mempalace-edge, a local MCP proxy
4.1 Shape
Do not put fallback logic in extensions/pi/mempalace.ts — opencode would need it again — not
because opencode lacks remote MCP (it has it, §9.1), but because offline-first writes are a client
concern that every harness would otherwise reimplement. Instead: a sidecar that is an MCP server.
pi (mempalace.ts, stdio) ─┐
opencode (mcp: type=local) ─┼──► mempalace-edge ──HTTPS MCP──► PRIMARY
mempalace CLI (local palace) ─┘ │ (stdio MCP server) mempalace serve
├─ child: mempalace-mcp --token --tls-cert
│ (local palace, always writable)
└─ outbox.sqlite (durable op queue)
mempalace-edge needs zero mempalace internals — it is an MCP-to-MCP proxy. It speaks stdio down
to a child mempalace-mcp and HTTPS up to the primary, reusing the RemoteMcpClient already written
in 96699f2 (extensions/pi/mempalace.ts:390, vendored from pi-extensions/mcp-loader.ts — mind the
MCP-STREAMABLE-HTTP-CLIENT-SYNC: v1 drift token if you copy it again).
Why this shape wins:
- One implementation for every harness. opencode's change is one line:
"command": ["mempalace-edge"]. pi's is one line. The CLI is untouched. - Tools never vanish from the tool list, so wake-up context injection keeps working offline — unlike today's fail-soft de-registration.
- Survives mempalace upgrades: coupled to the tool schema, not to chroma/HNSW.
- The remote transport is sessionless JSON-RPC today (per
96699f2's own note), so reconnect after an outage is cheap — there is no session to re-establish.
But per R1, that one-line change is conditional, not baked in. The edge binary is present in the
image (it costs nothing unused) and inserted into the path only when MEMPALACE_EDGE=1:
-
pi:
createClient()(extensions/pi/mempalace.ts:629) already branches on env at startup — add a third branch. Zero change to the default path. -
opencode: the MCP server entry is static JSON — but the templating step already exists and already implements the first two rungs of the §1.2 ladder.
rootfs/usr/local/lib/opencode-devbox/generate-config.py(run unconditionally fromentrypoint-user.sh:117) registersmempalaceas{"type":"remote", url, headers:{Authorization: Bearer …}}whenMEMPALACE_REMOTE_URLis set, else{"type":"local","command":["mempalace-mcp"]}when the binary is on PATH, else nothing — R5-compliant already. Its own comment states it uses the "same env contract as the mempalace.ts pi extension … so one shared MemPalace can serve pi + opencode + native", i.e. the two images are deliberately kept in step. Phase 2's opencode work is therefore a third branch in an existing script, not a new mechanism.⚠️ But the opt-in does not propagate to an existing container.
generate-config.pynever overwrites an existing config, and~/.config/opencodeis the named volumedevbox-opencode-config(docker-compose.yml:64,153) — so a config generated during solitary use survives recreate, and later settingMEMPALACE_REMOTE_URL/MEMPALACE_EDGEonly produces a non-loadedopencode.jsonc.proposedsidecar for manual merge (generate-config.py:203-244). Flipping the.envalone is a no-op there. Phase 1/2 docs must say: merge the sidecar, ordocker volume rmthe config volume, to adopt the change — and the same applies in reverse for R6 (reversibility). This asymmetry with pi — whosecreateClient()re-reads env every start — is the single biggest behavioural difference between the two harnesses under this RFC.✅ Fix shape (decided 2026-08-09): make the
mcp.mempalacesubtree env-authoritative. Four verified inputs:- pi is immune for a structural reason worth naming: its MemPalace wiring is code, not
config —
createClient()reads env at every startup, with no generated file in the path. pi's ownsettings.jsondoes sit on a preserved volume and does go stale, and pi-devbox already solved that properly atpi-devbox/entrypoint-user.sh:131-162:jq -s '.[0] * .[1]'deep-merge with template first, live second so the user's values always win and only missing keys are filled; arrays treated as leaves (a deliberately removed model is not re-added); rewrite only when the merge changes something; timestamped.bakfirst;PI_SETTINGS_MERGE=0to disable; invalid JSON on either side → skip, never clobber. That is the pattern to port. - The precedence must be inverted for this one subtree. pi's "live wins" is right for adding
new keys and wrong for a changed env value — and a changed
MEMPALACE_REMOTE_URLis the whole problem. Somcp.mempalaceneeds env-wins, which is only safe with a fingerprint: store a hash of what was last auto-generated and refresh only while the live value still matches it; otherwise fall back to the.proposedsidecar for that key.write_proposed(generate-config.py:203-244) already diffs rendered config against the live file, so this is an extension of existing logic at narrower granularity, not new machinery. - No higher-precedence layer exists to hide in. There is no
OPENCODE_CONFIG*env override in opencode's published schema (or anywhere inopencode-devbox/myconfigs), and MCP registration is global rather than project-scoped, so the real file must be written. - The fingerprint is the one genuinely new mechanism — neither repo has a managed-marker
convention for JSON; the only prior art is the markdown
<!-- pi-devbox:managed-block -->idiom.
This is a self-contained opencode-devbox change, independent of the rest of this RFC, and it is what turns Phase 1 into "flip
.env, restart" for both harnesses — hence Phase 1.5 in §8. - pi is immune for a structural reason worth naming: its MemPalace wiring is code, not
config —
4.2 Routing policy
service.py:29,54,72 already ships the exact three-way split we need — READ_TOOLS,
WRITE_TOOLS, MAINTENANCE_TOOLS ({mine, sync, reconnect}) — plus classify_tool().
| Class | Primary up | Primary down |
|---|---|---|
read (search, kg_query, diary_read, list_*, traverse…) |
query both, merge ranked lists, dedupe by drawer id | local only, response flagged degraded: true |
write (add_drawer, checkpoint, diary_write, kg_add…) |
apply local and enqueue → primary | apply local, enqueue, keep working |
maintenance (mine, sync, reconnect) |
never proxied — local only (§5, §7.2) | local only |
destructive (delete_drawer, delete_by_source, delete_tunnel…) |
admin-scoped only; tombstone, never hard-delete remotely | local only |
⚠️
classify_tool()must be treated as fail-closed.mempalace_kg_supersedeis a real MCP tool (3 references inmcp_server.py) but is absent fromWRITE_TOOLS→classify_tool()returns"unknown"for it. An edge proxy that routed "unknown" as read would silently drop supersede operations. Maintain our own tool→class table, default unknown ⇒ write, and file the upstream fix.
The merged-read trick is what makes Phase 3 optional. Both palaces return ranked results with distances; if the embedder identity matches (§7.4) the distances are comparable, so a union + re-sort + dedupe-by-id gives a correct combined result set. You get fleet-wide recall without replicating anything. Offline you simply see less.
KG reads are the exception: unioning triples is unsafe because invalidate/supersede are
order-dependent. Use primary-first, local-fallback (no union) for kg_* reads, and accept that an
offline KG answer is incomplete.
4.3 Outbox
Model it on daemon.py, which already implements a durable local job queue and is the closest
existing prior art (token auth via ensure_token, 0600, per-palace state dir, MAX_ATTEMPTS = 3,
recover_running, JOB_RETENTION_DAYS = 7). Its key primitive is worth copying verbatim:
CREATE UNIQUE INDEX IF NOT EXISTS idx_jobs_dedupe_active
ON jobs(dedupe_key) WHERE state IN ('queued', 'running');
A TOCTOU-safe, cross-process "at most one active job per key". Our outbox rows:
| Column | Purpose |
|---|---|
op_id |
sha256(origin_device, local_seq, tool_name, canonical_payload) — stable across retries, the primary's idempotency key |
local_seq |
Per-device monotonic counter → causal order for kg_invalidate/supersede replay |
origin_device, origin_label |
Provenance, stamped by edge (interim) or the primary (authoritative) — never by the agent (§7.3) |
tool_name, payload_json |
The MCP call to replay |
state, attempts, created_at, flushed_at, error_json |
Retry/audit |
Flush = drain in local_seq order, stop on first hard failure (preserve ordering), exponential
backoff, resume on reconnect. The primary de-dupes on op_id; add_drawer's own pre-write probe and
add_triple's open-fact guard make replay safe even if op_id tracking is lost.
write_routing.py is the upstream-shaped seam for this if we ever want it in core: its
WriteRoutingPolicy{DIRECT,PREFER,REQUIRE} → WriteRoutingTarget{DIRECT,DAEMON,BLOCKED} enums would
gain a REMOTE target. The module says it "changes no caller defaults by itself" — i.e. it exists
precisely to be extended.
4.4 Joining an existing primary (bootstrap)
Framing correction (2026-08-09). This is not "seed the primary from one chosen palace". Every container joins the same way, at any time, repeatedly — so a join is idempotent replay of local history, and the only real question per record type is what dedupes it.
| Record type | Dedupe on replay | Client work needed |
|---|---|---|
add_drawer / checkpoint drawers |
Server-side: content-addressed id + pre-write col.get probe → {"reason":"already_exists"}, no write (mcp_server.py:2593-2600) |
None. Just replay |
| KG open facts | Server-side: add_triple guard on (s,p,o) WHERE valid_to IS NULL |
None. Just replay |
| KG closed facts | None — the guard is scoped to open facts (§3.2) | Dedupe on (s,p,o,valid_from,valid_to) before sending |
| Diary entries | None whatsoever (§7.6) | Skip any local entry whose sha256(entry)[:12] suffix already exists remotely |
| Mined drawers | id is path-dependent → same content from two hosts = two rows | Out of scope: mined wings are local (§5) |
Existence checks available today, with no new server code:
| Tool | Kind | Fit |
|---|---|---|
get_drawer(id) |
exact id, clean not-found (mcp_server.py:3103-3114) |
The right check wherever ids are deterministic |
list_drawers(wing, room, since, before) |
metadata page | Bulk "what does this wing already hold" — the cheap way to collect existing diary id suffixes. Note since/before filter in Python, not in the backend where (chroma 1.5.7 rejects string $gte/$lt) |
check_duplicate(content, threshold) |
semantic, whole collection, no wing/room scoping | One embed + one HNSW query per call → the cost driver at thousands-of-records scale. Reserve it for diaries, where nothing cheaper works |
kg_query |
fact lookup | Redundant for open facts (the guard covers them); useful for closed ones |
Two containers on one host are the easy case, not the hard one. They share a bind-mounted palace, so it is one palace joining once, and content-addressing makes even a concurrent double-join harmless for everything except diaries. Belt and braces:
- Keep join state in the shared palace, not in the container — e.g.
<palace>/edge/bootstrap.jsonholding{target_url: {joined_at, high_water_local_seq}}. Both containers then see "this palace has already joined", a recreate does not repeat the work, and it is the same durable-state-next-to-the-data pattern as the outbox (§4.3). It must not live in a container-only path:~/.mempalaceis not preserved by default for solitary users (§1.2), which is exactly why the file belongs to the palace directory. - First join is a dry run. Bootstrap one palace, verify counts (
status,kg_stats, per-winglist_drawers) against expectations, then let the rest join. Ordering matters only because of diaries and closed facts; everything else is order-free.
The genuinely hard case is two different palaces holding overlapping mined content — the same repo
mined on a laptop and a workstation under different absolute paths. §5 excludes it by keeping mined wings
local; that exclusion is load-bearing, not tidiness.
5. Not everything should be global
Per-wing replication policy, declared in edge config:
| Policy | Wings | Rationale |
|---|---|---|
replicated |
diaries, wing_pi, checkpoints, KG, hand-authored notes |
Small, curated, content-addressed → already merge-safe |
local |
mined code/docs (e.g. the 13,597-drawer workspace wing on this host) |
Derived data, re-mineable from git, and carries exactly the path-dependent IDs that break merging (§3.3) |
This shrinks the hard problem to the layer that is already safe, and it is why mine must never be
remote: the primary serializes every request behind one lock (§7.5), so a bulk remote mine would
stall every other agent in the fleet.
Decided 2026-08-09: diaries are replicated. Cross-machine continuity is the entire point, and
diaries are the highest-value content in the palace to share ("given their value, I say go with (a)").
Accepted consequence, stated plainly because it follows from the primary being synlig, a work VM
(§8.1): personal diaries will live on employer infrastructure. They already quote internal hostnames,
paths, moods and the occasional token prefix (§6.1 threat 2) — so "don't put secrets in memory" stops
being advice and becomes a precondition.
The work/personal boundary is a property of the wing, not of the device. Rejected design
(2026-08-09): splitting the fleet into a work primary and a personal primary along machine lines. The
reasoning is decisive — pi-devbox/opencode-devbox are simultaneously work and home projects, and
personal machines get used for work-adjacent work, so the device where the work happened cannot
classify the project. That is precisely the axis this table already encodes, which is why
replicated/local per wing is the right knob and a second primary is not needed to express it.
Shape the config so multiple stores stay possible without paying for them now. Phase 1 keeps
MEMPALACE_REMOTE_URL a scalar (one primary). Per-wing targets — this same policy column, plus a
destination — belong to the edge config in Phase 2+, so nothing in the .env contract has to be
un-designed later.
6. Security model
The transport is largely solved; the gap is authorization, not cryptography.
6.1 Threats, in priority order
- Memory poisoning / persistent cross-machine prompt injection — the underrated one. Palace content is injected into agent context at wake-up. A shared palace means one careless or compromised container can plant instructions that every other agent in the fleet reads as trusted memory. Mitigation: server-stamped provenance on every synced record (§7.3.2), don't auto-inject wake-up content authored by devices outside a trusted set, keep an admin-only wing for anything instruction-shaped.
- Aggregation raises exfiltration impact. One primary holds every machine's diaries — which already quote internal hostnames, paths and token prefixes. Mitigation: per-wing ACL; don't put secrets in memory (the WAL redaction list exists for a reason); private-network-only exposure.
- Accidental mass deletion by a client (
sync,delete_by_source) — see §7.2. - Availability: the server is single-writer by design; one long operation blocks everyone.
6.2 Policy
| Control | Decision |
|---|---|
| Network posture | Primary never internet-exposed. Publish only onto the private overlay / existing tunnel (Pangolin/newt). ⚠️ The reflexive "bind loopback in the container" is the failure mode here — see Transport. |
| Transport | Decided 2026-08-09: terminate TLS in the existing Pangolin/newt tunnel, not in serve (cheaper than patching Python's TLS surface; one DNS record per service on the web hotel is the whole setup cost). ✅ Host/Origin policy verified by experiment on synlig 2026-08-10, 11/11 as predicted (runbook §2.4) — the summary: do not bind loopback behind the tunnel. enforce_host_pin = _http_is_loopback(host) (mcp_server.py:5367), so a loopback bind + a proxy forwarding Host: palace.example.com → 403, while a non-loopback bind → 200 (pin deliberately relaxed, "may sit behind a proxy that rewrites Host", :5362-5365); tokenless non-loopback binds require --allow-insecure (cli.py:1450). Prefer binding the docker0 gateway (e.g. 172.17.0.1) over 0.0.0.0: non-loopback, so the pin relaxes, yet reachable only from the host and its containers — so a newt container on the box reaches it and the LAN cannot. The Origin check is never relaxed: absent Origin is fine (every non-browser MCP client, incl. pi and opencode), a present non-loopback Origin is 403 with no override — so keep browser-based clients and Origin-injecting proxies out of the path. /healthz is Host/Origin-gated but token-free, so it works as the tunnel's liveness probe. |
| Authentication | Target: per-device bearer tokens with a server-side token → {device_id, scopes} registry — revoke one laptop, rotate without a fleet outage. Decided 2026-08-09: Phase 1 ships the single shared token ("iterate more feature rich but more complex solutions over time"), so per-device lands with Phase 4. Store in the existing .env.age flow, 0600 on disk. Consequence: until then the primary cannot tell devices apart, so origin_device stays client-asserted and advisory — nothing load-bearing may depend on it (§7.3.2). |
| Authorization | Per-device read/write wing globs. Server-side refusal of mine/sync/delete_* except for an admin device. --read-only gives a free observer tier. |
| Provenance/audit | origin_device (+ optional origin_label) + op_id on every record, stamped server-side from the authenticated credential — a client-asserted origin is a hint, not a fact (§7.3.2); server-side append-only op log. |
| Recovery | Deletes as tombstones; server-side backups with backups.py retention (MEMPALACE_MAX_BACKUPS). |
| DoS | Keep the 16 MiB cap; add rate limiting; no remote mine. |
7. Landmines — Phase 0 runbook
Do these before any cutover. 7.1 and 7.2 cause silent data loss; 7.6 causes silent duplication the first time a palace joins (§4.4).
7.1 MEMPALACE_PALACE_PATH ≠ --palace (silent empty KG)
# mcp_server.py:708-711 (_palace_flag_given = bool(_args.palace), line 325)
def _resolve_kg_path() -> str:
if _palace_flag_given:
return os.path.join(_config.palace_path, "knowledge_graph.sqlite3")
return DEFAULT_KG_PATH # knowledge_graph.py:49 → ~/.mempalace/knowledge_graph.sqlite3
cmd_serve always passes --palace. So the moment you start the server, the KG becomes
<palace>/knowledge_graph.sqlite3 — a different file from the live ~/.mempalace/knowledge_graph.sqlite3.
The vector store looks fine and the knowledge graph is silently empty.
Four stores, three location rules:
| Store | Location rule |
|---|---|
| drawers | backend-abstracted (BaseCollection) |
knowledge_graph.sqlite3 |
HOME-anchored unless --palace flag (knowledge_graph.py:49) |
hallways.json |
derived from palace_path, with legacy HOME fallback (hallways.py:73,83) |
known_entities.json |
HOME-anchored (miner.py:701) |
Action: mv all three files into the palace directory before first serve, and verify
kg_stats is non-empty afterwards. Corollary: a shared pgvector/qdrant backend shares drawers and
nothing else — it would leave the KG as fragmented as it is today. That is why the shared-backend
option is not the answer.
⚠️ Understated above, corrected 2026-08-10 while provisioning synlig: this is not a migration hazard,
it is the stock-default behaviour, and it is permanent. DEFAULT_PALACE_PATH is
~/.mempalace/palace (config.py:221) while DEFAULT_KG_PATH is
~/.mempalace/knowledge_graph.sqlite3 (knowledge_graph.py:49) — those differ out of the box, with no
custom path involved. So on every host, forever: serve (always passes --palace) uses the KG inside
the palace, while any CLI command run without --palace uses the HOME one. A one-time mv does not fix
that; it just relocates which of the two files is populated.
Better action — converge the two resolution rules onto one inode (and the only option on a greenfield primary, where there is nothing to move):
ln -sfn palace/knowledge_graph.sqlite3 ~/.mempalace/knowledge_graph.sqlite3
ln -sfn palace/known_entities.json ~/.mempalace/known_entities.json
# hallways.json deliberately NOT symlinked: already palace-derived, and its HOME
# path is a warning-only legacy probe that never auto-migrates (hallways.py:73-95)
Verified on synlig 2026-08-10, because the WAL behaviour was the load-bearing assumption: a dangling
symlink is created on first sqlite3.connect; -wal/-shm land next to the target (inside the palace
dir, so the palace stays a self-contained backup/bind-mount unit) and not beside the symlink; a write
through one path reads back through the other; same inode. Full record in
synlig-primary-runbook.md §2.3.
7.2 Never run mempalace sync against a shared palace
It classifies drawers whose source files are absent on the running host as orphans and deletes
them; _auto_detect_project_roots guesses roots from drawer metadata, so the blast radius is
data-dependent rather than obvious. From a laptop that lacks the repos, it is a fleet-wide wipe.
Action: edge blocks mempalace_sync from ever reaching the primary; document it; consider an
upstream --refuse-shared guard.
7.3 Provenance belongs to the sync boundary — not to the agent, and not to a solitary container
Today every drawer carries exactly {wing, room, source_file, added_by, filed_at, id_recipe}
(+chunk_index, parent_drawer_id); diaries add {hall, topic, type, agent, date}. added_by/agent
is the agent name (pi, mcp, checkpoint) — never the machine. So in a merged store you cannot
tell which host wrote a record, cannot audit, and cannot compute per-device high-water marks.
Verified 2026-08-09 (mcp_server.py:2578-2585 drawers, 3527-3536 diaries): those lists are
exhaustive — there is no session, PID or conversation field either. Two consequences, because both are
natural questions:
- Two concurrent pi sessions (the tmux pattern pi's own docs suggest) writing to one palace are indistinguishable. Nothing records which session produced which record.
- pi vs opencode is only accidentally distinguishable.
added_byis a free-form optional string (default"mcp";checkpointresolves explicit arg → diaryagent_name→"checkpoint"), andextensions/pi/mempalace.tsnever sets it foradd_drawer/checkpoint— it sets identity only for diaries (agent_namefrom$MEMPALACE_AGENT_NAME, defaultpi,:758).kg_addhas no attribution field at all. So the only real harness attribution today is the diary wing, and for drawers the value is whatever string an LLM happened to pass.
Design consequence: device + agent, never session. Provenance has exactly three consumers — poisoning
triage ("which box planted this?"), per-device high-water marks, and revocation — and none of them needs
session granularity; adding it would put a field on every record with no reader. Under §6's per-device
tokens, provenance granularity equals token granularity: one token per host makes two containers on
that host a single origin, while a token per container separates them. §7.3.4's <agent>/<label>/<device>
encoding keeps the two axes orthogonal, so pi-vs-opencode stays recoverable — but only if the edge
populates the agent segment, never the LLM.
Earlier drafts of this section said "stamp it everywhere, now, because it cannot be backfilled." That was wrong on both counts. See §7.3.3.
7.3.1 What can actually be stamped (mechanics)
The metadata schema is fixed — there is no free-form field — and tools/call whitelists arguments
to declared schema properties (mcp_server.py:4777, "Prevents callers from spoofing internal params
like added_by/source_file"), so an extra origin_host=… is silently dropped, not rejected.
| Surface | Provenance slot | Notes |
|---|---|---|
add_drawer, checkpoint |
added_by |
The only one. Free-form (strip_lone_surrogates only, not sanitize_name, so / and @ are legal). |
diary_write |
none usable | ⚠️ Never put a device in agent_name: :3504 does wing = f"wing_{agent_name}" → a separate wing per host, and diary_read filters {"agent": agent_name} (:3636) → diary_read("pi") then misses those entries. |
kg_add |
none | Only source_file/source_closet/source_drawer_id. Origin is inferable only via source_drawer_id → that drawer's added_by. |
added_by is also write-only today: absent from tool_search results, surfacing only via
get_drawer → _drawer_payload metadata. → Upstream asks: surface added_by in search results, and
give kg_add a provenance field.
7.3.2 Who should stamp it — a ladder of trust
Provenance answers "which device asserted this?" Only something that can verify the answer should write it. Ranked by trustworthiness:
| Stamper | Knows the device? | Verifiable? | Uniform? | Verdict |
|---|---|---|---|---|
| Agent (via skill) | No — must shell out to read env | No | No — per-call boilerplate, forgettable, improvisable | ❌ Worst possible place. Rejected. |
Client / mempalace-edge |
Yes, from host-supplied .env |
No — self-asserted | Yes — one line in a proxy | ⚠️ Acceptable interim |
| Primary, from the authenticated credential | Yes | Yes — bound to the token | Yes, for every synced record | ✅ Correct home |
The decisive point: a client-asserted origin is a hint, not a fact. The primary is the only party that can bind a write to an identity it verified. And under the §4 design every write reaches the primary through an authenticated channel — including offline ones, at outbox-flush time — so the server can stamp the complete set without any client cooperation. Provenance is a property of the sync channel, not of the record's author.
Blocker for the ✅ row, verified in 3.6.0: serve takes a single shared bearer token
(srv.auth_token, hmac.compare_digest, mcp_server.py:5291-5293) and the package contains zero
occurrences of any device/origin concept. Server-side stamping therefore requires the per-device
credentials of §6 — i.e. Phase 4, not Phase 1. Hence the phasing:
- Phase 2 (edge): edge may stamp
added_byfrom its configured env — self-asserted, advisory only, never load-bearing for authorization or destructive scoping. - Phase 4 (authz): per-device tokens land; the primary stamps authoritatively and the client-supplied value becomes redundant (and must be treated as untrusted input, not merely ignored).
7.3.3 Solitary containers should stamp nothing — and lose nothing by it
A pi-devbox or opencode-devbox running solitarily is a single-origin store by definition. Origin is
therefore a property of the whole palace, not of each record — so it can be assigned wholesale at
the moment the store stops being solitary: one --origin-device flag on the import/first-sync path
attributes every record from that palace to that device.
That dissolves the "impossible to backfill" argument. Per-record stamping is only necessary once records from multiple origins are interleaved in one store, which is exactly and only the primary. So:
- Solitary devboxes: no stamping, no config, no skill instruction, no behaviour change. This is strictly more compliant with R1 than the earlier draft, which quietly asked every solitary user to carry metadata for a feature they had opted out of.
- Migration is unaffected: bulk attribution at import is more reliable than per-record stamping, because it cannot be partially applied.
- Multi-harness on one host is solved in principle by
added_by= agent name (pivsopencode) — that is what the field is for, and it needs no device component. ⚠️ Corrected 2026-08-09: nothing actually sets it. The pi extension leavesadded_byat its default foradd_drawer/checkpoint, so in practice the value is whatever an LLM passed. The field is the right home; the client-side write that populates it is missing, and it belongs to the edge (the ⚠️ row of §7.3.2) — not to a skill instruction. - A palace on a shared host bind-mount (as tor-ms22's compose does) is still single-device under the "host owns the palace" model, so it too imports as one origin.
The one case bulk attribution cannot fix: a palace that was already merged from several devices without stamps. Preventing that is precisely why the primary must stamp from day one of Phase 4 — it is the only store where interleaving occurs.
7.3.4 Identity fields, when a stamper does exist
For the edge (interim) and the primary (authoritative) — never for agents:
| Field | Source | Rule |
|---|---|---|
origin_device |
uuidgen once, stored in the host's .env; later, issued with the device's token |
Opaque, stable, collision-free by construction. Compared, never parsed. |
origin_label |
hostname, same .env |
Human readability only. Never identity, uniqueness or scoping. Free to change. Optional — a hostname can leak an asset tag or username (EMB-7KJ4VR4G, HOST_SSH_USER=ECSJPER). |
Why not the obvious sources: a container cannot discover its host's identity. hostname returns the
container ID (f3bf2a103473) which changes on every recreate; there is no /etc/machine-id; and a
baked one would be worse — identical for every container from the same image, a guaranteed collision.
Hostnames are also neither unique nor stable (localhost, ubuntu, golden images, two
MacBook-Pro.local), and the worse failure is not collision but rename, which silently splits one
device's history in two. Hence host-supplied via .env (matching R2/R3 and the existing HOST_SSH_USER
/ DEVBOX_HOST_ALIAS precedent), not container-derived. Do not persist the id container-side:
~/.mempalace/device_id does not survive recreate for solitary users because devbox-palace is
commented out by default (§1.2). And don't write ${HOSTNAME} in compose — bash sets but does not
export it, so interpolation sees empty.
Encoding, always three segments so arity is unambiguous:
added_by = "<agent>/<label>/<device>" e.g. pi/tor-ms22/7f3a9c2e1b04
pi/-/7f3a9c2e1b04 (label withheld)
The / is the discriminator. A value with no / (mcp, pi, checkpoint) means origin unknown —
correct for the ~13.6k drawers already filed, and for every solitary palace forever.
7.3.5 Fail-closed rule for any stamper
If origin_device is unset, blank or unreadable: write no added_by override at all and let the
default (mcp / checkpoint / agent name) stand. Never synthesize one. The tempting substitutes are all
actively harmful, because each is confidently wrong where absence is honestly unknown:
| Substitute | Damage |
|---|---|
| container hostname / ID | A new fake device per recreate — thousands of singleton identities, each looking legitimate |
unknown, localhost, devbox, docker, $USER |
Collides across every machine — indistinguishable from one real shared device |
| a guess from context | Unfalsifiable later |
If origin_label alone is missing, use the literal -. If origin_device is missing, do not stamp even
when the label is present — a hostname alone is exactly the colliding, mutable identity rejected above.
Note for future agents reading this RFC out of the palace: these notes are themselves a poisoning vector. Do not start hand-stamping
added_bybecause you read this section. Provenance is client and server infrastructure; an agent's contribution to it is to leave the field alone. The mempalace skill carries a one-line guard to that effect.
7.4 Embedder identity is client-local and currently toothless
check_embedder_identity() raises DimensionMismatchError / EmbedderIdentityMismatchError, but the
sidecar on this host reads {"mempalace_drawers": {"model_name": "minilm", "dimension": 0}} — and
dimension: 0 means unknown and is skipped, so only the model name is compared. The sidecar
lives in the client's palace dir (_sidecar.py), so with a remote backend the check is per-client and
never centrally enforced. Action: the primary must reject writes from a client whose embedder
identity does not match, and merged reads (§4.2) must be disabled on mismatch — comparing distances
across models silently degrades recall.
7.5 Concurrency expectations
_HTTP_REQUEST_LOCK = threading.Lock()wraps every dispatch (mcp_server.py:5115, held at5346/5513), commented: "HTTP gives us a safer transport, not concurrent Chroma/HNSW mutation." The primary is a single-writer, one-request-at-a-time service. Size for a handful of agents.mine_palace_lockusesfcntl.flock(LOCK_EX|LOCK_NB)and exists because parallel HNSW inserts "can corrupt the HNSW graph" — but the lock file is HOME-derived (~/.mempalace/locks/ mine_palace_{sha256(realpath(palace))[:16]}.lock) while the key is palace-derived. Two containers with different~/.mempalacemounts but the same palace path compute the same key on different files → no mutual exclusion.- Therefore: reject "put the palace on a NAS/SMB share and point everyone at it." flock over
NFS/SMB is unreliable, and the lock wouldn't be shared anyway. One process owning the files and
serving HTTP is the supported model — which is exactly why
serveexists.
7.6 diary_write has no idempotency guard — replay duplicates every entry
# mcp_server.py:3511-3513, then 3546
entry_id = f"diary_{wing}_{now:%Y%m%d_%H%M%S%f}_{hashlib.sha256(entry.encode()).hexdigest()[:12]}"
...
col.add(ids=[entry_id], ...) # not upsert — and no col.get probe anywhere in tool_diary_write
Compare add_drawer 900 lines earlier, which probes col.get(ids=idempotency_probe_ids) and returns
{"reason": "already_exists"} without writing (:2593-2600). The contrast is the finding: the µs
timestamp makes every diary id unique by construction, and nothing checks the content. So §3's
conclusion — log the intent, replay the intent — is false for diaries; for them, replay is
duplication.
This matters more than it sounds, because diaries are replicated (§5) and are precisely the content a
join replays (§4.4).
Actions:
- Client-side, now: before replay, list the target's diary drawers for the wing, extract the 12-hex
sha256(entry)[:12]suffix from each existing id, and skip any local entry whose suffix already appears. The suffix is already content-addressed — no new key needs inventing. - Upstream ask: make
diary_writeprobe-and-skip on that suffix the wayadd_drawerdoes on its content hash. A small, self-contained patch in the sibling function's own idiom, and it closes the gap for every future writer instead of for one migration script. - Do not "fix" this by making the agent dedupe. An agent cannot know what the primary holds without a call, and per-call boilerplate is exactly the failure mode §7.3.2 rejects.
8. Phasing
| Phase | Effort | Deliverable |
|---|---|---|
| 0 — hygiene | hours | §7 runbook: converge the KG/entities store paths on synlig before first serve (§7.1 — done 2026-08-10, runbook §2.3), ban sync on shared palaces (§7.2), pi-devbox/.env.example:21)docker-compose.mempalace.yml turned out to be outright broken on 3.6.0, now fixed. Added 2026-08-09: settle the diary dedup approach and file its upstream ask (§7.6), and dry-run the join from one palace only (§4.4). No provenance work here — it is not backfill-critical (§7.3.3) and belongs to the stamper, not the agent |
| 1 — primary up | hours, no code | mempalace serve --token --tls-cert on a private-net host (reuse docker-compose.mempalace.yml — keep it a separate standalone project, R4 — and mind port 8765 vs pi-studio; tor-ms22 already moved to 8766). Repoint pi clients via MEMPALACE_REMOTE_URL/MEMPALACE_REMOTE_TOKEN. opencode clients can be repointed in the same breath — remote MCP is supported and generate-config.py already emits it (§9.1, §9.6), subject to the sidecar caveat in §4.1. Shared memory today, no offline. Decided 2026-08-09: primary = synlig, TLS at Pangolin, single shared token (§8.1) — mind the loopback Host-pin trap in §6.2. |
| 1.5 — opencode env propagation | hours | Make the mcp.mempalace subtree env-authoritative in generate-config.py, gated by a generated-value fingerprint (§4.1). Independent of the rest of this RFC. Without it, adopting or reverting the opt-in on an existing opencode container needs a manual sidecar merge or a docker volume rm — which also blocks R6 reversibility |
2 — mempalace-edge |
~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. Not "fixes opencode" — opencode's transport is already fine after Phase 1; what edge adds there is offline/local-first, since generate-config.py's switch is remote or local with no failover. Ships with the §1.2 opt-in wiring (compose + .env.example + a third branch in the existing generate-config.py) and must pass the R1 acceptance test. |
| 3 — pull replication | ~1 week | DEFERRED (2026-08-08). Server-side op-log with monotonic seq → each edge a full offline replica. Only needed if a laptop must hold everything offline. Accepted trade-off: offline recall = own writes + last-reachable state. |
| 4 — authz | days | Per-wing ACL, per-device scopes, audit log, token rotation |
Phase 1 is worth doing on its own — it is pure configuration and immediately ends KG fragmentation for online clients.
8.1 Deployment decisions (2026-08-09)
| Decision | Value | Note |
|---|---|---|
| Primary host | synlig — a VM in Xerces (work OpenStack cloud) | Chosen for always-on + good connectivity, not for work/personal reasons. Consequence: replicated content, diaries included, lives on employer infrastructure (§5, §6.1 threat 2) |
| TLS / ingress | Pangolin/newt, one DNS record per service on the web hotel | Not serve --tls-cert. See the bind-address/Host-pin coupling in §6.2 |
| Authentication | Single shared token for now | Per-device deferred to Phase 4; origin_device stays advisory until then |
| Diaries | replicated |
Highest-value cross-machine content; accepted placement consequence (§5) |
| Work/personal split | per-wing, not per-device | Rejects two primaries split along machine lines — the device cannot classify the project (§5) |
| Multi-store | Possible later, not now | MEMPALACE_REMOTE_URL stays a scalar in Phase 1; per-wing targets are an edge-config concern (Phase 2+) |
9. Open questions
-
Does opencode's MCP config support a remote/HTTP transport at all?RESOLVED 2026-08-09: YES. opencode's published JSON Schema (https://opencode.ai/config.json) defines$defs.McpRemoteConfig={type:"remote" (enum), url (required), headers?: {string:string}, oauth?, enabled?, timeout?}as a sibling ofMcpLocalConfiginsideConfig.properties.mcp.additionalProperties.anyOf. Soheaderscarries any bearer/API-key scheme, and setoauth: falseto stop opencode probing the URL for OAuth. The earlier "every sampled entry istype:local" observation was deployment evidence being read as schema evidence — the schema settles it in one fetch. Consequence: the edge proxy is not the only option for opencode. A plain Phase 1mempalace serve --tokencan be consumed directly, so nothing in the phasing depends on this question any more. What opencode still lacks is local-first writes and offline fallback —generate-config.py's switch is remote or local per container, with no failover — and that is the real Phase 2 justification for it. -
Upstream or local?
mempalace-edgeneeds no core changes, so it belongs in this repo (bin/+extensions/). Butorigin_devicestamping from the authenticated token, per-wing ACL, thekg_supersedeclassification fix, and async --refuse-sharedguard all want to go upstream (github.com/MemPalace/mempalace). -
Chunked drawers under merge.
Oversized content splits intoLargely resolved 2026-08-09: the last-chunk-only probe is deliberate, not an oversight. The code's own comment ({drawer_id}_chunk_NNNNNNwithparent_drawer_id, and the add-time idempotency probe checks only the last chunk id — a partially transferred chunk set may look "already present" and stay truncated.mcp_server.py:2586-2592) states that the last chunk's presence implies every earlier one landed because the batchedupsertis all-or-nothing, and it additionally probesdrawer_idalongside so a re-call with identical oversized content cannot duplicate a legacy pre-#1539 single-row write. The normal path is therefore safe by construction. What remains untested is the abnormal one: a chunk set left partial by a crash or kill mid-upsert, which would then look present and stay truncated. Worth a single fault-injection test before trusting bulk replay of oversized drawers. -
migrate.pyas a bootstrap tool.extract_drawers_from_sqlite()reads{id, document, metadata}straight out of chroma's SQLite (bypassing the chromadb API) and re-adds them with ids and metadata preserved and embeddings recomputed (migrate.py:326) — the right shape for a one-shot "seed the primary from an existing palace". Unverified:col.addbehaviour on id collision into a non-empty target. Test before relying on it. (exporter.pyis markdown-only, lossy, and has zero callers — not a transfer format.backups.pyis retention pruning only.) Reframed 2026-08-09: §4.4 makes MCP-level replay the primary join path — it reuses the server's own idempotency guards instead of trusting chroma-leveladdsemantics — somigrate.pyis now a fallback for bulk transfer rather than the plan, and its untested collision behaviour only matters if we reach for it. -
Does
_HTTP_REQUEST_LOCKstay held for the duration of an MCP-triggeredmine? Strongly suggested by the code shape; if yes, a remote mine makes the primary unusable for its duration (another argument for §5). -
How does opencode-devbox learn the opt-in?RESOLVED 2026-08-09: it already does. The image shipsrootfs/usr/local/lib/opencode-devbox/generate-config.py, invoked fromentrypoint-user.sh:117on every start, which auto-registers themempalaceMCP server — remote+bearer whenMEMPALACE_REMOTE_URL/MEMPALACE_REMOTE_TOKENare set, local stdio whenmempalace-mcpis on PATH, nothing otherwise — documented in that repo's.env.example:38-49and.env.shared.example:30-35. It also ships mempalace by default (see R5). So the config is templated at container start, not baked, and Phase 2 only adds a third branch. Two residual constraints replace the original unknown, and both are documented in §4.1: (a) the script never overwrites an existing config and~/.config/opencodeis a named volume, so an env flip yields only anopencode.jsonc.proposedsidecar — adopting or reverting the opt-in needs a manual merge or a volume removal; (b) it no-ops entirely unlessOPENCODE_PROVIDERis set. Prefer extending this script over inventing a parallel mechanism — treat it as the reference implementation, and check it before designing any pi-devbox-side mechanism with an opencode counterpart. -
Is a work VM the right long-term home for personal memory? Accepted with eyes open for Phase 1 (§8.1): synlig wins on availability and connectivity, and the alternative — splitting by machine — was rejected for sound reasons (§5). But the placement question survives the routing question: diaries are
replicated(decided), diaries are personal, and synlig is employer infrastructure. Revisit when per-wing targets exist (Phase 2+), at which point a second personal primary becomes an additive config change rather than a re-architecture. Until then, treat §6.1 threat 2 as live.
10. Evidence index
Re-verify without re-reading the package. Paths relative to
/opt/uv-tools/mempalace/lib/python3.13/site-packages/mempalace/ (mempalace 3.6.0) unless noted.
| Claim | Where |
|---|---|
| Server auth/TLS/read-only | cli.py:1448-1503; mcp_server.py:282-320, 5205-5215, 5284-5289 |
| Single global request lock | mcp_server.py:5115, 5346, 5513 |
KG path follows --palace flag only |
mcp_server.py:325, 708-711; knowledge_graph.py:49 |
KG DDL, no (s,p,o) uniqueness, open-fact guard |
knowledge_graph.py:163-178, 305-313 |
| ID recipes | ids.py:25, 56, 71; diary at mcp_server.py:3511-3514 |
| Diary write has no idempotency probe | mcp_server.py:3546 (col.add, no col.get) vs add_drawer's probe at 2593-2600 |
| Chunk probe is deliberate (all-or-nothing upsert) | mcp_server.py:2586-2592 |
| Metadata is exhaustive — no session/PID/conversation field | mcp_server.py:2578-2585 (drawers), 3527-3536 (diaries) |
| pi extension sets identity for diaries only | extensions/pi/mempalace.ts:758 (agent_name); no added_by set anywhere in the file |
| Anti-rebinding Host/Origin checks | mcp_server.py:5160-5194, 5276-5290, 5362-5367; cli.py:1450. Verified empirically — runbook §2.4 |
| Stock defaults already split the KG | config.py:221 (~/.mempalace/palace) vs knowledge_graph.py:49 (~/.mempalace/knowledge_graph.sqlite3) |
| Server token path (stable across restarts) | cli.py:_server_token_path — ~/.mempalace/server/sha256(realpath(palace))[:24]/token |
| pi-devbox non-destructive settings merge (pattern to port) | pi-devbox/entrypoint-user.sh:131-162 (jq -s '.[0] * .[1]', .bak, PI_SETTINGS_MERGE=0) |
No OPENCODE_CONFIG* env layer upstream |
absent from https://opencode.ai/config.json; zero hits in opencode-devbox, myconfigs |
Tool classification (+ kg_supersede gap) |
service.py:29, 54, 72-84 |
| Write-routing seam | write_routing.py:1-60 |
| Outbox prior art (queue DDL, token, retries) | daemon.py (_init_db, ensure_token, MAX_ATTEMPTS) |
sync is destructive pruning |
sync.py:1-12 |
| WAL redacts + has no reader | wal.py:74, _WAL_REDACT_KEYS |
| Client-side embeddings | backends/pgvector.py:6-8; embedding.py:6-11; _sidecar.py |
Dedup scoped by source_file |
dedup.py:get_source_groups |
| Store location rules | hallways.py:73,83; miner.py:701 |
| Mine lock (HOME-derived file) | palace.py:1090, 1123-1128 |
| Migrate as export primitive | migrate.py:extract_drawers_from_sqlite, :326 |
| pi remote transport + fail-soft | mempalace-toolkit/extensions/pi/mempalace.ts:11-12, 87, 108, 390, 629-641, 665-673 (commit 96699f2) |
| Server compose + port history | pi-devbox/docker-compose.mempalace.yml; pi-devbox/CHANGELOG.md v1.3.0; docker-compose-repo/tor-ms22/pi-devbox/ (df2c2ae) |
| Existing opt-in convention (§1.2) | pi-devbox/.env.example:12-23 ("MemPalace memory (local by default)", commented MEMPALACE_REMOTE_URL/_TOKEN); pi-devbox/docker-compose.yml:79-83 (devbox-palace volume commented out) |
| opencode-devbox does not wire mempalace | docker-compose-repo/{synlig,nyvaken,devbox-affection}/opencode-devbox/docker-compose.yml — zero mempalace references in any |
11. See also
ARCHITECTURE.md— producer side (how the palace gets fed); §6 upstream roadmapextensions/pi/README.md— pi bridge internalsSKILL.md— consumer-side protocol (search before answering, diary before exit)