Phase 0 on synlig: install, palace layout, verified Host/Origin policy

synlig is greenfield — no mempalace, no ~/.mempalace at all — so Phase 0
became "provision correctly from birth" rather than "migrate carefully".
Nothing is serving; no client config was touched.

Done:
- mempalace 3.6.0 installed via uv, pinned to the fleet version (the id
  recipes and idempotency probes this RFC leans on are version-specific).
- Embedder pre-warmed. This was the real unknown: the first embed pulls a
  79.3 MB ONNX model from the chroma CDN, and an egress-filtered work VM
  would have failed at the worst moment — the first client write. Pulled
  at ~20 MB/s, no proxy interference. Done in a throwaway palace so the
  real one never saw it.
- Palace at the stock default ~/.mempalace/palace, so no config file and
  no MEMPALACE_PALACE_PATH is needed on synlig at all.

Two corrections to the RFC, both from provisioning rather than reading:

- §7.1 was understated. DEFAULT_PALACE_PATH (~/.mempalace/palace) and
  DEFAULT_KG_PATH (~/.mempalace/knowledge_graph.sqlite3) already differ
  with stock defaults, so the KG split is out-of-the-box behaviour, not a
  consequence of a custom --palace, and it is permanent rather than
  one-time: serve always passes --palace, any CLI call without it uses
  the HOME path. A one-time mv does not fix that, it only picks which of
  the two files gets populated. Fixed instead by converging both rules on
  one inode via relative symlinks, and verified the load-bearing
  assumption: a dangling symlink is created on connect, -wal/-shm land
  next to the target (so the palace dir stays a self-contained backup
  unit, which is the part that mattered), cross-path read works, same
  inode. hallways.json deliberately left alone — already palace-derived,
  HOME path is a warning-only probe.
- §6.2 upgraded from "test early" to verified: 11/11 as predicted. The
  headline is that the safe-sounding reflex is the failure mode — loopback
  bind + proxy forwarding a public Host is 403, non-loopback is 200. Also
  confirmed Origin is never relaxed on either bind, and /healthz is
  Host/Origin-gated but token-free, so it works as the tunnel probe.
  Recommends binding the docker0 gateway over 0.0.0.0: non-loopback so
  the pin relaxes, but reachable only from the host and its containers.

New docs/synlig-primary-runbook.md carries the discovered facts about the
box, the evidence tables, an explicit "deliberately not done" list, and
tomorrow's ordered steps. New contrib/systemd/mempalace-serve.service
carries the bind rationale inline so nobody "fixes" it back to loopback;
staged on synlig with a .staged suffix so systemd cannot pick it up by
accident.

Flagged for tomorrow: synlig has no newt/tunnel client (docker ps shows
only the Gitea runner and digikam), so Pangolin on nyvaken cannot reach
it until one is added — easy to miss, because Pangolin will look healthy
from its own side.
This commit is contained in:
Joakim Persson
2026-08-10 00:14:06 +02:00
parent 7e51055c96
commit 3626946013
3 changed files with 247 additions and 3 deletions
+29 -3
View File
@@ -398,7 +398,7 @@ The transport is largely solved; **the gap is authorization, not cryptography.**
| Control | Decision |
| --- | --- |
| **Network posture** | Primary **never** internet-exposed. Publish only onto the private overlay / existing tunnel (Pangolin/newt). ⚠️ **The reflexive "bind loopback in the container" is the failure mode here — see Transport.** |
| **Transport** | **Decided 2026-08-09: terminate TLS in the existing Pangolin/newt tunnel**, not in `serve` (cheaper than patching Python's TLS surface; one DNS record per service on the web hotel is the whole setup cost). ⚠️ **Host-pinning is coupled to the bind address**, verified: `enforce_host_pin = _http_is_loopback(host)` (`mcp_server.py:5367`). On a **loopback** bind, `Host` is pinned to loopback literals + the bound host, so a proxy forwarding `Host: palace.example.com` gets **403 Forbidden** — either make the proxy rewrite `Host` to `127.0.0.1:<port>`, or bind the private interface instead. On a **non-loopback** bind the pin is deliberately **relaxed** ("*may sit behind a proxy that rewrites Host … lean on the Origin check + optional token instead*", `:5362-5365`), which is also why `cli.py:1450` makes a tokenless non-loopback bind require `--allow-insecure`. **The `Origin` check is never relaxed:** an absent `Origin` is allowed (every non-browser MCP client, incl. pi and opencode), but a *present* non-loopback `Origin` is 403 with no override — so keep browser-based clients and `Origin`-injecting proxies out of the path. |
| **Transport** | **Decided 2026-08-09: terminate TLS in the existing Pangolin/newt tunnel**, not in `serve` (cheaper than patching Python's TLS surface; one DNS record per service on the web hotel is the whole setup cost). **Host/Origin policy verified by experiment on synlig 2026-08-10, 11/11 as predicted** (runbook §2.4) — the summary: **do not bind loopback behind the tunnel.** `enforce_host_pin = _http_is_loopback(host)` (`mcp_server.py:5367`), so a loopback bind + a proxy forwarding `Host: palace.example.com` **403**, while a non-loopback bind → **200** (pin deliberately relaxed, "*may sit behind a proxy that rewrites Host*", `:5362-5365`); tokenless non-loopback binds require `--allow-insecure` (`cli.py:1450`). **Prefer binding the docker0 gateway (e.g. `172.17.0.1`) over `0.0.0.0`:** non-loopback, so the pin relaxes, yet reachable only from the host and its containers — so a newt container on the box reaches it and the LAN cannot. **The `Origin` check is never relaxed:** absent `Origin` is fine (every non-browser MCP client, incl. pi and opencode), a *present* non-loopback `Origin` is 403 with no override — so keep browser-based clients and `Origin`-injecting proxies out of the path. `/healthz` is Host/Origin-gated but token-free, so it works as the tunnel's liveness probe. |
| **Authentication** | Target: **per-device bearer tokens** with a server-side `token → {device_id, scopes}` registry — revoke one laptop, rotate without a fleet outage. **Decided 2026-08-09: Phase 1 ships the single shared token** ("*iterate more feature rich but more complex solutions over time*"), so per-device lands with Phase 4. Store in the existing `.env.age` flow, 0600 on disk. **Consequence: until then the primary cannot tell devices apart, so `origin_device` stays client-asserted and advisory — nothing load-bearing may depend on it (§7.3.2).** |
| **Authorization** | Per-device read/write **wing globs**. Server-side refusal of `mine`/`sync`/`delete_*` except for an admin device. `--read-only` gives a free observer tier. |
| **Provenance/audit** | `origin_device` (+ optional `origin_label`) + `op_id` on every record, stamped **server-side from the authenticated credential** — a client-asserted origin is a hint, not a fact (§7.3.2); server-side append-only op log. |
@@ -440,6 +440,30 @@ Four stores, **three** location rules:
nothing else** — it would leave the KG as fragmented as it is today. That is why the shared-backend
option is *not* the answer.
⚠️ **Understated above, corrected 2026-08-10 while provisioning synlig: this is not a migration hazard,
it is the stock-default behaviour, and it is permanent.** `DEFAULT_PALACE_PATH` is
`~/.mempalace/palace` (`config.py:221`) while `DEFAULT_KG_PATH` is
`~/.mempalace/knowledge_graph.sqlite3` (`knowledge_graph.py:49`) — **those differ out of the box**, with no
custom path involved. So on *every* host, forever: `serve` (always passes `--palace`) uses the KG inside
the palace, while any CLI command run without `--palace` uses the HOME one. A one-time `mv` does not fix
that; it just relocates which of the two files is populated.
**Better action — converge the two resolution rules onto one inode** (and the only option on a greenfield
primary, where there is nothing to move):
```sh
ln -sfn palace/knowledge_graph.sqlite3 ~/.mempalace/knowledge_graph.sqlite3
ln -sfn palace/known_entities.json ~/.mempalace/known_entities.json
# hallways.json deliberately NOT symlinked: already palace-derived, and its HOME
# path is a warning-only legacy probe that never auto-migrates (hallways.py:73-95)
```
Verified on synlig 2026-08-10, because the WAL behaviour was the load-bearing assumption: a **dangling**
symlink is created on first `sqlite3.connect`; `-wal`/`-shm` land next to the **target** (inside the palace
dir, so the palace stays a self-contained backup/bind-mount unit) and *not* beside the symlink; a write
through one path reads back through the other; **same inode**. Full record in
[`synlig-primary-runbook.md`](./synlig-primary-runbook.md) §2.3.
### 7.2 Never run `mempalace sync` against a shared palace
It classifies drawers whose source files are absent **on the running host** as orphans and deletes
@@ -658,7 +682,7 @@ join replays (§4.4).
| Phase | Effort | Deliverable |
| --- | --- | --- |
| **0 — hygiene** | hours | §7 runbook: move KG/hallways/entities **on synlig, before first `serve`** (§7.1), ban `sync` on shared palaces (§7.2), fix stale "unauthenticated" docs (incl. `pi-devbox/.env.example:21`). Added 2026-08-09: settle the **diary dedup** approach and file its upstream ask (§7.6), and **dry-run the join from one palace only** (§4.4). **No provenance work here** — it is not backfill-critical (§7.3.3) and belongs to the stamper, not the agent |
| **0 — hygiene** | hours | §7 runbook: converge the KG/entities store paths **on synlig before first `serve`** (§7.1**done 2026-08-10**, runbook §2.3), ban `sync` on shared palaces (§7.2), fix stale "unauthenticated" docs (incl. `pi-devbox/.env.example:21`). Added 2026-08-09: settle the **diary dedup** approach and file its upstream ask (§7.6), and **dry-run the join from one palace only** (§4.4). **No provenance work here** — it is not backfill-critical (§7.3.3) and belongs to the stamper, not the agent |
| **1 — primary up** | hours, **no code** | `mempalace serve --token --tls-cert` on a private-net host (reuse `docker-compose.mempalace.yml` — keep it a **separate standalone project**, R4 — and mind port 8765 vs pi-studio; tor-ms22 already moved to 8766). Repoint pi clients via `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN`. **opencode clients can be repointed in the same breath** — remote MCP is supported and `generate-config.py` already emits it (§9.1, §9.6), subject to the sidecar caveat in §4.1. **Shared memory today, no offline.** **Decided 2026-08-09: primary = synlig, TLS at Pangolin, single shared token (§8.1)** — mind the loopback Host-pin trap in §6.2. |
| **1.5 — opencode env propagation** | hours | Make the `mcp.mempalace` subtree env-authoritative in `generate-config.py`, gated by a generated-value fingerprint (§4.1). Independent of the rest of this RFC. Without it, adopting *or reverting* the opt-in on an existing opencode container needs a manual sidecar merge or a `docker volume rm` — which also blocks R6 reversibility |
| **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. **Not** "fixes opencode" — opencode's *transport* is already fine after Phase 1; what edge adds there is offline/local-first, since `generate-config.py`'s switch is remote **or** local with no failover. **Ships with the §1.2 opt-in wiring (compose + `.env.example` + a third branch in the existing `generate-config.py`) and must pass the R1 acceptance test.** |
@@ -762,7 +786,9 @@ Re-verify without re-reading the package. Paths relative to
| Chunk probe is deliberate (all-or-nothing upsert) | `mcp_server.py:2586-2592` |
| Metadata is exhaustive — no session/PID/conversation field | `mcp_server.py:2578-2585` (drawers), `3527-3536` (diaries) |
| pi extension sets identity for **diaries only** | `extensions/pi/mempalace.ts:758` (`agent_name`); no `added_by` set anywhere in the file |
| Host-pin coupled to bind address; Origin never relaxed | `mcp_server.py:5160-5194`, `5276-5290`, `5362-5367`; `cli.py:1450` |
| Anti-rebinding Host/Origin checks | `mcp_server.py:5160-5194`, `5276-5290`, `5362-5367`; `cli.py:1450`. **Verified empirically** — runbook §2.4 |
| Stock defaults already split the KG | `config.py:221` (`~/.mempalace/palace`) vs `knowledge_graph.py:49` (`~/.mempalace/knowledge_graph.sqlite3`) |
| Server token path (stable across restarts) | `cli.py:_server_token_path` — `~/.mempalace/server/sha256(realpath(palace))[:24]/token` |
| pi-devbox non-destructive settings merge (pattern to port) | `pi-devbox/entrypoint-user.sh:131-162` (`jq -s '.[0] * .[1]'`, `.bak`, `PI_SETTINGS_MERGE=0`) |
| No `OPENCODE_CONFIG*` env layer upstream | absent from `https://opencode.ai/config.json`; zero hits in `opencode-devbox`, `myconfigs` |
| Tool classification (+ `kg_supersede` gap) | `service.py:29`, `54`, `72-84` |