docs: reconcile the RFC-001 docs with what is actually deployed
Audit of every doc touching the global-palace rollout against the running
fleet. Each correction below was verified against the filesystem or the host,
not against another doc:
- synlig-primary-runbook: the decommission `rm -rf ~/.mempalace` now carries a
STOP block. That tree holds the fleet palace *and* the only copy of the
bearer token every client authenticates with; the old "empty today" comment
stopped being true when the palace was seeded on 2026-08-14. Adds an ordered
safe decommission, and drops count-based join verification.
- phase-1-exposure-runbook: new S3.8, how to verify a flip actually took --
the procedure that until now existed only in an untracked handover file.
Three claims that fail independently (env var / curl / the palace-path
discriminator) plus an explicit list of checks that produce FALSE POSITIVES:
drawer counts (both sides were seeded from the same palace, and `status`
counts chunks not drawers), write-then-read through the same transport, and
the `mempalace` CLI -- which has no remote support at all, so post-flip it
reads the dead local archive and reports success.
- rfc-001: status Draft -> Phases 0-1 implemented. Records that the join was a
file-level copy, which SIDESTEPPED the S7.6 diary-dedup question rather than
answering it -- so S7.6 remains a hard blocker for the second machine, which
is the one that will actually exercise merge semantics.
- ARCHITECTURE, SKILL, contrib/README, extensions/pi/README all claimed pi
feeds the palace automatically, unconditionally. That is gated on
mempalace-toolkit >= 29e660e and every deployed image predates it, so the
claim is currently false fleet-wide. Each site now states the gate plus a
check that inspects the *deployed* file rather than repo HEAD.
- extensions/pi/README: plaintext http://mempalace.lan example -> https
endpoint; the two transports are either/or (no dual-write, no local mirror);
the bridge fails CLOSED, so "the agent has no mempalace_* tools" is the
expected symptom of a server/token/DNS fault, not of a broken install.
- contrib/README: documents mempalace-serve.service, which this directory has
shipped since day one without explaining it (linger, the load-bearing
172.17.0.1 bind and why loopback is the unsafe-looking-safe option, the
token path, and an uninstall warning).
- Fixes a pre-existing stray ```sh fence that was swallowing S3.2's heading and
the token command into a code block.
Docs only; no behaviour change.
This commit is contained in:
Joakim Persson
+9
-4
@@ -32,10 +32,15 @@ So on a machine using opencode + the "docs-first palace hygiene" policy, three g
|
||||
The two wrappers in `bin/` close gaps **1** and **2**. Gap **3** is upstream work (see §6).
|
||||
|
||||
(Pi is a different story on gap 3: `bin/mempalace-pi-session` closes pi's
|
||||
version of gap 2 the same way `mempalace-session` closes opencode's, but pi's
|
||||
gap 3 is already closed **without** any upstream dependency — its bridge
|
||||
version of gap 2 the same way `mempalace-session` closes opencode's, and pi's
|
||||
gap 3 is closed **without** any upstream dependency — its bridge
|
||||
extension self-triggers the feed on `session_shutdown` and a debounced
|
||||
`agent_settled`. See §3's `mempalace-pi-session` subsection and
|
||||
`agent_settled`. **Version gate: extension-side feeding landed in
|
||||
mempalace-toolkit `29e660e` (2026-08-12), so a copy baked into an older
|
||||
container image cannot feed at all** — check the deployed file, not repo HEAD:
|
||||
`grep -c MEMPALACE_FEED "$(readlink -f ~/.pi/agent/extensions/mempalace.ts)"`.
|
||||
Zero means that machine still needs `contrib/`'s scheduled recipes. See §3's
|
||||
`mempalace-pi-session` subsection and
|
||||
[`extensions/pi/README.md`](extensions/pi/README.md).)
|
||||
|
||||
---
|
||||
@@ -433,7 +438,7 @@ These gaps should ideally close upstream, making the wrappers thinner or obsolet
|
||||
3. **Opencode harness in `hooks_cli.py`** — mempalace's hooks CLI only knows `claude-code` + `codex` today. Adding `opencode` would let the auto-save diary path work on opencode too. Pairs with #2 above.
|
||||
4. **SQLite mode for `mempalace mine --mode convos`** — if upstream ever adds direct SQLite ingest for opencode, `mempalace-session` loses its reason to exist (the export-to-JSONL dance goes away).
|
||||
|
||||
When #1 merges, retire `mempalace-docs` to a thin shim. When #2 + #3 land together, `mempalace-session` becomes a manual-only fallback (cron / backfill) while hooks handle live saves. (`mempalace-pi-session` has no equivalent entry here: pi's bridge extension already self-triggers the feed with no upstream dependency — see §3.)
|
||||
When #1 merges, retire `mempalace-docs` to a thin shim. When #2 + #3 land together, `mempalace-session` becomes a manual-only fallback (cron / backfill) while hooks handle live saves. (`mempalace-pi-session` has no equivalent entry here: pi's bridge extension already self-triggers the feed with no upstream dependency — see §3 — **provided the deployed copy is ≥ `29e660e` (2026-08-12); older baked images cannot feed, so `contrib/`'s schedulers remain load-bearing there**.)
|
||||
|
||||
Separately tracked in [`docs/rfc-001-global-palace.md`](docs/rfc-001-global-palace.md): moving from one palace *per machine* to a **single primary palace with per-machine local fallback** (`mempalace-edge`). That RFC also records upstream items of its own — server-side `origin_device` provenance stamped from a per-device credential, per-wing ACLs, a `mempalace_kg_supersede` tool-classification fix, and a guard against running `mempalace sync` on a shared palace.
|
||||
|
||||
|
||||
@@ -20,11 +20,17 @@ The `mempalace` skill covers *using* the palace (search, diary, KG). This skill
|
||||
|
||||
Both follow the same **stage-to-cache-then-mine** idiom — they curate input into `~/.cache/…/<wing>/`, then delegate to `mempalace mine`.
|
||||
|
||||
**Pi is out of scope for this skill.** The pi coding-agent has its own
|
||||
wrapper, `mempalace-pi-session`, but — unlike opencode — pi's bridge
|
||||
**Pi is out of scope for this skill — conditionally.** The pi coding-agent has
|
||||
its own wrapper, `mempalace-pi-session`, but — unlike opencode — pi's bridge
|
||||
extension (`extensions/pi/mempalace.ts`) invokes it automatically on session
|
||||
shutdown and a debounced mid-session tick, so a pi machine with that
|
||||
extension installed needs none of this skill's manual/scheduled recipe. See
|
||||
shutdown and a debounced mid-session tick, so a pi machine running
|
||||
**mempalace-toolkit ≥ `29e660e` (2026-08-12)** needs none of this skill's
|
||||
manual/scheduled recipe. Older installs — and any container image baked before
|
||||
that date — **do** still need it: having the extension is necessary but not
|
||||
sufficient, because the pre-`29e660e` extension has no feed path at all.
|
||||
Check the *deployed* file rather than the repo:
|
||||
`grep -c MEMPALACE_FEED "$(readlink -f ~/.pi/agent/extensions/mempalace.ts)"`
|
||||
— zero means fall back to the recipes below. See
|
||||
`extensions/pi/README.md` § Automatic transcript feeding in the same repo.
|
||||
|
||||
## When to Load This Skill
|
||||
|
||||
+76
-2
@@ -1,11 +1,18 @@
|
||||
# contrib/ — automation recipes for `mempalace-session` and `mempalace-pi-session`
|
||||
|
||||
Manual invocation of the session-mining wrappers is fine on a machine you actively drive. For long-running devboxes, a weekly automated mine keeps the palace fresh without thinking about it. This directory ships ready-to-use templates for two common scheduling mechanisms, for each wrapper.
|
||||
Manual invocation of the session-mining wrappers is fine on a machine you actively drive. For long-running devboxes, a weekly automated mine keeps the palace fresh without thinking about it. This directory ships ready-to-use templates for two common scheduling mechanisms, for each wrapper — plus `systemd/mempalace-serve.service`, which is not a mining job at all but the **shared-palace server** ([its own section below](#mempalace-serveservice--the-shared-palace-server)).
|
||||
|
||||
> **pi machines: check whether you need this at all.** If the pi bridge
|
||||
> extension (`extensions/pi/mempalace.ts`) is installed, it already feeds the
|
||||
> extension (`extensions/pi/mempalace.ts`) is installed **and is ≥ `29e660e`
|
||||
> (2026-08-12)**, it already feeds the
|
||||
> palace by itself on `session_shutdown` and a debounced `agent_settled` —
|
||||
> see [`extensions/pi/README.md` § Automatic transcript feeding](../extensions/pi/README.md#automatic-transcript-feeding).
|
||||
> ⚠️ **"Installed" is not enough** — the pre-`29e660e` extension has no feed
|
||||
> path at all, and a container image baked before that date ships exactly that
|
||||
> copy. Check the *deployed* file, not the repo clone:
|
||||
> `grep -c MEMPALACE_FEED "$(readlink -f ~/.pi/agent/extensions/mempalace.ts)"`
|
||||
> — zero means the templates below are **not** optional on that machine. As of
|
||||
> 2026-08-14 the entire pi-devbox fleet returns zero.
|
||||
> The templates below were written when scheduling was the *only* path for
|
||||
> both harnesses; that's still true for **opencode** (no such extension
|
||||
> exists), but for pi they're now a fallback — useful for a bare pi install
|
||||
@@ -29,11 +36,78 @@ Pick **one scheduler** (systemd *or* launchd *or* cron). The opencode and pi job
|
||||
| `cron/mempalace-session.cron` | opencode → palace | Mon 03:00 |
|
||||
| `cron/mempalace-pi-session.cron` | pi → palace | Tue 03:00 |
|
||||
| `cron/mempalace-session-devbox.cron` | opencode (devbox) → palace | Mon 03:00 |
|
||||
| `systemd/mempalace-serve.service` | **not a mining job** — runs the shared palace *server* | always-on |
|
||||
|
||||
The pi variants are drop-in copies of the opencode variants with script name and schedule updated; the install recipes below apply equally — just swap `mempalace-session` for `mempalace-pi-session` and the schedule day.
|
||||
|
||||
---
|
||||
|
||||
## `mempalace-serve.service` — the shared palace server
|
||||
|
||||
The odd one out in this directory: every other template *feeds* a palace on a schedule, this one
|
||||
**serves** a palace over HTTP so several machines can share it (RFC-001). **This unit currently runs
|
||||
the fleet primary on `synlig`** — serving since 2026-08-12, seeded 2026-08-14, and the palace behind
|
||||
it is the only copy. Read [`docs/rfc-001-global-palace.md`](../docs/rfc-001-global-palace.md) and
|
||||
[`docs/phase-1-exposure-runbook.md`](../docs/phase-1-exposure-runbook.md) before installing a second one.
|
||||
|
||||
It is a **user** unit (`systemctl --user`), so it dies with your login session unless lingering is
|
||||
enabled — that is the one `sudo` this recipe needs:
|
||||
|
||||
```sh
|
||||
# Install
|
||||
mkdir -p ~/.config/systemd/user
|
||||
cp contrib/systemd/mempalace-serve.service ~/.config/systemd/user/
|
||||
|
||||
sudo loginctl enable-linger "$USER" # else the server stops when you log out
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable --now mempalace-serve
|
||||
|
||||
# Verify — both lines matter
|
||||
curl -s 172.17.0.1:8765/healthz # -> ok
|
||||
curl -s localhost:8765/healthz # -> connection refused (exit 7), and that is CORRECT
|
||||
|
||||
# Logs
|
||||
journalctl --user -u mempalace-serve -f
|
||||
```
|
||||
|
||||
The unit refuses to start if `~/.mempalace/palace` does not exist (`ConditionPathExists`) — better a
|
||||
clear failure than a server quietly creating an empty palace somewhere unexpected.
|
||||
|
||||
**Why it binds `172.17.0.1` and not loopback**, since this looks backwards and is the single most
|
||||
load-bearing line in the file: mempalace pins the HTTP `Host` header to loopback literals *only on a
|
||||
loopback bind*, so a `127.0.0.1` server behind a reverse proxy 403s every proxied request — **and**
|
||||
token auto-minting is gated on the bind being non-loopback, so the "safe-looking" loopback bind starts
|
||||
with **no authentication at all and no warning**. `172.17.0.1` is the docker0 gateway: non-loopback
|
||||
(so the pin relaxes and a token is minted), reachable from this host and its containers, not from the
|
||||
LAN. Point the tunnel/proxy at **`http://172.17.0.1:8765`** — not `https://` — because there is no
|
||||
`--tls-cert` here; TLS belongs at the proxy. Targeting `https://` yields 502 from outside while local
|
||||
curl still says `ok`.
|
||||
|
||||
**The token.** There is deliberately no `--token` in the unit (units are world-readable). `serve`
|
||||
mints or reuses a `0600` token at `~/.mempalace/server/<sha256-prefix-of-palace-path>/token`, stable
|
||||
across restarts. Read it from there to configure clients.
|
||||
|
||||
> ⚠️ **Uninstall is where this unit differs from every other template here.** Stopping it is safe;
|
||||
> deleting its data is not.
|
||||
>
|
||||
> ```sh
|
||||
> systemctl --user disable --now mempalace-serve
|
||||
> rm ~/.config/systemd/user/mempalace-serve.service && systemctl --user daemon-reload
|
||||
> ```
|
||||
>
|
||||
> **Do not `rm -rf ~/.mempalace` on a host that has served the fleet.** That tree holds the shared
|
||||
> palace *and* the only copy of the bearer token every client authenticates with. For the same reason,
|
||||
> **never `rsync --delete` into `~/.mempalace`** — the token lives inside the tree you would be
|
||||
> syncing. And note palace directories cannot simply be moved: the directory name is a sha256 prefix
|
||||
> of its own path.
|
||||
>
|
||||
> Clients **fail closed** when this unit is down — they lose their palace tools entirely rather than
|
||||
> falling back to a local palace — so stopping it is visible, reversible, and loses no data.
|
||||
|
||||
Operational note: the server serializes every request behind one lock, so a wedged process is a
|
||||
fleet-wide outage. Hence `Restart=on-failure` with `TimeoutStopSec=30` — fail fast and let systemd
|
||||
recover it.
|
||||
|
||||
## systemd user timer (recommended on modern Linux)
|
||||
|
||||
**Why:** runs without the user logged in (with `loginctl enable-linger`), survives reboots, logs to `journalctl`, Persistent=true catches missed runs after the machine was off. No root required — it's a *user* unit.
|
||||
|
||||
@@ -5,6 +5,17 @@ Companion to [`rfc-001-global-palace.md`](./rfc-001-global-palace.md) (design +
|
||||
This doc covers only the step the other two leave open: **making the primary reachable** — runbook §4
|
||||
items 2 and 5.
|
||||
|
||||
> **Status 2026-08-14 — DONE. Exposed, seeded, and one client flipped.**
|
||||
> `https://mempalace.jordbo.se/mcp` has been serving since 2026-08-12 (§3.3–§3.6 are all ✅ below).
|
||||
> The palace was seeded 2026-08-14 15:07 from EMB-7KJ4VR4G (14,777 → 14,803 drawers), and that
|
||||
> machine's pi-devbox container is flipped and **verified end-to-end — see §3.8**, which is the
|
||||
> verification procedure that did not exist when the first flip was performed.
|
||||
> Still outstanding: the transcript feeder is **inert on every deployed image** (§3.7), and §7.6 is
|
||||
> still a hard blocker for the *second* machine to join (§4).
|
||||
|
||||
Original status, kept for the record — it contradicted this file's own ✅ section markers for two days,
|
||||
which is the failure mode a file-top status block invites:
|
||||
|
||||
**Status 2026-08-12 — Pangolin updated on nyvaken (done, yours). newt not yet installed on synlig.
|
||||
Nothing exposed. No client `.env` flipped.**
|
||||
|
||||
@@ -158,7 +169,6 @@ bound **to loopback**, receiving a proxy-forwarded foreign `Host:` (§2, verifie
|
||||
docker0-only bind you cannot get 403 from loopback, because you never get far enough to send a header.
|
||||
Refusal is the stronger signal of the two: it proves the loopback and LAN surface is not listening at all.
|
||||
If it *hangs* instead, or `ss` shows `0.0.0.0:8765`, stop — that is not this configuration.
|
||||
```sh
|
||||
|
||||
### 3.2 Collect the shared token
|
||||
|
||||
@@ -325,6 +335,94 @@ mempalace-pi-session --reason manual-check; echo "exit=$?" # exit=0 and a file
|
||||
cat ~/.pi/agent/mempalace-catchup.log # missing file = the entrypoint skipped
|
||||
```
|
||||
|
||||
### 3.8 Verify the flip actually took — ✅ done 2026-08-14 on EMB-7KJ4VR4G
|
||||
|
||||
§3.6 verifies the *endpoint* before you flip. §3.7 tells you *how* to flip. Neither verifies the thing
|
||||
you actually care about afterwards: **that the agent's palace tools are now talking to synlig.** That
|
||||
gap is why the first flip was "done" for an hour before anyone could say whether it had worked.
|
||||
|
||||
There are **three separate claims** here and they fail independently. Check them in order; each one is
|
||||
cheap and rules out a different fault.
|
||||
|
||||
**(a) Did the container receive the variable?** In a shell *inside* the container:
|
||||
|
||||
```sh
|
||||
env | grep MEMPALACE_REMOTE_URL # -> https://mempalace.jordbo.se/mcp
|
||||
```
|
||||
|
||||
⚠ Do **not** run a bare `env | grep MEMPALACE` — that prints the bearer token into your scrollback.
|
||||
If the variable is absent, the cause is almost always §3.7's edit not having been applied: `env_file`
|
||||
is read at container **create** time and baked into the container config, so **`docker compose up -d`
|
||||
is required; `docker compose restart` silently reuses the old config.** Evidence from 2026-08-14 —
|
||||
running container config-hash `3a55e09ac19e0118` vs compose-computed `8a7e0cd4a677a9c1`; a differing
|
||||
hash is what makes `up -d` recreate. `--force-recreate` is not needed.
|
||||
**Never `docker compose down -v`** to "pick up" a change: on pi-devbox that destroys seven named
|
||||
volumes, including `devbox-pi-config` (pi's config **and every session transcript**), `devbox-uv`, and
|
||||
`devbox-chroma-cache` (a large embedding-model re-download).
|
||||
|
||||
**(b) Is the server reachable and the token accepted?** Still inside the container — this proves the
|
||||
network path and the credential, independently of any agent:
|
||||
|
||||
```sh
|
||||
curl -s -X POST "$MEMPALACE_REMOTE_URL" \
|
||||
-H "Authorization: Bearer $MEMPALACE_REMOTE_TOKEN" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | head -c 200
|
||||
```
|
||||
|
||||
`401` = token problem (compare `md5sum` of the client value against synlig's token file — trailing
|
||||
whitespace from an editor is the classic cause). Connection failure = DNS/tunnel, not auth.
|
||||
|
||||
**(c) Did the agent's bridge actually switch?** This is the claim that matters and the one that cannot
|
||||
be checked from bash — there is no log file. Ask the agent running in the container for
|
||||
`mempalace_status` and read the palace path it reports:
|
||||
|
||||
```
|
||||
sqlite_integrity.palace = /home/ecsjper/.mempalace/palace # ← proves remote
|
||||
```
|
||||
|
||||
**This is the cheapest and strongest discriminator: that path cannot exist inside the container**,
|
||||
whose user is `developer` with `HOME=/home/developer` and whose local palace is
|
||||
`/home/developer/.mempalace/palace`. One call, no token in scrollback, no writes. If instead it
|
||||
reports the local path, the bridge fell back: `createClient()` reads `MEMPALACE_REMOTE_URL` **once at
|
||||
load**, so re-check (a) and restart the agent, not just the container.
|
||||
|
||||
#### What does *not* verify the flip — read this before inventing your own check
|
||||
|
||||
- **❌ A drawer count.** Central was seeded *from* the client's own palace, so both report ~14,777.
|
||||
Counts cannot tell the two apart. Worse, `mempalace status` counts **chunk rows, not logical
|
||||
drawers** (three drawers plus a 2-chunk diary presented as +9), so a delta does not even mean what
|
||||
it looks like. Never reason about palace identity or contents from a count.
|
||||
- **❌ Writing a drawer through the palace tools and reading it back through the same tools.** This
|
||||
succeeds *identically whether or not the flip worked* — both palaces are healthy and were seeded
|
||||
from the same source, so a write-then-read round-trips either way. It only becomes evidence if the
|
||||
drawer is read back over a **different transport** (the `curl` in (b), by drawer id) or is proven
|
||||
**absent** from the local sqlite. This is the same false-positive class as the CLI below, one layer
|
||||
up, and it is an easy trap to fall into precisely because it feels like an end-to-end test.
|
||||
- **❌ The `mempalace` CLI, in any form.** The CLI has **no remote support whatsoever** — its only
|
||||
selector is `--palace <path>` — so it reads and writes the LOCAL on-disk archive via
|
||||
`config.json`. After a flip that archive is dead, yet `mempalace status` / `mempalace search` will
|
||||
cheerfully report ~14,777 drawers and look exactly like success. It is a false-positive machine.
|
||||
**Corollary that bites later:** memories filed with the CLI after a flip land in the dead archive,
|
||||
not in central, and a transcript backfill must therefore be mined **on synlig**, where the CLI's
|
||||
local palace *is* the central one.
|
||||
|
||||
#### Expected failure behaviour, so you can recognise it
|
||||
|
||||
The bridge is **fail-closed, not fail-local.** If synlig is unreachable, DNS fails, or the token is
|
||||
rejected, the extension retries a bounded number of times, prints `mempalace-mcp unavailable after
|
||||
retries; continuing without palace tools`, and **does not register the palace tools at all**. It does
|
||||
not silently write to the local palace; there is no dual-write and no local mirror, and in remote mode
|
||||
no local `mempalace-mcp` process is spawned (verified: zero mempalace processes in the flipped
|
||||
container). So **"the agent has no `mempalace_*` tools" is the expected symptom of a server, token, or
|
||||
DNS fault** — not of a broken container. Diagnose with (b).
|
||||
|
||||
#### Rollback — ~30 seconds, loses nothing
|
||||
|
||||
Comment out `MEMPALACE_REMOTE_URL` in `.env`, then `docker compose up -d`. The bridge falls back to the
|
||||
local stdio palace, which is intact. Note the local archive is **frozen, not empty**: it stops at the
|
||||
moment of the flip, so anything the agent filed into central since then will not be there.
|
||||
|
||||
---
|
||||
|
||||
## 4. Still open
|
||||
@@ -334,7 +432,19 @@ cat ~/.pi/agent/mempalace-catchup.log # missing file = th
|
||||
path it must first rsync there. HTTPS would then be genuinely sufficient and mining would work from any
|
||||
network. Until then, mining is limited to devices that can reach synlig's SSH.
|
||||
- **Per-device tokens** — Phase 4. Until then `origin_device` is advisory (§1.1).
|
||||
- **§7.6 diary dedup** must be settled *before* the first §4.4 join; replay duplicates every entry.
|
||||
- **§7.6 diary dedup** must be settled *before* the **next** §4.4 join; replay duplicates every entry.
|
||||
⚠ **2026-08-14 — the first join did not resolve this, it SIDESTEPPED it.** The seed was a file-level
|
||||
copy of one palace, which replays no diaries and therefore cannot duplicate them. The success of
|
||||
that join is *not* evidence the replay path is safe — it never exercised it. §7.6 remains a hard
|
||||
blocker for the second machine, which is the one that will actually need merge semantics.
|
||||
- **The transcript feeder is inert on every deployed image**, so nothing is being mined automatically
|
||||
anywhere in the fleet (§3.7). Regaining it needs a **new tagged pi-devbox release** — its CI
|
||||
publishes only on `v*` tags and the latest tag *is* the currently deployed image — built on
|
||||
pi-devbox ≥ `7c00dd6` **and** mempalace-toolkit ≥ `29e660e`. Both are required: the first restores
|
||||
the container-start catch-up, the second is where extension-side feeding was implemented at all.
|
||||
- **Whether *native* pi on a flipped machine was also flipped** is a per-machine question nobody has
|
||||
checked. If the container is flipped and native pi is not, that machine writes into two different
|
||||
palaces and its memory silently splits in two. Verify per machine with the §3.8(c) path check.
|
||||
- **§7.2**: never run `mempalace sync` against the shared palace. Doubly true now that the pi/opencode
|
||||
feeders stage *inside* the palace root, which puts staged sources in scope for a sync of the palace dir.
|
||||
- **nyvaken's public FQDN and the web hotel's CNAME rules** — unverified (§3.4).
|
||||
|
||||
+100
-12
@@ -2,11 +2,12 @@
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **Status** | Draft — design agreed, not implemented |
|
||||
| **Status** | **Phases 0–1 implemented and verified — 2026-08-10 / 08-12 / 08-14.** Primary live + seeded, first client flipped. Phases 1.5, 2 and 4 outstanding; Phase 3 deferred by decision. |
|
||||
| **Created** | 2026-08-08 |
|
||||
| **Applies to** | mempalace 3.6.0, mempalace-toolkit @ `96699f2`, pi-devbox ≥ v1.3.0 |
|
||||
| **Decision** | Phases 0–2 + 4 in scope. **Phase 3 (full pull replication) explicitly deferred** — "a laptop that can reach its own stuff plus whatever it can reach" is good enough. **Centralization is strictly opt-in — solitary devbox operation remains the default and must not change (§1.1).** |
|
||||
| **Recon update** | **2026-08-09 — §9 Q1 and Q6 are RESOLVED, both in the permissive direction** (opencode supports remote MCP; opencode-devbox already templates the mempalace entry). Neither is a blocker. See those entries for evidence; §2, §4.1 and R5 were corrected accordingly. |
|
||||
| **Rollout update** | **2026-08-14 — Phase 1 is live.** Primary serving at `https://mempalace.jordbo.se/mcp` since 2026-08-12 (synlig, `mempalace-serve.service` under `systemctl --user`, palace `/home/ecsjper/.mempalace/palace`); **seeded 2026-08-14 15:07** from EMB-7KJ4VR4G's palace — 14,777 drawers / 9 wings / 16,337 embeddings / KG 46 entities, 34 triples (14,803 drawers by 17:00); **first client flipped and verified end-to-end** the same afternoon. ⚠️ Two things not to misread: the seed was a **file-level copy of one palace**, *not* the §4.4 MCP replay (see the Deviation note in §4.4), and it therefore **sidestepped §7.6 rather than resolving it — §7.6 remains a hard blocker for the *second* joiner (§8 Phase 0).** §2's and §4.1's predictions about the pi client were confirmed in production; §4.4's count-based verification advice was **wrong** and has been corrected. |
|
||||
|
||||
**Read this first if you are asked to "centralize MemPalace" / "sync palaces between machines".** Most of the
|
||||
hard-won facts below are non-obvious and two of them are actively destructive if you guess wrong
|
||||
@@ -88,8 +89,9 @@ Two of the three pieces already exist. This is not greenfield.
|
||||
| Piece | State | Evidence |
|
||||
| --- | --- | --- |
|
||||
| **Primary server** | ✅ **Exists.** `mempalace serve --host --port --token --tls-cert --tls-key --read-only --allow-insecure`. Bearer token compared with `hmac.compare_digest`, **mandatory** on non-loopback binds (unless `--allow-insecure`), TLS 1.2+ resolved *before* bind, Host-header pinning + `Origin` allowlist (anti-DNS-rebinding), 16 MiB body cap, token-free `/healthz`. | `cli.py:cmd_serve` (~1448); `mcp_server.py:5205-5215`, `5284-5289` |
|
||||
| **Remote client** | ⚠️ **Exists for pi only, and it is either/or.** `createClient()` picks stdio *or* HTTP once at process start. | `extensions/pi/mempalace.ts:629-641` |
|
||||
| **Fallback + resync** | ❌ **Absent everywhere.** On remote failure the pi bridge re-handshakes the same URL, then **de-registers all palace tools** and runs blind. | `extensions/pi/mempalace.ts:665-673` |
|
||||
| **Remote client** | ⚠️ **Exists for pi only, and it is either/or.** `createClient()` picks stdio *or* HTTP once at process start. ✅ **Confirmed in production 2026-08-14** on the first flipped client: the branch is a pure transport swap, and in remote mode **no local `mempalace-mcp` child is spawned at all** (zero mempalace processes in the flipped container). So **writes go only to the primary — no dual-write, no local mirror.** | `extensions/pi/mempalace.ts:629-641` |
|
||||
| **Fallback + resync** | ❌ **Absent everywhere.** On remote failure the pi bridge re-handshakes the same URL, then **de-registers all palace tools** and runs blind. ✅ **Confirmed 2026-08-14 — and the precise word is fail-*closed*, not fail-local.** After bounded retries it prints `mempalace-mcp unavailable after retries; continuing without palace tools` and registers nothing; it never silently falls back to the local palace, so a write cannot land in the wrong store. Operational corollary worth stating once: **"the palace tools vanished" is the expected symptom of a server / token / DNS fault**, not of a broken client. | `extensions/pi/mempalace.ts:665-673` |
|
||||
| **`mempalace` CLI** | ❌ **No remote support whatsoever.** Its only palace selector is `--palace <path>`; otherwise it resolves `palace_path` from the local `config.json`. **Verified 2026-08-14: on a flipped client the CLI still reads and writes the now-dead local archive — and cheerfully reports ~14,777 drawers while doing so.** It is a false-positive machine: never verify a flip with `mempalace status`/`search`, and never file memories with the CLI post-flip (they land in the archive, not the primary). Corollary: a **transcript backfill must be mined on the primary host**, where the CLI's local palace *is* the primary. | `cli.py` argument surface; verified on EMB-7KJ4VR4G |
|
||||
| **opencode client** | ✅ **Remote is supported and already wired.** opencode's published schema (`https://opencode.ai/config.json`, `$defs.McpRemoteConfig`) makes `{"type":"remote","url","headers","oauth"}` a first-class sibling of `McpLocalConfig`, `headers` being a free string→string map (so bearer is a convention, not a constraint). `opencode-devbox` already emits exactly that entry when `MEMPALACE_REMOTE_URL` is set. The all-`type:local` configs in `myconfigs` are a *deployment* fact, not a capability limit. | `generate-config.py:107-118`; hook at `entrypoint-user.sh:117`; schema `$defs.McpRemoteConfig` |
|
||||
| **Server compose** | ✅ Exists: `pi-devbox/docker-compose.mempalace.yml` (canonical) + a tor-ms22 derivative (`docker-compose-repo/tor-ms22/pi-devbox/`, `df2c2ae`, port 8766, uid 1000, binds the real `~/.mempalace`). Not enabled. | pi-devbox CHANGELOG v1.3.0 (2026-07-02) |
|
||||
|
||||
@@ -103,13 +105,19 @@ Two of the three pieces already exist. This is not greenfield.
|
||||
>
|
||||
> **✅ Done 2026-08-12** — `pi-devbox/.env.example` and
|
||||
> `mempalace-toolkit/extensions/pi/README.md` both now recommend `mempalace serve`, bind docker0 rather
|
||||
> than `0.0.0.0`/loopback, use an `https://` example URL, and state that the transport *is*
|
||||
> than `0.0.0.0`/loopback, and state that the transport *is*
|
||||
> authenticated. `docker-compose.mempalace.yml` audited too, and it was worse than stale — it was
|
||||
> **broken on 3.6.0 in both directions**: `--host 0.0.0.0` with no token in the environment makes the
|
||||
> server refuse to start (crash-looping under `restart: unless-stopped`), and once a token *is* supplied
|
||||
> the healthcheck's unauthenticated `tools/list` POST 401s, marking a healthy server unhealthy forever.
|
||||
> Fixed: token now required via `${MEMPALACE_REMOTE_TOKEN:?}` (fails fast at `up`), healthcheck switched
|
||||
> to the token-free `/healthz`.
|
||||
>
|
||||
> ⚠️ **Corrected 2026-08-14 — one sub-claim above was wrong for two days.** The `https://` example URL
|
||||
> landed in `pi-devbox/.env.example` only; `extensions/pi/README.md` still carried
|
||||
> `http://mempalace.lan:8765/mcp` until it was fixed on **2026-08-14**. Everything else in this block
|
||||
> checked out. Worth naming as a pattern, because it is cheap to repeat: **a doc's own ✅ is not evidence
|
||||
> the work landed — verify it per sub-claim and per file.**
|
||||
|
||||
### Two things that sound like the feature and are not
|
||||
|
||||
@@ -133,6 +141,11 @@ independently disqualifying:
|
||||
vectors are written to Postgres.*" Cross-architecture ONNX determinism (arm64 macOS vs x86-64
|
||||
Linux, different execution providers) is not guaranteed. → **Ship text + metadata, always re-embed
|
||||
on receipt.** Cheap and safe: the raw text is always stored as the chroma document.
|
||||
|
||||
⚠️ **Not absolute — see the 2026-08-14 deviation in §4.4.** A whole-palace file-level copy arm64 macOS →
|
||||
x86-64 Linux *did* preserve working search and an intact HNSW index. Re-embedding on receipt remains the
|
||||
rule for **merging** operations into an existing store; it is not a prohibition on cloning one palace
|
||||
wholesale, provided the embedder sidecar travels with it (§7.4).
|
||||
2. **KG rows cannot be copied.** `triples` has **no** `UNIQUE(subject,predicate,object,valid_from)` —
|
||||
only `id` is unique, and `make_triple_id` embeds `datetime.now()`. A row copy therefore duplicates
|
||||
every fact. But `add_triple()` guards at the application level (`SELECT id … WHERE subject=? AND
|
||||
@@ -315,6 +328,32 @@ precisely to be extended.
|
||||
container joins the same way, at any time, repeatedly — so a join is **idempotent replay of local
|
||||
history**, and the only real question per record type is *what dedupes it*.
|
||||
|
||||
> ⚠️ **Deviation (2026-08-14) — the first join did not follow this section.** It was precisely the thing
|
||||
> the framing correction says this is *not*: a **file-level copy of one chosen palace**, arm64 macOS →
|
||||
> x86-64 Linux, bypassing MCP replay entirely. It worked — search verified live on the primary, HNSW
|
||||
> index intact, KG intact (46 entities / 34 triples). Recorded here so the next operator neither repeats
|
||||
> it blind nor believes it is forbidden. **Four conditions made it safe, and all four must hold:**
|
||||
>
|
||||
> 1. **Same mempalace version (3.6.0) at both ends**, so the on-disk chroma/HNSW layout matched.
|
||||
> 2. **`mempalace_embedder.json` travelled with the palace**, so embedder identity matched. This is the
|
||||
> load-bearing one: a mismatched embedding model does **not** raise — it silently returns garbage
|
||||
> search results, and §7.4 explains why the guard cannot catch it (only the model *name* is compared,
|
||||
> and `dimension: 0` is skipped as unknown).
|
||||
> 3. **A single source palace, so no merge semantics were exercised at all.** This is exactly what makes
|
||||
> the method inapplicable to the second joiner.
|
||||
> 4. **Python's `sqlite3` online-backup API** (`src.backup(dst)`) for the two sqlite DBs — WAL-safe with
|
||||
> a live writer, and the only option available because **neither host has the `sqlite3` CLI**. The
|
||||
> rest of `palace/` was `rsync`ed with `--exclude 'chroma.sqlite3*'`.
|
||||
>
|
||||
> **This sidesteps the merge problem; it does not solve it.** The second machine to join still needs the
|
||||
> replay path and the dedupe keys tabulated below, and still needs §7.6 settled. Three traps found while
|
||||
> doing it: **never `rsync --delete` into `~/.mempalace`** — the server's bearer token lives *inside* that
|
||||
> tree at `~/.mempalace/server/<hash>/token` (§10) and there is no second copy; **palace paths cannot
|
||||
> move**, because the palace directory name is a sha256 prefix of its own path; and compare sizes with
|
||||
> **`stat -c%s`, not `du`** (APFS and ext4 disagree on block accounting, so `du` shows a spurious delta).
|
||||
> Quiesce the writer first — the source palace's own MCP server was live throughout, which is how a write
|
||||
> landed on the far side of the snapshot boundary and produced the false alarm described below.
|
||||
|
||||
| Record type | Dedupe on replay | Client work needed |
|
||||
| --- | --- | --- |
|
||||
| `add_drawer` / `checkpoint` drawers | **Server-side**: content-addressed id + pre-write `col.get` probe → `{"reason":"already_exists"}`, no write (`mcp_server.py:2593-2600`) | **None.** Just replay |
|
||||
@@ -342,9 +381,32 @@ for everything except diaries. Belt and braces:
|
||||
pattern as the outbox (§4.3). It must **not** live in a container-only path: `~/.mempalace` is not
|
||||
preserved by default for solitary users (§1.2), which is exactly why the file belongs to the palace
|
||||
directory.
|
||||
- **First join is a dry run.** Bootstrap one palace, verify counts (`status`, `kg_stats`, per-wing
|
||||
`list_drawers`) against expectations, *then* let the rest join. Ordering matters only because of diaries
|
||||
and closed facts; everything else is order-free.
|
||||
- **First join is a dry run.** Bootstrap one palace, verify it, *then* let the rest join. Ordering matters
|
||||
only because of diaries and closed facts; everything else is order-free.
|
||||
|
||||
> ⚠️ **Corrected 2026-08-14. This bullet used to say "verify counts (`status`, `kg_stats`, per-wing
|
||||
> `list_drawers`) against expectations". Do not verify a join by counts.** Two independent reasons,
|
||||
> both learned on the first real seed:
|
||||
>
|
||||
> 1. **`mempalace status` counts chunk rows, not logical drawers.** Three drawers plus one 2-chunk diary
|
||||
> presented as **+9**. A count delta cannot even tell you how many *records* moved.
|
||||
> 2. **Chunk counts and chunk-id sets legitimately differ between two palaces** whenever a drawer was
|
||||
> updated on either side: an update preserves `drawer_id`, **re-chunks to the new length, and deletes
|
||||
> the surplus chunk rows** (§9.3). A `…_chunk_000007` present on one side and absent on the other is
|
||||
> therefore the ordinary signature of an **edit**, not of loss.
|
||||
>
|
||||
> On 2026-08-14 that second mechanism produced a confident "the seed lost a chunk" conclusion that was
|
||||
> filed as a finding before being retracted. Counts **hid** the difference (14,777 vs 14,778 looked like
|
||||
> one lost row); an id-set diff **over-reported** it. The two methods fail in *opposite* directions, so
|
||||
> agreeing with either one alone proves nothing.
|
||||
>
|
||||
> **Verify by content.** For a sample of drawers, `get_drawer(<parent id>)` on *both* palaces and compare
|
||||
> the **reassembled `content`**. An id-set diff is a fine first pass to *find* candidates, but it must be
|
||||
> adjudicated by a content comparison before anyone concludes loss. Two cheap decisive checks once a
|
||||
> difference is real: **is one side a prefix of the other** (that, and only that, is truncation), and
|
||||
> **does the final chunk's length equal `len(content) - chunk_size * (n_chunks - 1)`** (800 in 3.6.0)? If
|
||||
> both sides satisfy the arithmetic for their own content, both are complete and you are looking at two
|
||||
> revisions, not damage.
|
||||
|
||||
The genuinely hard case is **two different palaces holding overlapping mined content** — the same repo
|
||||
mined on a laptop and a workstation under different absolute paths. §5 excludes it by keeping mined wings
|
||||
@@ -464,10 +526,20 @@ primary, where there is nothing to move):
|
||||
```sh
|
||||
ln -sfn palace/knowledge_graph.sqlite3 ~/.mempalace/knowledge_graph.sqlite3
|
||||
ln -sfn palace/known_entities.json ~/.mempalace/known_entities.json
|
||||
# hallways.json deliberately NOT symlinked: already palace-derived, and its HOME
|
||||
# path is a warning-only legacy probe that never auto-migrates (hallways.py:73-95)
|
||||
ln -sfn palace/hallways.json ~/.mempalace/hallways.json # added 2026-08-14, see note
|
||||
```
|
||||
|
||||
> **Updated 2026-08-14 — `hallways.json` is symlinked too now.** This block previously said it was
|
||||
> *deliberately* not symlinked, on the narrow grounds that it is already palace-derived and its HOME path
|
||||
> is a warning-only legacy probe that never auto-migrates (`hallways.py:73-95`). Both facts still hold —
|
||||
> but the goal changed from "symlink only what the code demands" to **"all real state lives under
|
||||
> `palace/`, so one copy of `palace/` is a complete copy"**, which is what let the 2026-08-14 file-level
|
||||
> seed (§4.4) treat the palace as a single self-contained unit. With all three links in place every
|
||||
> parent-level path resolves, and 3.6.0's *three different* resolution rules — palace-relative for the
|
||||
> served KG, HOME for the CLI KG, `dirname(palace_path)` for hallways, hardcoded HOME for
|
||||
> `known_entities.json` — converge on one set of files. Revert by deleting the symlink if it ever causes
|
||||
> trouble.
|
||||
|
||||
Verified on synlig 2026-08-10, because the WAL behaviour was the load-bearing assumption: a **dangling**
|
||||
symlink is created on first `sqlite3.connect`; `-wal`/`-shm` land next to the **target** (inside the palace
|
||||
dir, so the palace stays a self-contained backup/bind-mount unit) and *not* beside the symlink; a write
|
||||
@@ -675,6 +747,13 @@ duplication.
|
||||
This matters more than it sounds, because diaries are `replicated` (§5) and are precisely the content a
|
||||
join replays (§4.4).
|
||||
|
||||
> ⚠️ **Still open as of 2026-08-14 — and the first join did not test it.** The seed was a file-level copy
|
||||
> (§4.4 Deviation), which replays no diaries and therefore *cannot* duplicate them. **§7.6 was sidestepped
|
||||
> by method choice, not resolved.** Neither action below has been built. The moment a *second* palace
|
||||
> joins — by any replay-based route — this becomes live again, and it is the single most likely thing for a
|
||||
> future operator to get wrong, because the first join appears to have proved the path safe. It did not:
|
||||
> it avoided the path.
|
||||
|
||||
**Actions:**
|
||||
|
||||
1. **Client-side, now:** before replay, list the target's diary drawers for the wing, extract the 12-hex
|
||||
@@ -692,8 +771,8 @@ join replays (§4.4).
|
||||
|
||||
| Phase | Effort | Deliverable |
|
||||
| --- | --- | --- |
|
||||
| **0 — hygiene** | hours | §7 runbook: converge the KG/entities store paths **on synlig before first `serve`** (§7.1 — **done 2026-08-10**, runbook §2.3), ban `sync` on shared palaces (§7.2), ~~fix stale "unauthenticated" docs (incl. `pi-devbox/.env.example:21`)~~ — **done 2026-08-12**, and `docker-compose.mempalace.yml` turned out to be outright broken on 3.6.0, now fixed. Added 2026-08-09: settle the **diary dedup** approach and file its upstream ask (§7.6), and **dry-run the join from one palace only** (§4.4). **No provenance work here** — it is not backfill-critical (§7.3.3) and belongs to the stamper, not the agent |
|
||||
| **1 — primary up** | hours, **no code** | `mempalace serve --token --tls-cert` on a private-net host (reuse `docker-compose.mempalace.yml` — keep it a **separate standalone project**, R4 — and mind port 8765 vs pi-studio; tor-ms22 already moved to 8766). Repoint pi clients via `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN`. **opencode clients can be repointed in the same breath** — remote MCP is supported and `generate-config.py` already emits it (§9.1, §9.6), subject to the sidecar caveat in §4.1. **Shared memory today, no offline.** **Decided 2026-08-09: primary = synlig, TLS at Pangolin, single shared token (§8.1)** — mind the loopback Host-pin trap in §6.2. |
|
||||
| **0 — hygiene** | hours | §7 runbook: converge the KG/entities store paths **on synlig before first `serve`** (§7.1 — **done 2026-08-10**, runbook §2.3), ban `sync` on shared palaces (§7.2), ~~fix stale "unauthenticated" docs (incl. `pi-devbox/.env.example:21`)~~ — **done 2026-08-12**, and `docker-compose.mempalace.yml` turned out to be outright broken on 3.6.0, now fixed. Added 2026-08-09: settle the **diary dedup** approach and file its upstream ask (§7.6), and **dry-run the join from one palace only** (§4.4). ✅ **Both done 2026-08-14, with one asterisk that matters:** the join was dry-run *and* executed from one palace only — but **§7.6 was sidestepped, not settled** (a file-level copy replays no diaries, so it cannot duplicate them). The §7.6 client-side dedupe and its upstream ask are **still unbuilt and are hard blockers for the second joiner.** **No provenance work here** — it is not backfill-critical (§7.3.3) and belongs to the stamper, not the agent |
|
||||
| **1 — primary up** ✅ **done 2026-08-14** | hours, **no code** | `mempalace serve --token --tls-cert` on a private-net host (reuse `docker-compose.mempalace.yml` — keep it a **separate standalone project**, R4 — and mind port 8765 vs pi-studio; tor-ms22 already moved to 8766). Repoint pi clients via `MEMPALACE_REMOTE_URL`/`MEMPALACE_REMOTE_TOKEN`. **opencode clients can be repointed in the same breath** — remote MCP is supported and `generate-config.py` already emits it (§9.1, §9.6), subject to the sidecar caveat in §4.1. **Shared memory today, no offline.** **Decided 2026-08-09: primary = synlig, TLS at Pangolin, single shared token (§8.1)** — mind the loopback Host-pin trap in §6.2. |
|
||||
| **1.5 — opencode env propagation** | hours | Make the `mcp.mempalace` subtree env-authoritative in `generate-config.py`, gated by a generated-value fingerprint (§4.1). Independent of the rest of this RFC. Without it, adopting *or reverting* the opt-in on an existing opencode container needs a manual sidecar merge or a `docker volume rm` — which also blocks R6 reversibility |
|
||||
| **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. **Not** "fixes opencode" — opencode's *transport* is already fine after Phase 1; what edge adds there is offline/local-first, since `generate-config.py`'s switch is remote **or** local with no failover. **Ships with the §1.2 opt-in wiring (compose + `.env.example` + a third branch in the existing `generate-config.py`) and must pass the R1 acceptance test.** |
|
||||
| **3 — pull replication** | ~1 week | **DEFERRED (2026-08-08).** Server-side op-log with monotonic seq → each edge a full offline replica. Only needed if a laptop must hold *everything* offline. Accepted trade-off: offline recall = own writes + last-reachable state. |
|
||||
@@ -742,7 +821,16 @@ for online clients.
|
||||
re-call with identical oversized content cannot duplicate a legacy pre-#1539 single-row write. The
|
||||
normal path is therefore safe by construction. What remains untested is the abnormal one: a chunk set
|
||||
left partial by a crash or kill *mid-upsert*, which would then look present and stay truncated. Worth a
|
||||
single fault-injection test before trusting bulk replay of oversized drawers.
|
||||
single fault-injection test before trusting bulk replay of oversized drawers. **Still untested as of
|
||||
2026-08-14** — the first join did *not* exercise it.
|
||||
|
||||
⚠️ **But 2026-08-14 surfaced a benign mechanism that mimics it, and it will generate false alarms during
|
||||
any join verification (§4.4).** Updating a drawer preserves `drawer_id`, **re-chunks the new content, and
|
||||
deletes the surplus chunk rows**. So two palaces holding two *revisions* of one drawer legitimately
|
||||
differ in chunk count and chunk-id set, with zero data loss — and the shorter side returns a clean
|
||||
"not found" for the chunk ids it no longer needs, which looks exactly like the truncation this entry
|
||||
warns about. **The absence of a derived chunk id is not evidence of loss.** Distinguish them by prefix
|
||||
test and final-chunk arithmetic per §4.4, not by id-set membership.
|
||||
4. **`migrate.py` as a bootstrap tool.** `extract_drawers_from_sqlite()` reads `{id, document,
|
||||
metadata}` straight out of chroma's SQLite (bypassing the chromadb API) and re-`add`s them with ids
|
||||
and metadata preserved and **embeddings recomputed** (`migrate.py:326`) — the right shape for a
|
||||
|
||||
@@ -3,6 +3,18 @@
|
||||
Companion to [`rfc-001-global-palace.md`](./rfc-001-global-palace.md). Records what was actually done
|
||||
on the primary, with verified evidence, so the next session (or the next machine) does not re-derive it.
|
||||
|
||||
> **Status 2026-08-14 17:00 — SUPERSEDED IN PART. The primary is live, exposed, and seeded.**
|
||||
> Serving since 2026-08-12 at `https://mempalace.jordbo.se/mcp`. Seeded 2026-08-14 15:07 from
|
||||
> EMB-7KJ4VR4G's palace — 14,777 drawers / 9 wings / 16,337 embeddings / KG 46 entities, 34 triples,
|
||||
> now 14,803 drawers. One client (EMB-7KJ4VR4G's pi-devbox container) is flipped and verified
|
||||
> end-to-end. Both Phase 0 blockers below are cleared.
|
||||
>
|
||||
> **Read §3 "Deliberately NOT done" as a record of the 2026-08-10 state, not of today's** — every
|
||||
> item in it has since been done. And before running anything in §5 Rollback, read the warning at the
|
||||
> top of it: `~/.mempalace` on synlig is no longer disposable.
|
||||
|
||||
Original status, kept for the record:
|
||||
|
||||
**Status 2026-08-10 00:30 — Phase 0 prep complete. Not serving. Nothing exposed.**
|
||||
Blocked on two things, both deliberately left to Joakim: the Pangolin update on nyvaken, and one `sudo`.
|
||||
|
||||
@@ -62,10 +74,21 @@ ln -sfn palace/knowledge_graph.sqlite3 ~/.mempalace/knowledge_graph.sqlite3
|
||||
ln -sfn palace/known_entities.json ~/.mempalace/known_entities.json
|
||||
```
|
||||
|
||||
Relative targets, so a home-directory move survives. `hallways.json` is deliberately **not** symlinked:
|
||||
Relative targets, so a home-directory move survives. `hallways.json` was originally left **un**symlinked:
|
||||
it is already palace-derived, and its HOME path is a warning-only legacy probe (`hallways.py:73-95`) that
|
||||
never auto-migrates.
|
||||
|
||||
> **Update 2026-08-14 — `hallways.json` is now symlinked too**, during the seeding session:
|
||||
> ```sh
|
||||
> ln -sfn palace/hallways.json ~/.mempalace/hallways.json
|
||||
> ```
|
||||
> Rationale changed: the point is no longer "only symlink what the code demands" but *all real state
|
||||
> lives under `palace/` as a single backup unit*, so one `palace/` copy is a complete copy. All three
|
||||
> parent-level paths now resolve, which matters because mempalace 3.6.0 resolves these three paths
|
||||
> inconsistently (MCP server: palace-relative; KG CLI default: `~/.mempalace`; `hallways.json`:
|
||||
> `dirname(palace_path)`; `known_entities.json`: hardcoded `~`). Revert by deleting the symlink if it
|
||||
> ever causes trouble.
|
||||
|
||||
Verified the symlink assumption rather than trusting it (`python3 sqlite3` on synlig, temp dir):
|
||||
|
||||
| Check | Result |
|
||||
@@ -119,10 +142,21 @@ standing up a network-reachable service while you were asleep was not mine to de
|
||||
|
||||
## 3. Deliberately NOT done
|
||||
|
||||
> **⚠️ Historical — this section describes 2026-08-10 and is no longer true.** All five items were
|
||||
> done between 2026-08-12 and 2026-08-14. Kept because the *reasoning* for deferring them is still
|
||||
> the record of why the order was chosen. Current state per item is inlined below.
|
||||
|
||||
- **Nothing is serving.** No listener on 8765; no mempalace process. Re-verified at the end of the run.
|
||||
→ **Now serving** since 2026-08-12 (`mempalace-serve.service`, `systemctl --user`), reachable at
|
||||
`https://mempalace.jordbo.se/mcp` via newt/Pangolin.
|
||||
- **No client `.env` was touched.** Your working setup is exactly as you left it (R6: reversible).
|
||||
→ **One client flipped 2026-08-14**: four variables on EMB-7KJ4VR4G, `docker-compose.yaml` unchanged.
|
||||
Still reversible in ~30s (§3.8 of [`phase-1-exposure-runbook.md`](./phase-1-exposure-runbook.md)).
|
||||
- **No data joined.** The palace is empty. The §4.4 join needs the diary-dedup decision (§7.6) first —
|
||||
replaying diaries today duplicates them, and the primary is the one place that must stay clean.
|
||||
→ **Seeded 2026-08-14** from *one* palace by file-level copy. This sidestepped §7.6 rather than
|
||||
solving it: a file-level copy replays no diaries, so it cannot duplicate them. **§7.6 is still a
|
||||
hard blocker for the second machine to join.**
|
||||
- **nyvaken untouched.** Read nothing, changed nothing.
|
||||
- **No sudo.** `sudo -n` on synlig requires a password.
|
||||
|
||||
@@ -167,13 +201,48 @@ standing up a network-reachable service while you were asleep was not mine to de
|
||||
pointing any client at it.
|
||||
6. **Then, and only then**, Phase 1 client flip — one machine first, and remember opencode containers
|
||||
need the §4.1 sidecar merge (or Phase 1.5) before the `.env` takes effect.
|
||||
7. **Before the first join:** settle §7.6 diary dedup, then dry-run §4.4 from **one** palace and check
|
||||
counts.
|
||||
7. **Before the first join:** settle §7.6 diary dedup, then dry-run §4.4 from **one** palace.
|
||||
> **Correction 2026-08-14 — do NOT verify a join "by checking counts", which is what this item
|
||||
> originally said.** Counts are not evidence, in either direction. `mempalace status` counts
|
||||
> **chunk rows**, not logical drawers (3 drawers plus one 2-chunk diary presented as +9), and chunk
|
||||
> counts legitimately differ between two palaces whenever a drawer was updated on either side,
|
||||
> because an update re-chunks to the new length and deletes the surplus rows. Diffing chunk-id sets
|
||||
> is a useful first pass but **over**-reports: a chunk id present on one side only is the ordinary
|
||||
> signature of an edit, not of loss. This cost real time on 2026-08-14 — a missing
|
||||
> `chunk_000007` was read as a truncated seed, when in fact the two palaces held two revisions of
|
||||
> one drawer and nothing was lost. **Adjudicate by fetching the parent drawer on both sides and
|
||||
> comparing the reassembled `content`.**
|
||||
|
||||
## 5. Rollback
|
||||
|
||||
> **⚠️ STOP — 2026-08-14. Do not run this block as it was originally written.** `~/.mempalace` on
|
||||
> synlig is now the fleet primary. That tree holds the only central palace (14,803 drawers, seeded
|
||||
> from EMB-7KJ4VR4G) **and the server's bearer token** at `~/.mempalace/server/<hash>/token` — the
|
||||
> single credential every flipped client authenticates with, of which there is no second copy.
|
||||
> `rm -rf ~/.mempalace` destroys both. The original comment ("empty today — check before running once
|
||||
> it isn't") is far too soft for a destructive command someone runs under pressure, which is exactly
|
||||
> why it is being replaced rather than amended.
|
||||
|
||||
Stopping the service is safe and reversible on its own, and is the whole of what "rollback" should
|
||||
normally mean now:
|
||||
|
||||
```sh
|
||||
systemctl --user disable --now mempalace-serve # if enabled
|
||||
~/.local/bin/uv tool uninstall mempalace
|
||||
rm -rf ~/.mempalace # empty today — check before running once it isn't
|
||||
systemctl --user disable --now mempalace-serve # clients fail CLOSED — they lose the palace
|
||||
# tools; they do NOT fall back to a local palace
|
||||
```
|
||||
|
||||
To genuinely decommission the primary, in this order:
|
||||
|
||||
1. Flip every client back first (§3.8 of [`phase-1-exposure-runbook.md`](./phase-1-exposure-runbook.md),
|
||||
in reverse) so nothing is pointed at a host that is about to lose its palace.
|
||||
2. Copy `~/.mempalace/palace/` **and** the token file off the host, and verify the copy by comparing
|
||||
reassembled drawer `content`, not counts (see §4 item 7).
|
||||
3. Only then remove anything. Never `rsync --delete` into `~/.mempalace` — the token lives inside it.
|
||||
|
||||
The two destructive steps below were written on 2026-08-10, when `~/.mempalace` was genuinely empty.
|
||||
Kept for the record; **must not be run while the primary is live**:
|
||||
|
||||
```sh
|
||||
~/.local/bin/uv tool uninstall mempalace
|
||||
rm -rf ~/.mempalace # ⚠️ DESTROYS THE FLEET PALACE AND THE ONLY TOKEN
|
||||
```
|
||||
|
||||
+61
-5
@@ -40,7 +40,10 @@ dependencies (~300 MB).
|
||||
describes. Skipped on resume/fork (context is already in the thread).
|
||||
4. **Automatic transcript feeding** (`session_shutdown`, and a debounced
|
||||
`agent_settled`): stages + mines this pi installation's own session
|
||||
transcripts into the palace with no user action needed. Unlike the diary
|
||||
transcripts into the palace with no user action needed — **as of
|
||||
mempalace-toolkit `29e660e` (2026-08-12); see the version gate below,
|
||||
because "the extension is installed" does not imply "this copy can feed"**.
|
||||
Unlike the diary
|
||||
below, this needs no LLM turn — it's a subprocess + a tool call — so it
|
||||
*can* run on `session_shutdown` where the diary cannot. See
|
||||
[Automatic transcript feeding](#automatic-transcript-feeding).
|
||||
@@ -53,6 +56,26 @@ dependencies (~300 MB).
|
||||
|
||||
## Automatic transcript feeding
|
||||
|
||||
> **⚠️ Version gate — requires mempalace-toolkit ≥ `29e660e` (2026-08-12), and "installed" is not
|
||||
> the same question as "capable".** Feeding was added to this extension on 2026-08-12. A copy baked
|
||||
> into a container image built before that date has *no* feed path at all — its entire
|
||||
> `session_shutdown` handler is `client.stop()` — and it fails the only way a memory system must not:
|
||||
> silently, looking exactly like a healthy run with nothing to do.
|
||||
>
|
||||
> **Check the deployed artifact, never the repo.** `/opt/*` in an image is baked at build time and can
|
||||
> be days behind a bind-mounted clone, and `~/.pi/agent/extensions/mempalace.ts` is usually a symlink
|
||||
> *into* that baked copy:
|
||||
>
|
||||
> ```sh
|
||||
> grep -c MEMPALACE_FEED "$(readlink -f ~/.pi/agent/extensions/mempalace.ts)" # 0 = cannot feed
|
||||
> ```
|
||||
>
|
||||
> Zero hits means this machine needs the fallback recipes in
|
||||
> [`contrib/`](../../contrib/README.md) until it is rebuilt, regardless of what the toolkit repo's HEAD
|
||||
> looks like. Date the deployed copy with `stat` plus that content probe — not `git log`, which fails
|
||||
> with *"detected dubious ownership"* inside a root-owned `/opt` tree. As of 2026-08-14 the whole
|
||||
> pi-devbox fleet fails this check.
|
||||
|
||||
The bridge feeds this pi installation's own session transcripts into the
|
||||
palace by itself — no scheduler, no cron, no manual invocation. It fires on
|
||||
`session_shutdown` (covers quit, `/new`, `/resume`, `/fork`) and on a
|
||||
@@ -117,10 +140,27 @@ chosen at load time:
|
||||
lives wherever that process opens it (default `~/.mempalace`). This is the
|
||||
hardened path with per-request timeouts and respawn/self-heal (below).
|
||||
- **External** — set `MEMPALACE_REMOTE_URL` to a MemPalace HTTP endpoint (e.g.
|
||||
`http://mempalace.lan:8765/mcp`) and the bridge connects over HTTP instead,
|
||||
spawning no local process. Use this to share **one** palace across several
|
||||
harnesses/containers (pi + opencode + native). `MEMPALACE_REMOTE_TOKEN`, if
|
||||
set, is sent as `Authorization: Bearer <token>`.
|
||||
`https://mempalace.jordbo.se/mcp`, the live fleet primary — full path
|
||||
including `/mcp`, no trailing slash) and the bridge connects over HTTP
|
||||
instead, spawning no local process. Use this to share **one** palace across
|
||||
several harnesses/containers (pi + opencode + native).
|
||||
`MEMPALACE_REMOTE_TOKEN`, if set, is sent as `Authorization: Bearer <token>`.
|
||||
Use `https://` for anything crossing a network — the plaintext `http://`
|
||||
example that stood here until 2026-08-14 predated the reverse proxy.
|
||||
|
||||
The two transports are **either/or**, decided once at load time: with the URL
|
||||
set, writes go **only** to the remote palace. There is no dual-write, no
|
||||
local mirror, and no local `mempalace-mcp` process at all.
|
||||
|
||||
⚠️ **Consequence: once `MEMPALACE_REMOTE_URL` is set, the `mempalace` CLI on
|
||||
that machine is no longer a valid way to inspect or feed the palace the agent
|
||||
is using.** The CLI has no remote support whatsoever — its only selector is
|
||||
`--palace <path>` — so it reads and writes the LOCAL on-disk archive. After a
|
||||
flip that archive is frozen, yet `mempalace status` / `mempalace search`
|
||||
still report a plausible drawer count and look exactly like success: a
|
||||
false-positive machine. Memories filed with the CLI post-flip land in the
|
||||
dead archive, not in the shared palace. Use the agent's own palace tools
|
||||
(which go over HTTP), and mine backfills **on the palace host**.
|
||||
|
||||
Serve such an endpoint with `mempalace serve --host 172.17.0.1 --port 8765`
|
||||
(the `pi-devbox` / `opencode-devbox` repos ship a
|
||||
@@ -155,6 +195,22 @@ If `mempalace-mcp` can't be spawned (PATH missing, binary crashes at
|
||||
startup, …) the extension logs to stderr and returns early. pi keeps
|
||||
working without palace tools rather than refusing to start.
|
||||
|
||||
**In remote mode the triggers differ but the outcome is identical.** An
|
||||
unreachable server, a DNS failure, or an HTTP 401 from a wrong/expired token
|
||||
all end the same way: after bounded retries the extension prints
|
||||
`mempalace-mcp unavailable after retries; continuing without palace tools` and
|
||||
**does not register the palace tools**.
|
||||
|
||||
It is **fail-closed, not fail-local**: it does *not* quietly fall back to the
|
||||
local palace, so a remote outage can never scatter memories into a local copy
|
||||
nobody will look at again. The practical corollary, worth knowing before you
|
||||
debug the wrong layer: **"the agent has no `mempalace_*` tools" is the
|
||||
expected symptom of a server, token, or DNS fault**, not of a broken install.
|
||||
Diagnose it with a direct `curl` to `MEMPALACE_REMOTE_URL` — see
|
||||
[`docs/phase-1-exposure-runbook.md`](../../docs/phase-1-exposure-runbook.md)
|
||||
§3.8. The design rationale for de-registering rather than degrading is in
|
||||
[`docs/rfc-001-global-palace.md`](../../docs/rfc-001-global-palace.md) §2 and §4.1.
|
||||
|
||||
## Identity
|
||||
|
||||
`agent_name` for diary calls comes from `$MEMPALACE_AGENT_NAME`, defaulting
|
||||
|
||||
Reference in New Issue
Block a user