docs(synlig): the transcript inbox is the primary's third moving part; and survive an unset HOME
Runbook gaps found while fixing the 2026-08-15 feed failure: - §2.5 still titled "written but not installed" and still asserting "Not installed, not enabled" — false since 2026-08-12. A reader landing there got a flat contradiction of §4 item 3. Retitled, with the verified-2026-08-16 process line, and it now states the fact §2.6 depends on: the server is a NATIVE process (no mempalace container on synlig), so it sees host paths. - New §2.6 documents ~/mempalace-feed/<device>/: why transcripts cannot travel over the HTTPS leg at all, the three client variables, and why MEMPALACE_PI_REMOTE_PATH is the trap (its /data/feed default assumes a containerized server; here it must equal the ssh-target path, and a mismatch fails with rsync succeeding and only the mine failing). Plus the operational notes that cost time: dedup keys on the absolute path so the inbox path is load-bearing, grown sessions are purged+refiled by mtime, and a client-side MCP timeout is NOT a failed mine. - Header status: counts refreshed with an explicit "treat counts as timestamps". Also, bin/mempalace-pi-session: default HOME from the passwd database when it is unset. `docker run --entrypoint="" <image>` inherits no HOME when the image config declares none, and every default is HOME-anchored under `set -u`, so the script — including the palace-free --self-test — died with "HOME: unbound variable" in exactly the environment pi-devbox's smoke suite uses. pi-devbox v1.8.0 lost a release to the same assumption from the other side.
This commit is contained in:
Joakim Persson
@@ -123,6 +123,17 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# HOME can legitimately be unset: `docker run --entrypoint="" <image>` inherits
|
||||
# no HOME when the image config declares none (pi-devbox's does not — HOME is
|
||||
# normally set by its entrypoint, which --entrypoint="" skips), and every
|
||||
# default below is HOME-anchored under `set -u`, so the script died at line 1 of
|
||||
# real work with "HOME: unbound variable". Derive it from the passwd database —
|
||||
# exactly what python's expanduser() falls back to — so the script, and
|
||||
# especially the palace-free --self-test, runs in a bare container too.
|
||||
# pi-devbox v1.8.0 lost a release to this same "the image sets HOME" assumption.
|
||||
: "${HOME:=$(python3 -c 'import os, pwd; print(pwd.getpwuid(os.getuid()).pw_dir)' 2>/dev/null || echo /tmp)}"
|
||||
export HOME
|
||||
|
||||
# ── Defaults ─────────────────────────────────────────────────────────
|
||||
AGENT="${USER:-mempalace}"
|
||||
WING="wing_conversations"
|
||||
|
||||
@@ -10,6 +10,12 @@ on the primary, with verified evidence, so the next session (or the next machine
|
||||
> now 14,803 drawers. One client (EMB-7KJ4VR4G's pi-devbox container) is flipped and verified
|
||||
> end-to-end. Both Phase 0 blockers below are cleared.
|
||||
>
|
||||
> **Update 2026-08-16 00:20 — the transcript feed is live too, and it is the primary's third moving
|
||||
> part** (alongside the HTTPS tunnel and the palace itself). See §2.6: transcripts arrive over SSH into
|
||||
> `~/mempalace-feed/<device>/` and are mined *by this host's own server process*. 15,478 drawers as of
|
||||
> that check — but treat every count in this document as a timestamp, not a fact: `status` counts chunk
|
||||
> rows, and §4 item 7 explains why counts adjudicate nothing.
|
||||
>
|
||||
> **Read §3 "Deliberately NOT done" as a record of the 2026-08-10 state, not of today's** — every
|
||||
> item in it has since been done. And before running anything in §5 Rollback, read the warning at the
|
||||
> top of it: `~/.mempalace` on synlig is no longer disposable.
|
||||
@@ -132,7 +138,15 @@ Ran on synlig, loopback and docker0 binds, then stopped. **11/11 as predicted:**
|
||||
Test script kept at `/tmp/synlig-phase0-test.sh` on this container (ephemeral — re-create from the table
|
||||
above if needed; it starts, probes and stops the server, and asserts nothing is left listening).
|
||||
|
||||
### 2.5 A start unit, written but not installed
|
||||
### 2.5 A start unit — written, staged, and (since 2026-08-12) installed and running
|
||||
|
||||
> **This subsection describes 2026-08-10. The unit is now live.** Verified 2026-08-16 00:15:
|
||||
> `systemctl --user list-units` shows `mempalace-serve.service … loaded active running`, and the
|
||||
> process is
|
||||
> `~/.local/share/uv/tools/mempalace/bin/python -m mempalace.mcp_server --transport http
|
||||
> --host 172.17.0.1 --port 8765 --palace /home/ecsjper/.mempalace/palace`.
|
||||
> Note what that means and §2.6 depends on: **the server is a NATIVE process, not a container** — it
|
||||
> sees synlig's real filesystem paths, and `docker ps` on synlig lists no mempalace container.
|
||||
|
||||
`contrib/systemd/mempalace-serve.service` — user unit, follows the existing `contrib/systemd/` style,
|
||||
carries the bind rationale inline so nobody "fixes" it back to loopback. A copy is already staged on synlig
|
||||
@@ -140,6 +154,58 @@ at `~/.config/systemd/user/mempalace-serve.service.staged` — **the `.staged` s
|
||||
systemd only reads `*.service`, so the file cannot be activated by accident, not even by a stray
|
||||
`daemon-reload`. **Not** installed, **not** enabled: it needs one `sudo loginctl enable-linger`, and
|
||||
standing up a network-reachable service while you were asleep was not mine to decide.
|
||||
(Both were done on 2026-08-12 — §4 item 3 has the exact commands that were run.)
|
||||
|
||||
### 2.6 The transcript inbox — `~/mempalace-feed/<device>/` (added 2026-08-16)
|
||||
|
||||
Flipped clients write drawers over HTTPS, but their **session transcripts** cannot travel that way:
|
||||
`mempalace_mine` resolves its `source` path *in the server process*, so the server cannot see a
|
||||
client's staged exports. `mempalace-pi-session --mode remote` therefore rsyncs each client's stage into
|
||||
a per-device inbox here and then asks the server to mine its own local path:
|
||||
|
||||
```sh
|
||||
ls ~/mempalace-feed/ # one dir per device, e.g. emb-7kj4vr4g/
|
||||
ls ~/mempalace-feed/emb-7kj4vr4g/ # pi_<session-uuid>.jsonl, mtimes preserved
|
||||
```
|
||||
|
||||
Client side, that needs three variables — and **the third one is the trap**:
|
||||
|
||||
| Variable | Value for this fleet | Why |
|
||||
| --- | --- | --- |
|
||||
| `MEMPALACE_PI_SSH_TARGET` | `ecsjper@synlig:/home/ecsjper/mempalace-feed` | where rsync puts the files |
|
||||
| `MEMPALACE_PI_DEVICE` | e.g. `emb-7kj4vr4g` | inbox subdirectory per machine |
|
||||
| `MEMPALACE_PI_REMOTE_PATH` | `/home/ecsjper/mempalace-feed` | the inbox **as the server process sees it** |
|
||||
|
||||
The feeder's default for the third is `/data/feed`, which assumes a *containerized* palace server with
|
||||
the inbox bind-mounted there. **This primary is native (§2.5), so it only ever sees host paths and the
|
||||
value must equal the path half of the SSH target.** Get it wrong and the failure is quiet in the worst
|
||||
way: rsync succeeds, the files are all present here, and only the mine fails with
|
||||
`source directory not found: '/data/feed/<device>'`.
|
||||
|
||||
That is exactly what happened on 2026-08-15, and it went unnoticed for a session because the feeder
|
||||
decided success with `'"error"' in body` — MCP returns HTTP 200 with the tool's own JSON **escaped**
|
||||
inside `result.content[].text`, so those bytes are `\"error\"`, the substring never matched, and
|
||||
`~/.pi/agent/mempalace-catchup.log` printed `Done. Wing 'wing_conversations' updated.` directly under the
|
||||
error. Fixed in `6e1f4f3`: the envelope is parsed, `--self-test` pins that exact response body, and a
|
||||
preflight warning fires whenever the ship path and `MEMPALACE_PI_REMOTE_PATH` disagree.
|
||||
|
||||
**Operational notes for this inbox:**
|
||||
|
||||
- Dedup keys on the **absolute source path**, so the inbox path is load-bearing: it must stay stable, or
|
||||
every transcript re-files under its new name. Migrating it on 2026-08-15 (from the clients' old
|
||||
container-local stage paths, which arrived with the seed) cost a full re-mine plus
|
||||
`mempalace_delete_by_source` on 6 old paths — 651 drawers purged, 1243 re-filed. `wing_conversations`
|
||||
is now keyed entirely on `/home/ecsjper/mempalace-feed/<device>/…`.
|
||||
- A **grown** session is purged and re-filed for the same path (mtime-based), so re-feeding a live
|
||||
session refreshes it instead of duplicating it. That is why the inbox keeps whole transcripts rather
|
||||
than deltas — do not "tidy" it by deleting files the palace still references.
|
||||
- `mempalace_mine` over MCP can **exceed a client's request timeout while the server keeps working and
|
||||
finishes normally**. A client-side timeout is not a failed mine: check
|
||||
`SELECT COUNT(*) FROM embedding_metadata WHERE key='source_file' AND string_value LIKE '<inbox>%'`
|
||||
(read-only, `file:…?mode=ro`) before retrying anything.
|
||||
- Health check after any client recreate, from the client: the tail of
|
||||
`~/.pi/agent/mempalace-catchup.log` should end in `Done. Wing … updated.` with no `error:` line above
|
||||
it. With the fixed feeder a broken run exits 5 and names the reason.
|
||||
|
||||
## 3. Deliberately NOT done
|
||||
|
||||
|
||||
Reference in New Issue
Block a user