Both corrections come from the first real Phase 1 start on synlig (2026-08-12).
1. The Pangolin resource target was documented as bare `172.17.0.1:8765` with no
scheme, and the obvious guess from that is `https://` -- which cannot work.
`contrib/systemd/mempalace-serve.service` runs `serve --host 172.17.0.1
--port 8765` with no --tls-cert, so the primary speaks plaintext HTTP; TLS
terminates at Pangolin, which is the whole point of the RFC 6.2 decision.
Point a proxy at https:// and it attempts a TLS handshake against a plaintext
listener: 502 from outside, while `curl 172.17.0.1:8765/healthz` on the box
still says ok -- a confusing pair of symptoms. Now spelled `http://` with the
failure mode named, in the runbook and in the unit's comments.
2. `curl -s 127.0.0.1:8765/healthz` was documented as "expect 403". Wrong: the
real run returns empty. With a docker0-only bind nothing is listening on
loopback, so the connection is refused at TCP level before any header is sent
(%{http_code} -> 000, exit 7). The 403 is the *loopback-bind* case verified
2026-08-10 -- server on 127.0.0.1 answering a proxy-forwarded foreign Host.
Two distinct behaviours had been collapsed into one expectation in three
places (both runbooks and the unit).
Worth stating why the correction matters rather than just fixing it: refusal
is the *stronger* signal. A 403 proves only that a request was rejected; a
refused connection proves the loopback and LAN surface is not listening at
all. Someone who expected 403, saw silence, and "fixed" it by rebinding to
0.0.0.0 would have converted a correct configuration into an exposed one.
The docs now also say what to do if it hangs, or if ss shows 0.0.0.0:8765.
synlig is greenfield — no mempalace, no ~/.mempalace at all — so Phase 0
became "provision correctly from birth" rather than "migrate carefully".
Nothing is serving; no client config was touched.
Done:
- mempalace 3.6.0 installed via uv, pinned to the fleet version (the id
recipes and idempotency probes this RFC leans on are version-specific).
- Embedder pre-warmed. This was the real unknown: the first embed pulls a
79.3 MB ONNX model from the chroma CDN, and an egress-filtered work VM
would have failed at the worst moment — the first client write. Pulled
at ~20 MB/s, no proxy interference. Done in a throwaway palace so the
real one never saw it.
- Palace at the stock default ~/.mempalace/palace, so no config file and
no MEMPALACE_PALACE_PATH is needed on synlig at all.
Two corrections to the RFC, both from provisioning rather than reading:
- §7.1 was understated. DEFAULT_PALACE_PATH (~/.mempalace/palace) and
DEFAULT_KG_PATH (~/.mempalace/knowledge_graph.sqlite3) already differ
with stock defaults, so the KG split is out-of-the-box behaviour, not a
consequence of a custom --palace, and it is permanent rather than
one-time: serve always passes --palace, any CLI call without it uses
the HOME path. A one-time mv does not fix that, it only picks which of
the two files gets populated. Fixed instead by converging both rules on
one inode via relative symlinks, and verified the load-bearing
assumption: a dangling symlink is created on connect, -wal/-shm land
next to the target (so the palace dir stays a self-contained backup
unit, which is the part that mattered), cross-path read works, same
inode. hallways.json deliberately left alone — already palace-derived,
HOME path is a warning-only probe.
- §6.2 upgraded from "test early" to verified: 11/11 as predicted. The
headline is that the safe-sounding reflex is the failure mode — loopback
bind + proxy forwarding a public Host is 403, non-loopback is 200. Also
confirmed Origin is never relaxed on either bind, and /healthz is
Host/Origin-gated but token-free, so it works as the tunnel probe.
Recommends binding the docker0 gateway over 0.0.0.0: non-loopback so
the pin relaxes, but reachable only from the host and its containers.
New docs/synlig-primary-runbook.md carries the discovered facts about the
box, the evidence tables, an explicit "deliberately not done" list, and
tomorrow's ordered steps. New contrib/systemd/mempalace-serve.service
carries the bind rationale inline so nobody "fixes" it back to loopback;
staged on synlig with a .staged suffix so systemd cannot pick it up by
accident.
Flagged for tomorrow: synlig has no newt/tunnel client (docker ps shows
only the Gitea runner and digikam), so Pangolin on nyvaken cannot reach
it until one is added — easy to miss, because Pangolin will look healthy
from its own side.
Joakim Persson