docs: proxy target is http:// not https://; loopback probe refuses, it does not 403

Both corrections come from the first real Phase 1 start on synlig (2026-08-12).

1. The Pangolin resource target was documented as bare `172.17.0.1:8765` with no
   scheme, and the obvious guess from that is `https://` -- which cannot work.
   `contrib/systemd/mempalace-serve.service` runs `serve --host 172.17.0.1
   --port 8765` with no --tls-cert, so the primary speaks plaintext HTTP; TLS
   terminates at Pangolin, which is the whole point of the RFC 6.2 decision.
   Point a proxy at https:// and it attempts a TLS handshake against a plaintext
   listener: 502 from outside, while `curl 172.17.0.1:8765/healthz` on the box
   still says ok -- a confusing pair of symptoms. Now spelled `http://` with the
   failure mode named, in the runbook and in the unit's comments.

2. `curl -s 127.0.0.1:8765/healthz` was documented as "expect 403". Wrong: the
   real run returns empty. With a docker0-only bind nothing is listening on
   loopback, so the connection is refused at TCP level before any header is sent
   (%{http_code} -> 000, exit 7). The 403 is the *loopback-bind* case verified
   2026-08-10 -- server on 127.0.0.1 answering a proxy-forwarded foreign Host.
   Two distinct behaviours had been collapsed into one expectation in three
   places (both runbooks and the unit).

   Worth stating why the correction matters rather than just fixing it: refusal
   is the *stronger* signal. A 403 proves only that a request was rejected; a
   refused connection proves the loopback and LAN surface is not listening at
   all. Someone who expected 403, saw silence, and "fixed" it by rebinding to
   0.0.0.0 would have converted a correct configuration into an exposed one.
   The docs now also say what to do if it hangs, or if ss shows 0.0.0.0:8765.
This commit is contained in:
Joakim Persson
2026-08-13 16:01:02 +02:00
parent 00a95d1a2f
commit 08e344b047
3 changed files with 38 additions and 5 deletions
+12 -1
View File
@@ -52,5 +52,16 @@ WantedBy=default.target
# sudo loginctl enable-linger $USER # else the unit dies with your login
# systemctl --user daemon-reload
# systemctl --user enable --now mempalace-serve
# curl -s localhost:8765/healthz # -> 403 (expected! see bind note)
# curl -s 172.17.0.1:8765/healthz # -> ok
# curl -s localhost:8765/healthz # -> nothing: connection refused, exit 7.
# Corrected 2026-08-12 (the earlier "-> 403" here was wrong). With this
# docker0-only bind nothing listens on loopback, so the connection is
# refused before any header is sent. The 403 above is the *loopback-bind*
# case: server on 127.0.0.1 receiving a forwarded foreign Host header.
# Refusal is the stronger signal -- it proves loopback/LAN isn't listening.
#
# ── Reverse-proxy target: use http://, not https:// ──────────────────────────
# There is no --tls-cert below, so this server speaks PLAINTEXT HTTP. Point
# the tunnel/proxy resource at http://172.17.0.1:8765. Targeting https:// makes
# the proxy attempt a TLS handshake against a plaintext listener: 502 from
# outside, while curl on 172.17.0.1 still says ok. TLS belongs at the proxy.
+17 -2
View File
@@ -147,10 +147,19 @@ cd ~/.config/systemd/user && mv mempalace-serve.service.staged mempalace-serve.s
systemctl --user daemon-reload && systemctl --user enable --now mempalace-serve
curl -s 172.17.0.1:8765/healthz # expect ok
curl -s 127.0.0.1:8765/healthz # expect 403 — correct, not a bug (§2)
curl -s 127.0.0.1:8765/healthz # expect NOTHING — connection refused, exit 7 (see below)
ss -ltnp | grep 8765 # expect 172.17.0.1:8765 only
```
**The loopback probe returns empty, not 403** — corrected 2026-08-12 against the real run. Nothing is
listening on `127.0.0.1`, so the connection is refused at TCP level and `curl -s` prints nothing; check it
with `-w '%{http_code}'``000` and `$?``7`. The 403 belongs to a *different* configuration: server
bound **to loopback**, receiving a proxy-forwarded foreign `Host:` (§2, verified 2026-08-10). With a
docker0-only bind you cannot get 403 from loopback, because you never get far enough to send a header.
Refusal is the stronger signal of the two: it proves the loopback and LAN surface is not listening at all.
If it *hangs* instead, or `ss` shows `0.0.0.0:8765`, stop — that is not this configuration.
```sh
### 3.2 Collect the shared token
```sh
@@ -195,7 +204,13 @@ hostname). RFC §6.2 costed this as *"one DNS record per service on the web hote
### 3.5 Pangolin resource
- Target: newt site → `172.17.0.1:8765`, path `/mcp` (plus `/healthz` if you want the external probe).
- Target: newt site → **`http://172.17.0.1:8765`** — path `/mcp` (plus `/healthz` for the external probe).
- ⚠️ **The scheme is `http`, not `https`.** The primary runs `serve --host 172.17.0.1 --port 8765` with no
cert (`contrib/systemd/mempalace-serve.service`): TLS terminates **at Pangolin**, which is the entire
point of the §6.2 decision. Point the resource at `https://172.17.0.1:8765` and Pangolin attempts a TLS
handshake against a plaintext listener — you get a 502/Bad Gateway from outside while the server itself
looks perfectly healthy on `curl 172.17.0.1:8765/healthz`. (Got this wrong on the first attempt
2026-08-12, because this line used to omit the scheme.)
- **Auth: none at the Pangolin layer** (§1.1). TLS termination only.
- Do **not** attach an `Origin`-injecting proxy or browser client: a *present* non-loopback `Origin` is a
hard 403 with no override (runbook §2.4 B3).
+9 -2
View File
@@ -143,14 +143,21 @@ standing up a network-reachable service while you were asleep was not mine to de
its own side — which is also why "connected" is not yet proof it can reach the palace: verify
`172.17.0.1:8765/healthz` from *inside* newt's namespace, exposure runbook §3.3.) Since newt runs in
Docker here, the docker0 bind above is already correct for it.
3. **One sudo, then start** (the unit is already staged; just drop the suffix):
3. **One sudo, then start** (the unit is already staged; just drop the suffix). ✅ **done 2026-08-12**
linger enabled, unit enabled, `172.17.0.1:8765/healthz``ok`.
```sh
sudo loginctl enable-linger ecsjper
cd ~/.config/systemd/user && mv mempalace-serve.service.staged mempalace-serve.service
systemctl --user daemon-reload && systemctl --user enable --now mempalace-serve
curl -s 172.17.0.1:8765/healthz # ok
curl -s 127.0.0.1:8765/healthz # 403 — expected, not a bug (§2.4)
curl -s 127.0.0.1:8765/healthz # NOTHING — refused, exit 7 (not 403; see below)
ss -ltnp | grep 8765 # 172.17.0.1:8765 only
```
⚠ **Corrected 2026-08-12:** this line predicted `403`. The real run returned empty, which is *more*
reassuring. With the docker0-only bind nothing listens on loopback, so the connection is refused before
any header is sent (`%{http_code}` → `000`, `$?` → `7`). The 403 in §2.4 is the **loopback-bind** case:
a server on `127.0.0.1` answering a proxy-forwarded foreign `Host:`. Two different failures that were
collapsed into one expectation here.
4. **Collect the shared token** (auto-minted on first non-loopback start, stable across restarts):
```sh
cat ~/.mempalace/server/f5d849287f6d73f0141b29d7/token