joakimp/mempalace-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: why the tunnel and the feeder's SSH path are not redundant; newt done

Browse Source 
Asked "why do we need Pangolin if you proposed rsync/ssh?", and the runbook did
not actually answer it -- it stated both were needed without saying why neither
substitutes. New section 1.3:

  - Pangolin/HTTPS carries the MCP tool surface (search, add_drawer,
    diary_write, kg_*) -- every live tool call, from any MCP client.
  - SSH/rsync carries transcript *files* only, because mempalace_mine expands
    its source path server-side, so the server can only mine its own disk.

HTTPS alone is a palace you can query but cannot feed; SSH alone is files with
no query API. The rsync is not a transport preference, it is a workaround for
where `mine` resolves paths.

Records honestly that `ssh -L 8765:172.17.0.1:8765 synlig` *would* replace the
tunnel for MCP, and why we don't: synlig dials out (reaching for a dial-out
tunnel is itself the evidence inbound was unavailable), MCP clients want a
durable URL rather than a per-session forward, and the forward must be up on
every device before every session.

And the design's weak point, stated instead of glossed: the rsync runs
client -> synlig, so mining needs synlig's SSH reachable *from the client*. Were
that true everywhere, no tunnel would be needed for MCP either. Honest
expectation after Phase 1 is therefore: query/write from anywhere, mine only
from devices that can reach synlig's SSH. Section 4 now carries the upstream ask
that would close the gap -- have the feeder send content over MCP (add_drawer /
diary_write, which it already calls) instead of asking the server to mine a path
it must first rsync there.

New section 3.7, a live trap for the imminent client flip:
MEMPALACE_REMOTE_URL on its own does not degrade to local feeding, it *stops*
feeding. auto mode switches to remote as soon as the URL is set (:286) and
remote mode then exits 1 without MEMPALACE_PI_SSH_TARGET (:298-300), before
anything is staged or filed -- so a cron feeder just starts failing, and the
loudest symptom is silence. Two safe orders given: set all three variables in
one edit, or set URL+token and pin --mode local until the SSH target exists.

newt is installed on synlig and connected to Pangolin (done 2026-08-12), marked
here and in the synlig runbook's item 2; the blocker is now item 3, the one
sudo. Added the follow-up that "connected to Pangolin" only proves newt reached
nyvaken -- reaching the *palace* is a separate claim that fails independently,
so probe 172.17.0.1:8765/healthz from inside newt's namespace.

All seven code citations verified against the source at commit time.
This commit is contained in:
Joakim Persson
2026-08-13 00:15:57 +02:00
parent 29e660e18f
commit 00a95d1a2f
2 changed files with 83 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/phase-1-exposure-runbook.md
+71 -1
View File
@@ -75,6 +75,35 @@ Two implications people miss:
reviewable — a committed compose file — where duplicates are visible. That, not username hygiene in
`.env`, is the discipline this design actually asks of you.
### 1.3 "Then why Pangolin at all, if the feeder uses SSH?"
Because they are not alternatives — they carry different traffic, and neither substitutes for the other.
| | Pangolin/newt (HTTPS) | SSH + rsync |
| --- | --- | --- |
| Carries | the **MCP tool surface**: `search`, `add_drawer`, `diary_write`, `kg_*` — every live tool call | **transcript files only**, once per session or cron run |
| Used by | the pi extension, opencode `type:remote`, any MCP client | the feeder, internally (`bin/mempalace-pi-session:677-680`) |
| Needed because | clients need one stable URL, reachable from wherever they are | `mempalace_mine` expands its source path **server-side**, so the server can only mine files on its own disk |
HTTPS alone is a palace you can query but cannot feed. SSH alone is files shipped with no live query API.
The rsync is not a transport preference; it is a workaround for *where `mine` resolves paths*.
**Could SSH replace Pangolin?** Partly, and it is worth being honest about it:
`ssh -L 8765:172.17.0.1:8765 synlig` yields a working local MCP endpoint with no public HTTPS at all.
Three reasons this runbook does not do that:
1. **Direction.** synlig dials *out* through newt. That we reached for a dial-out tunnel rather than a
port-forward is itself the evidence that inbound was not available — a corporate host does not accept
connections from a phone on a foreign network.
2. **MCP clients want a durable URL**, not a per-session forwarded port. opencode `type:remote` takes a
URL; a forward that drops takes the tools down mid-session.
3. The forward must be up on **every device before every session**. Pangolin is up once.
**The weak point, stated plainly.** The rsync runs *client → synlig*, so it needs synlig's SSH reachable
**from the client**. Were that already true everywhere, no tunnel would be needed for MCP either. So the
honest expectation after Phase 1 is: **query and write from anywhere, mine only from devices that can
reach synlig's SSH** (corporate network / VPN / LAN). See §4 for the change that would remove that limit.
---
## 2. The bind trap, in full
@@ -131,7 +160,7 @@ cat ~/.mempalace/server/f5d849287f6d73f0141b29d7/token
Directory name is `sha256(realpath(palace))[:24]` — it changes if the palace path ever moves. Store via the
`.env.age` flow, 0600 (RFC §6.2).
### 3.3 newt on synlig
### 3.3 newt on synlig — ✅ done 2026-08-12 (installed, connected to Pangolin)
synlig runs Docker (Gitea Actions runner + digikam) but **no tunnel client** — runbook §4.2. Pangolin on
nyvaken cannot dial in; synlig must dial out. Add a `newt` container with the credentials Pangolin issues
@@ -144,6 +173,18 @@ host, before touching DNS.
> synlig has 7.8 GiB shared with a CI runner (runbook §1). newt is small, but do not colocate anything
> else here casually.
**Confirm next, now that newt is up.** "Connected to Pangolin" proves newt reached *nyvaken* — a different
claim from newt reaching *the palace*, and the two fail independently:
```sh
# from inside newt's namespace, not from the host
docker exec <newt-container> wget -qO- http://172.17.0.1:8765/healthz # expect ok
```
If that hangs or refuses while the Pangolin dashboard shows the site online, the tunnel is fine and the
*target* is wrong — look at the resource's upstream address (§3.5), not at newt. Note this check needs
§3.1 done first: if `mempalace-serve` is not running yet, it fails for that reason alone.
### 3.4 DNS at the web hotel
One CNAME: `mempalace` → **the same target your existing Pangolin resources use** (nyvaken's public
@@ -174,10 +215,39 @@ The 401 check matters as much as the 200: it is the only evidence that the thing
internet is not open. Then, and only then, Phase 1 client flip — **one machine first** (RFC §8), and
remember opencode containers need the §4.1 sidecar merge before their `.env` takes effect.
### 3.7 Flipping a client: set three variables, or none
⚠ **`MEMPALACE_REMOTE_URL` on its own does not degrade to local feeding — it stops feeding.** `auto` mode
switches to `remote` the moment the URL is set, and remote mode then refuses to run without an SSH target:
```sh
auto) if [[ -n "$REMOTE_URL" ]]; then MODE="remote"; else MODE="local"; fi ;; # :286
...
command -v rsync >/dev/null 2>&1 || { echo "error: rsync not found ..."; exit 3; } # :297
if [[ -z "$SSH_TARGET" ]]; then
echo "error: MEMPALACE_PI_SSH_TARGET unset (needed for --mode remote)" >&2; exit 1 # :298-300
fi
```
That exit happens **before anything is staged or filed**, and a cron-driven feeder will simply start
failing — the loudest symptom is silence, which is the hardest kind to notice. Two safe orders:
- **Both paths at once:** set `MEMPALACE_REMOTE_URL`, `MEMPALACE_REMOTE_TOKEN` **and**
`MEMPALACE_PI_SSH_TARGET` (plus `MEMPALACE_PI_DEVICE`, §1.2) in the same edit.
- **HTTPS first, mining later:** set the URL and token, and pin the feeder to `--mode local` until the SSH
target exists. Tools then read/write the shared palace while transcripts keep landing in the local one.
Either way, run the feeder once by hand and read its exit code before trusting the timer. This is
precisely the failure "one machine first" is meant to contain.
---
## 4. Still open
- **Feeding without SSH — the upstream ask that would close §1.3's gap.** Have the feeder send *content*
over MCP (`add_drawer` / `diary_write`, which it already calls) instead of asking the server to mine a
path it must first rsync there. HTTPS would then be genuinely sufficient and mining would work from any
network. Until then, mining is limited to devices that can reach synlig's SSH.
- **Per-device tokens** — Phase 4. Until then `origin_device` is advisory (§1.1).
- **§7.6 diary dedup** must be settled *before* the first §4.4 join; replay duplicates every entry.
- **§7.2**: never run `mempalace sync` against the shared palace. Doubly true now that the pi/opencode
docs/synlig-primary-runbook.md
+12 -8
View File
@@ -129,16 +129,20 @@ standing up a network-reachable service while you were asleep was not mine to de
## 4. Tomorrow, in order
> **2026-08-12: items 1–2 and 5 now have their own runbook —**
> [`phase-1-exposure-runbook.md`](./phase-1-exposure-runbook.md). Pangolin on nyvaken is updated (done).
> newt is still missing on synlig. That doc also records why per-device Pangolin users are the wrong layer,
> and an additional loopback finding: a loopback bind does not merely 403, it also silently starts the
> server with **no token at all** (auto-minting is gated on the bind being non-loopback).
> [`phase-1-exposure-runbook.md`](./phase-1-exposure-runbook.md). Pangolin on nyvaken is updated (done),
> and **newt is now installed on synlig and connected to Pangolin (done 2026-08-12)** — so the blocker is
> now item 3, the one `sudo`. That doc also records why per-device Pangolin users are the wrong layer, why
> the HTTPS tunnel and the feeder's SSH path are **not** redundant (§1.3), the client-flip variable trap
> (§3.7), and an additional loopback finding: a loopback bind does not merely 403, it also silently starts
> the server with **no token at all** (auto-minting is gated on the bind being non-loopback).
1. **Pangolin update on nyvaken** (yours). ✅ done 2026-08-12.
2. **⚠️ synlig has no tunnel client.** `docker ps` shows only the Gitea runner and digikam — no `newt`.
Pangolin on nyvaken cannot reach synlig by itself; synlig needs a newt container (or an equivalent
outbound tunnel) added. Easy to miss because Pangolin will look healthy on its own side. If newt runs
in Docker here, the docker0 bind above is already correct for it.
2. ~~**⚠️ synlig has no tunnel client.**~~ ✅ **done 2026-08-12** — newt installed and connected to Pangolin.
(Kept for the reasoning: `docker ps` showed only the Gitea runner and digikam. Pangolin on nyvaken
cannot reach synlig by itself; synlig had to dial out. Easy to miss because Pangolin looks healthy on
its own side — which is also why "connected" is not yet proof it can reach the palace: verify
`172.17.0.1:8765/healthz` from *inside* newt's namespace, exposure runbook §3.3.) Since newt runs in
Docker here, the docker0 bind above is already correct for it.
3. **One sudo, then start** (the unit is already staged; just drop the suffix):
```sh
sudo loginctl enable-linger ecsjper