c349d007e1a2e8977428f4126b0ac107c72702f3
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joakim Persson
|
ec436ed3ad |
docs: reconcile the RFC-001 docs with what is actually deployed
Audit of every doc touching the global-palace rollout against the running
fleet. Each correction below was verified against the filesystem or the host,
not against another doc:
- synlig-primary-runbook: the decommission `rm -rf ~/.mempalace` now carries a
STOP block. That tree holds the fleet palace *and* the only copy of the
bearer token every client authenticates with; the old "empty today" comment
stopped being true when the palace was seeded on 2026-08-14. Adds an ordered
safe decommission, and drops count-based join verification.
- phase-1-exposure-runbook: new S3.8, how to verify a flip actually took --
the procedure that until now existed only in an untracked handover file.
Three claims that fail independently (env var / curl / the palace-path
discriminator) plus an explicit list of checks that produce FALSE POSITIVES:
drawer counts (both sides were seeded from the same palace, and `status`
counts chunks not drawers), write-then-read through the same transport, and
the `mempalace` CLI -- which has no remote support at all, so post-flip it
reads the dead local archive and reports success.
- rfc-001: status Draft -> Phases 0-1 implemented. Records that the join was a
file-level copy, which SIDESTEPPED the S7.6 diary-dedup question rather than
answering it -- so S7.6 remains a hard blocker for the second machine, which
is the one that will actually exercise merge semantics.
- ARCHITECTURE, SKILL, contrib/README, extensions/pi/README all claimed pi
feeds the palace automatically, unconditionally. That is gated on
mempalace-toolkit >=
|
||
|
Joakim Persson
|
08e344b047 |
docs: proxy target is http:// not https://; loopback probe refuses, it does not 403
Both corrections come from the first real Phase 1 start on synlig (2026-08-12).
1. The Pangolin resource target was documented as bare `172.17.0.1:8765` with no
scheme, and the obvious guess from that is `https://` -- which cannot work.
`contrib/systemd/mempalace-serve.service` runs `serve --host 172.17.0.1
--port 8765` with no --tls-cert, so the primary speaks plaintext HTTP; TLS
terminates at Pangolin, which is the whole point of the RFC 6.2 decision.
Point a proxy at https:// and it attempts a TLS handshake against a plaintext
listener: 502 from outside, while `curl 172.17.0.1:8765/healthz` on the box
still says ok -- a confusing pair of symptoms. Now spelled `http://` with the
failure mode named, in the runbook and in the unit's comments.
2. `curl -s 127.0.0.1:8765/healthz` was documented as "expect 403". Wrong: the
real run returns empty. With a docker0-only bind nothing is listening on
loopback, so the connection is refused at TCP level before any header is sent
(%{http_code} -> 000, exit 7). The 403 is the *loopback-bind* case verified
2026-08-10 -- server on 127.0.0.1 answering a proxy-forwarded foreign Host.
Two distinct behaviours had been collapsed into one expectation in three
places (both runbooks and the unit).
Worth stating why the correction matters rather than just fixing it: refusal
is the *stronger* signal. A 403 proves only that a request was rejected; a
refused connection proves the loopback and LAN surface is not listening at
all. Someone who expected 403, saw silence, and "fixed" it by rebinding to
0.0.0.0 would have converted a correct configuration into an exposed one.
The docs now also say what to do if it hangs, or if ss shows 0.0.0.0:8765.
|
||
|
Joakim Persson
|
29e660e18f |
feeders: stage beside the palace, not in ~/.cache; document Phase 1 exposure
Staging default moves out of ~/.cache to <palace-root>/pi-stage (pi) and <palace-root>/opencode-stage (opencode), resolved with mempalace's own palace-path precedence ($MEMPALACE_PALACE_PATH -> $MEMPAL_PALACE_PATH -> ~/.mempalace/config.json -> ~/.mempalace/palace), then dirname. Why: the convos miner keys dedup on the *staged* path, so a wiped stage plus a sync scoped to include it prunes the drawers mined from those sources -- deleting memories, not a cache. Under ~/.cache that state was reachable by anything treating a cache as disposable. Staging inside the palace makes the coupling structural: the stage cannot be wiped without touching the palace itself. Overrides ($MEMPALACE_PI_STAGE / $MEMPALACE_SESSION_STAGE, --stage) are unchanged. Note the old default had never been created on any host, so this closed a latent hazard, not a live one. Measured, and the docs now claim only this much: sync prunes only within the scope it is given -- wing-only, 1299 scanned / 1299 out of scope / 0 removed; scoped at the palace root, 651 kept / 648 out of scope. The previous blanket "sync prunes every drawer" wording overstated it, which is a liability: the next reader disproves the overstatement and discards the real constraint with it. Also in this change: - cron log dir ~/.cache/mempalace-session -> ~/.cache/mempalace-logs. The stage left that namespace, so the old name now read as "the stage". - AGENTS.md: the convos miner *does* check mtime (verified against upstream convo_miner.py); the previous "no mtime check" claim was wrong. - smoke-test assertions use `mktemp -d` for --sessions-dir. One pointed at /tmp, which still held earlier synthetic transcripts, so a --dry-run exported a fake session into the real stage: --dry-run skips the mine, not the export. docs/phase-1-exposure-runbook.md -- the newt/DNS/auth step that RFC 001 and the synlig runbook leave open (runbook section 4, items 2 and 5). Port 8765 at /mcp, newt targets 172.17.0.1, and the authentication is the single shared bearer token (RFC 6.2, decided 2026-08-09) rather than per-device proxy users. The latter cannot work today: mempalace validates exactly one token, and Pangolin's SSO/PIN/password are browser-shaped while every client here is a headless JSON-RPC POST -- enabling that protection breaks the clients it protects. The per-device axis that *does* exist is the feeder's SSH key + per-device inbox. New finding recorded there: a loopback bind does not merely 403 behind a tunnel (already known, runbook 2.4) -- it also silently starts the server with no token at all, because auto-minting is gated on the bind being non-loopback. extensions/pi/README.md: the HTTP transport IS authenticated as of mempalace 3.6.0; the "sessionless and unauthenticated" note dated from the v1.3.0 era. Closes the RFC section 8 Phase-0 hygiene item. |
||
|
Joakim Persson
|
3626946013 |
Phase 0 on synlig: install, palace layout, verified Host/Origin policy
synlig is greenfield — no mempalace, no ~/.mempalace at all — so Phase 0 became "provision correctly from birth" rather than "migrate carefully". Nothing is serving; no client config was touched. Done: - mempalace 3.6.0 installed via uv, pinned to the fleet version (the id recipes and idempotency probes this RFC leans on are version-specific). - Embedder pre-warmed. This was the real unknown: the first embed pulls a 79.3 MB ONNX model from the chroma CDN, and an egress-filtered work VM would have failed at the worst moment — the first client write. Pulled at ~20 MB/s, no proxy interference. Done in a throwaway palace so the real one never saw it. - Palace at the stock default ~/.mempalace/palace, so no config file and no MEMPALACE_PALACE_PATH is needed on synlig at all. Two corrections to the RFC, both from provisioning rather than reading: - §7.1 was understated. DEFAULT_PALACE_PATH (~/.mempalace/palace) and DEFAULT_KG_PATH (~/.mempalace/knowledge_graph.sqlite3) already differ with stock defaults, so the KG split is out-of-the-box behaviour, not a consequence of a custom --palace, and it is permanent rather than one-time: serve always passes --palace, any CLI call without it uses the HOME path. A one-time mv does not fix that, it only picks which of the two files gets populated. Fixed instead by converging both rules on one inode via relative symlinks, and verified the load-bearing assumption: a dangling symlink is created on connect, -wal/-shm land next to the target (so the palace dir stays a self-contained backup unit, which is the part that mattered), cross-path read works, same inode. hallways.json deliberately left alone — already palace-derived, HOME path is a warning-only probe. - §6.2 upgraded from "test early" to verified: 11/11 as predicted. The headline is that the safe-sounding reflex is the failure mode — loopback bind + proxy forwarding a public Host is 403, non-loopback is 200. Also confirmed Origin is never relaxed on either bind, and /healthz is Host/Origin-gated but token-free, so it works as the tunnel probe. Recommends binding the docker0 gateway over 0.0.0.0: non-loopback so the pin relaxes, but reachable only from the host and its containers. New docs/synlig-primary-runbook.md carries the discovered facts about the box, the evidence tables, an explicit "deliberately not done" list, and tomorrow's ordered steps. New contrib/systemd/mempalace-serve.service carries the bind rationale inline so nobody "fixes" it back to loopback; staged on synlig with a .staged suffix so systemd cannot pick it up by accident. Flagged for tomorrow: synlig has no newt/tunnel client (docker ps shows only the Gitea runner and digikam), so Pangolin on nyvaken cannot reach it until one is added — easy to miss, because Pangolin will look healthy from its own side. |
||
 joakimp
|
6352373a1f |
fix(feeders): make post-mine repair opt-in, not default
The three feeder wrappers (mempalace-docs, mempalace-pi-session,
mempalace-session) unconditionally ran 'mempalace repair --yes' after
mining, controllable only via --no-repair opt-out. The contrib launchd
and systemd templates did not pass --no-repair, so every scheduled tick
invoked the destructive in-place HNSW rebuild.
This has bitten us twice:
- 2026-05-04 09:08: a kickstart triggered repair while an MCP
subprocess held the DB open; the live collection was wiped (0
drawers) and had to be restored from the palace.backup snapshot.
- 2026-05-05 10:00: post-mine repair crashed mid-rebuild with
'NotFoundError: Collection [<uuid>] does not exist' - chromadb's
rebuild recreated the collection under a new UUID while the code
still held the old handle. Live DB survived only by luck (crash
hit before the swap).
Fix: flip the default.
- New flag: --repair (opt-in). Prints a warning and sleeps 3s before
invoking 'mempalace repair --yes'.
- --no-repair is retained as a deprecated no-op alias for backward
compatibility with any scripts/units still passing it.
- Default behavior: no repair. Routine ChromaDB add() keeps HNSW
consistent; repair is a recovery op, not a maintenance tick.
Docs updated to match: README, SKILL, ARCHITECTURE, AGENTS,
contrib/README. Scheduling guidance now explicitly warns against
enabling --repair on cron/launchd/systemd-timer runs.
|
||
|
joakimp
|
53d96adc65 |
docs(contrib): scheduling templates for mempalace-pi-session
Drop-in equivalents of the opencode templates for each scheduler
mechanism:
systemd/mempalace-pi-session.{service,timer}
launchd/se.jordbo.mempalace-pi-session.plist
cron/mempalace-pi-session.cron
Schedule is staggered from the opencode jobs (Mon 03:00 -> Tue 03:00)
so machines running both don't race each other on the post-mine HNSW
repair step. Service unit uses ConditionPathExists=%h/.pi/agent/sessions
to no-op silently on machines that haven't used pi, matching the
opencode template's guard on ~/.local/share/opencode/opencode.db.
contrib/README.md grows a 'Templates at a glance' table so the set is
discoverable without reading the whole doc.
|
||
|
Joakim Persson
|
98baabe7a0 |
contrib: flag cron-not-installed as a common caveat
Minimal Debian/Ubuntu hosts (and most base container images) don't ship cron by default. `crontab: command not found` is the first thing a user hits if they try the cron path without installing it. Previous caveats block covered semantics (no Persistent, mail-drop stderr) but silently assumed cron was present. Add an explicit "check command -v crontab, apt install cron, or pick systemd" preflight to the caveats so the error is surfaced before the user runs into it. Caught during 2026-04-30 Phase 4 runtime validation on a Debian trixie host: `crontab -T` lint failed because cron wasn't installed, even though the underlying docker-exec shell command (the actual workload) ran fine. |
||
|
Joakim Persson
|
00ce8a7fa1 |
contrib: clarify when opencode-devbox bakes in the toolkit
Previous wording claimed opencode-devbox "bakes it in via
mempalace-toolkit" as if that were always true, but until
opencode-devbox v1.14.30b the image only shipped the mempalace
Python package, not the toolkit wrappers. Users following the
*-devbox scheduler docs on earlier images would hit
"mempalace-session: command not found" inside the container.
Rewrite the precondition to:
- Name the version where bake-in starts (v1.14.30b).
- Link to the upstream INSTALL_MEMPALACE_TOOLKIT build arg.
- Document the escape hatch for custom/older containers
(./install.sh --yes) and flag its ephemeral nature, so nobody
leans on the manual install as a long-term solution.
Caught during 2026-04-30 runtime validation of the *-devbox
systemd unit on a freshly-rebuilt container.
|
||
|
Joakim Persson
|
46bcce5a67 |
contrib: devbox-aware scheduler templates (host-side, docker exec)
On hosts running a long-lived opencode-devbox (or equivalent)
container, mempalace-session lives INSIDE the container, not on
the host. The existing contrib/* templates install a scheduler on
the machine that runs the tool; for the devbox case the scheduler
has to live on the host and reach into the container via
'docker exec'. This was noted in passing in contrib/README.md but
no templates were actually shipped for it.
Adds parallel *-devbox templates for systemd and cron:
contrib/systemd/mempalace-session-devbox.service
- Type=oneshot, same 2h TimeoutStartSec + low Nice as the direct
variant.
- Two Environment knobs (CONTAINER, CONTAINER_USER) default to
opencode-devbox/developer, overrideable via
'systemctl --user edit'.
- ExecCondition checks 'docker ps --filter name= --filter
status=running' so the unit no-ops cleanly when the container
is currently down. systemd reports this as a successful
'condition failed' state — no alert noise across dev cycles
of teardown/rebuild.
- ExecStart is plain /usr/bin/docker exec with no shell; systemd
does the env-var expansion.
- Stdout/stderr go to journalctl --user -u <unit> (nothing to
redirect, since docker exec surfaces container output to the
calling process).
contrib/systemd/mempalace-session-devbox.timer
- Mon 03:00 Persistent=true RandomizedDelaySec=30m, mirrors the
direct timer.
contrib/cron/mempalace-session-devbox.cron
- Equivalent shell-wrapped form for hosts using cron instead of
systemd. 'docker ps | grep -q .' short-circuits if the container
isn't running. Log goes to $HOME/.cache/mempalace-session/
cron-devbox.log on the HOST (outside the container) so it's
inspectable without dropping into the devbox.
contrib/README.md:
- Replaces the two-paragraph 'Running inside a container' note
with a proper section: preconditions, install recipes for both
the systemd and cron devbox variants, verify/uninstall commands,
customization via 'systemctl --user edit', behaviour when the
container is down.
- Chooser table gains a dedicated row pointing devbox users at
the *-devbox templates, and mentions the systemd vs cron pick
for that case.
- New 'When to pick devbox variants vs direct ones' table covers
the rare both-installed case (host mempalace AND in-container
mempalace see separate palaces — they don't cross-pollinate).
Top-level README.md 'Keeping it fresh' subsection gains a quick-start
block for the devbox variant alongside the existing Linux/macOS
quick-starts.
Tested: all four systemd units parse cleanly as INI via
configparser (sections + key=value pairs); validated file sizes
and locations match the layout described in docs. Runtime
validation (systemctl --user enable; actual docker exec) requires
a host with docker + an opencode-devbox container up — deferred
to the user's Mac/Linux boxes.
|
||
|
Joakim Persson
|
720245e010 |
Add macOS launchd template, bringing automation parity to macOS
Ship a launchd user agent plist alongside the existing systemd and
cron templates so macOS users can schedule mempalace-session without
falling back to cron. launchd is the macOS-native equivalent of a
systemd user timer: same scheduling model, same log conventions, same
single-instance guarantees.
- contrib/launchd/se.jordbo.mempalace-session.plist:
- Label uses reverse-DNS from the jordbo.se domain for consistency
with other user-installed launchd jobs; fork the prefix if reusing
this template in a different org.
- ProgramArguments points at /Users/USER/.local/bin/mempalace-session
(USER is substituted at install time, same pattern as
contrib/cron/).
- EnvironmentVariables.PATH covers ~/.local/bin, Apple Silicon
Homebrew, Intel Homebrew, and system defaults — launchd agents
get a minimal PATH by default and the wrapper needs to find
mempalace + python3.
- StartCalendarInterval matches systemd unit's schedule: Monday
03:00 local.
- RunAtLoad=false — load shouldn't trigger a run; schedule does.
- ProcessType=Background + LowPriorityIO=true + Nice=10 mirror
the systemd unit's Nice=10 + IOSchedulingClass=idle. macOS's
automatic App Nap and resource throttling for Background jobs
yields to interactive work cleanly.
- ExitTimeOut=7200 matches systemd's TimeoutStartSec=7200.
- StandardOut/ErrorPath under ~/Library/Logs/ so Console.app
surfaces them.
- contrib/README.md gains a full launchd section:
- Caveat table comparing to systemd (Persistent=true isn't quite
matched; RandomizedDelaySec has no equivalent; overlap prevention
is automatic).
- Install recipe using launchctl bootstrap (modern) with a fallback
note for legacy launchctl load -w on older macOS.
- Verify section shows launchctl list, launchctl print, log tails,
and launchctl kickstart for manual testing.
- Uninstall via launchctl bootout.
- Chooser table updated: macOS now explicitly points at launchd,
not cron.
- ARCHITECTURE.md §5, SKILL.md Quick automation pitch, and README.md
Keeping it fresh section all updated to mention the three scheduler
options and give per-platform quick-starts.
Plist XML validated with plistlib.
|
||
|
Joakim Persson
|
36845e14b2 |
Document the operational routine + ship automation templates
Until opencode session-stopping hooks land upstream, mempalace-session
is the entire mechanism that gets opencode conversations into the
palace — skip it and session history stays trapped in a local SQLite
DB, invisible to semantic search. Previous docs covered setup well
but were thin on when and how often to run it.
- ARCHITECTURE.md §5: replace the one-line 'When to re-mine' note with
a full Operational Routine section — triggers, cadence, relationship
to the session lifecycle, automation pointers, verification.
- SKILL.md: add an Operational Routine section aimed at agents —
when to suggest invoking the tool, cadence guidance, how to
distinguish this producer-side tool from the consumer-side
mempalace skill's in-session habits.
- README.md: add 'Keeping it fresh' subsection pointing at contrib/
and the full docs.
contrib/ ships three ready-to-use templates:
- systemd/mempalace-session.{service,timer} — user units with weekly
Mon 03:00 schedule, Persistent=true catch-up, RandomizedDelaySec for
fleet-wide jitter, ConditionPathExists guard for opencode-less boxes,
Nice+IOSchedulingClass=idle so it never fights interactive work.
- cron/mempalace-session.cron — sample crontab entry with log
redirection and clear USER-substitution instructions.
- README.md with install/verify/uninstall recipes for both, a chooser
table (systemd vs cron), container/devbox caveats, and tuning notes
(daily vs weekly vs monthly trade-offs).
The user's LATER-list item 'wrap mempalace-session in cron/systemd
timer for true auto-save coverage' is now actionable: a single
systemctl --user enable --now command stands it up.
|