6e8172d93a265947885c8134358cafc57ef14aa0
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joakim Persson
|
ec436ed3ad |
docs: reconcile the RFC-001 docs with what is actually deployed
Audit of every doc touching the global-palace rollout against the running
fleet. Each correction below was verified against the filesystem or the host,
not against another doc:
- synlig-primary-runbook: the decommission `rm -rf ~/.mempalace` now carries a
STOP block. That tree holds the fleet palace *and* the only copy of the
bearer token every client authenticates with; the old "empty today" comment
stopped being true when the palace was seeded on 2026-08-14. Adds an ordered
safe decommission, and drops count-based join verification.
- phase-1-exposure-runbook: new S3.8, how to verify a flip actually took --
the procedure that until now existed only in an untracked handover file.
Three claims that fail independently (env var / curl / the palace-path
discriminator) plus an explicit list of checks that produce FALSE POSITIVES:
drawer counts (both sides were seeded from the same palace, and `status`
counts chunks not drawers), write-then-read through the same transport, and
the `mempalace` CLI -- which has no remote support at all, so post-flip it
reads the dead local archive and reports success.
- rfc-001: status Draft -> Phases 0-1 implemented. Records that the join was a
file-level copy, which SIDESTEPPED the S7.6 diary-dedup question rather than
answering it -- so S7.6 remains a hard blocker for the second machine, which
is the one that will actually exercise merge semantics.
- ARCHITECTURE, SKILL, contrib/README, extensions/pi/README all claimed pi
feeds the palace automatically, unconditionally. That is gated on
mempalace-toolkit >=
|
||
|
Joakim Persson
|
29e660e18f |
feeders: stage beside the palace, not in ~/.cache; document Phase 1 exposure
Staging default moves out of ~/.cache to <palace-root>/pi-stage (pi) and <palace-root>/opencode-stage (opencode), resolved with mempalace's own palace-path precedence ($MEMPALACE_PALACE_PATH -> $MEMPAL_PALACE_PATH -> ~/.mempalace/config.json -> ~/.mempalace/palace), then dirname. Why: the convos miner keys dedup on the *staged* path, so a wiped stage plus a sync scoped to include it prunes the drawers mined from those sources -- deleting memories, not a cache. Under ~/.cache that state was reachable by anything treating a cache as disposable. Staging inside the palace makes the coupling structural: the stage cannot be wiped without touching the palace itself. Overrides ($MEMPALACE_PI_STAGE / $MEMPALACE_SESSION_STAGE, --stage) are unchanged. Note the old default had never been created on any host, so this closed a latent hazard, not a live one. Measured, and the docs now claim only this much: sync prunes only within the scope it is given -- wing-only, 1299 scanned / 1299 out of scope / 0 removed; scoped at the palace root, 651 kept / 648 out of scope. The previous blanket "sync prunes every drawer" wording overstated it, which is a liability: the next reader disproves the overstatement and discards the real constraint with it. Also in this change: - cron log dir ~/.cache/mempalace-session -> ~/.cache/mempalace-logs. The stage left that namespace, so the old name now read as "the stage". - AGENTS.md: the convos miner *does* check mtime (verified against upstream convo_miner.py); the previous "no mtime check" claim was wrong. - smoke-test assertions use `mktemp -d` for --sessions-dir. One pointed at /tmp, which still held earlier synthetic transcripts, so a --dry-run exported a fake session into the real stage: --dry-run skips the mine, not the export. docs/phase-1-exposure-runbook.md -- the newt/DNS/auth step that RFC 001 and the synlig runbook leave open (runbook section 4, items 2 and 5). Port 8765 at /mcp, newt targets 172.17.0.1, and the authentication is the single shared bearer token (RFC 6.2, decided 2026-08-09) rather than per-device proxy users. The latter cannot work today: mempalace validates exactly one token, and Pangolin's SSO/PIN/password are browser-shaped while every client here is a headless JSON-RPC POST -- enabling that protection breaks the clients it protects. The per-device axis that *does* exist is the feeder's SSH key + per-device inbox. New finding recorded there: a loopback bind does not merely 403 behind a tunnel (already known, runbook 2.4) -- it also silently starts the server with no token at all, because auto-minting is gated on the bind being non-loopback. extensions/pi/README.md: the HTTP transport IS authenticated as of mempalace 3.6.0; the "sessionless and unauthenticated" note dated from the v1.3.0 era. Closes the RFC section 8 Phase-0 hygiene item. |
||
 joakimp
|
6352373a1f |
fix(feeders): make post-mine repair opt-in, not default
The three feeder wrappers (mempalace-docs, mempalace-pi-session,
mempalace-session) unconditionally ran 'mempalace repair --yes' after
mining, controllable only via --no-repair opt-out. The contrib launchd
and systemd templates did not pass --no-repair, so every scheduled tick
invoked the destructive in-place HNSW rebuild.
This has bitten us twice:
- 2026-05-04 09:08: a kickstart triggered repair while an MCP
subprocess held the DB open; the live collection was wiped (0
drawers) and had to be restored from the palace.backup snapshot.
- 2026-05-05 10:00: post-mine repair crashed mid-rebuild with
'NotFoundError: Collection [<uuid>] does not exist' - chromadb's
rebuild recreated the collection under a new UUID while the code
still held the old handle. Live DB survived only by luck (crash
hit before the swap).
Fix: flip the default.
- New flag: --repair (opt-in). Prints a warning and sleeps 3s before
invoking 'mempalace repair --yes'.
- --no-repair is retained as a deprecated no-op alias for backward
compatibility with any scripts/units still passing it.
- Default behavior: no repair. Routine ChromaDB add() keeps HNSW
consistent; repair is a recovery op, not a maintenance tick.
Docs updated to match: README, SKILL, ARCHITECTURE, AGENTS,
contrib/README. Scheduling guidance now explicitly warns against
enabling --repair on cron/launchd/systemd-timer runs.
|
||
|
joakimp
|
53d96adc65 |
docs(contrib): scheduling templates for mempalace-pi-session
Drop-in equivalents of the opencode templates for each scheduler
mechanism:
systemd/mempalace-pi-session.{service,timer}
launchd/se.jordbo.mempalace-pi-session.plist
cron/mempalace-pi-session.cron
Schedule is staggered from the opencode jobs (Mon 03:00 -> Tue 03:00)
so machines running both don't race each other on the post-mine HNSW
repair step. Service unit uses ConditionPathExists=%h/.pi/agent/sessions
to no-op silently on machines that haven't used pi, matching the
opencode template's guard on ~/.local/share/opencode/opencode.db.
contrib/README.md grows a 'Templates at a glance' table so the set is
discoverable without reading the whole doc.
|
||
|
Joakim Persson
|
98baabe7a0 |
contrib: flag cron-not-installed as a common caveat
Minimal Debian/Ubuntu hosts (and most base container images) don't ship cron by default. `crontab: command not found` is the first thing a user hits if they try the cron path without installing it. Previous caveats block covered semantics (no Persistent, mail-drop stderr) but silently assumed cron was present. Add an explicit "check command -v crontab, apt install cron, or pick systemd" preflight to the caveats so the error is surfaced before the user runs into it. Caught during 2026-04-30 Phase 4 runtime validation on a Debian trixie host: `crontab -T` lint failed because cron wasn't installed, even though the underlying docker-exec shell command (the actual workload) ran fine. |
||
|
Joakim Persson
|
00ce8a7fa1 |
contrib: clarify when opencode-devbox bakes in the toolkit
Previous wording claimed opencode-devbox "bakes it in via
mempalace-toolkit" as if that were always true, but until
opencode-devbox v1.14.30b the image only shipped the mempalace
Python package, not the toolkit wrappers. Users following the
*-devbox scheduler docs on earlier images would hit
"mempalace-session: command not found" inside the container.
Rewrite the precondition to:
- Name the version where bake-in starts (v1.14.30b).
- Link to the upstream INSTALL_MEMPALACE_TOOLKIT build arg.
- Document the escape hatch for custom/older containers
(./install.sh --yes) and flag its ephemeral nature, so nobody
leans on the manual install as a long-term solution.
Caught during 2026-04-30 runtime validation of the *-devbox
systemd unit on a freshly-rebuilt container.
|
||
|
Joakim Persson
|
46bcce5a67 |
contrib: devbox-aware scheduler templates (host-side, docker exec)
On hosts running a long-lived opencode-devbox (or equivalent)
container, mempalace-session lives INSIDE the container, not on
the host. The existing contrib/* templates install a scheduler on
the machine that runs the tool; for the devbox case the scheduler
has to live on the host and reach into the container via
'docker exec'. This was noted in passing in contrib/README.md but
no templates were actually shipped for it.
Adds parallel *-devbox templates for systemd and cron:
contrib/systemd/mempalace-session-devbox.service
- Type=oneshot, same 2h TimeoutStartSec + low Nice as the direct
variant.
- Two Environment knobs (CONTAINER, CONTAINER_USER) default to
opencode-devbox/developer, overrideable via
'systemctl --user edit'.
- ExecCondition checks 'docker ps --filter name= --filter
status=running' so the unit no-ops cleanly when the container
is currently down. systemd reports this as a successful
'condition failed' state — no alert noise across dev cycles
of teardown/rebuild.
- ExecStart is plain /usr/bin/docker exec with no shell; systemd
does the env-var expansion.
- Stdout/stderr go to journalctl --user -u <unit> (nothing to
redirect, since docker exec surfaces container output to the
calling process).
contrib/systemd/mempalace-session-devbox.timer
- Mon 03:00 Persistent=true RandomizedDelaySec=30m, mirrors the
direct timer.
contrib/cron/mempalace-session-devbox.cron
- Equivalent shell-wrapped form for hosts using cron instead of
systemd. 'docker ps | grep -q .' short-circuits if the container
isn't running. Log goes to $HOME/.cache/mempalace-session/
cron-devbox.log on the HOST (outside the container) so it's
inspectable without dropping into the devbox.
contrib/README.md:
- Replaces the two-paragraph 'Running inside a container' note
with a proper section: preconditions, install recipes for both
the systemd and cron devbox variants, verify/uninstall commands,
customization via 'systemctl --user edit', behaviour when the
container is down.
- Chooser table gains a dedicated row pointing devbox users at
the *-devbox templates, and mentions the systemd vs cron pick
for that case.
- New 'When to pick devbox variants vs direct ones' table covers
the rare both-installed case (host mempalace AND in-container
mempalace see separate palaces — they don't cross-pollinate).
Top-level README.md 'Keeping it fresh' subsection gains a quick-start
block for the devbox variant alongside the existing Linux/macOS
quick-starts.
Tested: all four systemd units parse cleanly as INI via
configparser (sections + key=value pairs); validated file sizes
and locations match the layout described in docs. Runtime
validation (systemctl --user enable; actual docker exec) requires
a host with docker + an opencode-devbox container up — deferred
to the user's Mac/Linux boxes.
|
||
|
Joakim Persson
|
720245e010 |
Add macOS launchd template, bringing automation parity to macOS
Ship a launchd user agent plist alongside the existing systemd and
cron templates so macOS users can schedule mempalace-session without
falling back to cron. launchd is the macOS-native equivalent of a
systemd user timer: same scheduling model, same log conventions, same
single-instance guarantees.
- contrib/launchd/se.jordbo.mempalace-session.plist:
- Label uses reverse-DNS from the jordbo.se domain for consistency
with other user-installed launchd jobs; fork the prefix if reusing
this template in a different org.
- ProgramArguments points at /Users/USER/.local/bin/mempalace-session
(USER is substituted at install time, same pattern as
contrib/cron/).
- EnvironmentVariables.PATH covers ~/.local/bin, Apple Silicon
Homebrew, Intel Homebrew, and system defaults — launchd agents
get a minimal PATH by default and the wrapper needs to find
mempalace + python3.
- StartCalendarInterval matches systemd unit's schedule: Monday
03:00 local.
- RunAtLoad=false — load shouldn't trigger a run; schedule does.
- ProcessType=Background + LowPriorityIO=true + Nice=10 mirror
the systemd unit's Nice=10 + IOSchedulingClass=idle. macOS's
automatic App Nap and resource throttling for Background jobs
yields to interactive work cleanly.
- ExitTimeOut=7200 matches systemd's TimeoutStartSec=7200.
- StandardOut/ErrorPath under ~/Library/Logs/ so Console.app
surfaces them.
- contrib/README.md gains a full launchd section:
- Caveat table comparing to systemd (Persistent=true isn't quite
matched; RandomizedDelaySec has no equivalent; overlap prevention
is automatic).
- Install recipe using launchctl bootstrap (modern) with a fallback
note for legacy launchctl load -w on older macOS.
- Verify section shows launchctl list, launchctl print, log tails,
and launchctl kickstart for manual testing.
- Uninstall via launchctl bootout.
- Chooser table updated: macOS now explicitly points at launchd,
not cron.
- ARCHITECTURE.md §5, SKILL.md Quick automation pitch, and README.md
Keeping it fresh section all updated to mention the three scheduler
options and give per-platform quick-starts.
Plist XML validated with plistlib.
|
||
|
Joakim Persson
|
36845e14b2 |
Document the operational routine + ship automation templates
Until opencode session-stopping hooks land upstream, mempalace-session
is the entire mechanism that gets opencode conversations into the
palace — skip it and session history stays trapped in a local SQLite
DB, invisible to semantic search. Previous docs covered setup well
but were thin on when and how often to run it.
- ARCHITECTURE.md §5: replace the one-line 'When to re-mine' note with
a full Operational Routine section — triggers, cadence, relationship
to the session lifecycle, automation pointers, verification.
- SKILL.md: add an Operational Routine section aimed at agents —
when to suggest invoking the tool, cadence guidance, how to
distinguish this producer-side tool from the consumer-side
mempalace skill's in-session habits.
- README.md: add 'Keeping it fresh' subsection pointing at contrib/
and the full docs.
contrib/ ships three ready-to-use templates:
- systemd/mempalace-session.{service,timer} — user units with weekly
Mon 03:00 schedule, Persistent=true catch-up, RandomizedDelaySec for
fleet-wide jitter, ConditionPathExists guard for opencode-less boxes,
Nice+IOSchedulingClass=idle so it never fights interactive work.
- cron/mempalace-session.cron — sample crontab entry with log
redirection and clear USER-substitution instructions.
- README.md with install/verify/uninstall recipes for both, a chooser
table (systemd vs cron), container/devbox caveats, and tuning notes
(daily vs weekly vs monthly trade-offs).
The user's LATER-list item 'wrap mempalace-session in cron/systemd
timer for true auto-save coverage' is now actionable: a single
systemctl --user enable --now command stands it up.
|