joakimp/mempalace-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document the operational routine + ship automation templates

Browse Source 
Until opencode session-stopping hooks land upstream, mempalace-session
is the entire mechanism that gets opencode conversations into the
palace — skip it and session history stays trapped in a local SQLite
DB, invisible to semantic search. Previous docs covered setup well
but were thin on when and how often to run it.

- ARCHITECTURE.md §5: replace the one-line 'When to re-mine' note with
  a full Operational Routine section — triggers, cadence, relationship
  to the session lifecycle, automation pointers, verification.
- SKILL.md: add an Operational Routine section aimed at agents —
  when to suggest invoking the tool, cadence guidance, how to
  distinguish this producer-side tool from the consumer-side
  mempalace skill's in-session habits.
- README.md: add 'Keeping it fresh' subsection pointing at contrib/
  and the full docs.

contrib/ ships three ready-to-use templates:
- systemd/mempalace-session.{service,timer} — user units with weekly
  Mon 03:00 schedule, Persistent=true catch-up, RandomizedDelaySec for
  fleet-wide jitter, ConditionPathExists guard for opencode-less boxes,
  Nice+IOSchedulingClass=idle so it never fights interactive work.
- cron/mempalace-session.cron — sample crontab entry with log
  redirection and clear USER-substitution instructions.
- README.md with install/verify/uninstall recipes for both, a chooser
  table (systemd vs cron), container/devbox caveats, and tuning notes
  (daily vs weekly vs monthly trade-offs).

The user's LATER-list item 'wrap mempalace-session in cron/systemd
timer for true auto-save coverage' is now actionable: a single
systemctl --user enable --now command stands it up.
This commit is contained in:
Joakim Persson
2026-04-30 06:29:55 +00:00
parent e49d9285c4
commit 36845e14b2
7 changed files with 367 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ARCHITECTURE.md
+85 -1
View File
@@ -188,7 +188,91 @@ Both wrappers dedup via `mempalace mine`'s built-in key:
### When to re-mine
- `mempalace-docs`: after significant doc changes in a project.
- `mempalace-session`: opportunistically. Every few days catches new opencode sessions. Or wire to cron / systemd timer for true auto-save coverage (not yet done).
- `mempalace-session`: see the full **Operational Routine** below.
### Operational Routine (the `mempalace-session` workflow)
Until opencode grows session hooks and `hooks_cli.py` grows an opencode harness (see §6), **`mempalace-session` is the entire mechanism that gets opencode conversations into the palace.** Skip it and your session history exists only inside `~/.local/share/opencode/opencode.db` — a local SQLite file that's invisible to `mempalace_search`, vulnerable to volume wipes, and lost if the devbox is replaced.
That makes the routine worth codifying:
#### Triggers (when to run)
| Trigger | What to run | Why |
|---|---|---|
| **Substantive session you want preserved past `/exit`** | `mempalace-session --session <id>` | Targeted save before destructive action; see `~/.local/share/opencode/opencode.db` `session` table for the ID. |
| **Before a container recreate** | `mempalace-session` | The opencode DB lives in a named volume (`devbox-data`) so it normally survives, but a full mine right before is cheap insurance. |
| **Fresh machine, first provisioning** | `mempalace-session --dry-run` then `mempalace-session` | Backfills the whole corpus. Expect ~20 min / 60 sessions. |
| **Periodic sweep** | `mempalace-session` | Weekly catches anything you didn't explicitly save. Dedup is free, so running more often only costs the ~5 min repair. |
| **After upstream mempalace upgrade** | `mempalace-session` + `mempalace repair` | If the miner changed normalization or chunking, re-mine ensures the palace reflects current logic. Rare. |
#### Cadence (how often)
**Default: weekly.** Dedup is free on unchanged sessions, and `wing_conversations` growth is roughly linear in user activity. Weekly is frequent enough that searches almost always include recent context, and infrequent enough that the cost is negligible.
**Daily** is fine but wasteful — you'll pay the post-mine `repair` cost seven times more often than you need. If you want daily runs, add `--no-repair` and schedule a separate weekly repair.
**Monthly** is too infrequent. You'll search for "that thing we discussed last Tuesday" and miss it.
#### Relationship to the session lifecycle
`mempalace-session` is **offline, inter-session maintenance** — it runs between agent sessions, not during them. It does not replace the in-session habits from the consumer-side `mempalace` skill:
| Habit | When | Who |
|---|---|---|
| Wake-up search (load recent diary) | Agent session start | Agent, during session |
| Wind-down diary write | Agent session end | Agent, during session |
| `mempalace-session` mine | Between sessions (manual or scheduled) | Operator or automation |
The first two are live; the third is batched. They're complementary, not alternatives. A machine doing only wake-up/wind-down keeps a diary but loses the actual conversation turns. A machine doing only `mempalace-session` captures the raw turns but not the curated summaries. Do both.
#### Automation
Pick one:
1. **systemd user timer** (recommended on modern Linux). Survives reboots, optional `Persistent=true` catch-up, logs to `journalctl`, background I/O priority. Templates in [`contrib/systemd/`](contrib/systemd/).
2. **cron** (simpler, works anywhere). Templates in [`contrib/cron/`](contrib/cron/).
3. **Manual** — run `mempalace-session` opportunistically. Fine on machines where you're in and out frequently; less fine on long-running devboxes.
Install recipes, verification commands, and uninstall steps for all three are in [`contrib/README.md`](contrib/README.md).
Quick-start (systemd user timer):
```bash
mkdir -p ~/.config/systemd/user
cp contrib/systemd/*.{service,timer} ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now mempalace-session.timer
# Optional on headless boxes: keep timer running when logged out
sudo loginctl enable-linger "$USER"
systemctl --user list-timers mempalace-session.timer
```
Quick-start (cron):
```bash
sed "s|USER|$USER|g" contrib/cron/mempalace-session.cron \
| (crontab -l 2>/dev/null; cat) | crontab -
mkdir -p ~/.cache/mempalace-session
```
#### Verification
After any run (manual or scheduled), confirm the palace grew:
```bash
mempalace-session --dry-run # should list sessions
# Or from inside a live MCP client:
# mempalace_status — wing_conversations count
# mempalace_reconnect — refresh index after mine
```
A healthy run produces one of:
- **First run on fresh corpus**: several hundred to several thousand new drawers.
- **Incremental run**: zero to a few dozen new drawers (whatever grew since last run).
- **Rerun with no new activity**: zero new drawers, only the repair step runs.
A run that files far more drawers than expected may indicate a staging-dir wipe (forcing a full re-mine) — check `~/.cache/mempalace-session/<wing>/` modification times.
### Cost profile (reference)
README.md
+19
View File
@@ -70,6 +70,25 @@ mempalace-docs /workspace/my_project --dry-run
mempalace-docs /workspace/my_project
```
### Keeping it fresh (automation)
Manual invocation is fine while you're actively driving the machine, but long-running devboxes benefit from a weekly automated mine. [`contrib/`](contrib/) ships ready-to-install templates:
- **systemd user timer** (recommended on modern Linux): survives reboots, catches missed runs, logs to `journalctl`.
- **cron**: simpler, works everywhere, no user-unit awareness needed.
Quick-start (systemd, weekly Mon 03:00 local):
```bash
mkdir -p ~/.config/systemd/user
cp contrib/systemd/*.{service,timer} ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now mempalace-session.timer
sudo loginctl enable-linger "$USER" # optional, for headless boxes
```
See [`contrib/README.md`](contrib/README.md) for full install/verify/uninstall recipes, tuning, and devbox/container caveats. The full operational routine (triggers, cadence, verification) is in [`ARCHITECTURE.md`](ARCHITECTURE.md) §5.
### Containerized (devbox) notes
On a Docker-based devbox, the palace and opencode DB should live on named volumes so they survive container recreate:
SKILL.md
+51
View File
@@ -104,6 +104,57 @@ mempalace-session # stages + mines fresh
Staging is ephemeral by design; the palace is the source of truth.
## Operational Routine (when to invoke)
Until upstream opencode session hooks land, **`mempalace-session` is the entire mechanism** that gets opencode conversations into the palace. If the user's opencode history isn't showing up in `mempalace_search`, the most likely cause is "`mempalace-session` hasn't been run recently".
### Agent-level triggers
Suggest invoking the tool when any of these apply:
- User asks *"why can't you find our conversation from earlier?"* / *"you should remember this from last week"* → the palace may not have the opencode session yet. Check `mempalace_list_wings` for `wing_conversations`; if the last drawer there is older than the referenced conversation, run `mempalace-session`.
- User mentions an upcoming **container recreate / system migration / OS reinstall** → suggest a full mine first as a backup checkpoint. The opencode DB normally survives via named volumes, but an explicit mine is cheap insurance.
- User has just **set up a new machine** and asks about mempalace → part of the recipe is the initial backfill. Run `mempalace-session --dry-run` first to show scope, then the real mine.
- User completed a **substantive session they want preserved past `/exit`** → offer a targeted `mempalace-session --session <id>` right then.
- User asks *"how do I keep this up to date?"* → point at [`contrib/`](../../contrib/) (systemd timer or cron recipes).
### Cadence guidance
| Situation | Suggested cadence |
|---|---|
| Active devbox, daily opencode use | Weekly automated (systemd timer or cron) |
| Occasional opencode user | Monthly manual or weekly automated |
| Fresh machine / first setup | One-shot full backfill, then schedule |
| "I'm about to rebuild the container" | Run now, as a checkpoint |
| Automated daily mines | Pass `--no-repair` + schedule weekly repair separately |
Don't suggest running more often than daily — the post-mine HNSW repair (~5 min on 5k drawers) dominates cost, and session growth is slow enough that daily is already overkill.
### Relationship to the `mempalace` skill
The consumer-side `mempalace` skill covers in-session habits (wake-up search, wind-down diary). `mempalace-session` is **between-session maintenance** — it runs when no agent session is active, typically automated. Both are needed:
- Wake-up/wind-down → curated summaries (what mattered, what was decided)
- `mempalace-session` → raw turns (the actual words exchanged)
A machine running only one of these has half a memory. Agents loading the `mempalace` skill on an opencode machine should be aware this tool exists and point users at it if they ask about persistence.
### Quick automation pitch
If the user has a devbox and hasn't set up automation, offer it:
```bash
# One-time setup (modern Linux, systemd user timer)
cd ~/mempalace-toolkit
mkdir -p ~/.config/systemd/user
cp contrib/systemd/*.{service,timer} ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now mempalace-session.timer
sudo loginctl enable-linger "$USER" # optional, for headless boxes
```
Full install/verify/uninstall recipes for both systemd and cron are in [`contrib/README.md`](../../contrib/README.md). The quick-start above defaults to weekly runs on Monday 03:00 local time with a ≤30 min randomized delay.
## Failure Modes & Fixes
| Symptom | Cause | Fix |
contrib/README.md
+154
View File
@@ -0,0 +1,154 @@
# contrib/ — automation recipes for `mempalace-session`
Manual invocation of `mempalace-session` is fine on a machine you actively drive. For long-running devboxes, a weekly automated mine keeps the palace fresh without thinking about it. This directory ships ready-to-use templates for two common scheduling mechanisms.
> **Before using either**: confirm the toolkit is installed and the wrapper works —
> `mempalace-session --dry-run` should list qualifying sessions. If that errors, fix the install before scheduling.
Pick **one**. Running both would double-mine (harmless — dedup skips everything on the second run — but wastes wall time on the HNSW repair).
---
## systemd user timer (recommended on modern Linux)
**Why:** runs without the user logged in (with `loginctl enable-linger`), survives reboots, logs to `journalctl`, Persistent=true catches missed runs after the machine was off. No root required — it's a *user* unit.
**Install:**
```bash
mkdir -p ~/.config/systemd/user
cp contrib/systemd/mempalace-session.service ~/.config/systemd/user/
cp contrib/systemd/mempalace-session.timer ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now mempalace-session.timer
# Optional: keep the timer running when you log out (needed on headless servers)
sudo loginctl enable-linger "$USER"
```
**Verify:**
```bash
# Is the timer active and when will it next fire?
systemctl --user list-timers mempalace-session.timer
# Last run status + log tail
systemctl --user status mempalace-session.service
# Full run log (since today)
journalctl --user -u mempalace-session --since today
# Force a run right now (outside the schedule), for testing
systemctl --user start mempalace-session.service
```
**Uninstall:**
```bash
systemctl --user disable --now mempalace-session.timer
rm ~/.config/systemd/user/mempalace-session.{service,timer}
systemctl --user daemon-reload
```
### What the service does
- `Type=oneshot` — runs to completion, not a long-lived daemon.
- `ConditionPathExists=%h/.local/share/opencode/opencode.db` — skips silently on machines that haven't used opencode (no wasted boot-time runs).
- `ConditionPathExists=!%t/mempalace-session.lock` + `ExecStartPre/ExecStopPost` — soft mutual exclusion between overlapping runs.
- `Nice=10` + `IOSchedulingClass=idle` — background priority; won't interfere with interactive work.
- `TimeoutStartSec=7200` — 2 hour ceiling. The reference 60-session mine takes ~21 min; this is headroom for large corpora + slow disks.
### What the timer does
- `OnCalendar=Mon 03:00` — weekly, Monday 03:00 local time. Edit to taste (see `man systemd.time` for syntax).
- `Persistent=true` — if the machine was off at the scheduled time, run on next boot.
- `RandomizedDelaySec=30m` — jitters up to 30 minutes to avoid thundering-herd across a fleet.
---
## cron
**Why:** simpler, ubiquitous, works on any UNIX. No `loginctl enable-linger` dance, no user-units awareness required.
**Caveats:** no "persistent" semantics (a missed run while the machine was off stays missed); default cron output goes to mail or is silently dropped if no MTA.
**Install:**
```bash
# Edit the template first — replace USER with your actual username
sed "s|USER|$USER|g" contrib/cron/mempalace-session.cron > /tmp/mempalace-session.cron
# Append to your existing crontab (preserves any entries you already have)
(crontab -l 2>/dev/null; cat /tmp/mempalace-session.cron) | crontab -
rm /tmp/mempalace-session.cron
# Verify
crontab -l | grep mempalace
```
Ensure `~/.cache/mempalace-session/` exists so the log file can be written:
```bash
mkdir -p ~/.cache/mempalace-session
```
**Verify a run is happening:**
```bash
# Tail the log the cron entry writes to
tail -f ~/.cache/mempalace-session/cron.log
# Or force a run manually to prove the command is well-formed
mempalace-session
```
**Uninstall:**
```bash
crontab -e # remove the mempalace-session line by hand
```
---
## Which should I pick?
| Situation | Pick |
|---|---|
| Desktop / laptop, modern systemd-based distro | systemd user timer |
| Long-running devbox or server, wants "Persistent=true" catch-up | systemd user timer |
| macOS, BSD, or distro without systemd | cron |
| You already have a cron-based job scheduler on the box | cron |
| You want logs in `journalctl` rather than a file | systemd user timer |
If you're not sure, pick systemd. `Persistent=true` alone is worth it on any box that ever sleeps or reboots.
---
## Running inside a container (devbox)
Inside a Docker-based devbox, neither systemd nor cron typically runs by default. Two options:
1. **Schedule on the host, not the container** — have the host run `docker exec -u <user> <container> mempalace-session` on a timer. The container must be long-running (not per-invocation) for this to work.
2. **Run a systemd-in-container setup** — viable but usually not worth the complexity for this alone.
For most devbox users, a simple weekly manual run via `mempalace-session` (or a host-side cron that shells into the container) is the pragmatic choice. The tool is cheap enough that skipping a week costs nothing — dedup will catch up on the next run.
---
## Tuning
**Frequency.** Weekly is the default because:
- New sessions you care about are typically a handful per week per user.
- Dedup is free on unchanged sessions, so there's no cost to running daily other than the ~5 min post-mine repair.
- Weekly keeps the palace fresh enough that searches almost always return current context.
**Daily or more:** edit `OnCalendar=` or the cron DOW field. On a daily schedule, add `--no-repair` to the wrapper invocation and let a separate weekly unit handle repair — otherwise you repair 7× more often than you need.
**Monthly:** probably too infrequent. You'll search for "that thing we discussed last Tuesday" and miss it.
---
## See also
- [`../../ARCHITECTURE.md`](../../ARCHITECTURE.md) §5 — operational routine (triggers, cadence) in full context.
- [`../../SKILL.md`](../../SKILL.md) — the agent-side Operational Routine section for when an AI agent should suggest running this.
contrib/cron/mempalace-session.cron
+19
View File
@@ -0,0 +1,19 @@
# Sample crontab entry for mempalace-session.
#
# Runs a full opencode → MemPalace mine weekly (Mondays at 03:00 local).
# Dedup is free on subsequent runs (unchanged sessions skipped); only the
# post-mine HNSW repair has cost, so frequent runs are cheap.
#
# To install:
# (crontab -l 2>/dev/null; cat contrib/cron/mempalace-session.cron) | crontab -
#
# To remove, edit your crontab:
# crontab -e
#
# Adjust PATH below if ~/.local/bin is not the binary location on your box.
# Replace USER with your actual username, or use `$(whoami)` in-shell.
PATH=/home/USER/.local/bin:/usr/local/bin:/usr/bin:/bin
# m h dom mon dow command
0 3 * * 1 mempalace-session >> /home/USER/.cache/mempalace-session/cron.log 2>&1
contrib/systemd/mempalace-session.service
+23
View File
@@ -0,0 +1,23 @@
[Unit]
Description=Mine opencode session history into MemPalace
Documentation=https://gitea.jordbo.se/joakimp/mempalace-toolkit
# Only run if opencode has actually been used (avoids noise on idle machines)
ConditionPathExists=%h/.local/share/opencode/opencode.db
# Don't start if a previous run is still going
ConditionPathExists=!%t/mempalace-session.lock
[Service]
Type=oneshot
# The wrapper writes to ~/.cache/mempalace-session/ and the palace.
# Keep stdout/stderr in the journal — inspect with:
# journalctl --user -u mempalace-session --since today
ExecStart=%h/.local/bin/mempalace-session
# Belt-and-braces lock so two overlapping runs can't corrupt staging
ExecStartPre=/bin/sh -c 'touch %t/mempalace-session.lock'
ExecStopPost=/bin/sh -c 'rm -f %t/mempalace-session.lock'
# Protect against runaway runs (the 60-session reference mine takes ~21 min;
# give 2h of headroom for larger corpora and slow disks)
TimeoutStartSec=7200
# Low priority — this is background maintenance
Nice=10
IOSchedulingClass=idle
contrib/systemd/mempalace-session.timer
+16
View File
@@ -0,0 +1,16 @@
[Unit]
Description=Weekly opencode → MemPalace session mine
Documentation=https://gitea.jordbo.se/joakimp/mempalace-toolkit
[Timer]
# Every Monday at 03:00 local time. Adjust to taste.
# Use `systemctl --user list-timers mempalace-session.timer` to see next run.
OnCalendar=Mon 03:00
# If the machine was off at the scheduled time, run at next boot.
Persistent=true
# Randomize up to 30 minutes to avoid thundering-herd across machines.
RandomizedDelaySec=30m
AccuracySec=1m
[Install]
WantedBy=timers.target