Joakim Persson
720245e010
Ship a launchd user agent plist alongside the existing systemd and
cron templates so macOS users can schedule mempalace-session without
falling back to cron. launchd is the macOS-native equivalent of a
systemd user timer: same scheduling model, same log conventions, same
single-instance guarantees.
- contrib/launchd/se.jordbo.mempalace-session.plist:
- Label uses reverse-DNS from the jordbo.se domain for consistency
with other user-installed launchd jobs; fork the prefix if reusing
this template in a different org.
- ProgramArguments points at /Users/USER/.local/bin/mempalace-session
(USER is substituted at install time, same pattern as
contrib/cron/).
- EnvironmentVariables.PATH covers ~/.local/bin, Apple Silicon
Homebrew, Intel Homebrew, and system defaults — launchd agents
get a minimal PATH by default and the wrapper needs to find
mempalace + python3.
- StartCalendarInterval matches systemd unit's schedule: Monday
03:00 local.
- RunAtLoad=false — load shouldn't trigger a run; schedule does.
- ProcessType=Background + LowPriorityIO=true + Nice=10 mirror
the systemd unit's Nice=10 + IOSchedulingClass=idle. macOS's
automatic App Nap and resource throttling for Background jobs
yields to interactive work cleanly.
- ExitTimeOut=7200 matches systemd's TimeoutStartSec=7200.
- StandardOut/ErrorPath under ~/Library/Logs/ so Console.app
surfaces them.
- contrib/README.md gains a full launchd section:
- Caveat table comparing to systemd (Persistent=true isn't quite
matched; RandomizedDelaySec has no equivalent; overlap prevention
is automatic).
- Install recipe using launchctl bootstrap (modern) with a fallback
note for legacy launchctl load -w on older macOS.
- Verify section shows launchctl list, launchctl print, log tails,
and launchctl kickstart for manual testing.
- Uninstall via launchctl bootout.
- Chooser table updated: macOS now explicitly points at launchd,
not cron.
- ARCHITECTURE.md §5, SKILL.md Quick automation pitch, and README.md
Keeping it fresh section all updated to mention the three scheduler
options and give per-platform quick-starts.
Plist XML validated with plistlib.
186 lines
11 KiB
Markdown
186 lines
11 KiB
Markdown
# mempalace-toolkit
|
|
|
|
Producer-side tooling for [MemPalace](https://github.com/MemPalace/mempalace) — bridges that feed opencode session history and project documentation into the palace. Pairs with the consumer-side [`mempalace` agent skill](https://github.com/MemPalace/mempalace).
|
|
|
|
**What this repo contains:**
|
|
|
|
- `bin/mempalace-session` — exports [opencode](https://github.com/anomalyco/opencode) session history from its local SQLite DB to Claude Code JSONL, then mines it via `mempalace mine --mode convos`.
|
|
- `bin/mempalace-docs` — mines project directories into MemPalace while excluding source code, keeping the palace signal-dense.
|
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — **canonical spec**: architecture diagram, component details, setup recipe, operational notes, upstream-retirement roadmap.
|
|
- [`SKILL.md`](SKILL.md) — the companion agent skill, symlinked into `~/.agents/skills/opencode-mempalace-bridge/` on install.
|
|
|
|
**If you're just trying to get this working on a new machine → jump to [Setup](#setup).**
|
|
**If you want the full architecture story → read [`ARCHITECTURE.md`](ARCHITECTURE.md).**
|
|
|
|
---
|
|
|
|
## Why this exists
|
|
|
|
MemPalace is the agent memory layer. Its stock CLI has two gaps that bite on a machine running opencode with a docs-first palace policy:
|
|
|
|
1. **`mempalace mine` floods the palace with source code** — every `__init__` fragment, every generated file, hundreds of low-signal drawers per project. `mempalace-docs` fixes this by staging only documentation-class files (`*.md`, `*.yml`, `Dockerfile`, etc.) before mining.
|
|
2. **`mempalace mine --mode convos` can't read opencode's SQLite DB** — only file-based chat formats (Claude Code JSONL, Claude.ai JSON, ChatGPT, Slack, Codex). Opencode persists every turn in `~/.local/share/opencode/opencode.db` and has no upstream hook into mempalace's auto-save. `mempalace-session` fixes this by exporting each session to Claude Code JSONL before mining.
|
|
|
|
Both wrappers follow the same **stage-to-cache-then-mine** idiom. Neither reimplements the miner; they curate input and delegate.
|
|
|
|
Long-term, both should retire:
|
|
- `mempalace-docs` → retires when [MemPalace PR #1213](https://github.com/MemPalace/mempalace/pull/1213) (`exclude_patterns` in `mempalace.yaml`) merges.
|
|
- `mempalace-session` → retires when opencode session-stopping hooks ([PR #16598](https://github.com/anomalyco/opencode/pull/16598) et al.) merge **and** `hooks_cli.py` gains an `opencode` harness. Until both land, this repo fills the gap.
|
|
|
|
See [`ARCHITECTURE.md`](ARCHITECTURE.md) §6 for the full upstream roadmap.
|
|
|
|
---
|
|
|
|
## Setup
|
|
|
|
### Prerequisites
|
|
|
|
- [MemPalace](https://github.com/MemPalace/mempalace) CLI v3.3.3+
|
|
- Python 3 (stdlib `sqlite3` only — no extra deps)
|
|
- [opencode](https://github.com/anomalyco/opencode) with an active session DB at `~/.local/share/opencode/opencode.db` *(only needed for `mempalace-session`)*
|
|
|
|
### Install
|
|
|
|
```bash
|
|
git clone ssh://git@gitea.jordbo.se:2222/joakimp/mempalace-toolkit.git ~/mempalace-toolkit
|
|
cd ~/mempalace-toolkit
|
|
./install.sh
|
|
```
|
|
|
|
The installer symlinks `bin/*` into `~/.local/bin/` and optionally installs the agent skill into `~/.agents/skills/opencode-mempalace-bridge/`.
|
|
|
|
Ensure `~/.local/bin` is on `$PATH`:
|
|
|
|
```bash
|
|
export PATH="$HOME/.local/bin:$PATH"
|
|
```
|
|
|
|
### First mine
|
|
|
|
```bash
|
|
# One-time palace init (if not done)
|
|
mempalace init --yes
|
|
|
|
# Mine opencode session history into wing_conversations
|
|
mempalace-session --dry-run # preview qualifying sessions
|
|
mempalace-session # do it (~20 min per 60 sessions)
|
|
|
|
# Mine a project (docs only)
|
|
mempalace-docs /workspace/my_project --dry-run
|
|
mempalace-docs /workspace/my_project
|
|
```
|
|
|
|
### Keeping it fresh (automation)
|
|
|
|
Manual invocation is fine while you're actively driving the machine, but long-running devboxes benefit from a weekly automated mine. [`contrib/`](contrib/) ships ready-to-install templates:
|
|
|
|
- **systemd user timer** (recommended on Linux): survives reboots, catches missed runs, logs to `journalctl`.
|
|
- **launchd user agent** (recommended on macOS): native-equivalent — logs to `~/Library/Logs/`, single-instance guarantees, `ProcessType=Background` throttling.
|
|
- **cron**: simplest, works on BSD and systemd-less distros. No user-unit awareness needed.
|
|
|
|
Quick-start (Linux / systemd, weekly Mon 03:00 local):
|
|
|
|
```bash
|
|
mkdir -p ~/.config/systemd/user
|
|
cp contrib/systemd/*.{service,timer} ~/.config/systemd/user/
|
|
systemctl --user daemon-reload
|
|
systemctl --user enable --now mempalace-session.timer
|
|
sudo loginctl enable-linger "$USER" # optional, for headless boxes
|
|
```
|
|
|
|
Quick-start (macOS / launchd, same schedule):
|
|
|
|
```bash
|
|
sed "s|USER|$USER|g" contrib/launchd/se.jordbo.mempalace-session.plist \
|
|
> ~/Library/LaunchAgents/se.jordbo.mempalace-session.plist
|
|
mkdir -p ~/Library/Logs
|
|
launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/se.jordbo.mempalace-session.plist
|
|
launchctl enable "gui/$(id -u)/se.jordbo.mempalace-session"
|
|
```
|
|
|
|
See [`contrib/README.md`](contrib/README.md) for full install/verify/uninstall recipes, tuning, and devbox/container caveats. The full operational routine (triggers, cadence, verification) is in [`ARCHITECTURE.md`](ARCHITECTURE.md) §5.
|
|
|
|
### Containerized (devbox) notes
|
|
|
|
On a Docker-based devbox, the palace and opencode DB should live on named volumes so they survive container recreate:
|
|
|
|
- `devbox-palace` → `~/.mempalace/palace`
|
|
- `devbox-data` → `~/.local/share/opencode`
|
|
|
|
This repo is typically bind-mounted from the host, so code survives recreate and syncs via git. After a container recreate, `~/.local/bin` is wiped — just re-run `./install.sh` (idempotent) to relink.
|
|
|
|
---
|
|
|
|
## `mempalace-docs`
|
|
|
|
Docs-only MemPalace miner. Stages documentation files into a cache dir and runs `mempalace mine` against the cache — never against the raw project dir.
|
|
|
|
```bash
|
|
mempalace-docs <directory> # mine with wing = dirname
|
|
mempalace-docs <directory> --wing my_project # override wing name
|
|
mempalace-docs <directory> --agent alice # record agent on drawers
|
|
mempalace-docs <directory> --dry-run # list files, don't file
|
|
mempalace-docs <directory> --no-repair # skip post-mine repair
|
|
mempalace-docs --help
|
|
```
|
|
|
|
**What gets mined:** `*.md`, `*.mdx`, `*.rst`, `*.txt`, `*.yml`, `*.yaml`, `*.toml`, `*.json`, `*.sh`, `*.bash`, `*.zsh`, `*.fish`, `Dockerfile*`, `Makefile*`, `*.conf`, `*.cfg`, `*.ini`, `LICENSE*`, `COPYING*`, `NOTICE*`.
|
|
|
|
**What gets skipped:** `.py`, `.ts`, `.tsx`, `.js`, `.jsx`, `.go`, `.rs`, `.java`, `.cpp`, `.c`, `.rb`, `.kt`, `.swift`, build output directories (`.git`, `.venv`, `node_modules`, `__pycache__`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `dist`, `build`, `.next`, `target`, `coverage`), lockfiles.
|
|
|
|
**Rationale:** the palace is for *context and intent*. Agents already have `grep`/`glob`/`Read` for code — always authoritative, never stale. Embedding source code creates a parallel, lossier, drift-prone copy that pollutes semantic search for years.
|
|
|
|
---
|
|
|
|
## `mempalace-session`
|
|
|
|
Opencode → MemPalace session bridge. Reads `~/.local/share/opencode/opencode.db`, transforms each session into Claude Code JSONL, and files via `mempalace mine --mode convos`.
|
|
|
|
```bash
|
|
mempalace-session # mine all sessions (≥3 msgs)
|
|
mempalace-session --wing my_convos # custom wing (default: wing_conversations)
|
|
mempalace-session --session ses_abc123 # one session only
|
|
mempalace-session --since 2026-04-01 # only sessions updated on/after date
|
|
mempalace-session --min-messages 6 # stricter short-session filter
|
|
mempalace-session --db /custom/path/opencode.db # non-default DB location
|
|
mempalace-session --dry-run # export + list, skip mine
|
|
mempalace-session --no-repair # skip post-mine index repair
|
|
mempalace-session --help
|
|
```
|
|
|
|
**What gets exported per session:**
|
|
|
|
- Synthetic header injected as the first user turn (`[session: <title> | <dir> | <date>]`) so the palace can find sessions by topic, not just by ID.
|
|
- Each message → Claude Code JSONL line (`{"type": "user"|"assistant", "message": {"content": ...}}`).
|
|
- Tool calls → `tool_use` blocks. Known tools (`Bash`, `Read`, `Grep`, `Edit`, `Write`) get formatted summaries; unknown tools are JSON-serialized.
|
|
- Tool outputs → `tool_result` blocks in a follow-up human message, folded back into the assistant turn by the mempalace normalizer.
|
|
- `step-start` / `step-finish` parts are dropped as noise. `reasoning` parts are kept with a `[reasoning]` prefix.
|
|
|
|
**Dedup:** staging at `~/.cache/mempalace-session/<wing>/` with deterministic per-session filenames (`<slug>_<id>.jsonl`). The convos miner keys on `source_file`, so re-runs skip unchanged sessions. To force re-mining a session, delete its JSONL from the staging dir.
|
|
|
|
**Filter:** sessions with fewer than `--min-messages` messages (default 3) are skipped — drops throwaway `/exit`'d sessions that would otherwise flood the palace. On a reference 140-session corpus, 78 were filtered this way.
|
|
|
|
**Cost profile:** ~20 minutes per 60-session batch. Scales roughly linearly with message count. Dedup re-run: mine step instant, only the post-mine `repair` runs (~5 min on 5k drawers).
|
|
|
|
---
|
|
|
|
## Companion agent skill
|
|
|
|
Installing this repo symlinks `SKILL.md` into `~/.agents/skills/opencode-mempalace-bridge/SKILL.md`, where it's auto-discovered by opencode (and by Claude Code / Kiro if you run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils)).
|
|
|
|
The skill is the *short-form checklist* for agents — when to use which wrapper, failure modes, setup recipes, anti-patterns. The canonical reference is always [`ARCHITECTURE.md`](ARCHITECTURE.md); the skill points there for deep context.
|
|
|
|
The skill pairs with the consumer-side [`mempalace` skill](https://github.com/MemPalace/mempalace) — that one covers using the palace (search, diary, KG); this one covers feeding it.
|
|
|
|
**Colocated skill pattern.** The skill lives here (not in [`skillset`](https://gitea.jordbo.se/joakimp/skillset)) because it moves in lockstep with the wrappers it documents. `install.sh` drops a `.skill-source` marker file in the deployed skill directory so sibling tooling (skillset's `deploy-skills.sh`, cli_utils's `agents-sync.zsh`) can tell the directory is externally owned. See [`AGENTS.md`](AGENTS.md) for the full convention and how to adopt it for future colocated skills.
|
|
|
|
---
|
|
|
|
## See also
|
|
|
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — canonical spec: diagrams, setup recipe, failure modes, upstream roadmap.
|
|
- [`AGENTS.md`](AGENTS.md) — repo conventions for AI agents modifying this codebase.
|
|
- [MemPalace](https://github.com/MemPalace/mempalace) — the memory layer itself.
|
|
- [opencode](https://github.com/anomalyco/opencode) — the agent harness this bridges.
|
|
- [cli_utils](https://gitea.jordbo.se/joakimp/cli_utils) — sibling repo with shell quality-of-life tools (origin of these wrappers before the 2026-04-30 split).
|