Joakim Persson
36845e14b2
Until opencode session-stopping hooks land upstream, mempalace-session
is the entire mechanism that gets opencode conversations into the
palace — skip it and session history stays trapped in a local SQLite
DB, invisible to semantic search. Previous docs covered setup well
but were thin on when and how often to run it.
- ARCHITECTURE.md §5: replace the one-line 'When to re-mine' note with
a full Operational Routine section — triggers, cadence, relationship
to the session lifecycle, automation pointers, verification.
- SKILL.md: add an Operational Routine section aimed at agents —
when to suggest invoking the tool, cadence guidance, how to
distinguish this producer-side tool from the consumer-side
mempalace skill's in-session habits.
- README.md: add 'Keeping it fresh' subsection pointing at contrib/
and the full docs.
contrib/ ships three ready-to-use templates:
- systemd/mempalace-session.{service,timer} — user units with weekly
Mon 03:00 schedule, Persistent=true catch-up, RandomizedDelaySec for
fleet-wide jitter, ConditionPathExists guard for opencode-less boxes,
Nice+IOSchedulingClass=idle so it never fights interactive work.
- cron/mempalace-session.cron — sample crontab entry with log
redirection and clear USER-substitution instructions.
- README.md with install/verify/uninstall recipes for both, a chooser
table (systemd vs cron), container/devbox caveats, and tuning notes
(daily vs weekly vs monthly trade-offs).
The user's LATER-list item 'wrap mempalace-session in cron/systemd
timer for true auto-save coverage' is now actionable: a single
systemctl --user enable --now command stands it up.
175 lines
10 KiB
Markdown
175 lines
10 KiB
Markdown
# mempalace-toolkit
|
|
|
|
Producer-side tooling for [MemPalace](https://github.com/MemPalace/mempalace) — bridges that feed opencode session history and project documentation into the palace. Pairs with the consumer-side [`mempalace` agent skill](https://github.com/MemPalace/mempalace).
|
|
|
|
**What this repo contains:**
|
|
|
|
- `bin/mempalace-session` — exports [opencode](https://github.com/anomalyco/opencode) session history from its local SQLite DB to Claude Code JSONL, then mines it via `mempalace mine --mode convos`.
|
|
- `bin/mempalace-docs` — mines project directories into MemPalace while excluding source code, keeping the palace signal-dense.
|
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — **canonical spec**: architecture diagram, component details, setup recipe, operational notes, upstream-retirement roadmap.
|
|
- [`SKILL.md`](SKILL.md) — the companion agent skill, symlinked into `~/.agents/skills/opencode-mempalace-bridge/` on install.
|
|
|
|
**If you're just trying to get this working on a new machine → jump to [Setup](#setup).**
|
|
**If you want the full architecture story → read [`ARCHITECTURE.md`](ARCHITECTURE.md).**
|
|
|
|
---
|
|
|
|
## Why this exists
|
|
|
|
MemPalace is the agent memory layer. Its stock CLI has two gaps that bite on a machine running opencode with a docs-first palace policy:
|
|
|
|
1. **`mempalace mine` floods the palace with source code** — every `__init__` fragment, every generated file, hundreds of low-signal drawers per project. `mempalace-docs` fixes this by staging only documentation-class files (`*.md`, `*.yml`, `Dockerfile`, etc.) before mining.
|
|
2. **`mempalace mine --mode convos` can't read opencode's SQLite DB** — only file-based chat formats (Claude Code JSONL, Claude.ai JSON, ChatGPT, Slack, Codex). Opencode persists every turn in `~/.local/share/opencode/opencode.db` and has no upstream hook into mempalace's auto-save. `mempalace-session` fixes this by exporting each session to Claude Code JSONL before mining.
|
|
|
|
Both wrappers follow the same **stage-to-cache-then-mine** idiom. Neither reimplements the miner; they curate input and delegate.
|
|
|
|
Long-term, both should retire:
|
|
- `mempalace-docs` → retires when [MemPalace PR #1213](https://github.com/MemPalace/mempalace/pull/1213) (`exclude_patterns` in `mempalace.yaml`) merges.
|
|
- `mempalace-session` → retires when opencode session-stopping hooks ([PR #16598](https://github.com/anomalyco/opencode/pull/16598) et al.) merge **and** `hooks_cli.py` gains an `opencode` harness. Until both land, this repo fills the gap.
|
|
|
|
See [`ARCHITECTURE.md`](ARCHITECTURE.md) §6 for the full upstream roadmap.
|
|
|
|
---
|
|
|
|
## Setup
|
|
|
|
### Prerequisites
|
|
|
|
- [MemPalace](https://github.com/MemPalace/mempalace) CLI v3.3.3+
|
|
- Python 3 (stdlib `sqlite3` only — no extra deps)
|
|
- [opencode](https://github.com/anomalyco/opencode) with an active session DB at `~/.local/share/opencode/opencode.db` *(only needed for `mempalace-session`)*
|
|
|
|
### Install
|
|
|
|
```bash
|
|
git clone ssh://git@gitea.jordbo.se:2222/joakimp/mempalace-toolkit.git ~/mempalace-toolkit
|
|
cd ~/mempalace-toolkit
|
|
./install.sh
|
|
```
|
|
|
|
The installer symlinks `bin/*` into `~/.local/bin/` and optionally installs the agent skill into `~/.agents/skills/opencode-mempalace-bridge/`.
|
|
|
|
Ensure `~/.local/bin` is on `$PATH`:
|
|
|
|
```bash
|
|
export PATH="$HOME/.local/bin:$PATH"
|
|
```
|
|
|
|
### First mine
|
|
|
|
```bash
|
|
# One-time palace init (if not done)
|
|
mempalace init --yes
|
|
|
|
# Mine opencode session history into wing_conversations
|
|
mempalace-session --dry-run # preview qualifying sessions
|
|
mempalace-session # do it (~20 min per 60 sessions)
|
|
|
|
# Mine a project (docs only)
|
|
mempalace-docs /workspace/my_project --dry-run
|
|
mempalace-docs /workspace/my_project
|
|
```
|
|
|
|
### Keeping it fresh (automation)
|
|
|
|
Manual invocation is fine while you're actively driving the machine, but long-running devboxes benefit from a weekly automated mine. [`contrib/`](contrib/) ships ready-to-install templates:
|
|
|
|
- **systemd user timer** (recommended on modern Linux): survives reboots, catches missed runs, logs to `journalctl`.
|
|
- **cron**: simpler, works everywhere, no user-unit awareness needed.
|
|
|
|
Quick-start (systemd, weekly Mon 03:00 local):
|
|
|
|
```bash
|
|
mkdir -p ~/.config/systemd/user
|
|
cp contrib/systemd/*.{service,timer} ~/.config/systemd/user/
|
|
systemctl --user daemon-reload
|
|
systemctl --user enable --now mempalace-session.timer
|
|
sudo loginctl enable-linger "$USER" # optional, for headless boxes
|
|
```
|
|
|
|
See [`contrib/README.md`](contrib/README.md) for full install/verify/uninstall recipes, tuning, and devbox/container caveats. The full operational routine (triggers, cadence, verification) is in [`ARCHITECTURE.md`](ARCHITECTURE.md) §5.
|
|
|
|
### Containerized (devbox) notes
|
|
|
|
On a Docker-based devbox, the palace and opencode DB should live on named volumes so they survive container recreate:
|
|
|
|
- `devbox-palace` → `~/.mempalace/palace`
|
|
- `devbox-data` → `~/.local/share/opencode`
|
|
|
|
This repo is typically bind-mounted from the host, so code survives recreate and syncs via git. After a container recreate, `~/.local/bin` is wiped — just re-run `./install.sh` (idempotent) to relink.
|
|
|
|
---
|
|
|
|
## `mempalace-docs`
|
|
|
|
Docs-only MemPalace miner. Stages documentation files into a cache dir and runs `mempalace mine` against the cache — never against the raw project dir.
|
|
|
|
```bash
|
|
mempalace-docs <directory> # mine with wing = dirname
|
|
mempalace-docs <directory> --wing my_project # override wing name
|
|
mempalace-docs <directory> --agent alice # record agent on drawers
|
|
mempalace-docs <directory> --dry-run # list files, don't file
|
|
mempalace-docs <directory> --no-repair # skip post-mine repair
|
|
mempalace-docs --help
|
|
```
|
|
|
|
**What gets mined:** `*.md`, `*.mdx`, `*.rst`, `*.txt`, `*.yml`, `*.yaml`, `*.toml`, `*.json`, `*.sh`, `*.bash`, `*.zsh`, `*.fish`, `Dockerfile*`, `Makefile*`, `*.conf`, `*.cfg`, `*.ini`, `LICENSE*`, `COPYING*`, `NOTICE*`.
|
|
|
|
**What gets skipped:** `.py`, `.ts`, `.tsx`, `.js`, `.jsx`, `.go`, `.rs`, `.java`, `.cpp`, `.c`, `.rb`, `.kt`, `.swift`, build output directories (`.git`, `.venv`, `node_modules`, `__pycache__`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `dist`, `build`, `.next`, `target`, `coverage`), lockfiles.
|
|
|
|
**Rationale:** the palace is for *context and intent*. Agents already have `grep`/`glob`/`Read` for code — always authoritative, never stale. Embedding source code creates a parallel, lossier, drift-prone copy that pollutes semantic search for years.
|
|
|
|
---
|
|
|
|
## `mempalace-session`
|
|
|
|
Opencode → MemPalace session bridge. Reads `~/.local/share/opencode/opencode.db`, transforms each session into Claude Code JSONL, and files via `mempalace mine --mode convos`.
|
|
|
|
```bash
|
|
mempalace-session # mine all sessions (≥3 msgs)
|
|
mempalace-session --wing my_convos # custom wing (default: wing_conversations)
|
|
mempalace-session --session ses_abc123 # one session only
|
|
mempalace-session --since 2026-04-01 # only sessions updated on/after date
|
|
mempalace-session --min-messages 6 # stricter short-session filter
|
|
mempalace-session --db /custom/path/opencode.db # non-default DB location
|
|
mempalace-session --dry-run # export + list, skip mine
|
|
mempalace-session --no-repair # skip post-mine index repair
|
|
mempalace-session --help
|
|
```
|
|
|
|
**What gets exported per session:**
|
|
|
|
- Synthetic header injected as the first user turn (`[session: <title> | <dir> | <date>]`) so the palace can find sessions by topic, not just by ID.
|
|
- Each message → Claude Code JSONL line (`{"type": "user"|"assistant", "message": {"content": ...}}`).
|
|
- Tool calls → `tool_use` blocks. Known tools (`Bash`, `Read`, `Grep`, `Edit`, `Write`) get formatted summaries; unknown tools are JSON-serialized.
|
|
- Tool outputs → `tool_result` blocks in a follow-up human message, folded back into the assistant turn by the mempalace normalizer.
|
|
- `step-start` / `step-finish` parts are dropped as noise. `reasoning` parts are kept with a `[reasoning]` prefix.
|
|
|
|
**Dedup:** staging at `~/.cache/mempalace-session/<wing>/` with deterministic per-session filenames (`<slug>_<id>.jsonl`). The convos miner keys on `source_file`, so re-runs skip unchanged sessions. To force re-mining a session, delete its JSONL from the staging dir.
|
|
|
|
**Filter:** sessions with fewer than `--min-messages` messages (default 3) are skipped — drops throwaway `/exit`'d sessions that would otherwise flood the palace. On a reference 140-session corpus, 78 were filtered this way.
|
|
|
|
**Cost profile:** ~20 minutes per 60-session batch. Scales roughly linearly with message count. Dedup re-run: mine step instant, only the post-mine `repair` runs (~5 min on 5k drawers).
|
|
|
|
---
|
|
|
|
## Companion agent skill
|
|
|
|
Installing this repo symlinks `SKILL.md` into `~/.agents/skills/opencode-mempalace-bridge/SKILL.md`, where it's auto-discovered by opencode (and by Claude Code / Kiro if you run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils)).
|
|
|
|
The skill is the *short-form checklist* for agents — when to use which wrapper, failure modes, setup recipes, anti-patterns. The canonical reference is always [`ARCHITECTURE.md`](ARCHITECTURE.md); the skill points there for deep context.
|
|
|
|
The skill pairs with the consumer-side [`mempalace` skill](https://github.com/MemPalace/mempalace) — that one covers using the palace (search, diary, KG); this one covers feeding it.
|
|
|
|
**Colocated skill pattern.** The skill lives here (not in [`skillset`](https://gitea.jordbo.se/joakimp/skillset)) because it moves in lockstep with the wrappers it documents. `install.sh` drops a `.skill-source` marker file in the deployed skill directory so sibling tooling (skillset's `deploy-skills.sh`, cli_utils's `agents-sync.zsh`) can tell the directory is externally owned. See [`AGENTS.md`](AGENTS.md) for the full convention and how to adopt it for future colocated skills.
|
|
|
|
---
|
|
|
|
## See also
|
|
|
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — canonical spec: diagrams, setup recipe, failure modes, upstream roadmap.
|
|
- [`AGENTS.md`](AGENTS.md) — repo conventions for AI agents modifying this codebase.
|
|
- [MemPalace](https://github.com/MemPalace/mempalace) — the memory layer itself.
|
|
- [opencode](https://github.com/anomalyco/opencode) — the agent harness this bridges.
|
|
- [cli_utils](https://gitea.jordbo.se/joakimp/cli_utils) — sibling repo with shell quality-of-life tools (origin of these wrappers before the 2026-04-30 split).
|