Joakim Persson
954c3f2ebb
Producer-side MemPalace tooling: two bash wrappers that bridge opencode session history and project documentation into the palace. Originally developed in cli_utils (2026-04-28); split into its own repo on 2026-04-30 because the conceptual fit was weak — cli_utils is interactive shell tooling, while this is agent memory infrastructure with its own architecture, dependency surface, and growth trajectory. Contents: - bin/mempalace-docs — docs-only mining wrapper (originally a2ddcc9 in cli_utils), bridges the gap until MemPalace PR #1213 (exclude_patterns) merges upstream. - bin/mempalace-session — opencode → palace session bridge (originally dacca0e in cli_utils). Reads ~/.local/share/opencode/opencode.db, exports each session to Claude Code JSONL, mines via 'mempalace mine --mode convos'. Bridges the gap until opencode session-stopping hooks + an opencode harness in hooks_cli.py land upstream. - ARCHITECTURE.md — canonical spec: architecture diagram, component details, setup recipe, operational notes, upstream-retirement roadmap. Originally a4cf314 in cli_utils. - SKILL.md — companion agent skill (producer side). Pairs with the consumer-side mempalace skill. Symlinked into ~/.agents/skills/opencode-mempalace-bridge/ by install.sh. - install.sh — idempotent installer, also handles --uninstall. - AGENTS.md — repo conventions. History of the individual files is not preserved in this split; see cli_utils (gitea.jordbo.se/joakimp/cli_utils) commits a2ddcc9, dacca0e, and a4cf314 for the original authorship context.
154 lines
8.8 KiB
Markdown
154 lines
8.8 KiB
Markdown
# mempalace-toolkit
|
|
|
|
Producer-side tooling for [MemPalace](https://github.com/MemPalace/mempalace) — bridges that feed opencode session history and project documentation into the palace. Pairs with the consumer-side [`mempalace` agent skill](https://github.com/MemPalace/mempalace).
|
|
|
|
**What this repo contains:**
|
|
|
|
- `bin/mempalace-session` — exports [opencode](https://github.com/anomalyco/opencode) session history from its local SQLite DB to Claude Code JSONL, then mines it via `mempalace mine --mode convos`.
|
|
- `bin/mempalace-docs` — mines project directories into MemPalace while excluding source code, keeping the palace signal-dense.
|
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — **canonical spec**: architecture diagram, component details, setup recipe, operational notes, upstream-retirement roadmap.
|
|
- [`SKILL.md`](SKILL.md) — the companion agent skill, symlinked into `~/.agents/skills/opencode-mempalace-bridge/` on install.
|
|
|
|
**If you're just trying to get this working on a new machine → jump to [Setup](#setup).**
|
|
**If you want the full architecture story → read [`ARCHITECTURE.md`](ARCHITECTURE.md).**
|
|
|
|
---
|
|
|
|
## Why this exists
|
|
|
|
MemPalace is the agent memory layer. Its stock CLI has two gaps that bite on a machine running opencode with a docs-first palace policy:
|
|
|
|
1. **`mempalace mine` floods the palace with source code** — every `__init__` fragment, every generated file, hundreds of low-signal drawers per project. `mempalace-docs` fixes this by staging only documentation-class files (`*.md`, `*.yml`, `Dockerfile`, etc.) before mining.
|
|
2. **`mempalace mine --mode convos` can't read opencode's SQLite DB** — only file-based chat formats (Claude Code JSONL, Claude.ai JSON, ChatGPT, Slack, Codex). Opencode persists every turn in `~/.local/share/opencode/opencode.db` and has no upstream hook into mempalace's auto-save. `mempalace-session` fixes this by exporting each session to Claude Code JSONL before mining.
|
|
|
|
Both wrappers follow the same **stage-to-cache-then-mine** idiom. Neither reimplements the miner; they curate input and delegate.
|
|
|
|
Long-term, both should retire:
|
|
- `mempalace-docs` → retires when [MemPalace PR #1213](https://github.com/MemPalace/mempalace/pull/1213) (`exclude_patterns` in `mempalace.yaml`) merges.
|
|
- `mempalace-session` → retires when opencode session-stopping hooks ([PR #16598](https://github.com/anomalyco/opencode/pull/16598) et al.) merge **and** `hooks_cli.py` gains an `opencode` harness. Until both land, this repo fills the gap.
|
|
|
|
See [`ARCHITECTURE.md`](ARCHITECTURE.md) §6 for the full upstream roadmap.
|
|
|
|
---
|
|
|
|
## Setup
|
|
|
|
### Prerequisites
|
|
|
|
- [MemPalace](https://github.com/MemPalace/mempalace) CLI v3.3.3+
|
|
- Python 3 (stdlib `sqlite3` only — no extra deps)
|
|
- [opencode](https://github.com/anomalyco/opencode) with an active session DB at `~/.local/share/opencode/opencode.db` *(only needed for `mempalace-session`)*
|
|
|
|
### Install
|
|
|
|
```bash
|
|
git clone ssh://git@gitea.jordbo.se:2222/joakimp/mempalace-toolkit.git ~/mempalace-toolkit
|
|
cd ~/mempalace-toolkit
|
|
./install.sh
|
|
```
|
|
|
|
The installer symlinks `bin/*` into `~/.local/bin/` and optionally installs the agent skill into `~/.agents/skills/opencode-mempalace-bridge/`.
|
|
|
|
Ensure `~/.local/bin` is on `$PATH`:
|
|
|
|
```bash
|
|
export PATH="$HOME/.local/bin:$PATH"
|
|
```
|
|
|
|
### First mine
|
|
|
|
```bash
|
|
# One-time palace init (if not done)
|
|
mempalace init --yes
|
|
|
|
# Mine opencode session history into wing_conversations
|
|
mempalace-session --dry-run # preview qualifying sessions
|
|
mempalace-session # do it (~20 min per 60 sessions)
|
|
|
|
# Mine a project (docs only)
|
|
mempalace-docs /workspace/my_project --dry-run
|
|
mempalace-docs /workspace/my_project
|
|
```
|
|
|
|
### Containerized (devbox) notes
|
|
|
|
On a Docker-based devbox, the palace and opencode DB should live on named volumes so they survive container recreate:
|
|
|
|
- `devbox-palace` → `~/.mempalace/palace`
|
|
- `devbox-data` → `~/.local/share/opencode`
|
|
|
|
This repo is typically bind-mounted from the host, so code survives recreate and syncs via git. After a container recreate, `~/.local/bin` is wiped — just re-run `./install.sh` (idempotent) to relink.
|
|
|
|
---
|
|
|
|
## `mempalace-docs`
|
|
|
|
Docs-only MemPalace miner. Stages documentation files into a cache dir and runs `mempalace mine` against the cache — never against the raw project dir.
|
|
|
|
```bash
|
|
mempalace-docs <directory> # mine with wing = dirname
|
|
mempalace-docs <directory> --wing my_project # override wing name
|
|
mempalace-docs <directory> --agent alice # record agent on drawers
|
|
mempalace-docs <directory> --dry-run # list files, don't file
|
|
mempalace-docs <directory> --no-repair # skip post-mine repair
|
|
mempalace-docs --help
|
|
```
|
|
|
|
**What gets mined:** `*.md`, `*.mdx`, `*.rst`, `*.txt`, `*.yml`, `*.yaml`, `*.toml`, `*.json`, `*.sh`, `*.bash`, `*.zsh`, `*.fish`, `Dockerfile*`, `Makefile*`, `*.conf`, `*.cfg`, `*.ini`, `LICENSE*`, `COPYING*`, `NOTICE*`.
|
|
|
|
**What gets skipped:** `.py`, `.ts`, `.tsx`, `.js`, `.jsx`, `.go`, `.rs`, `.java`, `.cpp`, `.c`, `.rb`, `.kt`, `.swift`, build output directories (`.git`, `.venv`, `node_modules`, `__pycache__`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `dist`, `build`, `.next`, `target`, `coverage`), lockfiles.
|
|
|
|
**Rationale:** the palace is for *context and intent*. Agents already have `grep`/`glob`/`Read` for code — always authoritative, never stale. Embedding source code creates a parallel, lossier, drift-prone copy that pollutes semantic search for years.
|
|
|
|
---
|
|
|
|
## `mempalace-session`
|
|
|
|
Opencode → MemPalace session bridge. Reads `~/.local/share/opencode/opencode.db`, transforms each session into Claude Code JSONL, and files via `mempalace mine --mode convos`.
|
|
|
|
```bash
|
|
mempalace-session # mine all sessions (≥3 msgs)
|
|
mempalace-session --wing my_convos # custom wing (default: wing_conversations)
|
|
mempalace-session --session ses_abc123 # one session only
|
|
mempalace-session --since 2026-04-01 # only sessions updated on/after date
|
|
mempalace-session --min-messages 6 # stricter short-session filter
|
|
mempalace-session --db /custom/path/opencode.db # non-default DB location
|
|
mempalace-session --dry-run # export + list, skip mine
|
|
mempalace-session --no-repair # skip post-mine index repair
|
|
mempalace-session --help
|
|
```
|
|
|
|
**What gets exported per session:**
|
|
|
|
- Synthetic header injected as the first user turn (`[session: <title> | <dir> | <date>]`) so the palace can find sessions by topic, not just by ID.
|
|
- Each message → Claude Code JSONL line (`{"type": "user"|"assistant", "message": {"content": ...}}`).
|
|
- Tool calls → `tool_use` blocks. Known tools (`Bash`, `Read`, `Grep`, `Edit`, `Write`) get formatted summaries; unknown tools are JSON-serialized.
|
|
- Tool outputs → `tool_result` blocks in a follow-up human message, folded back into the assistant turn by the mempalace normalizer.
|
|
- `step-start` / `step-finish` parts are dropped as noise. `reasoning` parts are kept with a `[reasoning]` prefix.
|
|
|
|
**Dedup:** staging at `~/.cache/mempalace-session/<wing>/` with deterministic per-session filenames (`<slug>_<id>.jsonl`). The convos miner keys on `source_file`, so re-runs skip unchanged sessions. To force re-mining a session, delete its JSONL from the staging dir.
|
|
|
|
**Filter:** sessions with fewer than `--min-messages` messages (default 3) are skipped — drops throwaway `/exit`'d sessions that would otherwise flood the palace. On a reference 140-session corpus, 78 were filtered this way.
|
|
|
|
**Cost profile:** ~20 minutes per 60-session batch. Scales roughly linearly with message count. Dedup re-run: mine step instant, only the post-mine `repair` runs (~5 min on 5k drawers).
|
|
|
|
---
|
|
|
|
## Companion agent skill
|
|
|
|
Installing this repo symlinks `SKILL.md` into `~/.agents/skills/opencode-mempalace-bridge/SKILL.md`, where it's auto-discovered by opencode (and by Claude Code / Kiro if you run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils)).
|
|
|
|
The skill is the *short-form checklist* for agents — when to use which wrapper, failure modes, setup recipes, anti-patterns. The canonical reference is always [`ARCHITECTURE.md`](ARCHITECTURE.md); the skill points there for deep context.
|
|
|
|
The skill pairs with the consumer-side [`mempalace` skill](https://github.com/MemPalace/mempalace) — that one covers using the palace (search, diary, KG); this one covers feeding it.
|
|
|
|
---
|
|
|
|
## See also
|
|
|
|
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — canonical spec: diagrams, setup recipe, failure modes, upstream roadmap.
|
|
- [`AGENTS.md`](AGENTS.md) — repo conventions for AI agents modifying this codebase.
|
|
- [MemPalace](https://github.com/MemPalace/mempalace) — the memory layer itself.
|
|
- [opencode](https://github.com/anomalyco/opencode) — the agent harness this bridges.
|
|
- [cli_utils](https://gitea.jordbo.se/joakimp/cli_utils) — sibling repo with shell quality-of-life tools (origin of these wrappers before the 2026-04-30 split).
|