# mempalace-toolkit

Producer-side tooling for [MemPalace](https://github.com/MemPalace/mempalace) — bridges that feed opencode session history and project documentation into the palace. Pairs with the consumer-side [`mempalace` agent skill](https://github.com/MemPalace/mempalace).

**What this repo contains:**

- `bin/mempalace-session` — exports [opencode](https://github.com/anomalyco/opencode) session history from its local SQLite DB to Claude Code JSONL, then mines it via `mempalace mine --mode convos`.
- `bin/mempalace-docs` — mines project directories into MemPalace while excluding source code, keeping the palace signal-dense.
- [`ARCHITECTURE.md`](ARCHITECTURE.md) — **canonical spec**: architecture diagram, component details, setup recipe, operational notes, upstream-retirement roadmap.
- [`SKILL.md`](SKILL.md) — the companion agent skill, symlinked into `~/.agents/skills/opencode-mempalace-bridge/` on install.

**If you're just trying to get this working on a new machine → jump to [Setup](#setup).**
**If you want the full architecture story → read [`ARCHITECTURE.md`](ARCHITECTURE.md).**

---

## Why this exists

MemPalace is the agent memory layer. Its stock CLI has two gaps that bite on a machine running opencode with a docs-first palace policy:

1. **`mempalace mine` floods the palace with source code** — every `__init__` fragment, every generated file, hundreds of low-signal drawers per project. `mempalace-docs` fixes this by staging only documentation-class files (`*.md`, `*.yml`, `Dockerfile`, etc.) before mining.
2. **`mempalace mine --mode convos` can't read opencode's SQLite DB** — only file-based chat formats (Claude Code JSONL, Claude.ai JSON, ChatGPT, Slack, Codex). Opencode persists every turn in `~/.local/share/opencode/opencode.db` and has no upstream hook into mempalace's auto-save. `mempalace-session` fixes this by exporting each session to Claude Code JSONL before mining.

Both wrappers follow the same **stage-to-cache-then-mine** idiom. Neither reimplements the miner; they curate input and delegate.

Long-term, both should retire:
- `mempalace-docs` → retires when [MemPalace PR #1213](https://github.com/MemPalace/mempalace/pull/1213) (`exclude_patterns` in `mempalace.yaml`) merges.
- `mempalace-session` → retires when opencode session-stopping hooks ([PR #16598](https://github.com/anomalyco/opencode/pull/16598) et al.) merge **and** `hooks_cli.py` gains an `opencode` harness. Until both land, this repo fills the gap.

See [`ARCHITECTURE.md`](ARCHITECTURE.md) §6 for the full upstream roadmap.

---

## Setup

### Prerequisites

- [MemPalace](https://github.com/MemPalace/mempalace) CLI v3.3.3+
- Python 3 (stdlib `sqlite3` only — no extra deps)
- [opencode](https://github.com/anomalyco/opencode) with an active session DB at `~/.local/share/opencode/opencode.db` *(only needed for `mempalace-session`)*

### Install

```bash
git clone ssh://git@gitea.jordbo.se:2222/joakimp/mempalace-toolkit.git ~/mempalace-toolkit
cd ~/mempalace-toolkit
./install.sh
```

The installer symlinks `bin/*` into `~/.local/bin/` and optionally installs the agent skill into `~/.agents/skills/opencode-mempalace-bridge/`.

Ensure `~/.local/bin` is on `$PATH`:

```bash
export PATH="$HOME/.local/bin:$PATH"
```

### First mine

```bash
# One-time palace init (if not done)
mempalace init --yes

# Mine opencode session history into wing_conversations
mempalace-session --dry-run      # preview qualifying sessions
mempalace-session                # do it (~20 min per 60 sessions)

# Mine a project (docs only)
mempalace-docs /workspace/my_project --dry-run
mempalace-docs /workspace/my_project
```

### Keeping it fresh (automation)

Manual invocation is fine while you're actively driving the machine, but long-running devboxes benefit from a weekly automated mine. [`contrib/`](contrib/) ships ready-to-install templates:

- **systemd user timer** (recommended on Linux): survives reboots, catches missed runs, logs to `journalctl`.
- **launchd user agent** (recommended on macOS): native-equivalent — logs to `~/Library/Logs/`, single-instance guarantees, `ProcessType=Background` throttling.
- **cron**: simplest, works on BSD and systemd-less distros. No user-unit awareness needed.

Quick-start (Linux / systemd, weekly Mon 03:00 local):

```bash
mkdir -p ~/.config/systemd/user
cp contrib/systemd/*.{service,timer} ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now mempalace-session.timer
sudo loginctl enable-linger "$USER"   # optional, for headless boxes
```

Quick-start (macOS / launchd, same schedule):

```bash
sed "s|USER|$USER|g" contrib/launchd/se.jordbo.mempalace-session.plist \
  > ~/Library/LaunchAgents/se.jordbo.mempalace-session.plist
mkdir -p ~/Library/Logs
launchctl bootstrap "gui/$(id -u)" ~/Library/LaunchAgents/se.jordbo.mempalace-session.plist
launchctl enable "gui/$(id -u)/se.jordbo.mempalace-session"
```

See [`contrib/README.md`](contrib/README.md) for full install/verify/uninstall recipes, tuning, and devbox/container caveats. The full operational routine (triggers, cadence, verification) is in [`ARCHITECTURE.md`](ARCHITECTURE.md) §5.

### Containerized (devbox) notes

On a Docker-based devbox, the palace and opencode DB should live on named volumes so they survive container recreate:

- `devbox-palace` → `~/.mempalace/palace`
- `devbox-data` → `~/.local/share/opencode`

This repo is typically bind-mounted from the host, so code survives recreate and syncs via git. After a container recreate, `~/.local/bin` is wiped — just re-run `./install.sh` (idempotent) to relink.

---

## `mempalace-docs`

Docs-only MemPalace miner. Stages documentation files into a cache dir and runs `mempalace mine` against the cache — never against the raw project dir.

```bash
mempalace-docs <directory>                     # mine with wing = dirname
mempalace-docs <directory> --wing my_project   # override wing name
mempalace-docs <directory> --agent alice       # record agent on drawers
mempalace-docs <directory> --dry-run           # list files, don't file
mempalace-docs <directory> --no-repair         # skip post-mine repair
mempalace-docs --help
```

**What gets mined:** `*.md`, `*.mdx`, `*.rst`, `*.txt`, `*.yml`, `*.yaml`, `*.toml`, `*.json`, `*.sh`, `*.bash`, `*.zsh`, `*.fish`, `Dockerfile*`, `Makefile*`, `*.conf`, `*.cfg`, `*.ini`, `LICENSE*`, `COPYING*`, `NOTICE*`.

**What gets skipped:** `.py`, `.ts`, `.tsx`, `.js`, `.jsx`, `.go`, `.rs`, `.java`, `.cpp`, `.c`, `.rb`, `.kt`, `.swift`, build output directories (`.git`, `.venv`, `node_modules`, `__pycache__`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `dist`, `build`, `.next`, `target`, `coverage`), lockfiles.

**Rationale:** the palace is for *context and intent*. Agents already have `grep`/`glob`/`Read` for code — always authoritative, never stale. Embedding source code creates a parallel, lossier, drift-prone copy that pollutes semantic search for years.

---

## `mempalace-session`

Opencode → MemPalace session bridge. Reads `~/.local/share/opencode/opencode.db`, transforms each session into Claude Code JSONL, and files via `mempalace mine --mode convos`.

```bash
mempalace-session                                 # mine all sessions (≥3 msgs)
mempalace-session --wing my_convos                # custom wing (default: wing_conversations)
mempalace-session --session ses_abc123            # one session only
mempalace-session --since 2026-04-01              # only sessions updated on/after date
mempalace-session --min-messages 6                # stricter short-session filter
mempalace-session --db /custom/path/opencode.db   # non-default DB location
mempalace-session --dry-run                       # export + list, skip mine
mempalace-session --no-repair                     # skip post-mine index repair
mempalace-session --help
```

**What gets exported per session:**

- Synthetic header injected as the first user turn (`[session: <title> | <dir> | <date>]`) so the palace can find sessions by topic, not just by ID.
- Each message → Claude Code JSONL line (`{"type": "user"|"assistant", "message": {"content": ...}}`).
- Tool calls → `tool_use` blocks. Known tools (`Bash`, `Read`, `Grep`, `Edit`, `Write`) get formatted summaries; unknown tools are JSON-serialized.
- Tool outputs → `tool_result` blocks in a follow-up human message, folded back into the assistant turn by the mempalace normalizer.
- `step-start` / `step-finish` parts are dropped as noise. `reasoning` parts are kept with a `[reasoning]` prefix.

**Dedup:** staging at `~/.cache/mempalace-session/<wing>/` with deterministic per-session filenames (`<slug>_<id>.jsonl`). The convos miner keys on `source_file`, so re-runs skip unchanged sessions. To force re-mining a session, delete its JSONL from the staging dir.

**Filter:** sessions with fewer than `--min-messages` messages (default 3) are skipped — drops throwaway `/exit`'d sessions that would otherwise flood the palace. On a reference 140-session corpus, 78 were filtered this way.

**Cost profile:** ~20 minutes per 60-session batch. Scales roughly linearly with message count. Dedup re-run: mine step instant, only the post-mine `repair` runs (~5 min on 5k drawers).

---

## Companion agent skill

Installing this repo symlinks `SKILL.md` into `~/.agents/skills/opencode-mempalace-bridge/SKILL.md`, where it's auto-discovered by opencode (and by Claude Code / Kiro if you run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils)).

The skill is the *short-form checklist* for agents — when to use which wrapper, failure modes, setup recipes, anti-patterns. The canonical reference is always [`ARCHITECTURE.md`](ARCHITECTURE.md); the skill points there for deep context.

The skill pairs with the consumer-side [`mempalace` skill](https://github.com/MemPalace/mempalace) — that one covers using the palace (search, diary, KG); this one covers feeding it.

**Colocated skill pattern.** The skill lives here (not in [`skillset`](https://gitea.jordbo.se/joakimp/skillset)) because it moves in lockstep with the wrappers it documents. `install.sh` drops a `.skill-source` marker file in the deployed skill directory so sibling tooling (skillset's `deploy-skills.sh`, cli_utils's `agents-sync.zsh`) can tell the directory is externally owned. See [`AGENTS.md`](AGENTS.md) for the full convention and how to adopt it for future colocated skills.

---

## See also

- [`ARCHITECTURE.md`](ARCHITECTURE.md) — canonical spec: diagrams, setup recipe, failure modes, upstream roadmap.
- [`AGENTS.md`](AGENTS.md) — repo conventions for AI agents modifying this codebase.
- [MemPalace](https://github.com/MemPalace/mempalace) — the memory layer itself.
- [opencode](https://github.com/anomalyco/opencode) — the agent harness this bridges.
- [cli_utils](https://gitea.jordbo.se/joakimp/cli_utils) — sibling repo with shell quality-of-life tools (origin of these wrappers before the 2026-04-30 split).