joakimp/mempalace-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit — split out from cli_utils

Browse Source 
Producer-side MemPalace tooling: two bash wrappers that bridge opencode
session history and project documentation into the palace. Originally
developed in cli_utils (2026-04-28); split into its own repo on
2026-04-30 because the conceptual fit was weak — cli_utils is
interactive shell tooling, while this is agent memory infrastructure
with its own architecture, dependency surface, and growth trajectory.

Contents:
- bin/mempalace-docs — docs-only mining wrapper (originally a2ddcc9 in
  cli_utils), bridges the gap until MemPalace PR #1213 (exclude_patterns)
  merges upstream.
- bin/mempalace-session — opencode → palace session bridge (originally
  dacca0e in cli_utils). Reads ~/.local/share/opencode/opencode.db,
  exports each session to Claude Code JSONL, mines via
  'mempalace mine --mode convos'. Bridges the gap until opencode
  session-stopping hooks + an opencode harness in hooks_cli.py land
  upstream.
- ARCHITECTURE.md — canonical spec: architecture diagram, component
  details, setup recipe, operational notes, upstream-retirement
  roadmap. Originally a4cf314 in cli_utils.
- SKILL.md — companion agent skill (producer side). Pairs with the
  consumer-side mempalace skill. Symlinked into
  ~/.agents/skills/opencode-mempalace-bridge/ by install.sh.
- install.sh — idempotent installer, also handles --uninstall.
- AGENTS.md — repo conventions.

History of the individual files is not preserved in this split; see
cli_utils (gitea.jordbo.se/joakimp/cli_utils) commits a2ddcc9, dacca0e,
and a4cf314 for the original authorship context.
This commit is contained in:
Joakim Persson
2026-04-30 05:30:04 +00:00
commit 954c3f2ebb
8 changed files with 1438 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SKILL.md
+157
View File
@@ -0,0 +1,157 @@
---
name: opencode-mempalace-bridge
description: Set up the producer side of MemPalace — feed opencode session history and project docs into the palace via the cli_utils wrappers. Use when provisioning a new machine, when the user asks how palace feeding works, when opencode sessions aren't showing up in searches, or when a project needs docs-only mining. Pairs with the `mempalace` skill (consumer side).
---
# Opencode ↔ MemPalace Bridge (producer side)
## Overview
The `mempalace` skill covers *using* the palace (search, diary, KG). This skill covers *feeding* it — specifically, how to wire opencode session history and project docs into the palace on a new machine or after a container recreate.
**Authoritative source:** `/workspace/cli_utils/ARCHITECTURE.md` (also at `<cli_utils>/ARCHITECTURE.md` in the gitea repo). When in doubt, read that file — it's the canonical spec. This skill is the short-form checklist.
**Core idea:** two thin wrappers in `cli_utils/bin/` close gaps in the stock mempalace CLI:
| Gap | Wrapper |
| ---------------------------------------------------------------------------------------- | -------------------- |
| `mempalace mine` floods the palace with source code we don't want | `mempalace-docs` |
| `mempalace mine --mode convos` can't read opencode's SQLite DB | `mempalace-session` |
Both follow the same **stage-to-cache-then-mine** idiom — they curate input into `~/.cache/…/<wing>/`, then delegate to `mempalace mine`.
## When to Load This Skill
- User asks "how does the palace get fed?" or mentions setting up mempalace on a new machine.
- Opencode conversations are missing from palace searches (`wing_conversations` is empty or stale).
- A project needs to be mined but you want *docs only, no source code*.
- User asks about `mempalace-docs` or `mempalace-session`.
- After a container recreate on a devbox — the wrappers need reinstall.
- Planning to retire either wrapper once upstream PRs merge (see §6 of ARCHITECTURE.md).
## Setup Recipe (new machine)
Prerequisites: `opencode` installed with an active DB at `~/.local/share/opencode/opencode.db`, `mempalace` CLI v3.3.3+, Python 3 (stdlib `sqlite3` only — no extra deps).
```bash
# 1. Clone cli_utils (holds the two wrappers in bin/)
git clone <gitea-url>/cli_utils ~/cli_utils
cd ~/cli_utils
# 2. Install — symlinks bin/* into ~/.local/bin, adds loader to rc file
./install.sh
# 3. Verify ~/.local/bin is on PATH
which mempalace-session mempalace-docs
# 4. Initialize palace (one-time, platform-wide)
mempalace init --yes
# 5. Mine opencode session history into wing_conversations
mempalace-session --dry-run # preview: which sessions qualify?
mempalace-session # do it (~20 min per 60 sessions)
# 6. Mine project docs per project (docs only — no source code)
mempalace-docs /workspace/my_project --dry-run
mempalace-docs /workspace/my_project
# 7. If a long-lived MCP session is open, reconnect it
# (from inside the MCP client): mempalace_reconnect
```
### Containerized (devbox) specifics
Named Docker volumes preserve state across container recreate:
- `devbox-palace` → `~/.mempalace/palace`
- `devbox-data` → `~/.local/share/opencode`
Bind mount `/workspace/cli_utils` from the host — code survives recreate, syncs via gitea.
**After container recreate:** `~/.local/bin` is ephemeral. Just re-run `./install.sh` (idempotent) — everything else already persists.
## Key Operational Rules
### Always dry-run first on a cold system
```bash
mempalace-session --dry-run # shows qualifying sessions
mempalace-docs <dir> --dry-run # shows files that would be mined
```
A docs-heavy repo should produce ~5–10 drawers per file. >15 drawers/file on average = code leaked in; investigate.
### Dedup is free — re-running is safe
- `mempalace-docs`: dedup keyed on `source_file` path + `mtime`. Unchanged files skipped.
- `mempalace-session`: dedup keyed on `source_file` path alone (no mtime check for convos). Staging filenames are deterministic per session (`<slug>_<id>.jsonl`), so re-runs skip already-filed sessions.
Second run immediately after first → 0 new drawers, only the post-mine `repair` step runs (~5 min on 5k drawers).
### Incremental catch-up
```bash
mempalace-session --since 2026-04-20 # only recent sessions
mempalace-session --session ses_abc123 # one specific session
```
### Force re-mine
```bash
rm -rf ~/.cache/mempalace-session/<wing>/ # nukes staging dir
mempalace-session # stages + mines fresh
```
Staging is ephemeral by design; the palace is the source of truth.
## Failure Modes & Fixes
| Symptom | Cause | Fix |
| ---------------------------------------------------------- | ----------------------------------------------- | ------------------------------------------------------- |
| `mempalace-session: command not found` | `~/.local/bin` wiped (container recreate) | `cd ~/cli_utils && ./install.sh` |
| Sessions missing from palace | Fewer messages than `--min-messages` (default 3)| Lower threshold or `--session <id>` explicitly |
| "Error finding id" on search after mining | Stale HNSW index | `mempalace repair --yes` + `mempalace_reconnect` |
| Drawers doubled for a project | Someone ran raw `mempalace mine` alongside wrapper, or renamed wing mid-flight | Inspect `embedding_metadata` in `chroma.sqlite3`, purge duplicates by source prefix, then `mempalace repair` |
| Post-mine ChromaDB search returns stale results in MCP | MCP server caches old index | Call `mempalace_reconnect` from MCP |
| Opencode DB not at default path | Non-standard `XDG_DATA_HOME` or opencode config | `export OPENCODE_DB=/custom/path/opencode.db` or `--db` |
## What to File Under Which Wing
| Content type | Wing (convention) | Room | Tool |
| ----------------------------------- | ------------------------------ | ---------------- | ----------------------- |
| Opencode session transcripts | `wing_conversations` | auto (keyword) | `mempalace-session` |
| Project docs (md, yaml, Dockerfile) | `wing_<project-name>` | auto | `mempalace-docs` |
| Per-agent session diaries | `wing_<agent-name>` | `diary` | `mempalace_diary_write` (from the consumer-side `mempalace` skill) |
| Ad-hoc verbatim facts | any | any | `mempalace_add_drawer` |
## Cost Profile (reference)
From a 10-day opencode corpus (140 sessions / 1491 msgs / 4656 parts):
- Dry run: seconds.
- Full mine: ~21 min wall / ~38 min user CPU → 2378 drawers from 62 qualifying sessions.
- Dedup re-run: mine instant, repair ~5 min.
Budget **~20 minutes per 60-session batch**. Scales roughly linearly with message count.
## Anti-Patterns
- **Don't run `mempalace mine` directly on a project.** Use `mempalace-docs` — otherwise source code floods the palace.
- **Don't try to point `mempalace mine --mode convos` at `opencode.db` directly.** The convos miner reads files (txt/md/json/jsonl) only — no SQLite support. Use `mempalace-session` to export first.
- **Don't delete staging dirs unnecessarily.** They're dedup anchors; deleting means a forced re-mine of everything in that wing.
- **Don't forget `mempalace_reconnect`** after a mine from inside a live MCP session — otherwise search hits the stale index.
- **Don't mine with `--min-messages 0` or `1`** — 78 out of 140 sessions in reference corpus were throwaway `/exit`'d sessions that would flood the palace with noise. Default 3 is sensible.
## Upstream Roadmap (when to retire these wrappers)
- **[MemPalace PR #1213](https://github.com/MemPalace/mempalace/pull/1213)** merges → `mempalace-docs` becomes redundant (exclude patterns in `mempalace.yaml`). Retire to thin shim or delete.
- **Opencode session-stopping hooks merge** ([PR #16598](https://github.com/anomalyco/opencode/pull/16598) et al.) **AND** `hooks_cli.py` gains `opencode` harness → live auto-save works; `mempalace-session` becomes a manual-only backfill tool (cron / historic import).
- **SQLite mode lands in `mempalace mine --mode convos`** → `mempalace-session` loses its reason to exist entirely.
Check `ARCHITECTURE.md` §6 in `cli_utils/` for current upstream status before doing any retirement work.
## See Also
- `<cli_utils>/ARCHITECTURE.md` — **canonical spec** (diagrams, implementation notes, full troubleshooting).
- `<cli_utils>/README.md` — per-tool usage reference.
- `~/.agents/skills/mempalace/SKILL.md` — consumer-side skill (search, diary, KG) — *pair this skill with that one*.