joakimp
6352373a1f
The three feeder wrappers (mempalace-docs, mempalace-pi-session,
mempalace-session) unconditionally ran 'mempalace repair --yes' after
mining, controllable only via --no-repair opt-out. The contrib launchd
and systemd templates did not pass --no-repair, so every scheduled tick
invoked the destructive in-place HNSW rebuild.
This has bitten us twice:
- 2026-05-04 09:08: a kickstart triggered repair while an MCP
subprocess held the DB open; the live collection was wiped (0
drawers) and had to be restored from the palace.backup snapshot.
- 2026-05-05 10:00: post-mine repair crashed mid-rebuild with
'NotFoundError: Collection [<uuid>] does not exist' - chromadb's
rebuild recreated the collection under a new UUID while the code
still held the old handle. Live DB survived only by luck (crash
hit before the swap).
Fix: flip the default.
- New flag: --repair (opt-in). Prints a warning and sleeps 3s before
invoking 'mempalace repair --yes'.
- --no-repair is retained as a deprecated no-op alias for backward
compatibility with any scripts/units still passing it.
- Default behavior: no repair. Routine ChromaDB add() keeps HNSW
consistent; repair is a recovery op, not a maintenance tick.
Docs updated to match: README, SKILL, ARCHITECTURE, AGENTS,
contrib/README. Scheduling guidance now explicitly warns against
enabling --repair on cron/launchd/systemd-timer runs.
98 lines
6.1 KiB
Markdown
98 lines
6.1 KiB
Markdown
# AGENTS.md
|
|
|
|
## What this is
|
|
|
|
Producer-side tooling for [MemPalace](https://github.com/MemPalace/mempalace). Two thin wrappers in `bin/` plus the companion agent skill. Pairs with the consumer-side `mempalace` skill.
|
|
|
|
Read [`ARCHITECTURE.md`](ARCHITECTURE.md) first — it's the canonical spec for what this repo does and why.
|
|
|
|
## Structure
|
|
|
|
```
|
|
install.sh # Idempotent installer — symlinks bin/* into ~/.local/bin
|
|
# and SKILL.md into ~/.agents/skills/opencode-mempalace-bridge/
|
|
ARCHITECTURE.md # Canonical spec: diagrams, setup recipe, ops notes, upstream roadmap
|
|
README.md # Human-facing quickstart + per-tool usage reference
|
|
SKILL.md # Agent skill (symlinked into ~/.agents/skills/ on install)
|
|
bin/
|
|
mempalace-docs # Docs-only MemPalace miner (bash wrapper)
|
|
mempalace-session # Opencode session → MemPalace bridge (bash + inline Python)
|
|
```
|
|
|
|
## Conventions
|
|
|
|
- **Standalone executables** in `bin/` with `#!/usr/bin/env bash` shebang, no extension, `chmod +x`. Must work in non-interactive contexts (agent processes, cron, CI).
|
|
- **Thin wrappers only.** Neither tool reimplements the mempalace miner. Both follow the **stage-to-cache-then-mine** idiom: curate input to `~/.cache/…/<wing>/`, then delegate to `mempalace mine`.
|
|
- **Idempotent + dry-runnable.** Every tool supports `--dry-run`. Second invocation on unchanged input is a no-op (dedup via `source_file` path, optionally + `mtime`).
|
|
- **No external Python deps.** Stdlib only (`sqlite3`, `json`, `pathlib`). Inline in the bash wrapper via heredoc.
|
|
- Argument parsing: `--help`/`-h` first, then mode flags, then positional args.
|
|
- Comment sections use `# ── Section Name ──────` style (matches sibling `cli_utils` repo).
|
|
|
|
## Adding a new wrapper
|
|
|
|
A third wrapper would justify factoring a shared helper library. Until then, copy the pattern from `mempalace-session` (richest example):
|
|
|
|
1. Create `bin/<name>` with `#!/usr/bin/env bash` + `chmod +x`.
|
|
2. Implement `--help`, `--dry-run`, `--repair` flags (repair is opt-in; `--no-repair` kept as deprecated alias).
|
|
3. Stage to `~/.cache/<name>/<wing>/` with deterministic filenames.
|
|
4. Invoke `mempalace mine ...` (choose `--mode convos` if input is chat-like).
|
|
5. Do NOT end with `mempalace repair` unless `--repair` was explicitly passed. Repair is a destructive in-place HNSW rebuild and must never run on an unattended schedule.
|
|
6. Update `README.md` with usage + rationale.
|
|
7. Update `install.sh`? No — `bin/*` is auto-linked.
|
|
8. Update `ARCHITECTURE.md` if the wrapper fills a new architectural gap.
|
|
9. Update `SKILL.md` if agents should know when to invoke it.
|
|
|
|
## Testing
|
|
|
|
Manual only. Integration-shaped:
|
|
|
|
```bash
|
|
# Smoke test — does it parse args and list what would happen?
|
|
./bin/mempalace-session --help
|
|
./bin/mempalace-session --dry-run
|
|
|
|
# Real test on a single session (safe, deterministic)
|
|
./bin/mempalace-session --session ses_<id> --dry-run
|
|
./bin/mempalace-session --session ses_<id> # file into palace
|
|
mempalace_search "a phrase from that session" # verify visibility
|
|
./bin/mempalace-session --session ses_<id> # re-run → should skip
|
|
```
|
|
|
|
For `mempalace-docs`, test on a small repo (e.g. this one) first:
|
|
|
|
```bash
|
|
./bin/mempalace-docs "$PWD" --dry-run
|
|
```
|
|
|
|
## Gotchas
|
|
|
|
- `install.sh` is idempotent but interactive — use `--yes` in non-interactive contexts.
|
|
- `~/.local/bin` must be on `$PATH`. The installer warns if not.
|
|
- The companion skill lives at `~/.agents/skills/opencode-mempalace-bridge/SKILL.md` and is a **symlink into this repo**. Editing that file edits `SKILL.md` here. To propagate to Claude Code / Kiro, run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils).
|
|
- The opencode DB path defaults to `~/.local/share/opencode/opencode.db`. Override via `$OPENCODE_DB` or `--db`.
|
|
- The mempalace miner **skips symlinks** (as of v3.3.3 — `miner.py` line ~828). That's why the wrappers use `cp -p` / explicit file writes for staging, not symlinks.
|
|
- The convos miner dedups on `source_file` path only (no mtime check). Staging filenames must be stable per session; deleting a staged JSONL forces a re-mine.
|
|
- The docs miner dedups on `source_file` path + `mtime`. That's why staging uses `cp -p` (preserves mtime).
|
|
|
|
## Colocated skill pattern
|
|
|
|
This repo owns an agent skill (`SKILL.md`) that lives alongside the code it documents, rather than in a central skills repo like [`skillset`](https://gitea.jordbo.se/joakimp/skillset). The advantages: the skill moves in lockstep with the wrappers it explains, one `git clone` gets you the full producer-side setup, and retirement (when upstream gaps close) removes skill + code + docs in one commit.
|
|
|
|
The convention for making this coexist cleanly with sibling tooling:
|
|
|
|
1. **`install.sh` creates `~/.agents/skills/<name>/` as a real directory** containing a `SKILL.md` symlink back into this repo. It does **not** create a dir-symlink, because real dirs are the signal that sibling reconcilers (skillset's `deploy-skills.sh`, cli_utils's `agents-sync.zsh`) should leave the dir alone.
|
|
2. **`install.sh` drops a `.skill-source` marker file** at the root of the skill dir:
|
|
```
|
|
# skill-source: mempalace-toolkit
|
|
# repo: <absolute path>
|
|
# url: ssh://git@gitea.jordbo.se:2222/joakimp/mempalace-toolkit.git
|
|
```
|
|
This is a breadcrumb for humans and future tooling — it answers "who owns this skill dir?" at a glance. `deploy-skills.sh` and `agents-sync.zsh` don't read it today (their existing logic already handles external dirs correctly) but may surface it in status reports later.
|
|
3. **`install.sh --uninstall` removes the marker** (only if it still says `mempalace-toolkit`) and the now-empty skill dir.
|
|
|
|
If you add a third colocated skill from a new repo, follow the same convention. The marker format is shared; only the repo name changes.
|
|
|
|
## History
|
|
|
|
Split out from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils) on 2026-04-30. The wrappers originated there but the conceptual fit was weak (`cli_utils` is interactive shell tools; these are agent memory infrastructure). Some older diary entries and KG facts in the palace reference the original paths.
|