joakimp/mempalace-toolkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
mempalace-toolkit/AGENTS.md
T
joakimp 90e70fff61 docs: add Ecosystem diagram to README + update harness-extension conventions post-split 
Two small doc updates consolidating the session's architectural arc:

1. README.md gets a new 'Ecosystem' section (after the contents list,
   before 'Why this exists') showing the five-repo composition:
     myconfigs -> opencode-toolkit + pi-toolkit -> mempalace-toolkit
   Plus an ownership table clarifying which scope lives where. The
   diagram makes the opt-out pattern visible \u2014 opencode-devbox's slim
   container path skips mempalace-toolkit and still gets a functional
   stack.

2. AGENTS.md 'Adding a new harness extension' section was still written
   for the pre-split extensions/pi/ which had keybindings.json +
   settings.example.json + pi-env.zsh. Rewrote to reflect:
   - Bridge-only scope (harness-generic config goes in <harness>-toolkit).
   - 'Probe for the sibling toolkit' step replaces the old symlink-keybindings
     and template-settings steps.
   - Worked example now points at install_pi_extension + check_pi_toolkit
     rather than the four functions that moved out.
   - Explicitly names the pattern: opencode-toolkit + pi-toolkit as
     the two existing examples of the sibling-toolkit convention.
2026-05-05 17:50:13 +02:00

177 lines
11 KiB
Markdown
Raw Permalink Blame History

# AGENTS.md
## What this is
Producer-side tooling for [MemPalace](https://github.com/MemPalace/mempalace). Three thin wrappers in `bin/` (opencode + pi session feeders, a docs miner), a companion agent skill, and an `extensions/` tree with per-harness bridges (currently `pi/`). Pairs with the consumer-side `mempalace` skill.
Read [`ARCHITECTURE.md`](ARCHITECTURE.md) first — it's the canonical spec for what this repo does and why.
## Structure
```
install.sh # Idempotent installer — see "What install.sh does" below
ARCHITECTURE.md # Canonical spec: diagrams, setup recipe, ops notes, upstream roadmap
README.md # Human-facing quickstart + per-tool usage reference
SKILL.md # Agent skill (symlinked into ~/.agents/skills/ on install)
bin/
mempalace-docs # Docs-only MemPalace miner (bash wrapper)
mempalace-session # Opencode session → MemPalace bridge (bash + inline Python)
mempalace-pi-session # pi session → MemPalace bridge (bash + inline Python)
contrib/ # systemd / launchd / cron templates for scheduling feeders
extensions/
pi/ # pi↔mempalace MCP bridge (bridge-only; pi's own config is in the pi-toolkit repo)
mempalace.ts # Symlinked into ~/.pi/agent/extensions/ (MCP <→ pi glue)
README.md # Bridge internals, Type.Unsafe gotcha, pi+mempalace deploy recipe
```
## What `install.sh` does
Idempotent, safe to re-run. Always:
- Symlinks `bin/*` into `~/.local/bin/`.
- Creates `~/.agents/skills/opencode-mempalace-bridge/` with a `SKILL.md` symlink and a `.skill-source` marker.
Gated on pi being installed (`~/.pi/agent/extensions/` exists):
- Symlinks `extensions/pi/mempalace.ts` into `~/.pi/agent/extensions/`. Backs up any real file in the way.
Probes (never halt, `warn` + `return 0`):
- `~/.local/bin` is on `$PATH`.
- `~/.config/opencode/instructions/mempalace.md` exists (opencode wake-up protocol).
- `mempalace` is registered as an MCP server in `~/.config/opencode/opencode.json`.
- If pi is installed: pi-toolkit artifacts (`~/.pi/agent/keybindings.json` symlink, `~/.oh-my-zsh/custom/pi-env.zsh`) exist. Warns with a `git clone ssh://...pi-toolkit.git` pointer if missing.
All non-destructive: if something is already in place and points into this repo, prints "already linked" and moves on. If a non-symlink real file is in the way, backs it up with a timestamp.
**Not handled here any more** (split to [`pi-toolkit`](https://gitea.jordbo.se/joakimp/pi-toolkit) on 2026-05-05):
- `keybindings.json` symlink into `~/.pi/agent/`
- `pi-env.zsh` cp into `~/.oh-my-zsh/custom/`
- `settings.example.json` template + `check_pi_settings` probe
- `check_aws_env` probe
Those are pi-generic concerns. This toolkit installs **only** the pi↔mempalace MCP bridge on top of whatever pi-toolkit set up.
## Conventions
- **Standalone executables** in `bin/` with `#!/usr/bin/env bash` shebang, no extension, `chmod +x`. Must work in non-interactive contexts (agent processes, cron, CI).
- **Thin wrappers only.** Neither tool reimplements the mempalace miner. Both follow the **stage-to-cache-then-mine** idiom: curate input to `~/.cache/…/<wing>/`, then delegate to `mempalace mine`.
- **Idempotent + dry-runnable.** Every tool supports `--dry-run`. Second invocation on unchanged input is a no-op (dedup via `source_file` path, optionally + `mtime`).
- **No external Python deps.** Stdlib only (`sqlite3`, `json`, `pathlib`). Inline in the bash wrapper via heredoc.
- Argument parsing: `--help`/`-h` first, then mode flags, then positional args.
- Comment sections use `# ── Section Name ──────` style (matches sibling `cli_utils` repo).
## Adding a new wrapper
Three wrappers live happily as standalone scripts — no shared helper library yet, because each one's stage-to-cache logic differs enough that the common surface is thin (arg parsing + `mempalace mine` invocation). A fourth wrapper might tip the balance; re-evaluate then. Until then, copy the pattern from `mempalace-session` (richest example):
1. Create `bin/<name>` with `#!/usr/bin/env bash` + `chmod +x`.
2. Implement `--help`, `--dry-run`, `--repair` flags (repair is opt-in; `--no-repair` kept as deprecated alias).
3. Stage to `~/.cache/<name>/<wing>/` with deterministic filenames.
4. Invoke `mempalace mine ...` (choose `--mode convos` if input is chat-like).
5. Do NOT end with `mempalace repair` unless `--repair` was explicitly passed. Repair is a destructive in-place HNSW rebuild and must never run on an unattended schedule.
6. Update `README.md` with usage + rationale.
7. Update `install.sh`? No — `bin/*` is auto-linked.
8. Update `ARCHITECTURE.md` if the wrapper fills a new architectural gap.
9. Update `SKILL.md` if agents should know when to invoke it.
## Adding a new harness extension
`extensions/<harness>/` is the home for **bridges** — code that lives
inside an agent runtime (pi, claude-code, kiro, …) and talks to the
mempalace MCP server. Currently only `extensions/pi/` exists. If you
add a second one (e.g. `extensions/claude-code/`), follow the same
shape:
1. **One directory per harness.** Never mix harnesses in one dir.
2. **Bridge-only scope.** This toolkit owns the mempalace-side wiring;
harness-generic config (keybindings, env loaders, settings templates)
belongs in a sibling `<harness>-toolkit` repo, following the pattern
established by [`pi-toolkit`](https://gitea.jordbo.se/joakimp/pi-toolkit)
and [`opencode-toolkit`](https://gitea.jordbo.se/joakimp/opencode-toolkit).
That boundary is load-bearing for `opencode-devbox`'s slim container
path (mempalace opt-out, ~300 MB saved).
3. **A `README.md`** covering: what the bridge does, harness-specific
install path, debug knobs, and any gotchas (e.g. the pi `Type.Unsafe`
schema passthrough). May also hold the "Deploying <harness> with
mempalace" recipe since that straddles the two repos.
4. **Gate `install.sh` steps on the harness being present.** Detect via
a well-known path (pi uses `~/.pi/agent/extensions/`). Skip silently
on machines without that harness. Never force-install.
5. **Symlink the bridge code.** `mempalace.ts` (or equivalent) gets
symlinked into the harness's extensions directory so edits flow
through git. Back up any pre-existing real file to
`<path>.bak.YYYYMMDD-HHMMSS` before linking.
6. **Probe for the sibling toolkit.** After installing the bridge, check
whether the harness's own base config is in place (e.g. for pi-toolkit:
`~/.pi/agent/keybindings.json` symlink, `~/.oh-my-zsh/custom/pi-env.zsh`).
Warn with a `git clone` pointer if missing. `warn` + `return 0`, never
halt.
7. **Mirror in `--uninstall`.** Every symlink this repo creates must have
a matching removal step guarded by `link_if_into_repo`. Do **not**
touch sibling-toolkit-owned files — point the user at
`<harness>-toolkit/install.sh --uninstall` instead.
8. **Update the root `README.md`** — repo-contents list + Ecosystem
diagram's "Who owns what" table + Setup section's deploy summary.
9. **Update this file's Structure block** to list the new
`extensions/<harness>/` contents.
See `extensions/pi/README.md` and the `install_pi_extension` +
`check_pi_toolkit` functions in `install.sh` for a worked example.
## Testing
Manual only. Integration-shaped:
```bash
# Smoke test — does it parse args and list what would happen?
./bin/mempalace-session --help
./bin/mempalace-session --dry-run
# Real test on a single session (safe, deterministic)
./bin/mempalace-session --session ses_<id> --dry-run
./bin/mempalace-session --session ses_<id> # file into palace
mempalace_search "a phrase from that session" # verify visibility
./bin/mempalace-session --session ses_<id> # re-run → should skip
```
For `mempalace-docs`, test on a small repo (e.g. this one) first:
```bash
./bin/mempalace-docs "$PWD" --dry-run
```
## Gotchas
- `install.sh` is idempotent but interactive — use `--yes` in non-interactive contexts.
- `~/.local/bin` must be on `$PATH`. The installer warns if not.
- The companion skill lives at `~/.agents/skills/opencode-mempalace-bridge/SKILL.md` and is a **symlink into this repo**. Editing that file edits `SKILL.md` here. To propagate to Claude Code / Kiro, run `agents-sync` from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils).
- The opencode DB path defaults to `~/.local/share/opencode/opencode.db`. Override via `$OPENCODE_DB` or `--db`.
- The mempalace miner **skips symlinks** (as of v3.3.3 — `miner.py` line ~828). That's why the wrappers use `cp -p` / explicit file writes for staging, not symlinks.
- The convos miner dedups on `source_file` path only (no mtime check). Staging filenames must be stable per session; deleting a staged JSONL forces a re-mine.
- The docs miner dedups on `source_file` path + `mtime`. That's why staging uses `cp -p` (preserves mtime).
## Colocated skill pattern
This repo owns an agent skill (`SKILL.md`) that lives alongside the code it documents, rather than in a central skills repo like [`skillset`](https://gitea.jordbo.se/joakimp/skillset). The advantages: the skill moves in lockstep with the wrappers it explains, one `git clone` gets you the full producer-side setup, and retirement (when upstream gaps close) removes skill + code + docs in one commit.
The convention for making this coexist cleanly with sibling tooling:
1. **`install.sh` creates `~/.agents/skills/<name>/` as a real directory** containing a `SKILL.md` symlink back into this repo. It does **not** create a dir-symlink, because real dirs are the signal that sibling reconcilers (skillset's `deploy-skills.sh`, cli_utils's `agents-sync.zsh`) should leave the dir alone.
2. **`install.sh` drops a `.skill-source` marker file** at the root of the skill dir:
```
# skill-source: mempalace-toolkit
# repo: <absolute path>
# url: ssh://git@gitea.jordbo.se:2222/joakimp/mempalace-toolkit.git
```
This is a breadcrumb for humans and future tooling — it answers "who owns this skill dir?" at a glance. `deploy-skills.sh` and `agents-sync.zsh` don't read it today (their existing logic already handles external dirs correctly) but may surface it in status reports later.
3. **`install.sh --uninstall` removes the marker** (only if it still says `mempalace-toolkit`) and the now-empty skill dir.
If you add a third colocated skill from a new repo, follow the same convention. The marker format is shared; only the repo name changes.
## History
Split out from [`cli_utils`](https://gitea.jordbo.se/joakimp/cli_utils) on 2026-04-30. The wrappers originated there but the conceptual fit was weak (`cli_utils` is interactive shell tools; these are agent memory infrastructure). Some older diary entries and KG facts in the palace reference the original paths.
Reference in New Issue View Git Blame Copy Permalink