feat(skills): bake pi-extensions + mempalace fallback skills

The pi-toolkit global AGENTS.md tells every pi session to read ~/.agents/skills/pi-extensions/SKILL.md at start (the fork/recall under-utilisation fix), but that skill lived only in the private skillset repo — so the pointer dangled in any container started without skillset mounted. Bake fallbacks so the pointer always resolves. - pi-extensions (Option 1 + Option 2, layered): * Canonical skill promoted to the public pi-extensions package repo under skill/ (separate commit there); co-located with the code it documents. * rootfs/ carries a committed snapshot (the floor). * Dockerfile.variant copies /opt/pi-extensions/skill/ over the snapshot after the pinned clone, so a normal build ships the fresh package copy (recorded via PI_EXTENSIONS_REF) and an old-ref/mirror build still ships the snapshot. Helper evaluate-extension-usage.py travels with it. - mempalace (Option 2 only): snapshot in rootfs/. Its consumer skill has no public package home (mempalace-toolkit ships a different skill, opencode-mempalace-bridge), so no build-time refresh. - entrypoint links both (only-when-absent; mounted skillset still wins). - smoke-test: build-time presence + package-match check + runtime symlink assertions; readiness gate now waits on the last-linked skill. - docs: skills/VENDORED.md (provenance + refresh), README, AGENTS.md, CHANGELOG [Unreleased]. Note: shipped in the NEXT release; v1.2.0 (run 409) predates this.
2026-06-23 15:32:04 +02:00
parent d619a6e2ec
commit a7d6a7d235
9 changed files with 848 additions and 4 deletions
@@ -0,0 +1,301 @@
+---
+name: mempalace
+description: MemPalace agent memory protocol. Use on every session to maintain continuity across conversations — search before answering about past work, write diary entries before session ends, and mine new projects into the palace. Load this skill at session start.
+---
+
+# MemPalace Agent Memory Protocol
+
+## Overview
+
+MemPalace gives you persistent memory across sessions via an MCP server. It stores project knowledge (mined from files), conversation summaries (diary entries), and entity relationships (knowledge graph). Without this protocol, you have tools but no habits — and memory without habits is just storage.
+
+**Core principle:** Storage is not memory. Storage + protocol = memory.
+
+## When to Load This Skill
+
+- At the **start of every session** (proactively, before the user asks)
+- When the user mentions **past conversations, decisions, or work**
+- When working on a **new project or repository** for the first time
+- When the user asks about **people, projects, or relationships**
+
+## Session Lifecycle
+
+### Phase 1: Wake Up (session start)
+
+Run these immediately when a session begins, before responding to the user:
+
+1. **Load palace overview:**
+   ```
+   mempalace_status
+   ```
+   This returns wing/room counts, the AAAK spec, and the memory protocol reminder.
+
+2. **Read your recent diary:**
+   ```
+   mempalace_diary_read(agent_name="<your_agent_name>", last_n=5)
+   ```
+   Scan for context about recent sessions — what was worked on, what matters, what's pending.
+
+3. **Check the knowledge graph** for the user or active project if relevant:
+   ```
+   mempalace_kg_query(entity="<project_or_person>")
+   ```
+
+Do NOT announce this to the user. Just do it silently to orient yourself.
+
+### Phase 2: Active Session (during work)
+
+#### Search Before You Speak
+
+Before answering questions about past work, decisions, people, or projects:
+
+```
+mempalace_search(query="<keywords>", wing="<project>")
+```
+
+**Never guess about facts that might be in the palace.** Wrong is worse than slow. Say "let me check" and query.
+
+#### Mine New Projects
+
+When working on a new codebase for the first time:
+
+1. Check if it's already mined:
+   ```
+   mempalace_list_wings
+   ```
+
+2. **Decide what to mine — docs first, code never (by default).**
+
+   The palace is for *context and intent*, not code recall. Code is better read from the working tree via `Read`/`Grep`/`glob` — always authoritative, never stale. Embedding source code produces thousands of low-signal drawers (e.g. `def __init__(self, ...)` across every class) that pollute search for years.
+
+   **Mine by default:**
+   - `*.md`, `*.rst`, `*.txt` — docs, READMEs, CHANGELOGs, architecture notes
+   - `AGENTS.md`, `CLAUDE.md`, `CONTRIBUTING.md`, design/decision docs — highest signal per byte
+   - `*.sh`, `Dockerfile`, `Makefile`, entrypoints — small, intent-bearing
+   - `*.yml`, `*.yaml`, `*.toml`, selective `*.json` (`docker-compose`, `pyproject`, `mkdocs.yml`, CI workflows) — skip lockfiles
+
+   **Do NOT mine by default:**
+   - `*.py`, `*.ts`, `*.tsx`, `*.js`, `*.go`, `*.rs`, `*.java`, `*.cpp`, `*.c`, `*.rb` — raw source code
+   - Test files, fixtures, generated code
+   - `node_modules/`, `.venv/`, `__pycache__/`, `.mypy_cache/`, `.pytest_cache/`, `.ruff_cache/` (the miner respects `.gitignore` but double-check)
+
+   Exception: if a code file *is* the documentation (e.g. a heavily-commented reference script, or a protocol definition), file it manually via `mempalace_add_drawer`.
+
+3. **Before mining**, inspect the repo to estimate drawer count:
+   ```bash
+   # Quick audit — what will actually get mined?
+   find <dir> -type f \
+     -not -path '*/.git/*' -not -path '*/node_modules/*' \
+     -not -path '*/.venv/*' -not -path '*/__pycache__/*' \
+     \( -name '*.md' -o -name '*.sh' -o -name '*.yml' -o -name '*.yaml' \
+        -o -name '*.toml' -o -name 'Dockerfile*' -o -name 'Makefile' \) | wc -l
+   ```
+   A docs-heavy repo should produce ~5–10 drawers per file. If a mine produces >15 drawers/file on average, code leaked in — investigate.
+
+4. Run the mine:
+   ```bash
+   mempalace init --yes <directory>
+   mempalace mine <directory> --agent <your_agent_name>
+   ```
+
+   The miner currently lacks a `--docs-only` or `--exclude-ext` flag (as of v3.3.3). Until it does, either:
+   - (a) Add a `mempalace.yaml` at the repo root with explicit include globs, OR
+   - (b) Mine everything, then surgically remove code-sourced drawers via SQL on `~/.mempalace/palace/chroma.sqlite3` (delete by `embedding_metadata.source_file LIKE '%.py'`), followed by `mempalace repair --yes`.
+
+5. If the CLI miner misses a file you *do* want (e.g., `.zsh`, an undocumented extension), file it manually:
+   ```
+   mempalace_add_drawer(wing="<project>", room="<aspect>", content="<verbatim content>", source_file="<path>")
+   ```
+
+6. After mining, reconnect to pick up the new embeddings:
+   ```
+   mempalace_reconnect
+   ```
+   If search errors occur after mining ("Error finding id"), repair the index:
+   ```bash
+   mempalace repair --yes
+   ```
+
+#### Track Facts in the Knowledge Graph
+
+When you learn new facts about people, projects, or relationships:
+
+```
+mempalace_kg_add(subject="ProjectX", predicate="uses", object="PostgreSQL")
+mempalace_kg_add(subject="Alice", predicate="owns", object="ProjectX", valid_from="2026-01-15")
+```
+
+When facts change (ended, no longer true):
+
+```
+mempalace_kg_invalidate(subject="Alice", predicate="works_at", object="OldCorp", ended="2026-03-01")
+```
+
+#### Cross-Reference with Tunnels
+
+When content in one project relates to another, create a tunnel:
+
+```
+mempalace_create_tunnel(
+  source_wing="project_api", source_room="endpoints",
+  target_wing="project_db", target_room="schema",
+  label="API endpoints map to these DB tables"
+)
+```
+
+#### Feeding opencode session history (opencode + mempalace-toolkit only)
+
+MemPalace has no upstream integration with [opencode](https://github.com/anomalyco/opencode) as of v3.3.3 — `hooks_cli.py` only supports `claude-code` and `codex` harnesses. Opencode persists every turn in a local SQLite DB at `~/.local/share/opencode/opencode.db`, but nothing moves that data into the palace automatically.
+
+On a machine with opencode + the [`mempalace-toolkit`](https://gitea.jordbo.se/joakimp/mempalace-toolkit) installed, session history is fed into `wing_conversations` via `mempalace-session` — either manually, or on a weekly systemd user timer / cron schedule shipped in `mempalace-toolkit/contrib/`. If this is missing, opencode conversations exist only in the local SQLite DB and are invisible to `mempalace_search`.
+
+**How to tell if it's set up:**
+
+```
+mempalace_list_wings
+```
+
+If `wing_conversations` exists and has a drawer count comparable to the user's opencode session count, session feeding is working. If it's empty or suspiciously small, suggest:
+
+1. Check if the toolkit is installed: `which mempalace-session`.
+2. If installed, suggest running `mempalace-session --dry-run` to preview and `mempalace-session` to file.
+3. If not installed, point the user at `gitea.jordbo.se/joakimp/mempalace-toolkit` for setup.
+
+**Don't try to paper over the gap by dumping turn-level content into the palace manually via `mempalace_add_drawer`** — that reinvents what `mempalace-session` does with normalization and dedup. Use the tool.
+
+Full routine (triggers, cadence, automation) is in the [`opencode-mempalace-bridge`](https://gitea.jordbo.se/joakimp/mempalace-toolkit) skill and the toolkit's `ARCHITECTURE.md` §5. The two skills pair: this one (`mempalace`) covers using the palace; that one (`opencode-mempalace-bridge`) covers feeding it from opencode.
+
+### Phase 3: Wind Down (session end)
+
+**Always write a diary entry before the session ends.** This is the most important habit.
+
+```
+mempalace_diary_write(
+  agent_name="<your_agent_name>",
+  entry="<AAAK compressed summary>",
+  topic="session-summary"
+)
+```
+
+#### Why still write diaries when sessions may be mined automatically?
+
+On machines running opencode + `mempalace-toolkit`, every session is mined into `wing_conversations` on a weekly (or user-defined) schedule. A common and incorrect conclusion: *"since every turn is captured automatically, writing a diary entry is redundant."* It isn't.
+
+Session mining captures **what was said** (every turn, verbatim). A diary captures **what the session meant** — editorial judgment by the agent who lived it:
+
+- Lessons learned, patterns noticed, pending items rolled forward
+- Meta-observations that were never said aloud during the session
+- Aggregate counts (commits shipped, bugs fixed, hours spent)
+- A compressed, recency-scannable summary for the *next* agent's wake-up
+
+Mining raw turns cannot surface these because the words don't exist verbatim — they're the agent's reflection at wind-down. Think of the split as *release notes* (diary) vs. *git log with diffs* (session mine): a repo keeps both because they answer different questions. So does the palace.
+
+**Practical rule:** automated mining does not replace Phase 3. Both systems cover each other's failure modes — a skipped diary is recovered from the raw turns; a missed mine is recovered from the diary summary. For the full treatment (comparison table, retrieval patterns, token economics), see [`mempalace-toolkit/ARCHITECTURE.md` §5 → "Diary vs session mine: why keep both?"](https://gitea.jordbo.se/joakimp/mempalace-toolkit/src/branch/main/ARCHITECTURE.md#diary-vs-session-mine-why-keep-both).
+
+#### AAAK Diary Format
+
+Write diary entries in compressed AAAK format for efficiency. Structure:
+
+```
+SESSION:<date>|<what.you.worked.on>|
+TASKS:
+1.<task.description>→<outcome>|
+2.<task.description>→<outcome>|
+DISCOVERED:<unexpected.findings>|
+ENTITIES:<people.or.projects.encountered>|
+<importance: one to five stars>
+```
+
+Example:
+
+```
+SESSION:2026-04-28|api.refactor+db.migration|
+TASKS:
+1.refactored.auth.endpoints→split.into.3.modules|
+2.added.user.roles.migration→postgres.enum.type|
+DISCOVERED:legacy.session.table.unused.since.v2|
+ENTITIES:ProjectX;Alice(reviewer)|
+***
+```
+
+Rules:
+- Use dots instead of spaces within phrases
+- Use pipes as field separators
+- Use arrows for cause/effect or transitions
+- Stars indicate session importance (one to five)
+- Keep it tight — a future agent should get the gist in seconds
+
+#### What to Capture
+
+Prioritize recording:
+- **Decisions made** and their rationale
+- **Discoveries** — things that surprised you or that a future session needs to know
+- **Unfinished work** — what's pending, what was deferred
+- **User preferences** observed during the session
+- **Entities encountered** — people, projects, tools, services
+
+### Phase 4: Fact Updates
+
+If facts changed during the session, update the knowledge graph before writing the diary:
+
+```
+mempalace_kg_invalidate(subject="...", predicate="...", object="...", ended="<today>")
+mempalace_kg_add(subject="...", predicate="...", object="...", valid_from="<today>")
+```
+
+## Palace Structure
+
+### Wings
+
+Wings are top-level categories, typically one per project or domain:
+- Named after the project directory (e.g., `cli_utils`, `opencode_devbox`)
+- Agent diaries live in `wing_<agent_name>` (e.g., `wing_orchestrator`, `wing_pi`)
+
+#### Multi-harness palace
+
+A single palace can be fed by multiple coding-agent harnesses. On this machine the palace is shared between **opencode** and **pi** (Mario Zechner's pi-coding-agent). Implications:
+
+- **`wing_conversations` mixes sources.** Both harnesses' session feeders write into the same wing. To tell them apart, look at the `source_file` metadata on each drawer:
+  - `pi_<uuid>.jsonl` → pi session
+  - `<slug>_ses_<id>.jsonl` → opencode session
+  - The first chunk of each session also carries a `| source: opencode` or `| source: pi` marker in the synthetic header line.
+- **Other wings may belong to other harnesses.** For example `wing_pi` is pi's diary, not opencode's. Don't assume every diary entry was written by you — check `agent_name` on the entry.
+- **Session feeders run on different schedules.** Pi sessions are fed Tue 03:00, opencode sessions Mon 03:00. Recent sessions from either harness can lag the palace by up to a week, so absence-of-evidence in `wing_conversations` is not evidence-of-absence for recent work.
+- **Reading another harness's diary is useful.** When orienting after a gap, `mempalace_diary_read agent_name=pi` (or whichever sibling agent has been active) often gives a fresher picture than waiting for the conversations feeder to catch up.
+
+### Rooms
+
+Rooms are aspects within a wing:
+- `fzf`, `scripts`, `configuration`, `general` — whatever the miner detects
+- Diary entries go into rooms by topic tag
+
+### Drawers
+
+Drawers hold verbatim content — never summarized, always searchable.
+
+### Tunnels
+
+Cross-wing connections linking related content across projects.
+
+### Knowledge Graph
+
+Entity-relationship triples with temporal validity. Query with `mempalace_kg_query`, browse with `mempalace_kg_timeline`.
+
+## Troubleshooting
+
+| Problem | Fix |
+|---|---|
+| "No palace found" | Run `mempalace init <dir>` then `mempalace mine <dir>` |
+| "Error finding id" after mining | Run `mempalace repair --yes` then `mempalace_reconnect` |
+| Search returns irrelevant results | Use `max_distance=1.0` for stricter matching; add `wing` filter |
+| Miner skips file types | File manually with `mempalace_add_drawer` or use `--no-gitignore` |
+| Stale results after external changes | Call `mempalace_reconnect` |
+
+## Anti-Patterns
+
+- **Don't guess when you can search.** If a question touches past work, search first.
+- **Don't skip the diary.** A session without a diary entry is a session forgotten.
+- **Don't summarize drawer content.** File verbatim — the embedding model needs the original words.
+- **Don't mine .git directories or node_modules.** The CLI miner respects .gitignore by default.
+- **Don't create duplicate drawers.** Use `mempalace_check_duplicate` before adding manually.
+- **Don't treat the palace as a task list.** It's for knowledge and context, not todos.