6352373a1f2b397d225e13dbb0113d11aeee8b17
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 joakimp
|
6352373a1f |
fix(feeders): make post-mine repair opt-in, not default
The three feeder wrappers (mempalace-docs, mempalace-pi-session,
mempalace-session) unconditionally ran 'mempalace repair --yes' after
mining, controllable only via --no-repair opt-out. The contrib launchd
and systemd templates did not pass --no-repair, so every scheduled tick
invoked the destructive in-place HNSW rebuild.
This has bitten us twice:
- 2026-05-04 09:08: a kickstart triggered repair while an MCP
subprocess held the DB open; the live collection was wiped (0
drawers) and had to be restored from the palace.backup snapshot.
- 2026-05-05 10:00: post-mine repair crashed mid-rebuild with
'NotFoundError: Collection [<uuid>] does not exist' - chromadb's
rebuild recreated the collection under a new UUID while the code
still held the old handle. Live DB survived only by luck (crash
hit before the swap).
Fix: flip the default.
- New flag: --repair (opt-in). Prints a warning and sleeps 3s before
invoking 'mempalace repair --yes'.
- --no-repair is retained as a deprecated no-op alias for backward
compatibility with any scripts/units still passing it.
- Default behavior: no repair. Routine ChromaDB add() keeps HNSW
consistent; repair is a recovery op, not a maintenance tick.
Docs updated to match: README, SKILL, ARCHITECTURE, AGENTS,
contrib/README. Scheduling guidance now explicitly warns against
enabling --repair on cron/launchd/systemd-timer runs.
|
||
 Joakim Persson
|
2f703a8ebc |
Docs: explain diary vs session mine (why keep both)
Automated session mining could plausibly lead a user (or a future agent) to conclude that writing diary entries is redundant — mining captures every turn, so why also write a compressed summary at wind-down? That conclusion is wrong, and it's worth explaining why in the docs so both disciplines survive. ARCHITECTURE.md §5 gets a new subsection 'Diary vs session mine: why keep both?' that presents this as a first-class concept: - Comparison table — content, granularity, compression, authorship, signal density, retrieval pattern, and the question each answers. - The defining property of a diary entry: editorial judgment by the author. Captures meta-observations that were never said aloud during the session (lessons, patterns, pending items, aggregate counts). Mining raw turns can never surface these because the words don't exist verbatim. - Three practical scenarios where the distinction bites: wake-up token economics, 'what did we decide' vs 'what did we say', redundancy covering each other's failure modes. - Practical implications: don't skip either habit, let them specialize (diary = release notes; mine = git log). README.md gets a brief teaser in the 'First mine' area with a link to the canonical ARCHITECTURE.md section — enough for a skim reader to decide they want to keep writing diaries, and for a deep reader to know where to go for the full explanation. SKILL.md replaces the three-line 'Relationship to the mempalace skill' note with a compact version of the comparison table and a direct call-out of the 'session mining means I don't need diaries' misconception agents fall into. Points agents at ARCHITECTURE.md §5 for the full treatment when users ask the question. Cross-references verified: anchor slug for the new section is #diary-vs-session-mine-why-keep-both (standard slug rules: colon, spaces, punctuation removed/hyphenated). Both linking docs use the matching fragment. |
||
|
Joakim Persson
|
349a3a3d3d |
mempalace-session: make --dry-run dedup-aware
A --dry-run report showed all qualifying sessions without indicating
which would actually hit the palace on a real run. On a second run
against an already-mined corpus this was misleading — output said
'Exported 62 session(s)' but the real mine step would skip all 62.
The wrapper now queries the palace's chroma.sqlite3 (read-only, via
file:...?mode=ro URI) for source_file values under the staging dir,
then tags each exported session as [NEW] or [SKIP] during listing and
reports the split in the summary:
Exported 62 session(s) to ~/.cache/mempalace-session/wing_conversations
0 new → will be filed on mine
62 already filed → will be skipped (dedup by source_file)
--dry-run: no new sessions to mine. A real run would skip all 62.
Implementation notes:
- Classification is best-effort. If the palace is unreachable (fresh
install, moved, permission-denied, file missing) the wrapper falls
back to treating all exports as NEW — the real mine step still
delegates dedup to 'mempalace mine --mode convos' which is the
authoritative source of truth. Getting the classification wrong
in --dry-run is cosmetic; behaviour of a real run is unchanged.
- Palace path respects $MEMPALACE_PATH env var for non-default setups.
- Same classification also shown on a real (non-dry-run) mine so users
see upfront how much of the export set is actually new before the
miner runs.
Verified both directions:
- All-already-filed case (current box, 62 sessions in palace): reports
0 new, 62 skipped. --dry-run message correctly says 'would skip all'.
- Partial case (simulated by deleting one session's metadata from
palace): reports 1 new, 61 skipped. --dry-run message correctly
says 'would file 1 new'. Palace was restored from backup
immediately after the test.
README and SKILL.md both updated with the new dedup-aware output and
a direct answer to the FAQ 'will it mine the same sessions again?'
|
||
|
Joakim Persson
|
72e7019101 |
Fix two docs/UX errors found during Mac install (tor-ms22)
1. 'mempalace init --yes' without a dir argument fails — 'dir' is
required. The semantics were wrong too: 'mempalace init' is
per-project (sets up mempalace.yaml + entity detection in a specific
directory), not a one-time global init. The palace itself is
created lazily on first write, so neither mempalace-session nor
mempalace-docs requires any init step.
Removed the misleading 'One-time palace init' block from README.md,
ARCHITECTURE.md, and SKILL.md. Added a clarifying note:
'mempalace init <dir>' is per-project and optional (needed only to
customize the wing name or entity detection before mempalace-docs).
2. install.sh's 'Skipping <name>: <dest> exists and is not our symlink'
warning gave no actionable guidance. On the Mac, a leftover
~/.local/bin/mempalace-docs (likely from the pre-split cli_utils
days) was blocking the new install and the user had no easy way
to know what to do about it.
Expanded the warning to:
- Show whether the blocker is a symlink (and what it points at) or
a real file.
- Print the exact 'rm && ./install.sh' fix line.
- Track skipped count separately and flag it in the closing
summary so a scrolling user doesn't miss it.
Added matching troubleshooting paragraph to the README 'Install
mempalace-toolkit' section explaining the skip behaviour and
pointing at the installer's own message for the fix.
Smoke-tested the new skip-warning code path by temporarily replacing
~/.local/bin/mempalace-docs with a foreign symlink and re-running
install.sh — output is clear, specific, and restores cleanly.
|
||
|
Joakim Persson
|
25972b7499 |
README: document uv-based mempalace install + MCP wrapper pitfall
mempalace-toolkit's Prerequisites section assumed mempalace was already
installed but didn't explain how. The upstream mempalace repo only
shows pip install, which fights PEP 668 on modern distros and leaks
dependencies into system site-packages. The production pattern used
in opencode-devbox (uv tool install) is cleaner but wasn't documented
here.
Adds a full 'Installing mempalace itself (prerequisite)' section with
five subsections:
1. Why uv over pip — isolated venv, no PEP 668 fight, shim makes
the CLI accessible from any bash/zsh terminal without manual
venv activation.
2. Personal machine — with default paths
(shim in ~/.local/bin, venv under ~/.local/share/uv/tools/). Simple
one-liner plus PATH guidance. This is the recommended default.
3. System-wide / container install — the opencode-devbox pattern:
UV_TOOL_DIR=/opt/uv-tools + UV_TOOL_BIN_DIR=/usr/local/bin, with
the exact Dockerfile RUN step used in production (including the
python -c build-time sanity check). Cross-references
opencode-devbox/Dockerfile for the full canonical version.
4. MCP server wrapper — explains the 'missing venv when the container
was deployed' pitfall from the first opencode-devbox attempt:
with a non-default UV_TOOL_DIR, system python3 can't import
mempalace, so MCP configs of the form
["python3", "-m", "mempalace.mcp_server"]
fail silently with ModuleNotFoundError. Fix is a thin wrapper on
PATH that exec's the venv's own python. Shows the exact 3-line
shell wrapper from opencode-devbox/rootfs/usr/local/bin/
mempalace-mcp-server. Points at opencode-devbox/AGENTS.md
'Critical conventions' as the authoritative reference.
5. Verification checklist — /usr/local/bin/mempalace, MemPalace 3.3.3,
and a minimal
=======================================================
MemPalace Status — 4943 drawers
=======================================================
WING: cli_utils
ROOM: scripts 38 drawers
ROOM: fzf 25 drawers
ROOM: general 1 drawers
WING: opencode_devbox
ROOM: general 203 drawers
ROOM: configuration 3 drawers
WING: proxmox
ROOM: general 1046 drawers
WING: skillset
ROOM: general 1118 drawers
WING: wing_conversations
ROOM: technical 1775 drawers
ROOM: architecture 513 drawers
ROOM: planning 164 drawers
ROOM: problems 42 drawers
ROOM: general 6 drawers
ROOM: decisions 3 drawers
WING: wing_orchestrator
ROOM: diary 6 drawers
======================================================= smoke test that catches venv
mismatches by failing with a Python traceback instead of a clean
error message.
Renames the existing 'Install' section to 'Install mempalace-toolkit'
to disambiguate from the new mempalace install section — the toolkit's
own install.sh still works the same, just labeled more precisely.
ARCHITECTURE.md §4 prerequisites paragraph and SKILL.md prerequisites
block both cross-reference the new section with anchor links, so any
entry point into the docs leads the reader to the right recipe.
|
||
|
Joakim Persson
|
720245e010 |
Add macOS launchd template, bringing automation parity to macOS
Ship a launchd user agent plist alongside the existing systemd and
cron templates so macOS users can schedule mempalace-session without
falling back to cron. launchd is the macOS-native equivalent of a
systemd user timer: same scheduling model, same log conventions, same
single-instance guarantees.
- contrib/launchd/se.jordbo.mempalace-session.plist:
- Label uses reverse-DNS from the jordbo.se domain for consistency
with other user-installed launchd jobs; fork the prefix if reusing
this template in a different org.
- ProgramArguments points at /Users/USER/.local/bin/mempalace-session
(USER is substituted at install time, same pattern as
contrib/cron/).
- EnvironmentVariables.PATH covers ~/.local/bin, Apple Silicon
Homebrew, Intel Homebrew, and system defaults — launchd agents
get a minimal PATH by default and the wrapper needs to find
mempalace + python3.
- StartCalendarInterval matches systemd unit's schedule: Monday
03:00 local.
- RunAtLoad=false — load shouldn't trigger a run; schedule does.
- ProcessType=Background + LowPriorityIO=true + Nice=10 mirror
the systemd unit's Nice=10 + IOSchedulingClass=idle. macOS's
automatic App Nap and resource throttling for Background jobs
yields to interactive work cleanly.
- ExitTimeOut=7200 matches systemd's TimeoutStartSec=7200.
- StandardOut/ErrorPath under ~/Library/Logs/ so Console.app
surfaces them.
- contrib/README.md gains a full launchd section:
- Caveat table comparing to systemd (Persistent=true isn't quite
matched; RandomizedDelaySec has no equivalent; overlap prevention
is automatic).
- Install recipe using launchctl bootstrap (modern) with a fallback
note for legacy launchctl load -w on older macOS.
- Verify section shows launchctl list, launchctl print, log tails,
and launchctl kickstart for manual testing.
- Uninstall via launchctl bootout.
- Chooser table updated: macOS now explicitly points at launchd,
not cron.
- ARCHITECTURE.md §5, SKILL.md Quick automation pitch, and README.md
Keeping it fresh section all updated to mention the three scheduler
options and give per-platform quick-starts.
Plist XML validated with plistlib.
|
||
|
Joakim Persson
|
36845e14b2 |
Document the operational routine + ship automation templates
Until opencode session-stopping hooks land upstream, mempalace-session
is the entire mechanism that gets opencode conversations into the
palace — skip it and session history stays trapped in a local SQLite
DB, invisible to semantic search. Previous docs covered setup well
but were thin on when and how often to run it.
- ARCHITECTURE.md §5: replace the one-line 'When to re-mine' note with
a full Operational Routine section — triggers, cadence, relationship
to the session lifecycle, automation pointers, verification.
- SKILL.md: add an Operational Routine section aimed at agents —
when to suggest invoking the tool, cadence guidance, how to
distinguish this producer-side tool from the consumer-side
mempalace skill's in-session habits.
- README.md: add 'Keeping it fresh' subsection pointing at contrib/
and the full docs.
contrib/ ships three ready-to-use templates:
- systemd/mempalace-session.{service,timer} — user units with weekly
Mon 03:00 schedule, Persistent=true catch-up, RandomizedDelaySec for
fleet-wide jitter, ConditionPathExists guard for opencode-less boxes,
Nice+IOSchedulingClass=idle so it never fights interactive work.
- cron/mempalace-session.cron — sample crontab entry with log
redirection and clear USER-substitution instructions.
- README.md with install/verify/uninstall recipes for both, a chooser
table (systemd vs cron), container/devbox caveats, and tuning notes
(daily vs weekly vs monthly trade-offs).
The user's LATER-list item 'wrap mempalace-session in cron/systemd
timer for true auto-save coverage' is now actionable: a single
systemctl --user enable --now command stands it up.
|
||
|
Joakim Persson
|
3554f56bcc |
Update SKILL.md references after cli_utils split
The initial split copied SKILL.md verbatim from its pre-split location where it still referenced paths in cli_utils. Update all 10 stale references to point at mempalace-toolkit instead — canonical path, clone URL, bind-mount path, container-recreate recovery command, and See also links. |
||
|
Joakim Persson
|
954c3f2ebb |
Initial commit — split out from cli_utils
Producer-side MemPalace tooling: two bash wrappers that bridge opencode session history and project documentation into the palace. Originally developed in cli_utils (2026-04-28); split into its own repo on 2026-04-30 because the conceptual fit was weak — cli_utils is interactive shell tooling, while this is agent memory infrastructure with its own architecture, dependency surface, and growth trajectory. Contents: - bin/mempalace-docs — docs-only mining wrapper (originally a2ddcc9 in cli_utils), bridges the gap until MemPalace PR #1213 (exclude_patterns) merges upstream. - bin/mempalace-session — opencode → palace session bridge (originally dacca0e in cli_utils). Reads ~/.local/share/opencode/opencode.db, exports each session to Claude Code JSONL, mines via 'mempalace mine --mode convos'. Bridges the gap until opencode session-stopping hooks + an opencode harness in hooks_cli.py land upstream. - ARCHITECTURE.md — canonical spec: architecture diagram, component details, setup recipe, operational notes, upstream-retirement roadmap. Originally a4cf314 in cli_utils. - SKILL.md — companion agent skill (producer side). Pairs with the consumer-side mempalace skill. Symlinked into ~/.agents/skills/opencode-mempalace-bridge/ by install.sh. - install.sh — idempotent installer, also handles --uninstall. - AGENTS.md — repo conventions. History of the individual files is not preserved in this split; see cli_utils (gitea.jordbo.se/joakimp/cli_utils) commits a2ddcc9, dacca0e, and a4cf314 for the original authorship context. |