2e73a9eae8e6a0c106af27e3d8a97cd5f4e88f15
46 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joakim Persson
|
2e73a9eae8 |
docs: REMOTE_URL includes /mcp, and the entrypoint skips silently where the feeder exits 1
Two details needed before the first client flip. 1. MEMPALACE_REMOTE_URL is the full endpoint including /mcp with no trailing slash. The feeder POSTs to it verbatim (bin/mempalace-pi-session:718), and the server matches `path != "/mcp"` exactly, so a base URL or a trailing slash both 404 -- and a 404 here looks like a routing/proxy fault, not a config typo, which is a bad hour to spend. Matches the existing pi-devbox examples. For this fleet: https://mempalace.jordbo.se/mcp 2. The 3.7 trap has TWO symptoms, not one, and I had only documented the loud one. Direct runs / session-end / cron exit 1 with a clear error (:298-300). But pi-devbox's entrypoint-user.sh:134 checks the same condition and skips *quietly* -- and the skip happens before the subshell that writes ~/.pi/agent/mempalace-catchup.log, so there is not even an empty log to find. A container flipped with REMOTE_URL but no SSH_TARGET therefore contributes nothing to the palace and leaves no artifact explaining why. The skip is correct in itself (there is genuinely no inbox to ship to) but it is indistinguishable from a healthy run with nothing to do. Documented with two commands that tell those apart after a flip. |
||
|
Joakim Persson
|
4cb70ce3e1 |
docs: record the 302 auth-redirect fingerprint and the server's exact HTTP surface
Phase 1 exposure now verified end to end (2026-08-12): /healthz -> ok and an unauthenticated POST /mcp -> 401, both from a client container and from the primary itself. 3.4 and 3.6 marked passed. Two findings from the failure in between, worth more than a checkbox: 1. Leaving Pangolin's resource authentication on does NOT surface as 401 or 403. It is a 302 with `location: https://pangolin.jordbo.se/auth/resource/<uuid> ?redirect=...`, sent *before* the request is proxied -- so the palace never sees it and its journal stays silent, `curl -s` prints an empty body, and an MCP client gets non-JSON. That is 1.1's "Pangolin's HTTP auth is browser-shaped; the clients are not" arriving as a concrete symptom rather than an argument. Now recorded with the exact header shape and the two curl flags that reveal it (-D-, -w '%{redirect_url}'), plus the reading: a 302 is good news, because DNS, TLS and routing all worked and only auth intervened. 2. Read the server's routing to settle whether path-scoped proxy rules are sufficient. The entire HTTP surface is two exact paths (mcp_server.py:5299- 5318): GET /healthz (no token, Host/Origin gated) and POST /mcp (Bearer, compare_digest on the exact string). Everything else is a 404 from the palace itself. So path-scoped rules are tighter than a host-wide proxy and lose nothing. Two client-facing consequences: /mcp is matched exactly, so a trailing slash 404s -- configure clients without one; and there is no GET /mcp, no SSE, no session id, no DELETE, so this is plain JSON-RPC over POST, not MCP streamable-HTTP. A strict client opening with a GET handshake sees 404. Also means the verify step needs no initialize and no Accept: text/event-stream, which the old snippet left ambiguous. Also: run the 200 and 401 from two different networks, not one -- passing from only the primary leaves split-horizon DNS untested. |
||
|
Joakim Persson
|
08e344b047 |
docs: proxy target is http:// not https://; loopback probe refuses, it does not 403
Both corrections come from the first real Phase 1 start on synlig (2026-08-12).
1. The Pangolin resource target was documented as bare `172.17.0.1:8765` with no
scheme, and the obvious guess from that is `https://` -- which cannot work.
`contrib/systemd/mempalace-serve.service` runs `serve --host 172.17.0.1
--port 8765` with no --tls-cert, so the primary speaks plaintext HTTP; TLS
terminates at Pangolin, which is the whole point of the RFC 6.2 decision.
Point a proxy at https:// and it attempts a TLS handshake against a plaintext
listener: 502 from outside, while `curl 172.17.0.1:8765/healthz` on the box
still says ok -- a confusing pair of symptoms. Now spelled `http://` with the
failure mode named, in the runbook and in the unit's comments.
2. `curl -s 127.0.0.1:8765/healthz` was documented as "expect 403". Wrong: the
real run returns empty. With a docker0-only bind nothing is listening on
loopback, so the connection is refused at TCP level before any header is sent
(%{http_code} -> 000, exit 7). The 403 is the *loopback-bind* case verified
2026-08-10 -- server on 127.0.0.1 answering a proxy-forwarded foreign Host.
Two distinct behaviours had been collapsed into one expectation in three
places (both runbooks and the unit).
Worth stating why the correction matters rather than just fixing it: refusal
is the *stronger* signal. A 403 proves only that a request was rejected; a
refused connection proves the loopback and LAN surface is not listening at
all. Someone who expected 403, saw silence, and "fixed" it by rebinding to
0.0.0.0 would have converted a correct configuration into an exposed one.
The docs now also say what to do if it hangs, or if ss shows 0.0.0.0:8765.
|
||
|
Joakim Persson
|
00a95d1a2f |
docs: why the tunnel and the feeder's SSH path are not redundant; newt done
Asked "why do we need Pangolin if you proposed rsync/ssh?", and the runbook did
not actually answer it -- it stated both were needed without saying why neither
substitutes. New section 1.3:
- Pangolin/HTTPS carries the MCP tool surface (search, add_drawer,
diary_write, kg_*) -- every live tool call, from any MCP client.
- SSH/rsync carries transcript *files* only, because mempalace_mine expands
its source path server-side, so the server can only mine its own disk.
HTTPS alone is a palace you can query but cannot feed; SSH alone is files with
no query API. The rsync is not a transport preference, it is a workaround for
where `mine` resolves paths.
Records honestly that `ssh -L 8765:172.17.0.1:8765 synlig` *would* replace the
tunnel for MCP, and why we don't: synlig dials out (reaching for a dial-out
tunnel is itself the evidence inbound was unavailable), MCP clients want a
durable URL rather than a per-session forward, and the forward must be up on
every device before every session.
And the design's weak point, stated instead of glossed: the rsync runs
client -> synlig, so mining needs synlig's SSH reachable *from the client*. Were
that true everywhere, no tunnel would be needed for MCP either. Honest
expectation after Phase 1 is therefore: query/write from anywhere, mine only
from devices that can reach synlig's SSH. Section 4 now carries the upstream ask
that would close the gap -- have the feeder send content over MCP (add_drawer /
diary_write, which it already calls) instead of asking the server to mine a path
it must first rsync there.
New section 3.7, a live trap for the imminent client flip:
MEMPALACE_REMOTE_URL on its own does not degrade to local feeding, it *stops*
feeding. auto mode switches to remote as soon as the URL is set (:286) and
remote mode then exits 1 without MEMPALACE_PI_SSH_TARGET (:298-300), before
anything is staged or filed -- so a cron feeder just starts failing, and the
loudest symptom is silence. Two safe orders given: set all three variables in
one edit, or set URL+token and pin --mode local until the SSH target exists.
newt is installed on synlig and connected to Pangolin (done 2026-08-12), marked
here and in the synlig runbook's item 2; the blocker is now item 3, the one
sudo. Added the follow-up that "connected to Pangolin" only proves newt reached
nyvaken -- reaching the *palace* is a separate claim that fails independently,
so probe 172.17.0.1:8765/healthz from inside newt's namespace.
All seven code citations verified against the source at commit time.
|
||
|
Joakim Persson
|
29e660e18f |
feeders: stage beside the palace, not in ~/.cache; document Phase 1 exposure
Staging default moves out of ~/.cache to <palace-root>/pi-stage (pi) and <palace-root>/opencode-stage (opencode), resolved with mempalace's own palace-path precedence ($MEMPALACE_PALACE_PATH -> $MEMPAL_PALACE_PATH -> ~/.mempalace/config.json -> ~/.mempalace/palace), then dirname. Why: the convos miner keys dedup on the *staged* path, so a wiped stage plus a sync scoped to include it prunes the drawers mined from those sources -- deleting memories, not a cache. Under ~/.cache that state was reachable by anything treating a cache as disposable. Staging inside the palace makes the coupling structural: the stage cannot be wiped without touching the palace itself. Overrides ($MEMPALACE_PI_STAGE / $MEMPALACE_SESSION_STAGE, --stage) are unchanged. Note the old default had never been created on any host, so this closed a latent hazard, not a live one. Measured, and the docs now claim only this much: sync prunes only within the scope it is given -- wing-only, 1299 scanned / 1299 out of scope / 0 removed; scoped at the palace root, 651 kept / 648 out of scope. The previous blanket "sync prunes every drawer" wording overstated it, which is a liability: the next reader disproves the overstatement and discards the real constraint with it. Also in this change: - cron log dir ~/.cache/mempalace-session -> ~/.cache/mempalace-logs. The stage left that namespace, so the old name now read as "the stage". - AGENTS.md: the convos miner *does* check mtime (verified against upstream convo_miner.py); the previous "no mtime check" claim was wrong. - smoke-test assertions use `mktemp -d` for --sessions-dir. One pointed at /tmp, which still held earlier synthetic transcripts, so a --dry-run exported a fake session into the real stage: --dry-run skips the mine, not the export. docs/phase-1-exposure-runbook.md -- the newt/DNS/auth step that RFC 001 and the synlig runbook leave open (runbook section 4, items 2 and 5). Port 8765 at /mcp, newt targets 172.17.0.1, and the authentication is the single shared bearer token (RFC 6.2, decided 2026-08-09) rather than per-device proxy users. The latter cannot work today: mempalace validates exactly one token, and Pangolin's SSO/PIN/password are browser-shaped while every client here is a headless JSON-RPC POST -- enabling that protection breaks the clients it protects. The per-device axis that *does* exist is the feeder's SSH key + per-device inbox. New finding recorded there: a loopback bind does not merely 403 behind a tunnel (already known, runbook 2.4) -- it also silently starts the server with no token at all, because auto-minting is gated on the bind being non-loopback. extensions/pi/README.md: the HTTP transport IS authenticated as of mempalace 3.6.0; the "sessionless and unauthenticated" note dated from the v1.3.0 era. Closes the RFC section 8 Phase-0 hygiene item. |
||
|
Joakim Persson
|
3626946013 |
Phase 0 on synlig: install, palace layout, verified Host/Origin policy
synlig is greenfield — no mempalace, no ~/.mempalace at all — so Phase 0 became "provision correctly from birth" rather than "migrate carefully". Nothing is serving; no client config was touched. Done: - mempalace 3.6.0 installed via uv, pinned to the fleet version (the id recipes and idempotency probes this RFC leans on are version-specific). - Embedder pre-warmed. This was the real unknown: the first embed pulls a 79.3 MB ONNX model from the chroma CDN, and an egress-filtered work VM would have failed at the worst moment — the first client write. Pulled at ~20 MB/s, no proxy interference. Done in a throwaway palace so the real one never saw it. - Palace at the stock default ~/.mempalace/palace, so no config file and no MEMPALACE_PALACE_PATH is needed on synlig at all. Two corrections to the RFC, both from provisioning rather than reading: - §7.1 was understated. DEFAULT_PALACE_PATH (~/.mempalace/palace) and DEFAULT_KG_PATH (~/.mempalace/knowledge_graph.sqlite3) already differ with stock defaults, so the KG split is out-of-the-box behaviour, not a consequence of a custom --palace, and it is permanent rather than one-time: serve always passes --palace, any CLI call without it uses the HOME path. A one-time mv does not fix that, it only picks which of the two files gets populated. Fixed instead by converging both rules on one inode via relative symlinks, and verified the load-bearing assumption: a dangling symlink is created on connect, -wal/-shm land next to the target (so the palace dir stays a self-contained backup unit, which is the part that mattered), cross-path read works, same inode. hallways.json deliberately left alone — already palace-derived, HOME path is a warning-only probe. - §6.2 upgraded from "test early" to verified: 11/11 as predicted. The headline is that the safe-sounding reflex is the failure mode — loopback bind + proxy forwarding a public Host is 403, non-loopback is 200. Also confirmed Origin is never relaxed on either bind, and /healthz is Host/Origin-gated but token-free, so it works as the tunnel probe. Recommends binding the docker0 gateway over 0.0.0.0: non-loopback so the pin relaxes, but reachable only from the host and its containers. New docs/synlig-primary-runbook.md carries the discovered facts about the box, the evidence tables, an explicit "deliberately not done" list, and tomorrow's ordered steps. New contrib/systemd/mempalace-serve.service carries the bind rationale inline so nobody "fixes" it back to loopback; staged on synlig with a .staged suffix so systemd cannot pick it up by accident. Flagged for tomorrow: synlig has no newt/tunnel client (docker ps shows only the Gitea runner and digikam), so Pangolin on nyvaken cannot reach it until one is added — easy to miss, because Pangolin will look healthy from its own side. |
||
|
Joakim Persson
|
7e51055c96 |
rfc-001: join protocol, diary non-idempotency, deployment decisions
Second planning round. Three findings from verification, and the decisions that were blocking phasing. Verified in mempalace 3.6.0 and written up: - NEW §7.6 — diary_write has no idempotency guard at all. The id embeds datetime.now() at microsecond resolution and the write is a bare col.add (mcp_server.py:3546) with no col.get probe, in direct contrast to add_drawer's probe 900 lines earlier (:2593-2600). So §3's "log the intent, replay the intent" is false for diaries — replay duplicates. This lands on the critical path because diaries are `replicated` and are exactly what a join replays. - NEW §4.4 — join/bootstrap protocol, which the RFC simply lacked. Not "seed the primary from one palace": every container joins the same way, repeatedly, so a join is idempotent replay and the only question per record type is what dedupes it. Drawers and open KG facts need zero client bookkeeping; closed KG facts and diaries need client-side keys. Join state belongs in the palace dir, not the container (~/.mempalace is not preserved by default), which also makes two containers sharing one host's palace the easy case rather than a double-upload hazard. - §3.2 — the add_triple guard is scoped to `valid_to IS NULL`, so closed historical facts are unguarded. §3 read as unconditionally idempotent. - §7.3 — drawer/diary metadata is exhaustive: no session, PID or conversation field. Two concurrent pi sessions are indistinguishable, and pi-vs-opencode is only accidentally distinguishable because the pi extension never sets added_by (it sets identity for diaries only, mempalace.ts:758). Corrects the §7.3.3 bullet that claimed multi-harness attribution was already solved — the field is the right home, but nothing populates it. Design rule stated: device + agent, never session; provenance granularity equals token granularity. - §6.2 — Host-pinning is coupled to the bind address (enforce_host_pin = _http_is_loopback(host), :5367). The trap is the RFC's own "bind loopback" reflex: behind a proxy that forwards a public Host, a loopback bind 403s, while a non-loopback bind deliberately relaxes the pin. The Origin check is never relaxed. This replaces the warning I first wrote, which had it backwards. - §9.3 — largely resolved: the last-chunk-only probe is deliberate (batched upsert is all-or-nothing, :2586-2592). Only a crash mid-upsert remains untested. - §4.1 — fix shape for opencode's stale-config asymmetry: make the mcp.mempalace subtree env-authoritative, with a fingerprint so hand-edits still win. pi-devbox/entrypoint-user.sh:131-162 already has the jq deep-merge pattern to port; no OPENCODE_CONFIG* layer exists upstream, so the real file must be written. Now Phase 1.5. Decisions (new §8.1): primary = synlig; TLS at Pangolin; single shared token for now (so origin_device stays advisory); diaries replicated. Work/personal is per-wing, not per-device — two primaries split along machine lines is rejected, because pi-devbox/opencode-devbox are simultaneously work and home projects and the device where work happened cannot classify the project. MEMPALACE_REMOTE_URL stays scalar so multi-store remains additive later (new §9.7 keeps the placement question open). |
||
|
Joakim Persson
|
fdcd5871de |
rfc-001: resolve open questions 1 and 6 — both permissive
Recon before planning the implementation, and both blockers dissolved. Q1, does opencode support remote MCP: yes. Its published schema defines McpRemoteConfig (type/url/headers/oauth) as a sibling of McpLocalConfig, with headers as a free string→string map. The RFC had been treating "every sampled config in myconfigs is type:local" as evidence about the schema when it was only evidence about deployments. Consequence: the edge proxy is not the only option for opencode, so nothing in the phasing hangs on this. What opencode still lacks is offline/local-first, which is the honest Phase 2 argument. Q6, how opencode-devbox learns the opt-in: it already does. generate-config.py, run from entrypoint-user.sh:117, registers mempalace as remote+bearer when MEMPALACE_REMOTE_URL is set and local stdio otherwise, and its comment says it deliberately mirrors the mempalace.ts env contract. The image also ships mempalace by default, so R5's premise was wrong and is corrected. Phase 2 there is a third branch in an existing script, not a new mechanism. One new constraint found while confirming this, and it is the sharpest edge in the whole opt-in story: generate-config.py never overwrites an existing config and ~/.config/opencode is a named volume, so flipping the .env on a container that already has a config is a no-op — it only writes an opencode.jsonc.proposed sidecar. pi re-reads env every start; opencode does not. Documented in §4.1 and §9.6, and it applies symmetrically to R6 reversibility. |
||
|
Joakim Persson
|
052dbb8038 |
docs(rfc-001): provenance belongs to the sync boundary, not the agent
Reverses the previous §7.3 on review. It said "stamp added_by everywhere, now, because it cannot be backfilled" and was about to become a skill instruction telling agents to do it. Both halves were wrong. Wrong on ownership: provenance answers "which device asserted this?", so only a party that can verify the answer should write it. An agent must shell out to read env, can forget, and will improvise when the values are absent — the worst possible stamper, and its claim is unverifiable by anyone. Under the §4 design every write reaches the primary over an authenticated channel, including offline ones at outbox-flush time, so the primary can stamp the complete set with no client cooperation. Provenance is a property of the sync channel, not of the record's author. §7.3.2 adds the trust ladder; edge-side stamping is demoted to an advisory interim, because 3.6.0's serve takes a single shared bearer token (mcp_server.py:5291-5293) and the package has zero device/origin concept, so authoritative stamping needs the per-device credentials of §6 — Phase 4, not Phase 0. Wrong on backfill: a solitary devbox is a single-origin store by definition, so origin is a property of the whole palace and can be assigned wholesale at import (one --origin-device flag) at the moment it stops being solitary. Bulk attribution is strictly more reliable than per-record stamping since it cannot be partially applied. Per-record provenance is only needed where origins interleave, which is only the primary. Solitary containers therefore stamp nothing and lose nothing — more R1-compliant than the previous draft, which quietly asked users who had opted out to carry metadata for the feature. Multi-harness-on-one-host stays solved by added_by = agent name. Keeps the verified mechanics (fixed metadata schema, argument whitelisting at mcp_server.py:4777 silently dropping unknown fields, the diary agent_name → wing_pi@host trap, kg_add having no provenance slot, added_by absent from search results) and the identity findings (a container cannot discover its host's identity; hostnames are neither unique nor stable; rename splits one device's history in two). §7.3.5 keeps the fail-closed rule for whichever component does stamp. Phase 0 drops its provenance item accordingly, and §6.2 now specifies server-side stamping from the authenticated credential. Also flags in-document that these notes are a poisoning vector: a future agent reading them out of the palace must not conclude it should hand-stamp. |
||
 joakimp
|
661ee20b39 |
docs(rfc-001): require solitary-first operation, opt-in centralization
Adds §1.1 (R1–R6) and §1.2 as a hard constraint on the design rather than a preference. A shared palace is valuable only for the multi-machine / multi-container pattern; for most users of the published pi-devbox and opencode-devbox images it is useless overhead. Evidence: all three sampled opencode-devbox deployments contain zero mempalace references. Requirements: solitary operation stays the default and stays byte-identical (no extra process, no outbox, no network calls); opt-in via docker-compose.yml + .env only, never an image rebuild; credentials only in .env, never in compose, the image, a command line, or a log; no new required services (the primary stays a separate standalone compose project); degrade-not-fail where mempalace is absent; fully reversible. §1.2 extends the convention pi-devbox already ships (.env.example:12-23, "local by default" with commented MEMPALACE_REMOTE_URL/_TOKEN) into a three-state ladder — local stdio (default, unchanged) / direct remote (unchanged) / edge (new, MEMPALACE_EDGE=1) — instead of inventing a new mechanism. Notes that the devbox-palace volume coupling reverses under edge mode: the local palace holds the outbox, so persisting it becomes required rather than irrelevant. Consequence recorded in §4.1: mempalace-edge must be selected at registration time, not left always-in-path to decide by env at runtime, since that would insert a process and a failure mode into every solitary user's setup. pi branches in createClient(); opencode needs its static MCP JSON templated at container start, which is new open question §9.6. Also: adds an R1 acceptance test, marks pi-devbox/.env.example:21 as stale (advertises mempalace-mcp --transport http rather than mempalace serve --token/--tls-cert), and extends the evidence index. |
||
|
joakimp
|
35b1e3d81d |
docs: add RFC 001 — global palace with local fallback (mempalace-edge)
Design for moving from one MemPalace per machine per harness to a single primary palace with per-machine offline fallback, plus the source-verified archaeology behind it. Key decisions recorded: - Sync operations (MCP tool calls), not databases. Embeddings are computed client-side and are not portable across architectures; the KG `triples` table has no UNIQUE(subject,predicate,object,valid_from) and its ids embed datetime.now(), so row copies duplicate facts. Replaying tool calls is idempotent where it matters. - Implement as `mempalace-edge`, a local stdio MCP proxy (child mempalace-mcp + HTTPS to the primary + outbox.sqlite), not as per-harness patches. Needs zero mempalace internals, so it serves pi, opencode and the CLI alike and survives mempalace upgrades. - Merged reads (query both, re-sort, dedupe by drawer id) give fleet-wide recall without replication — which is why Phase 3 (pull replication) is deferred: "own writes plus whatever it can reach" is good enough. - Per-wing replication policy: curated/content-addressed wings replicate, mined code/docs stay local (derived, re-mineable, path-dependent ids). Also documents two silently destructive footguns to avoid during rollout: `--palace` vs MEMPALACE_PALACE_PATH (the KG follows the flag only, so `mempalace serve` can start with a silently empty knowledge graph), and `mempalace sync`, which is gitignore-aware drawer deletion rather than replication and would wipe fleet memory when run from a host lacking the repos. Notes that mempalace 3.6.0's `serve` already ships token auth + TLS, making the "unauthenticated, front it with a proxy" notes elsewhere stale. Includes an evidence index mapping each claim to file:line in mempalace 3.6.0, and four upstream candidates (origin_host provenance, per-wing ACL, mempalace_kg_supersede missing from service.py WRITE_TOOLS, and a sync --refuse-shared guard). |
||
 pi
|
96699f2a17 |
feat(pi-bridge): external MemPalace transport via MEMPALACE_REMOTE_URL
Let the pi<->mempalace bridge connect to a shared MemPalace over HTTP instead of always spawning a local mempalace-mcp: - Extract IMcpClient; rename McpClient -> StdioMcpClient (ctor command, arg-less start()). - Add RemoteMcpClient (vendored from pi-extensions/mcp-loader.ts): streamable-HTTP with AbortController timeouts, protocolVersion pinned 2024-11-05, alive/ensureAlive. mempalace-mcp --transport http is sessionless JSON-RPC today; session/SSE/404 branches retained for a future streamable-HTTP server. - createClient() selects transport from MEMPALACE_REMOTE_URL; MEMPALACE_REMOTE_TOKEN -> Authorization: Bearer. Lifecycle automation (wake-up, /mempalace-diary) unchanged. - scripts/check-mcp-client-sync.sh: drift guard vs canonical mcp-loader.ts. - README: document local-vs-external transport. Typechecks clean (strict); both transports smoke-tested against live mempalace-mcp. |
||
|
pi
|
e12b624cf7 |
feat(pi-ext): self-healing respawn + scoped init timeout for mempalace-mcp
A stall-kill (or any crash) of mempalace-mcp was a permanent latch: available flipped off and stayed off until pi restart. Now the next tool call transparently respawns the server and retries. - ensureAlive(): bounded respawn with capped exponential backoff (MEMPALACE_MCP_MAX_RESPAWNS, default 2; MEMPALACE_MCP_RESPAWN_BACKOFF_MS, default 1000). Respawn budget resets on any successful JSON-RPC response, so a recovered server regains full patience while a persistently-broken one hits the cap and stays down (no hot-loop). - Init timeout default raised 120000 -> 300000 (scoped to init only): a genuine virtiofs cold-open shouldn't be killed mid-progress only to respawn and re-pay the same cost. Per-call timeout stays 60000. - Concurrency hardening: generation counter so a late exit from a killed old process can't tear down a fresh respawn; explicit healthy flag replaces racy proc!=null liveness check. - README: document self-heal, new env vars, and why generous-init + bounded-respawn compose rather than overlap. |
||
|
joakimp
|
a3b8829991 |
feat(pi-ext): per-request timeout + stall-kill for mempalace-mcp
A wedged mempalace-mcp (classically an OrbStack virtiofs cold-open of a large chroma.sqlite3 / HNSW load) left the awaiting JSON-RPC promise pending forever, freezing the pi TUI uninterruptibly: ESC cancels the LLM stream, not a pending tool execute(). The JSON-RPC client now arms a per-request timer. On expiry it rejects the request AND kills the stalled child (SIGTERM->SIGKILL), so pi gets an error instead of hanging; the extension flips available=false so later calls fail fast (restart pi to retry). Per-REQUEST, not per-process: the long-lived server only dies on a genuine stall. Knobs: MEMPALACE_MCP_TIMEOUT_MS (default 60000), MEMPALACE_MCP_INIT_TIMEOUT_MS (default 120000), 0 = disable. This supersedes the planned standalone stdio-watchdog shim: the extension already owns request/response correlation, so a separate framing-reparsing shim is unnecessary. |
||
|
joakimp
|
ce09d25c97 |
Rename to @earendil-works/pi-coding-agent + earendil-works/pi URL
Pi moved to its new home at earendil-works on 2026-05-07 (https://pi.dev/news/2026/5/7/pi-has-a-new-home). Sweep: - extensions/pi/mempalace.ts: 'import type { ExtensionAPI } from "@mariozechner/pi-coding-agent"' -> @earendil-works/pi-coding-agent. - README and extensions/pi/README: github.com/mariozechner/pi-coding-agent URL refs -> github.com/earendil-works/pi. - install.sh: same URL substitution in the user-facing pointer line. Brew install references (`brew install pi-coding-agent`) left as-is: formula still works at 0.73.1, tap update tracked upstream at earendil-works/pi#2755. |
||
|
joakimp
|
90e70fff61 |
docs: add Ecosystem diagram to README + update harness-extension conventions post-split
Two small doc updates consolidating the session's architectural arc:
1. README.md gets a new 'Ecosystem' section (after the contents list,
before 'Why this exists') showing the five-repo composition:
myconfigs -> opencode-toolkit + pi-toolkit -> mempalace-toolkit
Plus an ownership table clarifying which scope lives where. The
diagram makes the opt-out pattern visible \u2014 opencode-devbox's slim
container path skips mempalace-toolkit and still gets a functional
stack.
2. AGENTS.md 'Adding a new harness extension' section was still written
for the pre-split extensions/pi/ which had keybindings.json +
settings.example.json + pi-env.zsh. Rewrote to reflect:
- Bridge-only scope (harness-generic config goes in <harness>-toolkit).
- 'Probe for the sibling toolkit' step replaces the old symlink-keybindings
and template-settings steps.
- Worked example now points at install_pi_extension + check_pi_toolkit
rather than the four functions that moved out.
- Explicitly names the pattern: opencode-toolkit + pi-toolkit as
the two existing examples of the sibling-toolkit convention.
|
||
|
joakimp
|
16915f0e55 |
refactor: split pi-generic config into pi-toolkit repo
Parallel to the opencode-toolkit split earlier today. Pi's own config (keybindings, shell env loader, settings template) moves to a new sibling repo so opencode-devbox's mempalace opt-out can build slim containers that include pi without dragging in chromadb + embedding models (~300 MB). What moved to pi-toolkit (https://gitea.jordbo.se/joakimp/pi-toolkit): - extensions/pi/keybindings.json (mosh/tmux newline fix) - extensions/pi/pi-env.zsh (sources ~/.config/pi/.env) - extensions/pi/settings.example.json (Bedrock template) - install.sh::install_pi_keybindings (symlink step) - install.sh::install_pi_env_loader (cp step + bash fallback) - install.sh::check_pi_settings (probe) - install.sh::check_aws_env (probe) What stays here (this is the pi\u2194mempalace bridge, mempalace-side): - extensions/pi/mempalace.ts (the MCP extension) - install.sh::install_pi_extension (symlink step) - NEW: install.sh::check_pi_toolkit (probe: warns if pi is installed but pi-toolkit's artifacts are missing, with git-clone pointer) install.sh shrank from 520 to 403 lines. Uninstall mirror correctly does NOT touch pi-toolkit-owned files (explicit comment). Docs updated: - extensions/pi/README.md: rewritten as 'pi\u2194MemPalace MCP bridge', recipe becomes 'Deploying pi with mempalace' (pi-toolkit step 3, this repo step 5). - AGENTS.md: Structure block + 'What install.sh does' section reflect the narrower scope and list the four things that moved out. - README.md: repo-contents line + Setup section's deploy summary. Verified on tor-ms22: full install\u2192uninstall\u2192reinstall lifecycle clean. After mempalace-toolkit uninstall, pi-toolkit artifacts (~/.pi/agent/keybindings.json, ~/.oh-my-zsh/custom/pi-env.zsh) remain intact \u2014 correctly untouched. check_pi_toolkit probe fires green when both exist. |
||
|
joakimp
|
3d3a0fb125 |
docs(pi): add opencode-toolkit pointer in deploy recipe step 6
Cross-reference the newly-extracted opencode-toolkit repo, which owns ~/.config/opencode/.env loading. The recipe now distinguishes between registering mempalace in opencode.json (still this repo's concern) and ensuring opencode's own env loader is in place (opencode-toolkit's). |
||
|
joakimp
|
118bd20fec |
feat(extensions/pi): ship pi-env.zsh shell loader
The loader that sources ~/.config/pi/.env into every shell was only
living in the myconfigs tor-ms22 backup \u2014 a fresh machine had nowhere
to get it from except copying by hand. Now canonical here.
- extensions/pi/pi-env.zsh: 20-line POSIX-compatible loader
(set -a; source ~/.config/pi/.env; set +a). Works in bash and zsh.
- install.sh install_pi_env_loader:
* oh-my-zsh detected (~/.oh-my-zsh/custom/ exists)
\u2192 cp into that dir (NOT symlink \u2014 that dir is typically part of
a dotfiles backup, and a symlink to mempalace-toolkit would
break when restored on another host).
\u2192 Idempotent: if target content matches repo, says 'already
installed'. If it differs, leaves user edits alone and points
at diff for manual reconcile.
* No oh-my-zsh \u2192 prints source-this-line snippet for ~/.zshrc or
~/.bashrc (derived from $SHELL). Does NOT auto-edit rc files.
- install.sh uninstall: only removes the copy if content still matches
repo. Local edits preserved.
- Docs:
* extensions/pi/README.md Environment setup section rewritten with
both install paths, step 5 of deploy recipe updated.
* AGENTS.md Structure block lists pi-env.zsh.
* Root README repo-contents line mentions it.
Verified on tor-ms22: install fresh \u2192 uninstall (content match \u2192 remove)
\u2192 reinstall \u2192 zsh -ic loads AWS vars correctly. Also tested bash fallback
path via HOME=/tmp/fake-home SHELL=/bin/bash \u2014 prints right .bashrc snippet.
|
||
|
joakimp
|
5d8f523cb3 |
docs(AGENTS): unstale wrapper count + new 'Adding a new harness extension' section
Two stale lines from the 2-wrapper era (we have three now), plus a missing convention doc for future harness extensions: - 'What this is': two wrappers \u2192 three + extensions/ tree - 'Adding a new wrapper': drop the 'third wrapper triggers helper lib' claim since three coexist fine as standalone scripts; keep the threshold at four, re-evaluate then. - New 'Adding a new harness extension' section codifies the extensions/ pattern so a future claude-code or kiro bridge follows the same shape (per-harness dir, symlink-vs-template rules, gated install steps, probe-don't-halt, uninstall mirror, README + Structure updates). |
||
|
joakimp
|
71c335148a |
docs(pi): full 'new machine' deploy recipe in extensions/pi/README
Consolidates the step-by-step recipe that's been living in diary entries and session chat into the canonical pi bring-up doc. Covers: 0. Prerequisites (zsh+oh-my-zsh, uv, tmux 3.2+, AWS creds) 1. Dotfiles: myconfigs provision (tmux CSI-u, ~/.config/pi/.env, zsh loader) 2. pi install (upstream brew/npm) 3. mempalace CLI (uv tool install) + mempalace-toolkit install.sh 4. pi settings bootstrap (start without --model, region prefix table) 5. AWS env verification (git-crypt unlock gotcha) 6. Opencode MCP registration pointer (if applicable) 7. First run + wake-up injection smoke test + Verification checklist + uninstall Root README.md adds a short summary box in the Setup section pointing at the full recipe, so readers coming in from the front door find the pi path immediately but the details stay with the files they install. Covers: macOS + Linux. Works for homelab / work-macos / any myconfigs profile that ships .config/pi/ + pi-env.zsh. |
||
|
joakimp
|
79e0692dac |
docs: update AGENTS.md structure + ARCHITECTURE.md see-also for pi bring-up
AGENTS.md Structure block was stale (listed only 2 bin/ wrappers, no extensions/, no contrib/). Added full tree + a new 'What install.sh does' section enumerating all steps, gates, and probes so maintainers see at a glance what the installer touches. ARCHITECTURE.md is scoped to the producer side (feeding the palace); pi extension is consumer side, so out of scope for the main body. Added a pointer in the See also section so readers can find extensions/pi/README.md. |
||
|
joakimp
|
75876e5c41 |
fix(install): silence AWS probe when pi settings.json absent
check_aws_env was warning about missing AWS_PROFILE/AWS_REGION even on fresh machines with no settings.json yet \u2014 but at that point we don't know which provider the user will pick, so the warning is noise. check_pi_settings already tells the user to bootstrap settings.json; the AWS probe now stays quiet until it has evidence (amazon-bedrock in settings.json) that AWS creds are actually needed. |
||
|
joakimp
|
854ae41f65 |
feat(extensions/pi): keybindings symlink + settings template + AWS/pi probes
Round out the pi bring-up story so a fresh machine can reach a working pi+mempalace install with just `git clone && ./install.sh`: - extensions/pi/keybindings.json: generic mosh/tmux newline fix (shift+enter, ctrl+j, alt+j). Safe on any machine — not region/account-specific. Symlinked into ~/.pi/agent/. - extensions/pi/settings.example.json: template for `settings.json` so pi can start without --provider/--model. NOT symlinked — pi rewrites settings.json at runtime (lastChangelogVersion bumps), which would dirty the repo. Installer prints the cp + edit hint. - install.sh: new install_pi_keybindings + uninstall mirror; new check_pi_settings probe (warns if settings.json missing); new check_aws_env probe (warns if AWS_PROFILE/AWS_REGION unset and settings.json selects amazon-bedrock). All new steps gated on pi being installed (~/.pi/agent/extensions/ exists). - extensions/pi/README.md: documents keybindings rationale, settings bootstrap, and the recommended ~/.config/pi/.env + ~/.oh-my-zsh/custom/pi-env.zsh env layout (paired with the myconfigs commit 884e329 that split AWS vars out of ~/.config/opencode/.env). Verified on tor-ms22: full install → uninstall → reinstall cycle, new shell loads AWS_PROFILE/AWS_REGION from the new pi-env.zsh hook. Works on macOS and Linux (plain ln -s, POSIX bash). |
||
|
joakimp
|
ef1d022fbc |
feat(extensions): version-control pi mempalace extension + install.sh symlink
The pi coding-agent extension at ~/.pi/agent/extensions/mempalace.ts was living only on tor-ms22, including hand-edited fixes (Type.Unsafe schema-passthrough for MCP tool parameters). One disk wipe away from losing it, and no way to reproduce the install on a new machine. - extensions/pi/mempalace.ts: canonical copy (matches tor-ms22 byte-for-byte) - extensions/pi/README.md: what it does, the schema-passthrough gotcha, debugging knobs - install.sh: new install_pi_extension step — gated on ~/.pi/agent/extensions/ existing, backs up any real file in the way, idempotent re-runs, mirror block in uninstall. Works on macOS and Linux (plain ln -s, readlink -f). - README.md: mention extensions/pi/ in the repo-contents list and in the Setup section Verified on tor-ms22: install (backs up existing real file) → uninstall (removes symlink) → reinstall (clean symlink). Re-runs are no-ops. |
||
|
joakimp
|
6352373a1f |
fix(feeders): make post-mine repair opt-in, not default
The three feeder wrappers (mempalace-docs, mempalace-pi-session,
mempalace-session) unconditionally ran 'mempalace repair --yes' after
mining, controllable only via --no-repair opt-out. The contrib launchd
and systemd templates did not pass --no-repair, so every scheduled tick
invoked the destructive in-place HNSW rebuild.
This has bitten us twice:
- 2026-05-04 09:08: a kickstart triggered repair while an MCP
subprocess held the DB open; the live collection was wiped (0
drawers) and had to be restored from the palace.backup snapshot.
- 2026-05-05 10:00: post-mine repair crashed mid-rebuild with
'NotFoundError: Collection [<uuid>] does not exist' - chromadb's
rebuild recreated the collection under a new UUID while the code
still held the old handle. Live DB survived only by luck (crash
hit before the swap).
Fix: flip the default.
- New flag: --repair (opt-in). Prints a warning and sleeps 3s before
invoking 'mempalace repair --yes'.
- --no-repair is retained as a deprecated no-op alias for backward
compatibility with any scripts/units still passing it.
- Default behavior: no repair. Routine ChromaDB add() keeps HNSW
consistent; repair is a recovery op, not a maintenance tick.
Docs updated to match: README, SKILL, ARCHITECTURE, AGENTS,
contrib/README. Scheduling guidance now explicitly warns against
enabling --repair on cron/launchd/systemd-timer runs.
|
||
|
joakimp
|
53d96adc65 |
docs(contrib): scheduling templates for mempalace-pi-session
Drop-in equivalents of the opencode templates for each scheduler
mechanism:
systemd/mempalace-pi-session.{service,timer}
launchd/se.jordbo.mempalace-pi-session.plist
cron/mempalace-pi-session.cron
Schedule is staggered from the opencode jobs (Mon 03:00 -> Tue 03:00)
so machines running both don't race each other on the post-mine HNSW
repair step. Service unit uses ConditionPathExists=%h/.pi/agent/sessions
to no-op silently on machines that haven't used pi, matching the
opencode template's guard on ~/.local/share/opencode/opencode.db.
contrib/README.md grows a 'Templates at a glance' table so the set is
discoverable without reading the whole doc.
|
||
|
joakimp
|
14d253f929 |
feat(session): tag opencode staging headers with '| source: opencode'
Complement to the mempalace-pi-session feeder: now that a second source mines into wing_conversations, every session's synthetic header carries an explicit source tag so the LLM can discriminate at read time when searches return first-chunk content: [session: <title> | <directory> | <YYYY-MM-DD> | source: opencode] The primary disambiguator in search results remains source_file basename (opencode: '<slug>_<id>.jsonl', pi: 'pi_<uuid>.jsonl'), which is present in every chunk's metadata regardless of where the search hit landed in the session. This header tag is a cosmetic second signal on first-chunk hits. Caveat: existing drawers keep their old header — mempalace mine dedups by source_file path, which didn't change, so old opencode sessions are not re-mined. They are implicitly opencode (the only pre-pi source). |
||
|
joakimp
|
9450a45194 |
feat(pi-session): add mempalace-pi-session feeder for pi coding-agent sessions
Parallel to mempalace-session, this wrapper walks ~/.pi/agent/sessions/
JSONL files and mines qualifying sessions into wing_conversations via
'mempalace mine --mode convos'.
Design choices mirror mempalace-session:
- Export-stage-mine idiom with deterministic per-session staging paths
under ~/.cache/mempalace-pi-session/<wing>/, so 'mempalace mine' dedup
on source_file makes re-runs idempotent.
- --dry-run classifies each export as [NEW] or [SKIP] by matching staging
path against the palace's already-filed source_files.
- --min-messages filter skips throwaway single-prompt sessions.
Pi-specific parsing:
- Pi JSONL is a typed tree (id/parentId) per docs/session-format.md;
this walks in file order, which is correct for the overwhelmingly
linear case and harmlessly duplicative on branched sessions (palace
semantic dedup handles it).
- Roles mapped to Claude Code JSONL shape:
user -> {type:user, content:text}
assistant -> {type:assistant, content:[text, tool_use]}
toolResult-> {type:human, content:[tool_result]} (folded back by normalizer)
bashExecution/custom(display)/branchSummary/compactionSummary
-> rendered as text annotations
- thinking blocks and image blocks dropped (noise / palace is text-only).
Source labelling:
- Staging filenames prefixed 'pi_<uuid>.jsonl' so every drawer's
source_file metadata (visible in search results) unambiguously
identifies the harness. Opencode's convention ('<slug>_<id>.jsonl')
is preserved to keep the existing 19k+ drawers deduped.
- Inline synthetic header on first chunk:
[session: <title> | <cwd> | <date> | source: pi]
as a secondary signal.
|
||
|
Joakim Persson
|
98baabe7a0 |
contrib: flag cron-not-installed as a common caveat
Minimal Debian/Ubuntu hosts (and most base container images) don't ship cron by default. `crontab: command not found` is the first thing a user hits if they try the cron path without installing it. Previous caveats block covered semantics (no Persistent, mail-drop stderr) but silently assumed cron was present. Add an explicit "check command -v crontab, apt install cron, or pick systemd" preflight to the caveats so the error is surfaced before the user runs into it. Caught during 2026-04-30 Phase 4 runtime validation on a Debian trixie host: `crontab -T` lint failed because cron wasn't installed, even though the underlying docker-exec shell command (the actual workload) ran fine. |
||
|
Joakim Persson
|
00ce8a7fa1 |
contrib: clarify when opencode-devbox bakes in the toolkit
Previous wording claimed opencode-devbox "bakes it in via
mempalace-toolkit" as if that were always true, but until
opencode-devbox v1.14.30b the image only shipped the mempalace
Python package, not the toolkit wrappers. Users following the
*-devbox scheduler docs on earlier images would hit
"mempalace-session: command not found" inside the container.
Rewrite the precondition to:
- Name the version where bake-in starts (v1.14.30b).
- Link to the upstream INSTALL_MEMPALACE_TOOLKIT build arg.
- Document the escape hatch for custom/older containers
(./install.sh --yes) and flag its ephemeral nature, so nobody
leans on the manual install as a long-term solution.
Caught during 2026-04-30 runtime validation of the *-devbox
systemd unit on a freshly-rebuilt container.
|
||
|
Joakim Persson
|
46bcce5a67 |
contrib: devbox-aware scheduler templates (host-side, docker exec)
On hosts running a long-lived opencode-devbox (or equivalent)
container, mempalace-session lives INSIDE the container, not on
the host. The existing contrib/* templates install a scheduler on
the machine that runs the tool; for the devbox case the scheduler
has to live on the host and reach into the container via
'docker exec'. This was noted in passing in contrib/README.md but
no templates were actually shipped for it.
Adds parallel *-devbox templates for systemd and cron:
contrib/systemd/mempalace-session-devbox.service
- Type=oneshot, same 2h TimeoutStartSec + low Nice as the direct
variant.
- Two Environment knobs (CONTAINER, CONTAINER_USER) default to
opencode-devbox/developer, overrideable via
'systemctl --user edit'.
- ExecCondition checks 'docker ps --filter name= --filter
status=running' so the unit no-ops cleanly when the container
is currently down. systemd reports this as a successful
'condition failed' state — no alert noise across dev cycles
of teardown/rebuild.
- ExecStart is plain /usr/bin/docker exec with no shell; systemd
does the env-var expansion.
- Stdout/stderr go to journalctl --user -u <unit> (nothing to
redirect, since docker exec surfaces container output to the
calling process).
contrib/systemd/mempalace-session-devbox.timer
- Mon 03:00 Persistent=true RandomizedDelaySec=30m, mirrors the
direct timer.
contrib/cron/mempalace-session-devbox.cron
- Equivalent shell-wrapped form for hosts using cron instead of
systemd. 'docker ps | grep -q .' short-circuits if the container
isn't running. Log goes to $HOME/.cache/mempalace-session/
cron-devbox.log on the HOST (outside the container) so it's
inspectable without dropping into the devbox.
contrib/README.md:
- Replaces the two-paragraph 'Running inside a container' note
with a proper section: preconditions, install recipes for both
the systemd and cron devbox variants, verify/uninstall commands,
customization via 'systemctl --user edit', behaviour when the
container is down.
- Chooser table gains a dedicated row pointing devbox users at
the *-devbox templates, and mentions the systemd vs cron pick
for that case.
- New 'When to pick devbox variants vs direct ones' table covers
the rare both-installed case (host mempalace AND in-container
mempalace see separate palaces — they don't cross-pollinate).
Top-level README.md 'Keeping it fresh' subsection gains a quick-start
block for the devbox variant alongside the existing Linux/macOS
quick-starts.
Tested: all four systemd units parse cleanly as INI via
configparser (sections + key=value pairs); validated file sizes
and locations match the layout described in docs. Runtime
validation (systemctl --user enable; actual docker exec) requires
a host with docker + an opencode-devbox container up — deferred
to the user's Mac/Linux boxes.
|
||
|
Joakim Persson
|
4dcd2959ec |
docs+installer: how to register mempalace with opencode MCP, probe for it
Two missing pieces bundled here:
1. The README described the MCP server wrapper (mempalace-mcp-server,
a 3-line shell script that exec's the venv's python) as the
canonical answer on a uv-tool install. That is over-engineered and
out of step with the live reference box: opencode.json there uses
['mempalace-mcp'] directly, which is the shim uv tool install
already creates. Rewrote the section to put the simple canonical
answer first and demote the wrapper to a legacy-fallback sidebar.
New 'Registering mempalace with opencode' section covers:
- The one-entry JSON stanza to paste into the mcp object.
- A full minimal opencode.json for someone starting fresh (with
the 'instructions' array pointing at the wake-up protocol).
- Custom palace path variant.
- Claude Code one-liner (claude mcp add mempalace -- mempalace-mcp).
- Pointer at 'mempalace mcp' subcommand which prints the currently-
recommended snippets — useful when upstream updates conventions.
- Troubleshooting table (tools absent / server unavailable /
ModuleNotFoundError) with per-symptom fixes.
The legacy-fallback subsection explains what the wrapper script
was for (the pip-install → uv-tool-install transition era), shows
its 3-line implementation for completeness, and is explicit about
not using it for new installs.
Verification checklist updated: now runs 'which mempalace-mcp' and
'mempalace-mcp --help' against the shim, not the wrapper.
2. install.sh gains a matching probe: after the existing wake-up
protocol check, it grep-checks ~/.config/opencode/opencode.json
for 'mempalace' + 'mempalace-mcp' substrings. Present → clean
success line. Missing → actionable warning that prints the exact
JSON stanza to add plus a pointer to the README anchor and the
'mempalace mcp' CLI helper. Skipped when opencode.json doesn't
exist (non-opencode hosts).
Textual grep rather than strict JSON parse: we don't want to
hard-depend on python/jq at install time, and the two substrings
together are specific enough to avoid false positives.
Prerequisites list gets a 5th bullet flagging MCP registration as
required and pointing at the new section + the installer probe.
Smoke-tested both scenarios on the reference box — config with
mempalace entry present (all ticks green), config with entry removed
(actionable warning, correct snippet shown), config restored to
original state.
|
||
|
Joakim Persson
|
60c2a4abec |
install.sh: probe for mempalace wake-up protocol, warn if missing
The mempalace skill is only useful if an agent loads its wake-up
protocol at session start. Without ~/.config/opencode/instructions/
mempalace.md, the skill is reachable but never auto-runs — agents
forget to search before answering and to write a diary at wind-down.
The failure mode is silent: no error, no warning, just gradual
memory degradation.
install.sh now probes for the file after its regular installation
steps and prints an actionable warning if missing:
! Wake-up protocol NOT installed at /path/to/.../mempalace.md
Without it, the mempalace skill is loadable but never auto-
runs at session start. Install via the skillset repo:
git clone .../skillset.git ~/skillset
cd ~/skillset && ./deploy-skills.sh --bootstrap
(if skillset is already cloned, just run the --bootstrap step)
If the file is present: prints a matching success line. If the host
doesn't even have ~/.config/opencode/, the check is skipped entirely
(non-opencode machine → no warning to display).
README Prerequisites gains a 4th bullet pointing at skillset's
--bootstrap as the canonical source for the wake-up protocol, so
anyone reading the docs without running install.sh also learns
about the dependency.
The wake-up file is shipped by skillset, not this toolkit. Rationale:
the file bootstraps the 'mempalace' skill (which lives in skillset)
and applies to any harness, not just opencode-plus-toolkit machines.
Cross-referenced via the skillset gitea URL in both the installer
message and the README.
Smoke-tested present + missing scenarios on the reference box —
cleanly detects both, does not hard-fail on missing.
|
||
|
Joakim Persson
|
2f703a8ebc |
Docs: explain diary vs session mine (why keep both)
Automated session mining could plausibly lead a user (or a future agent) to conclude that writing diary entries is redundant — mining captures every turn, so why also write a compressed summary at wind-down? That conclusion is wrong, and it's worth explaining why in the docs so both disciplines survive. ARCHITECTURE.md §5 gets a new subsection 'Diary vs session mine: why keep both?' that presents this as a first-class concept: - Comparison table — content, granularity, compression, authorship, signal density, retrieval pattern, and the question each answers. - The defining property of a diary entry: editorial judgment by the author. Captures meta-observations that were never said aloud during the session (lessons, patterns, pending items, aggregate counts). Mining raw turns can never surface these because the words don't exist verbatim. - Three practical scenarios where the distinction bites: wake-up token economics, 'what did we decide' vs 'what did we say', redundancy covering each other's failure modes. - Practical implications: don't skip either habit, let them specialize (diary = release notes; mine = git log). README.md gets a brief teaser in the 'First mine' area with a link to the canonical ARCHITECTURE.md section — enough for a skim reader to decide they want to keep writing diaries, and for a deep reader to know where to go for the full explanation. SKILL.md replaces the three-line 'Relationship to the mempalace skill' note with a compact version of the comparison table and a direct call-out of the 'session mining means I don't need diaries' misconception agents fall into. Points agents at ARCHITECTURE.md §5 for the full treatment when users ask the question. Cross-references verified: anchor slug for the new section is #diary-vs-session-mine-why-keep-both (standard slug rules: colon, spaces, punctuation removed/hyphenated). Both linking docs use the matching fragment. |
||
|
Joakim Persson
|
349a3a3d3d |
mempalace-session: make --dry-run dedup-aware
A --dry-run report showed all qualifying sessions without indicating
which would actually hit the palace on a real run. On a second run
against an already-mined corpus this was misleading — output said
'Exported 62 session(s)' but the real mine step would skip all 62.
The wrapper now queries the palace's chroma.sqlite3 (read-only, via
file:...?mode=ro URI) for source_file values under the staging dir,
then tags each exported session as [NEW] or [SKIP] during listing and
reports the split in the summary:
Exported 62 session(s) to ~/.cache/mempalace-session/wing_conversations
0 new → will be filed on mine
62 already filed → will be skipped (dedup by source_file)
--dry-run: no new sessions to mine. A real run would skip all 62.
Implementation notes:
- Classification is best-effort. If the palace is unreachable (fresh
install, moved, permission-denied, file missing) the wrapper falls
back to treating all exports as NEW — the real mine step still
delegates dedup to 'mempalace mine --mode convos' which is the
authoritative source of truth. Getting the classification wrong
in --dry-run is cosmetic; behaviour of a real run is unchanged.
- Palace path respects $MEMPALACE_PATH env var for non-default setups.
- Same classification also shown on a real (non-dry-run) mine so users
see upfront how much of the export set is actually new before the
miner runs.
Verified both directions:
- All-already-filed case (current box, 62 sessions in palace): reports
0 new, 62 skipped. --dry-run message correctly says 'would skip all'.
- Partial case (simulated by deleting one session's metadata from
palace): reports 1 new, 61 skipped. --dry-run message correctly
says 'would file 1 new'. Palace was restored from backup
immediately after the test.
README and SKILL.md both updated with the new dedup-aware output and
a direct answer to the FAQ 'will it mine the same sessions again?'
|
||
|
Joakim Persson
|
72e7019101 |
Fix two docs/UX errors found during Mac install (tor-ms22)
1. 'mempalace init --yes' without a dir argument fails — 'dir' is
required. The semantics were wrong too: 'mempalace init' is
per-project (sets up mempalace.yaml + entity detection in a specific
directory), not a one-time global init. The palace itself is
created lazily on first write, so neither mempalace-session nor
mempalace-docs requires any init step.
Removed the misleading 'One-time palace init' block from README.md,
ARCHITECTURE.md, and SKILL.md. Added a clarifying note:
'mempalace init <dir>' is per-project and optional (needed only to
customize the wing name or entity detection before mempalace-docs).
2. install.sh's 'Skipping <name>: <dest> exists and is not our symlink'
warning gave no actionable guidance. On the Mac, a leftover
~/.local/bin/mempalace-docs (likely from the pre-split cli_utils
days) was blocking the new install and the user had no easy way
to know what to do about it.
Expanded the warning to:
- Show whether the blocker is a symlink (and what it points at) or
a real file.
- Print the exact 'rm && ./install.sh' fix line.
- Track skipped count separately and flag it in the closing
summary so a scrolling user doesn't miss it.
Added matching troubleshooting paragraph to the README 'Install
mempalace-toolkit' section explaining the skip behaviour and
pointing at the installer's own message for the fix.
Smoke-tested the new skip-warning code path by temporarily replacing
~/.local/bin/mempalace-docs with a foreign symlink and re-running
install.sh — output is clear, specific, and restores cleanly.
|
||
|
Joakim Persson
|
d69e95d422 |
install.sh: set executable bit
The initial commit created install.sh with mode 0644, so a fresh clone (e.g. on tor-ms22) hit 'permission denied: ./install.sh' and needed a manual chmod +x or 'bash install.sh' workaround before first run. This is a pure permission change (same content hash); git tracks the execute bit in the tree, so this fixes it for every future clone. bin/mempalace-docs and bin/mempalace-session were already 0755 because they carried over from their original cli_utils commits — install.sh was new in the split-out commit and missed the +x that the write-path doesn't apply by default. |
||
|
Joakim Persson
|
25972b7499 |
README: document uv-based mempalace install + MCP wrapper pitfall
mempalace-toolkit's Prerequisites section assumed mempalace was already
installed but didn't explain how. The upstream mempalace repo only
shows pip install, which fights PEP 668 on modern distros and leaks
dependencies into system site-packages. The production pattern used
in opencode-devbox (uv tool install) is cleaner but wasn't documented
here.
Adds a full 'Installing mempalace itself (prerequisite)' section with
five subsections:
1. Why uv over pip — isolated venv, no PEP 668 fight, shim makes
the CLI accessible from any bash/zsh terminal without manual
venv activation.
2. Personal machine — with default paths
(shim in ~/.local/bin, venv under ~/.local/share/uv/tools/). Simple
one-liner plus PATH guidance. This is the recommended default.
3. System-wide / container install — the opencode-devbox pattern:
UV_TOOL_DIR=/opt/uv-tools + UV_TOOL_BIN_DIR=/usr/local/bin, with
the exact Dockerfile RUN step used in production (including the
python -c build-time sanity check). Cross-references
opencode-devbox/Dockerfile for the full canonical version.
4. MCP server wrapper — explains the 'missing venv when the container
was deployed' pitfall from the first opencode-devbox attempt:
with a non-default UV_TOOL_DIR, system python3 can't import
mempalace, so MCP configs of the form
["python3", "-m", "mempalace.mcp_server"]
fail silently with ModuleNotFoundError. Fix is a thin wrapper on
PATH that exec's the venv's own python. Shows the exact 3-line
shell wrapper from opencode-devbox/rootfs/usr/local/bin/
mempalace-mcp-server. Points at opencode-devbox/AGENTS.md
'Critical conventions' as the authoritative reference.
5. Verification checklist — /usr/local/bin/mempalace, MemPalace 3.3.3,
and a minimal
=======================================================
MemPalace Status — 4943 drawers
=======================================================
WING: cli_utils
ROOM: scripts 38 drawers
ROOM: fzf 25 drawers
ROOM: general 1 drawers
WING: opencode_devbox
ROOM: general 203 drawers
ROOM: configuration 3 drawers
WING: proxmox
ROOM: general 1046 drawers
WING: skillset
ROOM: general 1118 drawers
WING: wing_conversations
ROOM: technical 1775 drawers
ROOM: architecture 513 drawers
ROOM: planning 164 drawers
ROOM: problems 42 drawers
ROOM: general 6 drawers
ROOM: decisions 3 drawers
WING: wing_orchestrator
ROOM: diary 6 drawers
======================================================= smoke test that catches venv
mismatches by failing with a Python traceback instead of a clean
error message.
Renames the existing 'Install' section to 'Install mempalace-toolkit'
to disambiguate from the new mempalace install section — the toolkit's
own install.sh still works the same, just labeled more precisely.
ARCHITECTURE.md §4 prerequisites paragraph and SKILL.md prerequisites
block both cross-reference the new section with anchor links, so any
entry point into the docs leads the reader to the right recipe.
|
||
|
Joakim Persson
|
720245e010 |
Add macOS launchd template, bringing automation parity to macOS
Ship a launchd user agent plist alongside the existing systemd and
cron templates so macOS users can schedule mempalace-session without
falling back to cron. launchd is the macOS-native equivalent of a
systemd user timer: same scheduling model, same log conventions, same
single-instance guarantees.
- contrib/launchd/se.jordbo.mempalace-session.plist:
- Label uses reverse-DNS from the jordbo.se domain for consistency
with other user-installed launchd jobs; fork the prefix if reusing
this template in a different org.
- ProgramArguments points at /Users/USER/.local/bin/mempalace-session
(USER is substituted at install time, same pattern as
contrib/cron/).
- EnvironmentVariables.PATH covers ~/.local/bin, Apple Silicon
Homebrew, Intel Homebrew, and system defaults — launchd agents
get a minimal PATH by default and the wrapper needs to find
mempalace + python3.
- StartCalendarInterval matches systemd unit's schedule: Monday
03:00 local.
- RunAtLoad=false — load shouldn't trigger a run; schedule does.
- ProcessType=Background + LowPriorityIO=true + Nice=10 mirror
the systemd unit's Nice=10 + IOSchedulingClass=idle. macOS's
automatic App Nap and resource throttling for Background jobs
yields to interactive work cleanly.
- ExitTimeOut=7200 matches systemd's TimeoutStartSec=7200.
- StandardOut/ErrorPath under ~/Library/Logs/ so Console.app
surfaces them.
- contrib/README.md gains a full launchd section:
- Caveat table comparing to systemd (Persistent=true isn't quite
matched; RandomizedDelaySec has no equivalent; overlap prevention
is automatic).
- Install recipe using launchctl bootstrap (modern) with a fallback
note for legacy launchctl load -w on older macOS.
- Verify section shows launchctl list, launchctl print, log tails,
and launchctl kickstart for manual testing.
- Uninstall via launchctl bootout.
- Chooser table updated: macOS now explicitly points at launchd,
not cron.
- ARCHITECTURE.md §5, SKILL.md Quick automation pitch, and README.md
Keeping it fresh section all updated to mention the three scheduler
options and give per-platform quick-starts.
Plist XML validated with plistlib.
|
||
|
Joakim Persson
|
36845e14b2 |
Document the operational routine + ship automation templates
Until opencode session-stopping hooks land upstream, mempalace-session
is the entire mechanism that gets opencode conversations into the
palace — skip it and session history stays trapped in a local SQLite
DB, invisible to semantic search. Previous docs covered setup well
but were thin on when and how often to run it.
- ARCHITECTURE.md §5: replace the one-line 'When to re-mine' note with
a full Operational Routine section — triggers, cadence, relationship
to the session lifecycle, automation pointers, verification.
- SKILL.md: add an Operational Routine section aimed at agents —
when to suggest invoking the tool, cadence guidance, how to
distinguish this producer-side tool from the consumer-side
mempalace skill's in-session habits.
- README.md: add 'Keeping it fresh' subsection pointing at contrib/
and the full docs.
contrib/ ships three ready-to-use templates:
- systemd/mempalace-session.{service,timer} — user units with weekly
Mon 03:00 schedule, Persistent=true catch-up, RandomizedDelaySec for
fleet-wide jitter, ConditionPathExists guard for opencode-less boxes,
Nice+IOSchedulingClass=idle so it never fights interactive work.
- cron/mempalace-session.cron — sample crontab entry with log
redirection and clear USER-substitution instructions.
- README.md with install/verify/uninstall recipes for both, a chooser
table (systemd vs cron), container/devbox caveats, and tuning notes
(daily vs weekly vs monthly trade-offs).
The user's LATER-list item 'wrap mempalace-session in cron/systemd
timer for true auto-save coverage' is now actionable: a single
systemctl --user enable --now command stands it up.
|
||
|
Joakim Persson
|
e49d9285c4 |
install.sh: drop .skill-source marker in deployed skill dir
The skill directory at ~/.agents/skills/opencode-mempalace-bridge/ is a real dir containing a single SKILL.md symlink back into this repo — the 'colocated skill' pattern. Sibling reconcilers (skillset's deploy-skills.sh, cli_utils's agents-sync.zsh) already handle external dirs correctly via their existing 'leave real dirs alone' policies, but a machine-readable marker makes ownership explicit: # skill-source: mempalace-toolkit # repo: <absolute path> # url: ssh://git@gitea.jordbo.se:2222/joakimp/mempalace-toolkit.git The marker is the convention for any external repo that wants to ship a colocated skill. The name is generic (.skill-source, not .managed-by-mempalace-toolkit) so a second colocated skill from a different repo can reuse the same file name; the first line identifies the owner. --uninstall now also removes the marker (only if it still says mempalace-toolkit) and the now-empty skill dir. AGENTS.md + README.md describe the pattern and point at sibling docs in cli_utils/AGENTS-SYNC.md and skillset/README.md that mirror the convention. |
||
|
Joakim Persson
|
3554f56bcc |
Update SKILL.md references after cli_utils split
The initial split copied SKILL.md verbatim from its pre-split location where it still referenced paths in cli_utils. Update all 10 stale references to point at mempalace-toolkit instead — canonical path, clone URL, bind-mount path, container-recreate recovery command, and See also links. |
||
|
Joakim Persson
|
7bd5314419 | Merge remote-tracking branch 'origin/main' | ||
|
joakimp
|
82acd674c0 | Initial commit | ||
|
Joakim Persson
|
954c3f2ebb |
Initial commit — split out from cli_utils
Producer-side MemPalace tooling: two bash wrappers that bridge opencode session history and project documentation into the palace. Originally developed in cli_utils (2026-04-28); split into its own repo on 2026-04-30 because the conceptual fit was weak — cli_utils is interactive shell tooling, while this is agent memory infrastructure with its own architecture, dependency surface, and growth trajectory. Contents: - bin/mempalace-docs — docs-only mining wrapper (originally a2ddcc9 in cli_utils), bridges the gap until MemPalace PR #1213 (exclude_patterns) merges upstream. - bin/mempalace-session — opencode → palace session bridge (originally dacca0e in cli_utils). Reads ~/.local/share/opencode/opencode.db, exports each session to Claude Code JSONL, mines via 'mempalace mine --mode convos'. Bridges the gap until opencode session-stopping hooks + an opencode harness in hooks_cli.py land upstream. - ARCHITECTURE.md — canonical spec: architecture diagram, component details, setup recipe, operational notes, upstream-retirement roadmap. Originally a4cf314 in cli_utils. - SKILL.md — companion agent skill (producer side). Pairs with the consumer-side mempalace skill. Symlinked into ~/.agents/skills/opencode-mempalace-bridge/ by install.sh. - install.sh — idempotent installer, also handles --uninstall. - AGENTS.md — repo conventions. History of the individual files is not preserved in this split; see cli_utils (gitea.jordbo.se/joakimp/cli_utils) commits a2ddcc9, dacca0e, and a4cf314 for the original authorship context. |