mempalace-toolkit

joakimp/mempalace-toolkit

Fork 0

Commit Graph

Author	SHA1	Message	Date
joakimp	a361b71c40	ship: don't trust the mtime the exporter deliberately backdates rsync --update skips a file whose mtime is not strictly newer than the receiver's. The stage file's mtime IS the source transcript's mtime (os.utime() at :903, "preserve session mtime for dedup stability"), so re-exporting a session that has not been appended to since its last ship produces a mtime that is not newer than what's already at the receiver — exactly the case a redactor upgrade needs to ship, because content differs while mtime does not. --update reports success and sends nothing. Reported and patched by pi@mbp-m1-2020 (evt_20260827T211925_9674a31da0b4, artifact art_20260827T211839_fa52af563105, sha256 f217e47e…), measured live: a scrubbed re-export of a dormant session (pi_01a03022-…f542) sat unshipped in the palace host's inbox while every local signal reported a clean stage, saved only because a host-side sweep happened to rewrite the remote copy independently that same day. --checksum compares content and ignores size/mtime entirely. Dropping --update outright was considered and rejected: rsync's default quick check already transfers on a SIZE difference alone, which is why the observed case (33-byte placeholder vs a 43-byte token) would have been masked as "fixed" by a change that only works until a redaction whose placeholder happens to match the secret's length. os.utime() at :903 is untouched — its backdating is a separate, load-bearing design call for dedup stability, out of scope for this fix. Added scripts/test-rsync-ship-idempotency.sh: ships a file, rewrites its content to an EQUAL-LENGTH string while restoring the original mtime (what os.utime() does), ships again, asserts the receiver's sha256 changed. Equal length is deliberate, not cosmetic — mismatched lengths would pass via the quick check alone and prove nothing about --checksum specifically; this is the same reasoning that ruled out dropping --update. Verified the test discriminates: passes against today's --checksum, fails against --update (checked by temporarily substituting the flag in a copy, not committed). Runs offline — a local rsync destination path exercises the same size/mtime/checksum comparison as the ssh transfer, no palace or network needed.	2026-08-27 23:29:30 +02:00
pi	553d86570c	provenance: stamp device+harness at the edge, not in the agent's head RFC 001 §7.3.2 ranks "agent stamps provenance via a skill instruction" as the ❌ worst possible place — per-call boilerplate, forgettable, improvisable. It was right, and we had shipped exactly that: the mempalace skill told the agent to pass added_by="<harness>@<device>" by hand. Measured on the shared palace, 199 rows had reached it unresolvable, 10 of them filed by the very agent that wrote the instruction, in a drawer about host provenance. The trigger was a cross-host misattribution: a session on tor-ms22 read its own diary, could not tell that the entries were written on EMB-7KJ4VR4G, and reported another machine's verification as this one's. Move the same convention into the ⚠️ edge row, where it is uniform and unforgettable (§7.3.5): * extensions/pi/mempalace.ts defaults the writer field on every tool that has one — added_by (add_drawer, checkpoint), agent (mine), from_agent (event_append), created_by (artifact_put) — from $MEMPALACE_PI_DEVICE. An explicit value always wins, so filing for another device stays possible. The allowlist is per tool, never blanket: 3.8.0's dispatcher hard-rejects undeclared args with -32602, so injecting added_by into diary_write or kg_add (which have no such property) would break the call outright. * mine gets miner@<device> when the caller invokes it, but <harness>@<device> for the bridge's own transcript feed — bulk extraction is not agent-authored memory, and that keeps the pi/opencode/miner taxonomy honest. * diary_write has no metadata slot at all, and the device must never go in agent_name (wing = f"wing_{agent_name}" would splinter the diary per host). So the entry TEXT carries an AAAK field, HOST:<device>\|SESSION:… — which is also the only channel a READER sees: search projects a fixed key set and diary_read returns content, so no metadata fix, not even a server-authoritative one, would have prevented the misattribution. * The wake-up block now states the device and warns that diary_read interleaves every machine's diary. * R1: doubly gated on MEMPALACE_PI_DEVICE and MEMPALACE_REMOTE_URL, so a solitary devbox stamps nothing and behaves exactly as before — which is also the correct semantics per §7.3.3. Version the reconciler that was living only on synlig (bin/ + contrib/systemd/), add --dry-run, and teach it two new rules: diary_host_marker reads the HOST: field, and sibling_chunk propagates a resolved origin across a drawer's chunks (a text marker lands in chunk 0 only, so a 5-chunk diary entry would otherwise stamp 1 and leave 4 blank). --dry-run against the real palace before deploying earned its keep twice, and scripts/test-device-stamp.sh pins both findings with the strings it found: HOST: was ALREADY in use with a composite grammar (HOST:emb-7kj4vr4g.f1d3c3f89e3e.v1.8.3.pi0.84.2) and for bare container ids, so an unvalidated rule invented devices like "f1d3c3f89e3e.pi0.84.2"; and HOST: also carries a different SENSE elsewhere (HOST:exec.via.ssh-controlmaster->…, meaning where I was executing). Validating against the known-device set both refuses those and recovers the composite entries correctly. A marker convention inherits every prior meaning of its own name. Deployed and verified on synlig: device 14,217 → 14,317, integrity ok, idempotent on immediate re-run, no invented device values. RFC updates: §7.3.1 corrected (the arg whitelist is a hard -32602 in 3.8.0, not a silent drop; get_drawer DOES return metadata, search structurally cannot; triples and logstream live in separate databases the stamper cannot reach), §7.3.5 added (what is deployed, including the divergence from §7.3.4's opaque origin_device — tor-ms22 vs tor-ms22-native is that cost already visible), and Phase 4 now carries per-device tokens motivated FIRST by revocation, with the finding that tokens are the cheap half: core holds one scalar auth_token and has zero device concept, so authoritative stamping needs a component we own.	2026-08-25 22:26:46 +02:00
pi	96699f2a17	feat(pi-bridge): external MemPalace transport via MEMPALACE_REMOTE_URL Let the pi<->mempalace bridge connect to a shared MemPalace over HTTP instead of always spawning a local mempalace-mcp: - Extract IMcpClient; rename McpClient -> StdioMcpClient (ctor command, arg-less start()). - Add RemoteMcpClient (vendored from pi-extensions/mcp-loader.ts): streamable-HTTP with AbortController timeouts, protocolVersion pinned 2024-11-05, alive/ensureAlive. mempalace-mcp --transport http is sessionless JSON-RPC today; session/SSE/404 branches retained for a future streamable-HTTP server. - createClient() selects transport from MEMPALACE_REMOTE_URL; MEMPALACE_REMOTE_TOKEN -> Authorization: Bearer. Lifecycle automation (wake-up, /mempalace-diary) unchanged. - scripts/check-mcp-client-sync.sh: drift guard vs canonical mcp-loader.ts. - README: document local-vs-external transport. Typechecks clean (strict); both transports smoke-tested against live mempalace-mcp.	2026-07-02 13:09:29 +02:00

Author

SHA1

Message

Date

joakimp

a361b71c40

ship: don't trust the mtime the exporter deliberately backdates

rsync --update skips a file whose mtime is not strictly newer than the
receiver's. The stage file's mtime IS the source transcript's mtime
(os.utime() at :903, "preserve session mtime for dedup stability"), so
re-exporting a session that has not been appended to since its last
ship produces a mtime that is not newer than what's already at the
receiver — exactly the case a redactor upgrade needs to ship, because
content differs while mtime does not. --update reports success and
sends nothing.

Reported and patched by pi@mbp-m1-2020 (evt_20260827T211925_9674a31da0b4,
artifact art_20260827T211839_fa52af563105, sha256 f217e47e…), measured
live: a scrubbed re-export of a dormant session (pi_01a03022-…f542)
sat unshipped in the palace host's inbox while every local signal
reported a clean stage, saved only because a host-side sweep happened
to rewrite the remote copy independently that same day.

--checksum compares content and ignores size/mtime entirely. Dropping
--update outright was considered and rejected: rsync's default quick
check already transfers on a SIZE difference alone, which is why the
observed case (33-byte placeholder vs a 43-byte token) would have been
masked as "fixed" by a change that only works until a redaction whose
placeholder happens to match the secret's length. os.utime() at :903
is untouched — its backdating is a separate, load-bearing design call
for dedup stability, out of scope for this fix.

Added scripts/test-rsync-ship-idempotency.sh: ships a file, rewrites
its content to an EQUAL-LENGTH string while restoring the original
mtime (what os.utime() does), ships again, asserts the receiver's
sha256 changed. Equal length is deliberate, not cosmetic — mismatched
lengths would pass via the quick check alone and prove nothing about
--checksum specifically; this is the same reasoning that ruled out
dropping --update. Verified the test discriminates: passes against
today's --checksum, fails against --update (checked by temporarily
substituting the flag in a copy, not committed).

Runs offline — a local rsync destination path exercises the same
size/mtime/checksum comparison as the ssh transfer, no palace or
network needed.

2026-08-27 23:29:30 +02:00

553d86570c

provenance: stamp device+harness at the edge, not in the agent's head

RFC 001 §7.3.2 ranks "agent stamps provenance via a skill instruction" as the
❌ worst possible place — per-call boilerplate, forgettable, improvisable. It
was right, and we had shipped exactly that: the mempalace skill told the agent
to pass added_by="<harness>@<device>" by hand. Measured on the shared palace,
199 rows had reached it unresolvable, 10 of them filed by the very agent that
wrote the instruction, in a drawer about host provenance. The trigger was a
cross-host misattribution: a session on tor-ms22 read its own diary, could not
tell that the entries were written on EMB-7KJ4VR4G, and reported another
machine's verification as this one's.

Move the same convention into the ⚠️ edge row, where it is uniform and
unforgettable (§7.3.5):

* extensions/pi/mempalace.ts defaults the writer field on every tool that has
  one — added_by (add_drawer, checkpoint), agent (mine), from_agent
  (event_append), created_by (artifact_put) — from $MEMPALACE_PI_DEVICE. An
  explicit value always wins, so filing for another device stays possible. The
  allowlist is per tool, never blanket: 3.8.0's dispatcher hard-rejects
  undeclared args with -32602, so injecting added_by into diary_write or kg_add
  (which have no such property) would break the call outright.
* mine gets miner@<device> when the caller invokes it, but <harness>@<device>
  for the bridge's own transcript feed — bulk extraction is not agent-authored
  memory, and that keeps the pi/opencode/miner taxonomy honest.
* diary_write has no metadata slot at all, and the device must never go in
  agent_name (wing = f"wing_{agent_name}" would splinter the diary per host).
  So the entry TEXT carries an AAAK field, HOST:<device>|SESSION:… — which is
  also the only channel a READER sees: search projects a fixed key set and
  diary_read returns content, so no metadata fix, not even a
  server-authoritative one, would have prevented the misattribution.
* The wake-up block now states the device and warns that diary_read interleaves
  every machine's diary.
* R1: doubly gated on MEMPALACE_PI_DEVICE and MEMPALACE_REMOTE_URL, so a
  solitary devbox stamps nothing and behaves exactly as before — which is also
  the correct semantics per §7.3.3.

Version the reconciler that was living only on synlig (bin/ + contrib/systemd/),
add --dry-run, and teach it two new rules: diary_host_marker reads the HOST:
field, and sibling_chunk propagates a resolved origin across a drawer's chunks
(a text marker lands in chunk 0 only, so a 5-chunk diary entry would otherwise
stamp 1 and leave 4 blank).

--dry-run against the real palace before deploying earned its keep twice, and
scripts/test-device-stamp.sh pins both findings with the strings it found:
HOST: was ALREADY in use with a composite grammar
(HOST:emb-7kj4vr4g.f1d3c3f89e3e.v1.8.3.pi0.84.2) and for bare container ids, so
an unvalidated rule invented devices like "f1d3c3f89e3e.pi0.84.2"; and HOST:
also carries a different SENSE elsewhere (HOST:exec.via.ssh-controlmaster->…,
meaning where I was executing). Validating against the known-device set both
refuses those and recovers the composite entries correctly. A marker convention
inherits every prior meaning of its own name.

Deployed and verified on synlig: device 14,217 → 14,317, integrity ok,
idempotent on immediate re-run, no invented device values.

RFC updates: §7.3.1 corrected (the arg whitelist is a hard -32602 in 3.8.0, not
a silent drop; get_drawer DOES return metadata, search structurally cannot;
triples and logstream live in separate databases the stamper cannot reach),
§7.3.5 added (what is deployed, including the divergence from §7.3.4's opaque
origin_device — tor-ms22 vs tor-ms22-native is that cost already visible), and
Phase 4 now carries per-device tokens motivated FIRST by revocation, with the
finding that tokens are the cheap half: core holds one scalar auth_token and has
zero device concept, so authoritative stamping needs a component we own.

2026-08-25 22:26:46 +02:00

96699f2a17

feat(pi-bridge): external MemPalace transport via MEMPALACE_REMOTE_URL

Let the pi<->mempalace bridge connect to a shared MemPalace over HTTP instead
of always spawning a local mempalace-mcp:
- Extract IMcpClient; rename McpClient -> StdioMcpClient (ctor command, arg-less start()).
- Add RemoteMcpClient (vendored from pi-extensions/mcp-loader.ts): streamable-HTTP
  with AbortController timeouts, protocolVersion pinned 2024-11-05, alive/ensureAlive.
  mempalace-mcp --transport http is sessionless JSON-RPC today; session/SSE/404
  branches retained for a future streamable-HTTP server.
- createClient() selects transport from MEMPALACE_REMOTE_URL; MEMPALACE_REMOTE_TOKEN
  -> Authorization: Bearer. Lifecycle automation (wake-up, /mempalace-diary) unchanged.
- scripts/check-mcp-client-sync.sh: drift guard vs canonical mcp-loader.ts.
- README: document local-vs-external transport.

Typechecks clean (strict); both transports smoke-tested against live mempalace-mcp.

2026-07-02 13:09:29 +02:00

3 Commits