982b00100180a624a9172e77734ce371a228853b
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 joakimp
|
982b001001 |
docs: RFC 003 §7.11/§7.12 — delivered is not read, and the mailbox carries obligations not news
Both measured 2026-08-26 by peer devices, and the second one measured on me: the operator had to point at an event id before I read a report that had been addressed to me for two hours. The mailbox was working correctly the whole time. §7.12 is the structural one. Candidates are drawn with status="open", so an event carrying a TERMINAL status is not a mailbox candidate at all, whoever it is addressed to. A task.reply written to a named device to share a finding is therefore delivered to nobody, ever — and neither is any event.ack. So the most natural inter-device message, "here is something you should know", is exactly the shape that gets no delivery. Two things that do work: address it as a directed status="open" ask (correct when a response is actually wanted), or accept it as pull-only and pair it with a drawer, which the peer's search will surface. A terminal report plus an expectation of attention does not. §7.11 is the peer's finding, and it explains the other half of why that report sat unread: delivery is deliverAs "steer" with deliberately no triggerTurn, and the poll fires on agent_settled — i.e. when the agent is IDLE, with no inference running. The text is queued for the next turn, so the human is the trigger. Measured on another device: a delivery landed at ~20:50Z and sat visibly unreacted-to until the operator asked "do I have to nudge you?". Not a defect; the no-triggerTurn decision was deliberate and stands. But the delivered text explains how to CLOSE an ask and never says when it will be SEEN, so the one reader who needs that fact — the human watching the window — is the one not told. Recorded with the three fixes that do not wake a model, as decision 8. §7.4 is upgraded from inferred to measured, on two devices independently, by a two-arm control: a to_agent='*' event with status='open' survives the raw candidate filter and therefore reaches the exclusion branch, which no previously written broadcast (all non-open) ever did. Raw query returned it, derived owed set did not, delivery named only the directed arm. Two arms differing only in to_agent is what makes it evidence rather than an absence. And it carries a correction of my own record, in place and dated: I had filed that this branch COULD NOT be exercised, because every broadcast in the log is status='ready' and the protocol forbids writing an open one. Wrong in an instructive way — the protocol forbids it as PRODUCTION TRAFFIC, which does not forbid planting one as a labelled, self-closed control. Where a rule appears to block a measurement, check whether it blocks the use or the test. |
||
|
joakimp
|
bfe9c5cd4f |
mailbox: join the owed set on hlc, not seq, before a second replica exists
deriveOwed decides "is this ask still owed?" by asking whether one of my own terminal replies is LATER than the ask. It compared `seq` — this database's arrival rowid. On a single hub that is global order, so it was correct; the comment above it already said hlc was the durable key "once mesh_peers reports actual peers". Making the switch now, while one replica means the two orderings agree, costs nothing; making it later means changing the rule while two machines already disagree about order. The bug being pre-empted is specific: with a second replica the same event gets a different `seq` in each database, because arrival order is not authorship order. A reply authored after its ask can arrive first and take the lower seq; the join then concludes "no later reply exists" and an already-answered ask reappears as owed — permanently, on that machine. isStrictlyAfter() prefers `hlc` when both events carry one and falls back to `seq` otherwise (a server predating the field, or an un-backfilled row). hlc is rendered fixed-width, <unix_ms:13 digits>-<counter:6 hex>-<replica_id>, so a plain string comparison IS the causal comparison, with the replica id as final tiebreak. Verified before writing the code that the field is actually on the wire — event_list returns it per event (logstream.py:593) — because a fallback that never fires would have made this a no-op dressed as a fix. `created_at` stays rejected, and the reason is now written down where the decision is: it is server-generated at second precision, so ties are routine, and a tie can suppress an UNANSWERED ask. Both remaining failure modes are on the noisy-but-visible side — an answered item resurfacing is annoying, an unanswered ask going silent defeats the mailbox. Tested: 18 cases against the extracted comparator — hlc later/earlier/equal, same-ms counter ties in hex (0x10 vs 0x9, which is where a non-padded format would break), cross-replica tiebreak, both mesh reorderings, every seq-fallback path, and degenerate input (null seq, numeric hlc, empty strings, no keys) which must never claim "answered". All pass. Real-data check on the positive-control pair: seq 26/27 carry hlc ...3071857/...3574085, so the orderings agree today and the switch is a no-op now and correct later. The cursor keeps the opposite ordering ON PURPOSE — since_event_id is local-arrival ordered so a tail consumer still sees late-arriving remote ops whose hlc is older (hlc.py:19-21). RFC 003 §9.3 now says so explicitly, because that asymmetry looks like a bug worth "fixing" and is not. |
||
|
joakimp
|
d4d8bb6109 |
docs: retention direction for the coordination log, and drop the operator's name from RFC 002 §5
RFC 003 §9.1 — retention is settled in direction (rotate old traffic out of
the way, logrotate-style, moved aside rather than destroyed) and the sketch
records the parts that are not obvious:
- Tier artifacts before events. Events are a few KB; artifacts are capped at
4 MiB and stored in-row, so moving artifact CONTENT cold while keeping the
kind/sha256/size/created_by stub reclaims nearly all the space and keeps the
audit trail ("what was handed over, by whom, verified how") intact.
- If events rotate at all, the unit is the correlation_id THREAD whose latest
event is terminal — never the row. Archiving an ask while leaving its reply
(or the reverse) breaks the owed-set join, and both failure modes are bad:
an ask that can never be cleared resurfaces as owed forever, or a reply is
orphaned from what it answered.
- Never rotate an event that is still owed. Owed-ness is DERIVED at read time,
so an unanswered ask is indistinguishable from a stale one except by that
derivation — a purely time-based sweep would discard the live obligations of
a machine that has merely been offline for a month, which is precisely the
case this log exists to serve.
- Rotation invalidates held cursors: since_event_id RAISES on an unknown id
(§7.5), so archiving an event a watcher holds as its resume point turns its
next poll into an error. Either announce rotation ahead of live cursors, or
teach the anchor lookup to fall back to created_at/hlc.
- Archive → verify (row counts, artifact sha256) → only then DELETE + VACUUM,
with a --dry-run that reports in thread units.
RFC 002 §5: "Open decisions for ALC" → "Open decisions". RFC 001 never names
the operator anywhere; impersonal is the mature precedent and ALC was never a
real identifier in the first place (it is the AAAK spec's illustrative code for
"Alice", copy-forwarded into ~700 diary entries without verification).
Same edit also removes two device names and a hostname from §5.3, which is
host inventory and belongs in the private fleet repo, not a public one. The
mechanism it teaches — a palace in a Docker named volume dies on the next
container recreate, so census before flipping — is unchanged and is the part
that mattered.
|
||
|
joakimp
|
e1cc7592a1 |
docs: write the RFC 003 that the code has been citing all along, plus an operator-facing fleet-memory guide
`logstream.py`'s module docstring is headed "Agent coordination event log for MemPalace (RFC 003)" and enumerates five "Design constraints (RFC 003)". Comments cite "RFC 003 phase 5", "RFC 003 suggested defaults" and "the first RFC 003 dogfood". Every event/artifact tool description cites RFC 003. The document has never existed — confirmed by searching this repo and the primary host. So this is a retrospective spec: it transcribes what the implementation already believes, and records what it does NOT do. docs/rfc-003-coordination-log.md, verified line-by-line against mempalace 3.8.0 (every claim cites file.py:LINE, indexed in §10 for re-verification). The parts that are not visible from the tool descriptions: - No idempotency guard on event_append or put_artifact (§7.1). The replication path checks `id OR (origin_replica, origin_seq)` before applying; the client path checks nothing. So peer replay is safe and CLIENT RETRY IS NOT — a retried append forks a coordination thread into two ids. This makes the fleet's "a timeout is not a failure, verify before retrying" rule load-bearing rather than advisory. - Coordination traffic is deliberately exempt from BOTH palace locks (§4) — _HTTP_LOCK_FREE_TOOLS and _PEER_WRITER_EXEMPT_TOOLS, each with its own rationale in-source. "One large mine blocks every client" is true of drawer writes and false of coordination writes. - `mempalace sync` never touches the log, and no DELETE FROM events exists anywhere (§2, §9.1) — answering for the logstream a question RFC 001 §7.2 left open, and making the log permanent and unbounded. - from_agent is shape-validated and never authenticated; there is no read scoping at all (§6). The log authenticates the fleet, not the agent. That is simultaneously the security limitation and the only way to positive-control the mailbox. - Three distinct orderings — seq (local arrival rowid), origin_seq (author's counter), hlc (fleet-wide, lexicographically sortable) — and origin_replica identifies the PALACE, not the writer, which is why from_agent/to_agent carry the whole distinction between machines (§3.2). - GET /logstream/events does not exist and never did (§7.8). An earlier measurement saw it 404 and blamed the reverse proxy; that inference was right for /sync/* and /logstream/stream and wrong for this one. Corrected in extensions/pi/README.md §3 too, in place, dated. docs/fleet-memory.md is the operator-facing companion the repo lacked entirely: the front-door README had zero mentions of coordination, so the channel was undiscoverable unless a message happened to arrive. It covers the five stores and what each is for (drawers/wings/rooms, diaries, KG, palace graph, coordination log), what a central palace buys a fleet — awareness, non-repetition of expensive work, and retractions that travel — and a decision flow for drawer vs KG vs event. Six mermaid diagrams, all rendered and inspected as images, not merely parsed: the first pass produced a truncated state label from an HTML entity and a self-loop that drew a meaningless dotted lasso. Validation says "no syntax error"; only looking says "correct". Also: a Documentation table in the top-level README so all of the above is reachable from the front door. Deliberately host-agnostic, per synlig-primary-runbook.md's precedent — no device names, hostnames or operator names in either new document. |