provenance: stamp device+harness at the edge, not in the agent's head

RFC 001 §7.3.2 ranks "agent stamps provenance via a skill instruction" as the
❌ worst possible place — per-call boilerplate, forgettable, improvisable. It
was right, and we had shipped exactly that: the mempalace skill told the agent
to pass added_by="<harness>@<device>" by hand. Measured on the shared palace,
199 rows had reached it unresolvable, 10 of them filed by the very agent that
wrote the instruction, in a drawer about host provenance. The trigger was a
cross-host misattribution: a session on tor-ms22 read its own diary, could not
tell that the entries were written on EMB-7KJ4VR4G, and reported another
machine's verification as this one's.

Move the same convention into the ⚠️ edge row, where it is uniform and
unforgettable (§7.3.5):

* extensions/pi/mempalace.ts defaults the writer field on every tool that has
  one — added_by (add_drawer, checkpoint), agent (mine), from_agent
  (event_append), created_by (artifact_put) — from $MEMPALACE_PI_DEVICE. An
  explicit value always wins, so filing for another device stays possible. The
  allowlist is per tool, never blanket: 3.8.0's dispatcher hard-rejects
  undeclared args with -32602, so injecting added_by into diary_write or kg_add
  (which have no such property) would break the call outright.
* mine gets miner@<device> when the caller invokes it, but <harness>@<device>
  for the bridge's own transcript feed — bulk extraction is not agent-authored
  memory, and that keeps the pi/opencode/miner taxonomy honest.
* diary_write has no metadata slot at all, and the device must never go in
  agent_name (wing = f"wing_{agent_name}" would splinter the diary per host).
  So the entry TEXT carries an AAAK field, HOST:<device>|SESSION:… — which is
  also the only channel a READER sees: search projects a fixed key set and
  diary_read returns content, so no metadata fix, not even a
  server-authoritative one, would have prevented the misattribution.
* The wake-up block now states the device and warns that diary_read interleaves
  every machine's diary.
* R1: doubly gated on MEMPALACE_PI_DEVICE and MEMPALACE_REMOTE_URL, so a
  solitary devbox stamps nothing and behaves exactly as before — which is also
  the correct semantics per §7.3.3.

Version the reconciler that was living only on synlig (bin/ + contrib/systemd/),
add --dry-run, and teach it two new rules: diary_host_marker reads the HOST:
field, and sibling_chunk propagates a resolved origin across a drawer's chunks
(a text marker lands in chunk 0 only, so a 5-chunk diary entry would otherwise
stamp 1 and leave 4 blank).

--dry-run against the real palace before deploying earned its keep twice, and
scripts/test-device-stamp.sh pins both findings with the strings it found:
HOST: was ALREADY in use with a composite grammar
(HOST:emb-7kj4vr4g.f1d3c3f89e3e.v1.8.3.pi0.84.2) and for bare container ids, so
an unvalidated rule invented devices like "f1d3c3f89e3e.pi0.84.2"; and HOST:
also carries a different SENSE elsewhere (HOST:exec.via.ssh-controlmaster->…,
meaning where I was executing). Validating against the known-device set both
refuses those and recovers the composite entries correctly. A marker convention
inherits every prior meaning of its own name.

Deployed and verified on synlig: device 14,217 → 14,317, integrity ok,
idempotent on immediate re-run, no invented device values.

RFC updates: §7.3.1 corrected (the arg whitelist is a hard -32602 in 3.8.0, not
a silent drop; get_drawer DOES return metadata, search structurally cannot;
triples and logstream live in separate databases the stamper cannot reach),
§7.3.5 added (what is deployed, including the divergence from §7.3.4's opaque
origin_device — tor-ms22 vs tor-ms22-native is that cost already visible), and
Phase 4 now carries per-device tokens motivated FIRST by revocation, with the
finding that tokens are the cheap half: core holds one scalar auth_token and has
zero device concept, so authoritative stamping needs a component we own.
This commit is contained in:
pi
2026-08-25 22:26:46 +02:00
parent 0fe64c480e
commit 553d86570c
6 changed files with 600 additions and 20 deletions
+206
View File
@@ -0,0 +1,206 @@
#!/usr/bin/env python3
"""Stamp provenance onto MemPalace drawers: `device` (which machine) and
`agent_kind` (which harness: pi / opencode / miner / cli), plus a *_source key
recording HOW each was determined so an inference is never mistaken for a fact.
Why this exists: mempalace core (3.7.1 through 3.8.0) records neither, and the
single shared bearer token means the server cannot tell clients apart either --
core's `auth_token` is one scalar string (mcp_server.py:7685) and the package
contains zero occurrences of any device/origin concept. The feeder does encode
both facts incidentally -- device in the per-device inbox directory, harness in
the transcript filename prefix (pi_*.jsonl) -- so this recovers them.
Idempotent: only fills keys that are absent, so re-running is cheap and never
downgrades an authoritative value. It must run on a timer, not once: live
re-mining REPLACES metadata rows and silently drops earlier stamps.
RFC 001 7.3.2 places provenance at the sync boundary, never on the agent. As of
2026-08-25 the pi bridge (extensions/pi/mempalace.ts) implements the interim
"edge" row: it defaults added_by/agent/from_agent/created_by to
`<harness>@<device>` from $MEMPALACE_PI_DEVICE, so rule `agent_at_device` below
reads both directly instead of inferring them. This script's remaining job is
(a) historic rows, (b) rows re-mined after a stamp, and (c) diary entries, which
have no writer field at all -- see `diary_host_marker`.
These stamps are self-asserted and therefore ADVISORY (7.3.2 Phase 2): a hint,
never load-bearing for authorization or destructive scoping. When per-device
tokens land (Phase 4) the primary can stamp authoritatively and overwrite with
device_source='token'; the *_source key is what makes that upgrade lossless.
Usage: mempalace-device-stamp [--dry-run]
--dry-run report what would be stamped, write nothing. Use this after
changing any rule below -- the palace is shared by the whole fleet.
"""
import sqlite3, os, re, sys, collections
DRY_RUN = '--dry-run' in sys.argv[1:]
FEED = os.path.expanduser('~/mempalace-feed')
DB = os.path.expanduser('~/.mempalace/palace/chroma.sqlite3')
HOSTPATH = [('/Users/ECSJPER/', 'emb-7kj4vr4g'), ('/Users/joakim/', 'mbp-m1-2020'),
('/home/jper/', 'tor-ms22'), ('/home/ecsjper/src', 'synlig')]
DEV_FRAG = [('emb7kj', 'emb-7kj4vr4g'), ('mbp', 'mbp-m1-2020'), ('tor-ms22', 'tor-ms22')]
KNOWN_DEV = set(os.listdir(FEED)) if os.path.isdir(FEED) else set()
base2dev, dupes = {}, set()
if os.path.isdir(FEED):
for dev in sorted(KNOWN_DEV):
d = os.path.join(FEED, dev)
if not os.path.isdir(d): continue
for f in os.listdir(d):
if f in base2dev and base2dev[f] != dev: dupes.add(f)
base2dev[f] = dev
def split_agent(who):
"""'pi@emb-7kj4vr4g' -> ('pi', 'emb-7kj4vr4g'); the forward convention."""
if who and '@' in who:
h, d = who.split('@', 1)
if d in KNOWN_DEV or re.match(r'^[a-z0-9][a-z0-9.-]*$', d or ''):
return h.strip().lower() or None, d.strip().lower() or None
return None, None
def resolve_device(sf, added_by, agent, legacy_machine, doc=None):
for who in (added_by, agent):
_, d = split_agent(who)
if d: return d, 'agent_at_device'
if sf:
if '/mempalace-feed/' in sf:
return sf.split('/mempalace-feed/', 1)[1].split('/')[0], 'inbox_path'
m = re.match(r'^([a-z0-9][a-z0-9-]*):', sf)
if m and (m.group(1) in KNOWN_DEV): return m.group(1), 'source_prefix'
b = os.path.basename(sf)
if b in base2dev and b not in dupes: return base2dev[b], 'basename_index'
for pref, dev in HOSTPATH:
if sf.startswith(pref): return dev, 'host_path_heuristic'
if legacy_machine:
return legacy_machine.strip().lower(), 'legacy_source_machine'
if doc:
# Diary entries have NO writer field: diary_write exposes only
# agent_name, and a device there would become the wing name
# (wing = f"wing_{agent_name}"), splitting the diary per host and
# hiding entries from diary_read. So the pi bridge stamps the device
# into the entry TEXT as an AAAK field -- `HOST:<device>|SESSION:...`.
# That is also the only channel a *reader* ever sees, since search
# projects a fixed key set and diary_read returns content, never
# metadata. Anchored to a field boundary so it cannot match prose.
#
# MUST validate against KNOWN_DEV, for two reasons found by dry-running
# this rule against the real palace before deploying it:
# 1. `HOST:` was ALREADY in use in older diary entries with a richer
# grammar -- `HOST:emb-7kj4vr4g.f1d3c3f89e3e.v1.8.3.pi0.84.2`
# (host.container.image.pi) and bare container ids like
# `HOST:2efe2b06f480`. Taking the whole match would invent devices
# like "f1d3c3f89e3e.pi0.84.2" and fragment every device query, so
# read the first dotted segment and keep it only if it is a real
# device. A container id resolves to nothing, which is correct:
# containers are not devices and change on every recreate.
# 2. `HOST:` also carries a DIFFERENT SENSE in some entries -- e.g.
# `HOST:exec.via.ssh-controlmaster->alpserv-2(...)` means "the box I
# was executing on", not "the box that wrote this". Validation
# rejects it, so the two senses cannot be conflated.
m = re.search(r'(?:^|\|)\s*HOST:\s*([a-z0-9][a-z0-9._-]*)', doc, re.I)
if m:
cand = m.group(1).strip().lower().rstrip('.')
for probe in (cand, cand.split('.')[0]):
if probe in KNOWN_DEV:
return probe, 'diary_host_marker'
for who in (added_by, agent):
if who:
for frag, dev in DEV_FRAG:
if frag in who.lower(): return dev, 'agent_name_heuristic'
return None, None
def resolve_kind(sf, added_by, agent, ingest_mode, doc=None):
for who in (added_by, agent):
h, _ = split_agent(who)
if h: return h, 'agent_at_device'
if sf and '/mempalace-feed/' in sf:
# Filename shape identifies the harness, but the two shapes differ and a
# naive "text before the first _" is WRONG for opencode:
# pi_<uuid>.jsonl -> pi
# <slug>_ses_<id>.jsonl -> opencode (leading segment is an arbitrary slug)
b = os.path.basename(sf)
if re.match(r'^pi_', b): return 'pi', 'transcript_prefix'
if '_ses_' in b: return 'opencode', 'transcript_prefix'
if doc:
# Each session's first chunk carries a synthetic header marker.
m = re.search(r'\|\s*source:\s*(pi|opencode)\b', doc)
if m: return m.group(1).lower(), 'header_marker'
for who in (agent, added_by):
if not who: continue
w = who.lower()
if 'opencode' in w: return 'opencode', 'agent_name'
if w == 'pi' or w.startswith('pi-') or w.startswith('pi_') or 'pi-devbox' in w:
return 'pi', 'agent_name'
if w in ('mcp', 'checkpoint'): return 'pi', 'agent_name_weak'
if w == 'mempalace': return 'miner', 'added_by_default'
if ingest_mode: return 'miner', 'ingest_mode'
return None, None
c = sqlite3.connect(DB, timeout=60)
c.execute('pragma busy_timeout=60000')
meta = collections.defaultdict(dict)
for rid, k, sv in c.execute(
"select id,key,string_value from embedding_metadata where key in "
"('source_file','added_by','agent','device','agent_kind','ingest_mode','source_machine',"
"'parent_drawer_id','chroma:document')"):
meta[rid][k] = sv
dev_rows, kind_rows = [], []
for rid, m in meta.items():
if not m.get('device'):
d, how = resolve_device(m.get('source_file'), m.get('added_by'), m.get('agent'),
m.get('source_machine'), m.get('chroma:document'))
if d: dev_rows.append((rid, d, how))
if not m.get('agent_kind'):
k, how = resolve_kind(m.get('source_file'), m.get('added_by'), m.get('agent'),
m.get('ingest_mode'), m.get('chroma:document'))
if k: kind_rows.append((rid, k, how))
# --- Propagate within a drawer -------------------------------------------
# Every chunk of one drawer was produced by ONE write call, so they share an
# origin by construction. Without this, any doc-text rule (HOST: marker, header
# marker) resolves only the chunk that happens to contain the marker -- a
# 5-chunk diary entry would end up 1 stamped and 4 blank. Fill-only: never
# overrides a row that resolved on its own evidence, and only propagates when
# the drawer's known chunks agree.
def propagate(rows, key):
known = {rid: v for rid, v, _ in rows}
for rid, m in meta.items():
if m.get(key) and rid not in known:
known[rid] = m[key] # already-stamped siblings are evidence too
by_drawer = collections.defaultdict(set)
for rid, m in meta.items():
p = m.get('parent_drawer_id')
if p and rid in known: by_drawer[p].add(known[rid])
added = []
for rid, m in meta.items():
p = m.get('parent_drawer_id')
if not p or rid in known or m.get(key): continue
vals = by_drawer.get(p)
if vals and len(vals) == 1: # unanimous, else leave blank
added.append((rid, next(iter(vals)), 'sibling_chunk'))
return rows + added
dev_rows = propagate(dev_rows, 'device')
kind_rows = propagate(kind_rows, 'agent_kind')
cur = c.cursor()
def put(rows, key, skey):
if not rows or DRY_RUN: return
cur.executemany("insert or replace into embedding_metadata (id,key,string_value) values (?,?,?)",
[(r, key, v) for r, v, _ in rows])
cur.executemany("insert or replace into embedding_metadata (id,key,string_value) values (?,?,?)",
[(r, skey, h) for r, _, h in rows])
put(dev_rows, 'device', 'device_source')
put(kind_rows, 'agent_kind', 'agent_kind_source')
if not DRY_RUN: c.commit()
tot = lambda k: c.execute("select count(*) from embedding_metadata where key=?", (k,)).fetchone()[0]
print("%sdevice+%d %s | agent_kind+%d %s | totals device=%d agent_kind=%d" % (
"DRY RUN would stamp " if DRY_RUN else "stamped ",
len(dev_rows), dict(collections.Counter(h for _, _, h in dev_rows)) or {},
len(kind_rows), dict(collections.Counter(v for _, v, _ in kind_rows)) or {},
tot('device'), tot('agent_kind')))
if DRY_RUN:
print(" by new rule:", dict(collections.Counter(
"%s->%s" % (how, v) for _, v, how in dev_rows
if how in ('diary_host_marker', 'sibling_chunk', 'agent_at_device'))))
@@ -0,0 +1,8 @@
[Unit]
Description=Stamp device provenance onto MemPalace drawers (fills what core 3.7.1 does not record)
After=mempalace-serve.service
[Service]
Type=oneshot
ExecStart=%h/.local/bin/mempalace-device-stamp
Nice=10
@@ -0,0 +1,10 @@
[Unit]
Description=Hourly device-provenance stamp for MemPalace
[Timer]
OnCalendar=hourly
RandomizedDelaySec=300
Persistent=true
[Install]
WantedBy=timers.target
+161 -18
View File
@@ -611,7 +611,8 @@ natural questions:
`extensions/pi/mempalace.ts` **never sets it** for `add_drawer`/`checkpoint` — it sets identity only for `extensions/pi/mempalace.ts` **never sets it** for `add_drawer`/`checkpoint` — it sets identity only for
diaries (`agent_name` from `$MEMPALACE_AGENT_NAME`, default `pi`, `:758`). `kg_add` has no attribution diaries (`agent_name` from `$MEMPALACE_AGENT_NAME`, default `pi`, `:758`). `kg_add` has no attribution
field at all. So the *only* real harness attribution today is the diary wing, and for drawers the value field at all. So the *only* real harness attribution today is the diary wing, and for drawers the value
is whatever string an LLM happened to pass. is whatever string an LLM happened to pass. **✅ Fixed 2026-08-25: the bridge now defaults the writer
field on every write path that has one, and marks diary entries in-text — §7.3.5.**
**Design consequence: device + agent, never session.** Provenance has exactly three consumers — poisoning **Design consequence: device + agent, never session.** Provenance has exactly three consumers — poisoning
triage ("which box planted this?"), per-device high-water marks, and revocation — and none of them needs triage ("which box planted this?"), per-device high-water marks, and revocation — and none of them needs
@@ -627,18 +628,43 @@ was wrong on both counts.** See §7.3.3.
#### 7.3.1 What can actually be stamped (mechanics) #### 7.3.1 What can actually be stamped (mechanics)
The metadata schema is **fixed** — there is no free-form field — and `tools/call` **whitelists arguments The metadata schema is **fixed** — there is no free-form field — and `tools/call` **whitelists arguments
to declared schema properties** (`mcp_server.py:4777`, *"Prevents callers from spoofing internal params to declared schema properties** (*"Prevents callers from spoofing internal params like
like added_by/source_file"*), so an extra `origin_host=…` is **silently dropped, not rejected**. added_by/source_file"*), so an extra `origin_host=…` never reaches storage.
> **Corrected 2026-08-25 against 3.8.0:** that whitelist is no longer a silent drop. `mcp_server.py:6440`
> now returns a hard JSON-RPC error — `-32602 Unknown parameter '<k>' for tool <name>` — for any argument
> outside the tool's declared properties. Practical consequence for any stamper: **injection must be
> allowlisted per tool, never blanket.** Adding `added_by` to a `diary_write` or `kg_add` call no longer
> gets quietly ignored; it fails the whole call.
| Surface | Provenance slot | Notes | | Surface | Provenance slot | Notes |
| --- | --- | --- | | --- | --- | --- |
| `add_drawer`, `checkpoint` | **`added_by`** | The only one. Free-form (`strip_lone_surrogates` only, *not* `sanitize_name`, so `/` and `@` are legal). | | `add_drawer`, `checkpoint` | **`added_by`** | The only one. Free-form (`strip_lone_surrogates` only, *not* `sanitize_name`, so `/` and `@` are legal). `checkpoint` resolves explicit arg → `diary.agent_name` → literal `"checkpoint"`, so **omitting it yields a permanently unattributable drawer** — no `@`, no source path, no rule can recover it later. |
| `diary_write` | **none usable** | ⚠️ **Never** put a device in `agent_name`: `:3504` does `wing = f"wing_{agent_name}"` → a separate wing per host, and `diary_read` filters `{"agent": agent_name}` (`:3636`) → `diary_read("pi")` then **misses** those entries. | | `mine` | **`agent`** | Reaches `added_by` on every filed drawer (`miner.py:1432`, `convo_miner.py:129,248`). Defaults to `"mempalace"`. |
| `kg_add` | **none** | Only `source_file`/`source_closet`/`source_drawer_id`. Origin is inferable only via `source_drawer_id` → that drawer's `added_by`. | | `event_append`, `artifact_put` | **`from_agent` / `created_by`, plus a free-form `metadata` dict stored verbatim** | The best-provisioned surfaces in the whole API — and they also carry their own `origin_replica` column. If a future record type needs structured provenance, this is the shape to copy. |
| `diary_write` | **none in metadata — the entry TEXT is the only channel** | ⚠️ **Never** put a device in `agent_name`: `wing = f"wing_{agent_name}"` → a separate wing per host, and `diary_read` filters `{"agent": agent_name}` → `diary_read("pi")` then **misses** those entries. `sanitize_name` does *not* block `@`, so this fails silently rather than loudly. Since 2026-08-25 the edge instead prefixes the entry with an AAAK field, `HOST:<device>|SESSION:…` (§7.3.5). |
| `kg_add` | **none** | Only `source_file`/`source_closet`/`source_drawer_id`. Origin is inferable only via `source_drawer_id` → that drawer's `added_by`, so **always pass `source_drawer_id`**. |
`added_by` is also **write-only today**: absent from `tool_search` results, surfacing only via **Metadata is write-only to *readers*, and that asymmetry decides where provenance must live.** `added_by`
`get_drawer` → `_drawer_payload` metadata. → Upstream asks: **surface `added_by` in search results**, and and any stamped `device` **are** returned by `get_drawer` — `_response_safe_meta`/`_safe_meta`
**give `kg_add` a provenance field**. (`mcp_server.py:1688-1707`) is a `None`-coercion guard, *not* a key filter, so an earlier claim that the
read path strips provenance was wrong. But `search` results are assembled from a **fixed key list**
(`searcher.py:1822-1839`: drawer_id, text, wing, room, source_file, source_path, created_at, authored_at,
similarity, distance, effective_distance, closet_boost, matched_via) and `diary_read` returns content —
so **no amount of correct metadata is visible to an agent doing a search or reading its own diary.**
That is not a cosmetic gap: it is exactly how the 2026-08-25 cross-host misattribution happened
(drawer_pi-devbox_landmines_fa0002a4). Metadata serves the three consumers below; **reader-visible
attribution needs the record's own text.** → Upstream asks: **surface `added_by`/`device` in search
results**, **give `kg_add` a provenance field**, and **give `diary_write` an `added_by` that does not feed
the wing name**.
**Triples and coordination events live in different databases, which bounds any stamper's reach.**
`knowledge_graph.sqlite3` (`triples`: id, subject, predicate, object, valid_from, valid_to, confidence,
source_closet, source_file, source_drawer_id, adapter_name, extracted_at) and `logstream.sqlite3`
(`events`, `artifacts`) sit beside `chroma.sqlite3` in the palace directory. A stamper written against
Chroma's `embedding_metadata` — which is what `bin/mempalace-device-stamp` is — **cannot reach them at
all.** As of 2026-08-25 that is 156 triples and 11 events/3 artifacts, i.e. small enough to leave, but it
must be stated rather than assumed: *the palace is three stores, and provenance coverage is per store.*
#### 7.3.2 Who should stamp it — a ladder of trust #### 7.3.2 Who should stamp it — a ladder of trust
@@ -648,24 +674,53 @@ write it. Ranked by trustworthiness:
| Stamper | Knows the device? | Verifiable? | Uniform? | Verdict | | Stamper | Knows the device? | Verifiable? | Uniform? | Verdict |
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
| **Agent (via skill)** | No — must shell out to read env | No | No — per-call boilerplate, forgettable, improvisable | ❌ **Worst possible place.** Rejected. | | **Agent (via skill)** | No — must shell out to read env | No | No — per-call boilerplate, forgettable, improvisable | ❌ **Worst possible place.** Rejected. |
| **Client / `mempalace-edge`** | Yes, from host-supplied `.env` | No — self-asserted | Yes — one line in a proxy | ⚠️ Acceptable **interim** | | **Client / `mempalace-edge`** | Yes, from host-supplied `.env` | No — self-asserted | Yes — one line in a proxy | ⚠️ Acceptable **interim** — **implemented 2026-08-25, §7.3.5** |
| **Primary, from the authenticated credential** | Yes | **Yes** — bound to the token | Yes, for every synced record | ✅ **Correct home** | | **Primary, from the authenticated credential** | Yes | **Yes** — bound to the token | Yes, for every synced record | ✅ **Correct home** |
> **The ❌ row was not hypothetical.** Between 2026-08-23 and 08-25 the mempalace skill did instruct the
> agent to pass `added_by="<harness>@<device>"` by hand, and it behaved exactly as this table predicts:
> hand-filed drawers came out as `checkpoint`/`mcp`/`pi` whenever the instruction was not recalled, and
> the agent that *wrote the instruction* then filed its own provenance drawer without it. Measured on
> 2026-08-25: 199 rows the stamper had already seen and could not resolve, of which 10 were that drawer.
> Moving the same one-line convention into the bridge (§7.3.5) removed the failure mode without changing
> the values written — which is the point of the ladder: **the value was never the problem, the writer
> was.**
The decisive point: **a client-asserted origin is a hint, not a fact.** The primary is the only party The decisive point: **a client-asserted origin is a hint, not a fact.** The primary is the only party
that can bind a write to an identity it verified. And under the §4 design *every* write reaches the that can bind a write to an identity it verified. And under the §4 design *every* write reaches the
primary through an authenticated channel — including offline ones, at outbox-flush time — so the server primary through an authenticated channel — including offline ones, at outbox-flush time — so the server
can stamp the complete set without any client cooperation. **Provenance is a property of the sync can stamp the complete set without any client cooperation. **Provenance is a property of the sync
channel, not of the record's author.** channel, not of the record's author.**
Blocker for the ✅ row, verified in 3.6.0: `serve` takes a **single shared bearer token** Blocker for the ✅ row, verified in 3.6.0 and **re-verified in 3.8.0 on 2026-08-25**: `serve` takes a
(`srv.auth_token`, `hmac.compare_digest`, `mcp_server.py:5291-5293`) and the package contains **zero** **single shared bearer token** — one scalar string, `auth_token = os.environ.get("MEMPALACE_MCP_HTTP_TOKEN")`
occurrences of any device/origin concept. Server-side stamping therefore *requires* the per-device compared with one `hmac.compare_digest` (`mcp_server.py:7137-7140`, `:7685`) — and the package still
credentials of §6 — i.e. **Phase 4**, not Phase 1. Hence the phasing: contains **zero** occurrences of any device/origin concept. Two consequences worth stating plainly,
because "just issue per-device tokens now" sounds like a shortcut and is not:
- **Per-device tokens are not a config change.** There is no `token → {device, scopes}` registry to
populate; the code holds one string. Multiple tokens require a code change.
- **Tokens are the cheap half.** Even with N tokens, *nothing would stamp*: an origin field would have to
be added to six write paths (`add_drawer`, `checkpoint`, `diary_write`, `kg_add`, `event_append`,
`artifact_put`) across **three** separate databases (§7.3.1). And `mempalace` is **upstream MIT**
(PyPI `mempalace`, github.com/MemPalace/mempalace) consumed via `uv tool install` — so this is an
upstream PR or a carried fork, on a package that moved 3.7.1→3.8.0 inside one week.
- **The tunnel cannot substitute.** Pangolin *terminates TLS and nothing more* (§6.2), and "a Pangolin
user per container with credentials in each `.env`" was considered and rejected in the exposure runbook
§1.2 — its HTTP auth is browser-shaped (SSO, resource PIN), not client-shaped.
So the realistic ✅ implementation is **a small authenticating rewriter we own in front of the palace**:
it holds `token → {device, label}`, and rewrites the JSON-RPC `params` to set `added_by` from the
*verified* token. That is `mempalace-edge` relocated from the client to the primary. It must track tool
schemas to stay safe (§7.3.1: `-32602` on undeclared args; `diary_write` has no slot), which is precisely
why it is Phase 4 work and not a quick win. Hence the phasing:
- **Phase 2 (edge):** edge may stamp `added_by` from its configured env — self-asserted, **advisory - **Phase 2 (edge):** edge may stamp `added_by` from its configured env — self-asserted, **advisory
only**, never load-bearing for authorization or destructive scoping. only**, never load-bearing for authorization or destructive scoping. **Implemented 2026-08-25 (§7.3.5).**
- **Phase 4 (authz):** per-device tokens land; the primary stamps authoritatively and the client-supplied - **Phase 4 (authz):** per-device tokens land; the primary stamps authoritatively and the client-supplied
value becomes redundant (and must be treated as untrusted input, not merely ignored). value becomes redundant (and must be treated as untrusted input, not merely ignored). The upgrade is
**lossless because every stamp carries a `*_source` key** — an authoritative pass simply overwrites with
`device_source='token'`, so nothing done in Phase 2 has to be undone.
#### 7.3.3 Solitary containers should stamp nothing — and lose nothing by it #### 7.3.3 Solitary containers should stamp nothing — and lose nothing by it
@@ -687,6 +742,9 @@ from *multiple* origins are interleaved in one store, which is exactly and only
actually sets it.** The pi extension leaves `added_by` at its default for `add_drawer`/`checkpoint`, so actually sets it.** The pi extension leaves `added_by` at its default for `add_drawer`/`checkpoint`, so
in practice the value is whatever an LLM passed. The field is the right home; the client-side write that in practice the value is whatever an LLM passed. The field is the right home; the client-side write that
populates it is missing, and it belongs to the edge (the ⚠️ row of §7.3.2) — not to a skill instruction. populates it is missing, and it belongs to the edge (the ⚠️ row of §7.3.2) — not to a skill instruction.
**✅ Closed 2026-08-25: the edge write now exists (§7.3.5), and it carries the device as well as the
harness — `<harness>@<device>` — because in a shared palace the harness axis alone cannot answer "which
box planted this?".**
- **A palace on a shared host bind-mount** (as tor-ms22's compose does) is still single-*device* under - **A palace on a shared host bind-mount** (as tor-ms22's compose does) is still single-*device* under
the "host owns the palace" model, so it too imports as one origin. the "host owns the palace" model, so it too imports as one origin.
@@ -744,6 +802,91 @@ when the label is present — a hostname alone is exactly the colliding, mutable
> and server infrastructure; an agent's contribution to it is to leave the field alone. The mempalace > and server infrastructure; an agent's contribution to it is to leave the field alone. The mempalace
> skill carries a one-line guard to that effect. > skill carries a one-line guard to that effect.
#### 7.3.5 What is actually deployed (2026-08-25) — the interim, end to end
The ⚠️ row of §7.3.2, built after a cross-host misattribution made the cost concrete. Two writers, one
reconciler, and a deliberate divergence from §7.3.4 that is recorded rather than hidden.
**1. The edge stamps, uniformly** — `extensions/pi/mempalace.ts`. Every mempalace tool call passes through
one `execute()` wrapper, so the bridge fills in the writer field the caller omitted:
| Tool | Field defaulted | Value |
| --- | --- | --- |
| `add_drawer`, `checkpoint` | `added_by` | `<harness>@<device>` |
| `mine` (caller-invoked) | `agent` | `miner@<device>` — bulk machine-extracted content is not agent-authored memory, and keeping the harness segment `miner` preserves the pi/opencode/miner taxonomy while still recording the box |
| `mine` (the bridge's own transcript feed) | `agent` | `<harness>@<device>` — these *are* this harness's own sessions |
| `event_append` | `from_agent` | `<harness>@<device>` |
| `artifact_put` | `created_by` | `<harness>@<device>` |
| `diary_write`, `checkpoint.diary` | *entry text* | prefixes `HOST:<device>|` |
| `kg_add` | — | nothing; no slot exists (§7.3.1). Pass `source_drawer_id` instead |
An explicitly supplied value always wins, so filing on behalf of another device stays possible. The
allowlist is per tool, never blanket — §7.3.1's `-32602`. `<harness>` is `$MEMPALACE_AGENT_NAME` (default
`pi`), `<device>` is `$MEMPALACE_PI_DEVICE`, host-supplied per container per §1.2.
**R1 compliance:** the stamping is doubly gated on `MEMPALACE_PI_DEVICE` **and** `MEMPALACE_REMOTE_URL`.
A solitary devbox sets neither, so it stamps nothing and its behaviour is unchanged — which is also the
correct semantics per §7.3.3: a solitary store is single-origin, and origin belongs to the whole palace.
**2. The diary marker is in the TEXT, and that is the point.** `diary_write` has no metadata slot, but the
deeper reason is §7.3.1's read asymmetry: `search` projects a fixed key set and `diary_read` returns
content, so **metadata is invisible to the agent who will later read the entry.** A metadata-only fix,
even a server-authoritative one, would not have prevented the misattribution it was built for. The marker
is an AAAK field (`HOST:tor-ms22|SESSION:2026-08-25|…`), so it is machine-parseable *and* the first thing
a reader sees. The wake-up block now also states the device and warns that `diary_read` interleaves every
machine's diary.
**3. The primary reconciles** — `bin/mempalace-device-stamp` + `contrib/systemd/mempalace-device-stamp.{service,timer}`,
hourly on synlig. It fills `device` and `agent_kind` where absent, each paired with a `*_source` key
recording *how* it was determined, so an inference is never mistaken for a fact. It must stay on a timer,
not run once: **live re-mining replaces metadata rows and silently drops earlier stamps.** Its remaining
job after the edge exists is historic rows, re-mined rows, and diary entries. Coverage on 2026-08-25:
`agent_kind` 28,817/30,661 (94%), `device` 14,157 (46%) — the deficit is dominated by ~16k `/workspace`
project mines that are *deliberately* unattributed, because `/workspace/myconfigs/<host>/…` names the
machine a config **belongs to**, not the machine that mined it, and `/workspace` looks identical on every
devbox. Conflating subject with source would be worse than a blank.
**Two traps found by dry-running the reconciler before deploying it** (`--dry-run` exists for this reason;
the palace is shared, so a wrong rule invents device names that then have to be un-invented across 30k
rows, and `scripts/test-device-stamp.sh` pins both):
- **`HOST:` was already in use, with a different grammar** — older entries carry a composite fingerprint,
`HOST:emb-7kj4vr4g.f1d3c3f89e3e.v1.8.3.pi0.84.2` (device.container.image.pi), and some carry a bare
container id, `HOST:2efe2b06f480`. Taking the match whole would have invented devices like
`f1d3c3f89e3e.pi0.84.2`. The rule therefore validates against the known-device set (the feed inboxes)
and falls back to the first dotted segment — which *recovers* the composite entries correctly and
refuses container ids, exactly as §7.3.4 requires (containers are not devices).
- **`HOST:` also carries a different SENSE** — e.g. `HOST:exec.via.ssh-controlmaster->alpserv-2(…)`,
meaning "the box I was executing on", not "the box that wrote this". Validation rejects it, so the two
senses cannot be conflated. **A marker convention inherits every prior meaning of its own name.**
One further mechanism: chunks. A drawer's chunks are one write call, hence one origin — but a text marker
lands only in the chunk that contains it, so a 5-chunk diary entry would stamp 1 and leave 4 blank. The
reconciler propagates a resolved value across `parent_drawer_id` siblings (`device_source='sibling_chunk'`)
when they agree, and never overrides a row that resolved on its own evidence.
**Divergence from §7.3.4, stated openly.** §7.3 refers to an `<agent>/<label>/<device>` encoding that
§7.3.4 never actually defines — §7.3.4 specifies two *fields* (`origin_device`, an opaque uuid; and
`origin_label`, a hostname for readability only, explicitly "never identity"). What is deployed is
**`<harness>@<label>`**: two segments, hostname-derived, no opaque id. Why, and what it costs:
- There is nowhere to put a second field. `added_by` is one free-form string (§7.3.1), so a uuid could
only be carried *instead of* the readable label, in the one slot that also has to carry the harness.
- A uuid would have to be provisioned and kept in each host's `.env`; the label already exists there as
`MEMPALACE_PI_DEVICE`, is already the feed inbox name, and is therefore already the join key for the
`inbox_path` rule that resolves 13,350 rows.
- **The cost is real and already visible.** §7.3.4 warns that the worse failure of hostname identity is
not collision but *rename*, silently splitting one device's history. The palace holds `tor-ms22` (4,680
rows) **and** `tor-ms22-native` (3,826) — one physical machine, two device values, because the label
encoded an access mode. Under §7.3.4's design those would share an `origin_device` and differ only in
`origin_label`, which is exactly the property we gave up.
So: `<harness>@<label>` is the honest Phase-2 shape given one string to write into, and the `*_source`
keys are what make the Phase-4 upgrade to verified identity lossless. §7.3 should be read as promising
**orthogonality of the harness and device axes**, which `@` delivers — not the three-segment encoding it
appears to cite. → Fix §7.3's forward reference, or define the three-segment form in §7.3.4, when Phase 4
settles which identity the primary issues.
### 7.4 Embedder identity is client-local and currently toothless ### 7.4 Embedder identity is client-local and currently toothless
`check_embedder_identity()` raises `DimensionMismatchError` / `EmbedderIdentityMismatchError`, but the `check_embedder_identity()` raises `DimensionMismatchError` / `EmbedderIdentityMismatchError`, but the
@@ -815,7 +958,7 @@ join replays (§4.4).
| **1.5 — opencode env propagation** | hours | Make the `mcp.mempalace` subtree env-authoritative in `generate-config.py`, gated by a generated-value fingerprint (§4.1). Independent of the rest of this RFC. Without it, adopting *or reverting* the opt-in on an existing opencode container needs a manual sidecar merge or a `docker volume rm` — which also blocks R6 reversibility | | **1.5 — opencode env propagation** | hours | Make the `mcp.mempalace` subtree env-authoritative in `generate-config.py`, gated by a generated-value fingerprint (§4.1). Independent of the rest of this RFC. Without it, adopting *or reverting* the opt-in on an existing opencode container needs a manual sidecar merge or a `docker volume rm` — which also blocks R6 reversibility |
| **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. **Not** "fixes opencode" — opencode's *transport* is already fine after Phase 1; what edge adds there is offline/local-first, since `generate-config.py`'s switch is remote **or** local with no failover. **Ships with the §1.2 opt-in wiring (compose + `.env.example` + a third branch in the existing `generate-config.py`) and must pass the R1 acceptance test.** | | **2 — `mempalace-edge`** | ~1 week | The actual ask: local-first writes + outbox flush + merged reads + per-wing policy. **Not** "fixes opencode" — opencode's *transport* is already fine after Phase 1; what edge adds there is offline/local-first, since `generate-config.py`'s switch is remote **or** local with no failover. **Ships with the §1.2 opt-in wiring (compose + `.env.example` + a third branch in the existing `generate-config.py`) and must pass the R1 acceptance test.** |
| **3 — pull replication** | ~1 week | **DEFERRED (2026-08-08).** Server-side op-log with monotonic seq → each edge a full offline replica. Only needed if a laptop must hold *everything* offline. Accepted trade-off: offline recall = own writes + last-reachable state. | | **3 — pull replication** | ~1 week | **DEFERRED (2026-08-08).** Server-side op-log with monotonic seq → each edge a full offline replica. Only needed if a laptop must hold *everything* offline. Accepted trade-off: offline recall = own writes + last-reachable state. |
| **4 — authz** | days | Per-wing ACL, per-device scopes, audit log, token rotation | | **4 — authz** | days–week | Per-wing ACL, per-device scopes, audit log, token rotation. **Two things fold in here, decided 2026-08-25 (§7.3.2):** (a) **per-device bearer tokens, motivated first by revocation** — today one shared token covers 4+ devices and every container on them, so a single compromised or decommissioned laptop cannot be cut off without rotating the whole fleet, and that is a standing risk *independent* of provenance; (b) **authoritative server-side provenance rides along free once (a) exists** — but only via a component **we** own, because upstream core holds one scalar `auth_token` and has zero device concept, so the shape is a small authenticating rewriter in front of the palace that maps `token → {device, label}` and sets `added_by` from the verified credential. Do **not** sequence (b) before (a): tokens are the cheap half, and stamping without a verified identity is what Phase 2 already does more simply. Prerequisite understanding: §7.3.1 (`-32602` on undeclared args; three separate databases; `diary_write` has no slot) and §7.3.5 (what the interim already covers, and the `*_source` key that makes the upgrade lossless) |
Phase 1 is worth doing on its own — it is pure configuration and immediately ends KG fragmentation Phase 1 is worth doing on its own — it is pure configuration and immediately ends KG fragmentation
for online clients. for online clients.
@@ -826,7 +969,7 @@ for online clients.
| --- | --- | --- | | --- | --- | --- |
| **Primary host** | **synlig** — a VM in Xerces (work OpenStack cloud) | Chosen for always-on + good connectivity, *not* for work/personal reasons. Consequence: replicated content, diaries included, lives on employer infrastructure (§5, §6.1 threat 2) | | **Primary host** | **synlig** — a VM in Xerces (work OpenStack cloud) | Chosen for always-on + good connectivity, *not* for work/personal reasons. Consequence: replicated content, diaries included, lives on employer infrastructure (§5, §6.1 threat 2) |
| **TLS / ingress** | **Pangolin/newt**, one DNS record per service on the web hotel | Not `serve --tls-cert`. See the bind-address/Host-pin coupling in §6.2 | | **TLS / ingress** | **Pangolin/newt**, one DNS record per service on the web hotel | Not `serve --tls-cert`. See the bind-address/Host-pin coupling in §6.2 |
| **Authentication** | **Single shared token** for now | Per-device deferred to Phase 4; `origin_device` stays advisory until then | | **Authentication** | **Single shared token** for now | Per-device deferred to Phase 4; `origin_device` stays advisory until then. **Re-verified 2026-08-25 on 3.8.0:** still one scalar `auth_token` (`mcp_server.py:7685`), so per-device is a code change, not configuration — and the shared token is a **revocation gap**, now recorded as the primary motivation for Phase 4 (§8) rather than a provenance detail |
| **Diaries** | **`replicated`** | Highest-value cross-machine content; accepted placement consequence (§5) | | **Diaries** | **`replicated`** | Highest-value cross-machine content; accepted placement consequence (§5) |
| **Work/personal split** | **per-wing, not per-device** | Rejects two primaries split along machine lines — the device cannot classify the project (§5) | | **Work/personal split** | **per-wing, not per-device** | Rejects two primaries split along machine lines — the device cannot classify the project (§5) |
| **Multi-store** | Possible later, not now | `MEMPALACE_REMOTE_URL` stays a scalar in Phase 1; per-wing targets are an edge-config concern (Phase 2+) | | **Multi-store** | Possible later, not now | `MEMPALACE_REMOTE_URL` stays a scalar in Phase 1; per-wing targets are an edge-config concern (Phase 2+) |
+89 -2
View File
@@ -660,6 +660,80 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
let available = false; let available = false;
const agentName = process.env.MEMPALACE_AGENT_NAME ?? "pi"; const agentName = process.env.MEMPALACE_AGENT_NAME ?? "pi";
// --- Edge provenance (RFC 001 §7.3.2, Phase 2) -------------------------
// Provenance belongs to the sync boundary, not to the agent. §7.3.2 ranks
// "agent via skill" as the ❌ worst possible stamper — per-call boilerplate,
// forgettable, improvisable — and the client/edge as the ⚠️ acceptable
// interim until per-device tokens let the primary stamp authoritatively
// (Phase 4). So the bridge stamps, uniformly, from host-supplied env, and no
// skill instruction or LLM discipline is involved. Being self-asserted it is
// ADVISORY: a hint, never load-bearing for authz or destructive scoping.
//
// R1 (solitary operation stays unchanged) is why this is doubly gated: it is
// inert unless the host both labels the device AND points this client at a
// shared palace. A solitary devbox is single-origin by definition (§7.3.3),
// so it stamps nothing and loses nothing.
const device = process.env.MEMPALACE_PI_DEVICE?.trim();
const shared = Boolean(process.env.MEMPALACE_REMOTE_URL?.trim());
const stampProvenance = device && shared;
// Which parameter carries "who wrote this", per tool. An ALLOWLIST, never a
// blanket default: mempalace 3.8.0's dispatcher rejects undeclared arguments
// with -32602 (mcp_server.py:6440 — it used to drop them silently), so
// injecting into a tool that has no such property would break every call.
// diary_write and kg_add have no writer slot at all and are handled below.
const WRITER_PARAM: Record<string, string> = {
mempalace_add_drawer: "added_by",
mempalace_checkpoint: "added_by",
mempalace_mine: "agent",
mempalace_event_append: "from_agent",
mempalace_artifact_put: "created_by",
};
// `mine` produces bulk machine-extracted content, not agent-authored memory.
// Labelling its harness segment `miner` keeps the existing pi/opencode/miner
// taxonomy honest while still recording which box did the mining.
const MINER_TOOLS = new Set(["mempalace_mine"]);
const identity = (toolName: string) =>
`${MINER_TOOLS.has(toolName) ? "miner" : agentName}@${device}`;
// diary_write has NO usable writer parameter (§7.3.1: "none usable") and the
// device must never go in agent_name — `wing = f"wing_{agent_name}"`, so that
// splinters the diary into one wing per host and hides entries from
// diary_read. The entry TEXT is the only channel left, and it is also the
// only one a *reader* ever sees: search results project a fixed key set and
// diary_read returns content, so neither ever shows metadata. A metadata-only
// fix would not have prevented the 2026-08-25 cross-host misattribution.
// Prefixing (not appending) keeps the marker in chunk 0 and in the reader's
// first line. AAAK is pipe-separated, so `HOST:x|SESSION:…` stays in-dialect.
const HOST_MARKER = /(^|\|)\s*HOST:/;
const markEntry = (v: unknown): unknown =>
typeof v === "string" && v.trim() !== "" && !HOST_MARKER.test(v)
? `HOST:${device}|${v}`
: v;
/** Stamp origin into a tool's arguments in place. Never overwrites a value
* the caller set explicitly — an explicit argument wins, so a deliberate
* re-file on behalf of another device stays possible. */
const applyProvenance = (toolName: string, args: Record<string, unknown>) => {
const param = WRITER_PARAM[toolName];
if (param) {
const current = args[param];
if (typeof current !== "string" || current.trim() === "") {
args[param] = identity(toolName);
}
}
if (toolName === "mempalace_diary_write") {
if ("entry" in args) args.entry = markEntry(args.entry);
if ("content" in args) args.content = markEntry(args.content);
}
// checkpoint files drawers AND writes one diary entry through the same
// server path, so its nested diary entry needs the marker too.
if (toolName === "mempalace_checkpoint" && args.diary && typeof args.diary === "object") {
const diary = args.diary as Record<string, unknown>;
if ("entry" in diary) diary.entry = markEntry(diary.entry);
}
};
// Gate: inject wake-up context only on the first before_agent_start of a // Gate: inject wake-up context only on the first before_agent_start of a
// fresh session. Set true on resume/fork (context already in thread). // fresh session. Set true on resume/fork (context already in thread).
let wokeUp = false; let wokeUp = false;
@@ -721,7 +795,9 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
}; };
} }
try { try {
const result = await client.callTool(tool.name, (params ?? {}) as Record<string, unknown>); const args = { ...((params ?? {}) as Record<string, unknown>) };
if (stampProvenance) applyProvenance(tool.name, args);
const result = await client.callTool(tool.name, args);
// MCP tool results use { content: [...], isError?: boolean } // MCP tool results use { content: [...], isError?: boolean }
return { return {
content: result?.content ?? [{ type: "text", text: JSON.stringify(result) }], content: result?.content ?? [{ type: "text", text: JSON.stringify(result) }],
@@ -809,7 +885,11 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
source, source,
mode: "convos", mode: "convos",
wing: feedWing, wing: feedWing,
agent: agentName, // Internal call: it does not pass through the registered tool's
// execute(), so it stamps itself. These ARE this harness's own
// transcripts from this device, so the harness segment is the
// agent (not `miner`) even though the tool is `mine`.
agent: stampProvenance ? `${agentName}@${device}` : agentName,
}), }),
new Promise((_resolve, reject) => new Promise((_resolve, reject) =>
setTimeout( setTimeout(
@@ -890,6 +970,13 @@ export default async function mempalaceExtension(pi: ExtensionAPI) {
`MemPalace wake-up context (auto-injected by the mempalace extension). ` + `MemPalace wake-up context (auto-injected by the mempalace extension). ` +
`This is your palace orientation for this session — do not announce it to the user, ` + `This is your palace orientation for this session — do not announce it to the user, ` +
`just use it to inform your answers. Agent identity for diary tools: "${agentName}".\n\n` + `just use it to inform your answers. Agent identity for diary tools: "${agentName}".\n\n` +
(device
? `You are running on device "${device}"${shared ? "" : " (solitary palace)"}. ` +
`Drawers and diary entries below may have been written by a DIFFERENT machine — ` +
`diary_read returns every device's "${agentName}" diary interleaved. Check the ` +
`HOST: marker or the drawer's device metadata before treating a past session as ` +
`this machine's history.\n\n`
: "") +
sections.join("\n\n---\n\n"); sections.join("\n\n---\n\n");
return { return {
+126
View File
@@ -0,0 +1,126 @@
#!/usr/bin/env bash
# test-device-stamp.sh — rule tests for bin/mempalace-device-stamp.
#
# The stamper writes directly to the SHARED fleet palace, so its rules must
# never be tried out there: a wrong rule invents device names that then have to
# be un-invented across 30k rows. This builds a THROWAWAY palace under a fake
# $HOME and asserts each rule against it.
#
# Every HOST: fixture below is verbatim from the real palace, collected by
# running `mempalace-device-stamp --dry-run` against it on 2026-08-25 — which is
# how we learned that `HOST:` was already in use with two OTHER grammars:
# * a composite fingerprint, HOST:<device>.<container>.<image>.<pi-version>
# * a different SENSE entirely: HOST:exec.via.ssh-controlmaster->alpserv-2,
# meaning "the box I was executing on", not "the box that wrote this".
# Hence the KNOWN_DEV validation the tests below pin down.
#
# Usage: scripts/test-device-stamp.sh (exit 0 = all rules behave)
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
STAMPER="$REPO_ROOT/bin/mempalace-device-stamp"
FAKE="$(mktemp -d)"
trap 'rm -rf "$FAKE"' EXIT
python3 -m py_compile "$STAMPER"
# Devices exist iff they have a feed inbox — that is what KNOWN_DEV reads.
mkdir -p "$FAKE/.mempalace/palace" \
"$FAKE/mempalace-feed/tor-ms22" \
"$FAKE/mempalace-feed/emb-7kj4vr4g" \
"$FAKE/mempalace-feed/mbp-m1-2020"
FAKE="$FAKE" python3 - <<'PYEOF'
import os, sqlite3
db = os.path.join(os.environ['FAKE'], '.mempalace/palace/chroma.sqlite3')
c = sqlite3.connect(db)
c.execute("create table embedding_metadata (id text, key text, string_value text, primary key(id,key))")
docs = {
# --- the marker the pi bridge writes (extensions/pi/mempalace.ts) --------
'new_convention': 'HOST:tor-ms22|SESSION:2026-08-25|did.things',
# --- the older composite grammar already in the palace ------------------
'composite_v183': 'HOST:emb-7kj4vr4g.f1d3c3f89e3e.v1.8.3.pi0.84.2|SESSION:x',
'composite_om': 'HOST:emb-7kj4vr4g.f1d3c3f89e3e.pi0.84.2.om3.0.4.bedrock|SESSION:x',
'composite_devbox': 'HOST:emb-7kj4vr4g.devbox.pi0.84.2.om.699ccc7|SESSION:x',
# --- must NOT invent a device ------------------------------------------
'container_only': 'HOST:f1d3c3f89e3e.pi0.84.2|SESSION:x', # container id, not a device
'container_bare': 'HOST:2efe2b06f480|SESSION:x', # ditto, and it changes per recreate
'garbage_str': 'HOST:str|SESSION:x',
'other_sense': 'x|HOST:exec.via.ssh-controlmaster->alpserv-2(Alpine,user=joakim)|y',
'prose': 'we discussed the HOST:tor-ms22 marker convention', # not at a field boundary
}
rows = []
for k, doc in docs.items():
rows += [(k, 'chroma:document', doc), (k, 'parent_drawer_id', 'd_' + k), (k, 'added_by', 'pi')]
# Chunk propagation: one write call => one origin. Marker lands in chunk 0 only.
rows += [('chunk0', 'chroma:document', 'HOST:tor-ms22|SESSION:y'), ('chunk0', 'parent_drawer_id', 'multi')]
rows += [('chunk1', 'chroma:document', 'continuation, no marker'), ('chunk1', 'parent_drawer_id', 'multi')]
rows += [('chunk2', 'chroma:document', 'more continuation'), ('chunk2', 'parent_drawer_id', 'multi')]
# An existing stamp must survive untouched and seed its blank sibling.
rows += [('keep0', 'device', 'mbp-m1-2020'), ('keep0', 'device_source', 'inbox_path'),
('keep0', 'parent_drawer_id', 'kept'), ('keep0', 'chroma:document', 'x')]
rows += [('keep1', 'parent_drawer_id', 'kept'), ('keep1', 'chroma:document', 'y')]
# The edge convention on a normal drawer.
rows += [('edge0', 'added_by', 'pi@tor-ms22'), ('edge0', 'chroma:document', 'z'),
('edge0', 'parent_drawer_id', 'edged')]
# Defaults that carry no device: must stay blank rather than guess.
rows += [('dflt0', 'added_by', 'checkpoint'), ('dflt0', 'chroma:document', 'q'),
('dflt0', 'parent_drawer_id', 'dflt')]
c.executemany("insert into embedding_metadata values (?,?,?)", rows)
c.commit()
PYEOF
# --dry-run must write nothing.
HOME="$FAKE" python3 "$STAMPER" --dry-run >/dev/null
if [ -n "$(FAKE="$FAKE" python3 -c "
import os,sqlite3
db=os.path.join(os.environ['FAKE'],'.mempalace/palace/chroma.sqlite3')
print(''.join(r[0] for r in sqlite3.connect(db).execute(
\"select id from embedding_metadata where key='device' and id!='keep0'\")))")" ]; then
echo "FAIL: --dry-run wrote stamps" >&2
exit 1
fi
HOME="$FAKE" python3 "$STAMPER" >/dev/null
SECOND="$(HOME="$FAKE" python3 "$STAMPER")" # idempotence: a second pass adds nothing
FAKE="$FAKE" SECOND="$SECOND" python3 - <<'PYEOF'
import os, sqlite3, sys
db = os.path.join(os.environ['FAKE'], '.mempalace/palace/chroma.sqlite3')
c = sqlite3.connect(db)
got = {r[0]: r[1] for r in c.execute("select id,string_value from embedding_metadata where key='device'")}
src = {r[0]: r[1] for r in c.execute("select id,string_value from embedding_metadata where key='device_source'")}
want = {
'new_convention': 'tor-ms22', 'composite_v183': 'emb-7kj4vr4g',
'composite_om': 'emb-7kj4vr4g', 'composite_devbox': 'emb-7kj4vr4g',
'container_only': None, 'container_bare': None, 'garbage_str': None,
'other_sense': None, 'prose': None,
'chunk0': 'tor-ms22', 'chunk1': 'tor-ms22', 'chunk2': 'tor-ms22',
'keep0': 'mbp-m1-2020', 'keep1': 'mbp-m1-2020',
'edge0': 'tor-ms22', 'dflt0': None,
}
want_src = {
'new_convention': 'diary_host_marker', 'composite_v183': 'diary_host_marker',
'chunk1': 'sibling_chunk', 'chunk2': 'sibling_chunk', 'keep1': 'sibling_chunk',
'keep0': 'inbox_path', # pre-existing stamp not rewritten
'edge0': 'agent_at_device',
}
bad = 0
for k, exp in sorted(want.items()):
act = got.get(k)
if act != exp:
print("FAIL %-17s device expected=%s got=%s" % (k, exp, act)); bad += 1
for k, exp in sorted(want_src.items()):
act = src.get(k)
if act != exp:
print("FAIL %-17s source expected=%s got=%s" % (k, exp, act)); bad += 1
if 'device+0' not in os.environ['SECOND']:
print("FAIL not idempotent, second pass said: %s" % os.environ['SECOND'].strip()); bad += 1
print("device-stamp rules: %s (%d checks)" % ("PASS" if not bad else "%d FAILURES" % bad,
len(want) + len(want_src) + 1))
sys.exit(1 if bad else 0)
PYEOF