diff --git a/CHANGELOG.md b/CHANGELOG.md index 0e83bb7..38d03af 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,8 +13,27 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`). ## Unreleased -_Nothing yet. Entries land here as work merges, and this heading is renamed to -the version at tag time (see AGENTS.md, *Release-day checklist* step 3)._ +**`credential-incident-response` §5/§6 corrected — a stated mechanism was wrong, +and this is the second time in three days this section named a wrong reason +for a zero.** Docs only. + +§5 said `embedding_metadata.string_value` holds "metadata fields only". Measured +false on chroma 1.5.9 with a disposable sentinel drawer (pi@tor-ms22, +2026-08-30): the document text is ALSO there, under key `chroma:document` — one +row in `fts_content` and one in `embedding_metadata` for the same drawer. The +scan order in §5 is unchanged (scan `fts_content` directly, raw bytes as +backstop) but the stated REASON is fixed: a zero from `string_value` needs a +different explanation (key filter, query shape, escaping), not "it's +structurally blind". §6 already warns against explaining a zero with an +unverified mechanism; this was exactly that failure, in the file that carries +the warning. + +§6's row-gone/bytes-gone claim is now backed by the same sentinel measurement +rather than asserted: `delete_by_source` took both `fts_content` (1->0) and +`embedding_metadata` (1->0) to zero, while raw bytes stayed 4->4 until VACUUM. +Also records how the measurement got unblocked at all — not a better +instrument, a disposable sentinel drawer instead of testing deletion on real +data. --- diff --git a/rootfs/usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md b/rootfs/usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md index fb066fb..e6a6858 100644 --- a/rootfs/usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md +++ b/rootfs/usr/local/share/pi-devbox/skills/credential-incident-response/SKILL.md @@ -116,14 +116,24 @@ shared palace as incident response. High blast radius, low actual benefit. ## 5. Finding a secret in a Chroma palace — three targets, in this order -1. `embedding_fulltext_search_content.c0` — **where document text actually is** -2. `embedding_metadata.string_value` — metadata fields only +1. `embedding_fulltext_search_content.c0` — document text +2. `embedding_metadata.string_value` — metadata fields, **and a second copy of + the document text** under key `chroma:document` 3. raw byte scan of every `*.sqlite3` — backstop, covers FTS pages and free space -Scanning only (2) is the classic false clean: hundreds of thousands of rows, -zero hits, and the secret sitting in (1) the whole time. Semantic search proves -nothing about absence — it returns top-k. For completeness, enumerate by filing -window (`list_drawers(since=T, before=T+1m)`), since one mine shares a minute. +**Correction, measured on chroma 1.5.9 with a sentinel drawer:** one row in (1) +AND one row in (2) for the same drawer, so **(2) is not structurally +content-blind** — an earlier version of this section said it held "metadata +fields only", and that was wrong. Scan (1) and (3) regardless: (1) is the direct +target. But if a `string_value` query returns zero for a value you know is in a +drawer, the cause is a key filter, a query shape or escaping — *not* structural +absence, and the difference matters because the false explanation is what makes +the zero feel safe. See §6: do not explain a zero with a mechanism you have not +read from source. + +Semantic search proves nothing about absence — it returns top-k. For +completeness, enumerate by filing window (`list_drawers(since=T, before=T+1m)`), +since one mine shares a minute. Value-agnostic sweeps (uuid / 40-hex / `NAME=VALUE`) drown in false positives at fleet scale — 608 candidates, mostly session UUIDs and git SHAs. Name-anchoring @@ -199,10 +209,16 @@ extractor and making it look as strong as the union — a self-test artifact tha has already fooled an agent here. And never gate on `$?` when the tool has a lock-skip or no-op path that also exits 0; judge the reported line. -**Row-gone is not bytes-gone.** A correct sqlite `DELETE` leaves the payload in -freelist pages until `VACUUM`, so deletion effectiveness is *two* numbers: rows -removed, and a raw byte scan of the `.sqlite3`. One aggregate figure reported as -"erased" has only measured "unretrievable". +**Row-gone is not bytes-gone.** Measured, same sentinel drawer: after +`delete_by_source` the row count went 1 -> 0 in *both* the FTS content table and +`embedding_metadata`, while the raw byte count stayed 4 -> 4 — sqlite does not +zero freed pages, so the payload sits in free space until `VACUUM`. Deletion +effectiveness is therefore *two* numbers, and each direction has a trap: one +aggregate figure reported as "erased" has only measured "unretrievable", while a +raw byte scan used as the acceptance gate reads a CORRECT, complete deletion as a +failure. (Note how this was measured: the blocker was never a better instrument, +it was the subject — file your own disposable sentinel and delete that, instead +of testing deletion on real data.) ## 7. Choosing scopes: derive them from measured consumers