skills: correct the credential-incident-response \u00a75 premise about chroma metadata
\u00a75 said embedding_metadata.string_value holds "metadata fields only". False, measured directly: chroma also stores a copy of the document text there, under key chroma:document. Confirmed with a disposable sentinel drawer (pi@tor-ms22, 2026-08-30): one row in fts_content AND one row in embedding_metadata for the same drawer. This was a real mistake in shipped guidance, not a nitpick: this section's own scanning advice was written to guard against explaining a zero with a mechanism nobody verified from source, and the section itself did exactly that -- I downgraded a census to "a floor" on the strength of a metadata-blind claim I never checked against chroma's actual storage layout. The practical scan order is unchanged (fts_content is still the direct target, raw bytes are still the backstop); only the stated REASON for a metadata zero changes: it needs a different explanation now (key filter, query shape, escaping), not "structurally absent". \u00a76's row-gone/bytes-gone claim is upgraded from asserted to measured, same sentinel: delete_by_source took both fts_content and embedding_metadata 1->0, raw bytes stayed 4->4 (freed pages persist until VACUUM). Also records the method that unblocked the measurement: not a better instrument, a disposable sentinel drawer instead of risking real fleet data. No image behaviour changes.
This commit is contained in:
+21
-2
@@ -13,8 +13,27 @@ Pre-v1.0.0 tags followed the pi npm version (`v{pi_version}[letter]`).
|
|||||||
|
|
||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
_Nothing yet. Entries land here as work merges, and this heading is renamed to
|
**`credential-incident-response` §5/§6 corrected — a stated mechanism was wrong,
|
||||||
the version at tag time (see AGENTS.md, *Release-day checklist* step 3)._
|
and this is the second time in three days this section named a wrong reason
|
||||||
|
for a zero.** Docs only.
|
||||||
|
|
||||||
|
§5 said `embedding_metadata.string_value` holds "metadata fields only". Measured
|
||||||
|
false on chroma 1.5.9 with a disposable sentinel drawer (pi@tor-ms22,
|
||||||
|
2026-08-30): the document text is ALSO there, under key `chroma:document` — one
|
||||||
|
row in `fts_content` and one in `embedding_metadata` for the same drawer. The
|
||||||
|
scan order in §5 is unchanged (scan `fts_content` directly, raw bytes as
|
||||||
|
backstop) but the stated REASON is fixed: a zero from `string_value` needs a
|
||||||
|
different explanation (key filter, query shape, escaping), not "it's
|
||||||
|
structurally blind". §6 already warns against explaining a zero with an
|
||||||
|
unverified mechanism; this was exactly that failure, in the file that carries
|
||||||
|
the warning.
|
||||||
|
|
||||||
|
§6's row-gone/bytes-gone claim is now backed by the same sentinel measurement
|
||||||
|
rather than asserted: `delete_by_source` took both `fts_content` (1->0) and
|
||||||
|
`embedding_metadata` (1->0) to zero, while raw bytes stayed 4->4 until VACUUM.
|
||||||
|
Also records how the measurement got unblocked at all — not a better
|
||||||
|
instrument, a disposable sentinel drawer instead of testing deletion on real
|
||||||
|
data.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -116,14 +116,24 @@ shared palace as incident response. High blast radius, low actual benefit.
|
|||||||
|
|
||||||
## 5. Finding a secret in a Chroma palace — three targets, in this order
|
## 5. Finding a secret in a Chroma palace — three targets, in this order
|
||||||
|
|
||||||
1. `embedding_fulltext_search_content.c0` — **where document text actually is**
|
1. `embedding_fulltext_search_content.c0` — document text
|
||||||
2. `embedding_metadata.string_value` — metadata fields only
|
2. `embedding_metadata.string_value` — metadata fields, **and a second copy of
|
||||||
|
the document text** under key `chroma:document`
|
||||||
3. raw byte scan of every `*.sqlite3` — backstop, covers FTS pages and free space
|
3. raw byte scan of every `*.sqlite3` — backstop, covers FTS pages and free space
|
||||||
|
|
||||||
Scanning only (2) is the classic false clean: hundreds of thousands of rows,
|
**Correction, measured on chroma 1.5.9 with a sentinel drawer:** one row in (1)
|
||||||
zero hits, and the secret sitting in (1) the whole time. Semantic search proves
|
AND one row in (2) for the same drawer, so **(2) is not structurally
|
||||||
nothing about absence — it returns top-k. For completeness, enumerate by filing
|
content-blind** — an earlier version of this section said it held "metadata
|
||||||
window (`list_drawers(since=T, before=T+1m)`), since one mine shares a minute.
|
fields only", and that was wrong. Scan (1) and (3) regardless: (1) is the direct
|
||||||
|
target. But if a `string_value` query returns zero for a value you know is in a
|
||||||
|
drawer, the cause is a key filter, a query shape or escaping — *not* structural
|
||||||
|
absence, and the difference matters because the false explanation is what makes
|
||||||
|
the zero feel safe. See §6: do not explain a zero with a mechanism you have not
|
||||||
|
read from source.
|
||||||
|
|
||||||
|
Semantic search proves nothing about absence — it returns top-k. For
|
||||||
|
completeness, enumerate by filing window (`list_drawers(since=T, before=T+1m)`),
|
||||||
|
since one mine shares a minute.
|
||||||
|
|
||||||
Value-agnostic sweeps (uuid / 40-hex / `NAME=VALUE`) drown in false positives at
|
Value-agnostic sweeps (uuid / 40-hex / `NAME=VALUE`) drown in false positives at
|
||||||
fleet scale — 608 candidates, mostly session UUIDs and git SHAs. Name-anchoring
|
fleet scale — 608 candidates, mostly session UUIDs and git SHAs. Name-anchoring
|
||||||
@@ -199,10 +209,16 @@ extractor and making it look as strong as the union — a self-test artifact tha
|
|||||||
has already fooled an agent here. And never gate on `$?` when the tool has a
|
has already fooled an agent here. And never gate on `$?` when the tool has a
|
||||||
lock-skip or no-op path that also exits 0; judge the reported line.
|
lock-skip or no-op path that also exits 0; judge the reported line.
|
||||||
|
|
||||||
**Row-gone is not bytes-gone.** A correct sqlite `DELETE` leaves the payload in
|
**Row-gone is not bytes-gone.** Measured, same sentinel drawer: after
|
||||||
freelist pages until `VACUUM`, so deletion effectiveness is *two* numbers: rows
|
`delete_by_source` the row count went 1 -> 0 in *both* the FTS content table and
|
||||||
removed, and a raw byte scan of the `.sqlite3`. One aggregate figure reported as
|
`embedding_metadata`, while the raw byte count stayed 4 -> 4 — sqlite does not
|
||||||
"erased" has only measured "unretrievable".
|
zero freed pages, so the payload sits in free space until `VACUUM`. Deletion
|
||||||
|
effectiveness is therefore *two* numbers, and each direction has a trap: one
|
||||||
|
aggregate figure reported as "erased" has only measured "unretrievable", while a
|
||||||
|
raw byte scan used as the acceptance gate reads a CORRECT, complete deletion as a
|
||||||
|
failure. (Note how this was measured: the blocker was never a better instrument,
|
||||||
|
it was the subject — file your own disposable sentinel and delete that, instead
|
||||||
|
of testing deletion on real data.)
|
||||||
|
|
||||||
## 7. Choosing scopes: derive them from measured consumers
|
## 7. Choosing scopes: derive them from measured consumers
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user