skills: fix the half of the negative-result rule that was wrong
pi-devbox-environment already warned that "a negative result is usually your
own filter" — baked, symlinked in at every container start, authored by an
earlier session. It survived every recreate, was available all of a later
session, and was violated five times anyway. So the gap was never persistence.
That section also closed with "a positive result needs no such scepticism — it
carries its own evidence." That is false, and it aimed the scepticism budget
one way only. Three of those five errors were positives:
- an SSH handshake SUCCEEDED and greeted me as joakimp while I believed I was
probing gitea.egl.lan — `Host gitea*` had rewritten HostName
- a 401 that was a genuine answer from an issuer which never minted the token
- a "regression" produced by diffing against a value my own -p 2222 flag set
Adds the three missing false-negative rows, replaces the false claim with the
"a positive result only proves what you actually asked" subsection, and records
the two habits that actually caught these: `ssh -G` to learn which rule
captured a hostname, and declaring the expected result before running a check.
The cross-cutting form goes in pi-global-AGENTS.append.md rather than in a
skill, because it has to fire without a task description matching it — being
loadable on demand is exactly what failed. Across all five errors, none was
caught by re-reading my reasoning; every one was caught by a second measurement
that disagreed.
This commit is contained in:
@@ -70,3 +70,41 @@ rather than merely confusing you:
|
||||
local disk, so `mempalace search` can return older and different results than
|
||||
the MCP tools while both look correct. Use the MCP tools for the central
|
||||
palace; the CLI only for a local one.
|
||||
|
||||
## Before you file a finding: second measurement, different route
|
||||
|
||||
This is here rather than in a skill because it has to fire *without* a matching
|
||||
task description, and because the version of it that lived only in a skill was
|
||||
violated five times in one session by an agent that had the skill available.
|
||||
|
||||
**Any claim you are about to record as fact — in a drawer, a diary entry, a
|
||||
coordination event, or a report to the user — needs a second measurement taken
|
||||
by a different route.** Not a re-read of your reasoning: re-reading has caught
|
||||
zero of these. A disagreeing measurement has caught all of them.
|
||||
|
||||
The two shapes that get filed as fact and are not:
|
||||
|
||||
- **A negative result** (`401`, connection refused, zero rows, "not found") is
|
||||
first a claim about *your filter*, not about the world. Wrong host, wrong port,
|
||||
wrong table, capped output.
|
||||
- **A positive result** proves only what your command *actually asked*. An SSH
|
||||
handshake can succeed against the wrong host (`ssh -G` tells you which rule
|
||||
captured the name); a `401` can be a real answer from an issuer that never
|
||||
minted the credential.
|
||||
|
||||
Cheapest habit that works: **write the expected result next to each check before
|
||||
running it**, then diff. Expectations declared up front turn a silent wrong
|
||||
assumption into a visible mismatch. And if you cannot think of a second route to
|
||||
the same fact, you do not have a finding — you have a hypothesis, so label it as
|
||||
one.
|
||||
|
||||
## Handling an exposed credential
|
||||
|
||||
If a task touches a leaked secret, a token rotation, "is this credential still
|
||||
live?", whether to delete stored content, or which scopes a new token needs:
|
||||
**read `~/.agents/skills/credential-incident-response/SKILL.md` first.** One rule
|
||||
is load-bearing enough to state here: **probe the issuing provider before doing
|
||||
anything else** — most "exposed" credentials in a long-lived fleet are already
|
||||
dead, and the ones that are live are often far more privileged than assumed.
|
||||
Severity first, cleanup second, and prefer **revocation over deletion** for
|
||||
anything already replicated.
|
||||
|
||||
Reference in New Issue
Block a user