Files
pi-extensions/test
Joakim Persson b3dc979aba secret-guard: keep secret env values out of model context, session files and the palace feed
Measured 2026-10-12 on emb-7kj4vr4g. A new session ran `env | grep …` and
tried to redact the output itself with a sed. The sed only matched when TOKEN
came after the `=`, so MEMPALACE_REMOTE_TOKEN=<value> went through. That raw
value was found in 12 local session files back to 2026-08-16, and each of those
sessions had sent it to its model provider. The palace stayed clean only
because mempalace-toolkit's feeder scrubs transcripts before staging. Nothing
guarded the step from tool output to model context.

The new extension ports tier 1 of mempalace_redact.py exactly: the exact
values of env vars whose name says secret are replaced with <redacted:NAME>.
Matching a known value cannot hit anything that is not that secret.
- tool_result: content text, structuredContent (always returned when content
  changes, so it is not dropped) and details.
- context: the same pass over every request, for user `!` output, pasted text
  and resumed sessions that already hold a raw value.
- tool_call: block a write/edit whose new text contains a live secret's
  placeholder, i.e. writing a redacted .env back. The reason says to use bash
  and let the shell expand "$NAME".
PI_SECRET_GUARD=off makes it inert.

Tests (16, two-sided): selection, ordering, redaction, deep walk with
identity, every handler, the off switch. Two parity tests run
mempalace_redact.py on the same synthetic env and compare the selected names
and the output text. Thirteen mutations, one per rule, are each caught. Two
mutations initially survived and exposed fixture gaps, which are now fixed.
Full suite: 45/45.

End-to-end in real pi (erdc/deepseek-v4.1-flash). The secret was synthetic,
and a test-only probe on before_provider_request counted it in each payload.
- control without the guard: 1 raw copy sent to the provider, 1 in the
  session file;
- with the guard: 0 sent and 0 stored, placeholder in both;
- resumed control session: 1 sent without the guard, 0 with it;
- interactive `!echo $SECRET`: 0 sent, plus a TUI warning;
- a write of the placeholder: blocked, file sha unchanged.
Not covered, as documented: live progress (tool_execution_update), the stored
entry of a user `!` command, secrets not in env (no shape rules), and pi-task
children, which run --no-extensions.
2026-10-12 14:35:49 +02:00
..