Joakim Persson
3760d90998
A tier's model can be unusable for reasons that have nothing to do with the
task, typically an expired AWS SSO session behind an amazon-bedrock profile.
pi reports that as a normal run: rc 0, agent_end, willRetry=false. The failure
shows only as stopReason="error" + errorMessage on the assistant message
(measured 2026-10-12). pi-task turned it into "no fenced json envelope",
which points at the wrong layer.
- Always surface it, first in the problem list:
"child model error (<provider>/<id>): ...".
- New, optional: pi-task.fallbackProfiles.<effort> = {provider, id[, thinking]}
in ~/.pi/agent/settings.json. It is tried ONCE, and only when the primary
ended in an auth-class error, made zero tool calls, left the boundary
diff empty, and at least 30 s of wall_s remain. A write task that failed
after acting is never re-run on a half-edited tree.
- A fallback without `thinking` inherits the primary's rather than "off".
A fallback identical to the primary is ignored, with a warning.
- The key lives in pi-task's own namespace because the fork tool does not
fall back. pi-fork's parser silently drops unknown profile keys.
- Reporting: "[FALLBACK]" on the verdict line plus a warning line.
result.json gains fallback{} and attempts[]; metrics.model/provider name
the model that produced the verdict. Audit keeps raw.primary.ndjson,
argv.fallback.json and boundary-after.primary.json. The second attempt
gets only the remaining wall budget, so the task tool's (wall_s+90) s kill
still holds.
- Unchanged: exit 0 = PASS / 1 = FAIL, and the closing "audit:" line.
selftest: 14 -> 29 checks, two-sided for every gate (fixtures mirror the
measured event shape; the SSO string is verbatim from a real expired
session). Each gate was mutation-checked: removing it turns its line BAD.
End-to-end, emb-7kj4vr4g, with fake AWS keys in the environment:
- the old pi-task gives FAIL "no fenced json envelope";
- the new one, per tier, gives PASS [FALLBACK] with the correct answer and
real tool calls;
- with real credentials: PASS on the primary, fallback configured but unused;
- with no fallback configured: FAIL, model error listed first.
37 lines
3.2 KiB
JSON
37 lines
3.2 KiB
JSON
{
|
|
"_comment": "Template for ~/.pi/agent/settings.json. Copy to that path and adjust for your region/account — this file is NOT symlinked by install.sh because pi rewrites settings.json at runtime (lastChangelogVersion bumps), which would dirty the repo. This template exists so a fresh machine can start pi without --model by copying + editing.",
|
|
"_comment_models": "The 'eu.' prefix on Bedrock model IDs is an inference-profile prefix tied to the AWS region. Must match your AWS_REGION. For us-east use 'us.anthropic.*'; for bare Anthropic provider (non-Bedrock) use the raw 'anthropic:claude-*' IDs. Run `pi --list-models` to see what your credentials can actually invoke.",
|
|
|
|
"defaultProvider": "amazon-bedrock",
|
|
"defaultModel": "eu.anthropic.claude-opus-5",
|
|
"enabledModels": [
|
|
"eu.anthropic.claude-opus-5",
|
|
"eu.anthropic.claude-sonnet-5",
|
|
"eu.anthropic.claude-opus-4-8",
|
|
"eu.anthropic.claude-haiku-4-5-20251001-v1:0"
|
|
],
|
|
|
|
"observational-memory": {
|
|
"model": {
|
|
"provider": "amazon-bedrock",
|
|
"id": "eu.anthropic.claude-haiku-4-5-20251001-v1:0"
|
|
},
|
|
"debugLog": false
|
|
},
|
|
|
|
"pi-fork": {
|
|
"_comment_extensions": "extensions:[] makes every fork child run with `pi --no-extensions`. This is a CAPABILITY FLOOR, not a style preference: the mempalace bridge is a pi EXTENSION, so with [] a fork physically cannot file drawers or diary entries under your identity. Measured on mbp-m1-2020 2026-09-01 and 2026-09-06: forks ignored their brief, answered in the user's voice, fabricated self-referential measurements, and one filed a diary entry as agent_name=pi (landing in wing_pi, where a wing-scoped diary_read never sees it). Root cause is upstream and by design, not a bug to wait out: the child is handed getHeader()+getBranch(), i.e. the WHOLE active session branch, with the brief appended as the final user message and the system prompt untouched (pi-fork src/index.ts) - so in a long session the parent narrative outweighs the task. Proven by control: the SAME model as the 'fast' profile below (haiku, thinking off) obeyed the identical brief perfectly when run as `pi -p --mode json --session-id <fresh> --no-extensions`, i.e. with no inherited context. COST of []: no palace search and no recall inside a fork. Set to null to restore normal extension loading.",
|
|
"extensions": [],
|
|
"defaultEffort": "balanced",
|
|
"effortProfiles": {
|
|
"fast": { "provider": "amazon-bedrock", "id": "eu.anthropic.claude-haiku-4-5-20251001-v1:0", "thinking": "off" },
|
|
"balanced": { "provider": "amazon-bedrock", "id": "eu.anthropic.claude-sonnet-5", "thinking": "low" },
|
|
"deep": { "provider": "amazon-bedrock", "id": "eu.anthropic.claude-opus-5", "thinking": "high" }
|
|
}
|
|
},
|
|
|
|
"pi-task": {
|
|
"_comment_fallbackProfiles": "Optional, read by pi-task (the `task` tool) only - the fork tool ignores it. Add \"fallbackProfiles\": { \"fast\": { \"provider\": \"<provider>\", \"id\": \"<model>\", \"thinking\": \"low\" }, \"balanced\": {...}, \"deep\": {...} } to retry a task ONCE on another model when the effortProfiles primary fails with an auth-class error (e.g. expired AWS SSO) before any tool call or file change. Omitted `thinking` inherits the primary's. See README 'Model fallback'."
|
|
}
|
|
}
|