feat(pi-task): optional model fallback when the primary fails auth before acting
A tier's model can be unusable for reasons that have nothing to do with the
task, typically an expired AWS SSO session behind an amazon-bedrock profile.
pi reports that as a normal run: rc 0, agent_end, willRetry=false. The failure
shows only as stopReason="error" + errorMessage on the assistant message
(measured 2026-10-12). pi-task turned it into "no fenced json envelope",
which points at the wrong layer.
- Always surface it, first in the problem list:
"child model error (<provider>/<id>): ...".
- New, optional: pi-task.fallbackProfiles.<effort> = {provider, id[, thinking]}
in ~/.pi/agent/settings.json. It is tried ONCE, and only when the primary
ended in an auth-class error, made zero tool calls, left the boundary
diff empty, and at least 30 s of wall_s remain. A write task that failed
after acting is never re-run on a half-edited tree.
- A fallback without `thinking` inherits the primary's rather than "off".
A fallback identical to the primary is ignored, with a warning.
- The key lives in pi-task's own namespace because the fork tool does not
fall back. pi-fork's parser silently drops unknown profile keys.
- Reporting: "[FALLBACK]" on the verdict line plus a warning line.
result.json gains fallback{} and attempts[]; metrics.model/provider name
the model that produced the verdict. Audit keeps raw.primary.ndjson,
argv.fallback.json and boundary-after.primary.json. The second attempt
gets only the remaining wall budget, so the task tool's (wall_s+90) s kill
still holds.
- Unchanged: exit 0 = PASS / 1 = FAIL, and the closing "audit:" line.
selftest: 14 -> 29 checks, two-sided for every gate (fixtures mirror the
measured event shape; the SSO string is verbatim from a real expired
session). Each gate was mutation-checked: removing it turns its line BAD.
End-to-end, emb-7kj4vr4g, with fake AWS keys in the environment:
- the old pi-task gives FAIL "no fenced json envelope";
- the new one, per tier, gives PASS [FALLBACK] with the correct answer and
real tool calls;
- with real credentials: PASS on the primary, fallback configured but unused;
- with no fallback configured: FAIL, model error listed first.
This commit is contained in:
@@ -28,5 +28,9 @@
|
||||
"balanced": { "provider": "amazon-bedrock", "id": "eu.anthropic.claude-sonnet-5", "thinking": "low" },
|
||||
"deep": { "provider": "amazon-bedrock", "id": "eu.anthropic.claude-opus-5", "thinking": "high" }
|
||||
}
|
||||
},
|
||||
|
||||
"pi-task": {
|
||||
"_comment_fallbackProfiles": "Optional, read by pi-task (the `task` tool) only - the fork tool ignores it. Add \"fallbackProfiles\": { \"fast\": { \"provider\": \"<provider>\", \"id\": \"<model>\", \"thinking\": \"low\" }, \"balanced\": {...}, \"deep\": {...} } to retry a task ONCE on another model when the effortProfiles primary fails with an auth-class error (e.g. expired AWS SSO) before any tool call or file change. Omitted `thinking` inherits the primary's. See README 'Model fallback'."
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user