task tool + fork-gate: put the "pi-task, not fork" rule where the decision is made

The rule was correct and written down twice (global AGENTS.md, this skill)
and was still violated by agents that had just read it: on 2026-09-17 all
five fork briefs in one session carried "do not", one returned confident
verbatim quotes that did not exist, and four had disjoint write boundaries
fork cannot enforce (re-run as pi-task, they passed). Three mechanisms,
none of them wording:

  1. fork is a TOOL — its self-recommending description ("implementation,
     testing, review…") is in the model's face every turn; pi-task was a CLI
     to be remembered and reached through bash with a hand-written JSON.
  2. The skill is gone after the first compaction; the tool list never is.
     The asymmetry widens in exactly the long sessions where fork is worst.
  3. Friction: one string vs a spec file + bash + reading result.json.

extensions/task.ts registers pi-task as the `task` tool. Flat parameters
build the spec; the decision rule sits in the description and in
promptGuidelines (appended to the system prompt, so compaction cannot remove
it). Before spending a model run it rejects the two spec errors that make a
boundary violation certain — write_allowed not an exact subset of roots, and
a writable root nested in a watched-only root (the parent's porcelain would
change every time; pi-task keys deltas by root string) — and it serialises
sibling tasks whose roots overlap (parallel siblings saw each other's writes
as violations, 2026-09-17). Returns the CLI's own parent-facing report; a
FAIL verdict is a result, only a CLI refusal is an error.

extensions/fork-gate.ts is a tool_call hook that BLOCKS a fork whose brief
contains a prohibition, a write boundary, or a clause-initial file-changing
imperative, and returns as the reason the exact task(...) to make instead.
Wording, not intent — the message says so and how to rephrase a genuinely
read-only brief. PI_FORK_GATE=off logs instead; /ext disables.

Evidence: test/fork-gate.test.mjs is two-sided (15 must-block incl. the real
shapes, 10 must-pass incl. "Write a summary…", "Report which files were
modified…", "Give me an update…"); the classifier redirects 5/5 of the real
briefs from the motivating session. test/task.test.mjs pins root overlap and
the pre-launch validation. Live in `pi -p`: the fork was intercepted before
any child spawned (no /tmp/pi-fork-* dir) and the model received the
redirect; task returned PASS with an evidence pointer and audit dir, a
budget-overrun returned a FAIL result (isError=false), and a nested-root spec
was rejected with no audit dir created.

skill/SKILL.md: Part 1 now opens with "decide the rung before the brief"
(the table, the three-question pre-flight, the roots contract, overlap,
what isolation does not fix); the ladder section and quick reference no
longer say pi-task "will never appear in your tool list". package.json gains
"type": "module" and a test script.
This commit is contained in:
2026-09-19 16:41:23 +02:00
parent 2610545c83
commit 25c1265681
8 changed files with 699 additions and 22 deletions
+76
View File
@@ -0,0 +1,76 @@
// Two-sided test of the fork-gate classifier: briefs that MUST be redirected
// to pi-task and briefs that MUST stay forkable. A gate that only ever blocks
// has not been shown to discriminate. Runs the shipped .ts directly (node >= 23
// strips types natively; the only import is type-only).
import { test } from "node:test";
import assert from "node:assert/strict";
import { classifyForkBrief, blockReason, GATE_RULES } from "../extensions/fork-gate.ts";
// Real shapes from the sessions that motivated the gate (2026-09-17, tor-ms22).
const MUST_BLOCK = [
["prohibition", "Migrate docs/level-01-basics. Do NOT modify files outside docs/level-01-basics/ or tutorials/01-basics/."],
["prohibition", "Read the repo and report. Don't change anything."],
["prohibition", "You must not commit."],
["prohibition", "Never touch the inventory."],
["prohibition", "Please refrain from editing mkdocs.yml"],
["boundary", "Only touch the three files listed above."],
["boundary", "Edit only the README; nothing else."],
["boundary", "This is a read-only review of the role."],
["boundary", "Stay within roles/static_site/ for all changes."],
["boundary", "Changes are limited to the docs directory"],
["mutation", "Implement the FQCN sweep across docs/ and run mkdocs --strict."],
["mutation", "Fix the eleven failing playbooks under solutions/."],
["mutation", "Review the playbooks, then commit the result on develop."],
["mutation", "Rewrite Exercise 5 to use the inventory plugin."],
["mutation", "Please update docs/getting-started/structure.md from the real tree."],
];
// Fork's legitimate uses: read-only exploration against this conversation, and
// opinions. Includes phrasings that a naive verb list would misfire on.
const MUST_PASS = [
"Find where the runner config is written to disk and report file:line.",
"Which function creates the temp files for the session? Report the call chain.",
"Compare the three staging options we discussed and give an independent opinion.",
"Summarize the architecture of the static_site role in 200 words.",
"Write a summary of what the last five commits changed.",
"Give me an update on how the mailbox derivation handles ready-status events.",
"Report which files were modified by CI run 23 according to the logs.",
"Explain the difference between compose and keyed_groups in the openstack inventory plugin.",
"List every place the 60000 ms default is documented; return paths and line numbers.",
"Is the fixed deadline applied in the feed path? Quote the code with a pointer.",
];
test("every hazardous brief is blocked with the expected class", () => {
for (const [kind, brief] of MUST_BLOCK) {
const v = classifyForkBrief(brief);
assert.equal(v.block, true, `should block: ${brief}`);
assert.equal(v.kind, kind, `wrong class for: ${brief} (matched "${v.matched}")`);
assert.ok(v.matched && v.matched.length > 0, "reports what matched");
}
});
test("every legitimate exploration brief passes", () => {
for (const brief of MUST_PASS) {
const v = classifyForkBrief(brief);
assert.equal(v.block, false, `false positive on: ${brief} (${v.kind}: "${v.matched}")`);
}
});
test("non-string / empty briefs never block (gate must not break fork on odd input)", () => {
for (const x of [undefined, null, 42, "", {}, []]) {
assert.equal(classifyForkBrief(x).block, false);
}
});
test("block reason carries the redirect: task(...) call, CLI fallback, sequential-roots rule", () => {
const r = blockReason(classifyForkBrief(MUST_BLOCK[0][1]));
for (const needle of ["task(id=", "write_allowed=", "/opt/pi-toolkit/bin/pi-task run", "overlap", "matches wording, not intent"]) {
assert.ok(r.includes(needle), `reason lacks "${needle}"`);
}
assert.ok(r.includes('"Do NOT"'), "reason names the words that fired");
});
test("rules are ordered prohibition → boundary → mutation and each has a why", () => {
assert.deepEqual(GATE_RULES.map((r) => r.kind), ["prohibition", "boundary", "mutation"]);
for (const r of GATE_RULES) assert.ok(r.why.length > 20);
});
+73
View File
@@ -0,0 +1,73 @@
// Tests for the pure parts of task.ts: root overlap and the pre-launch spec
// validation that turns a guaranteed false FAIL into an immediate error.
// task.ts imports typebox / @earendil-works/pi-ai at runtime (only resolvable
// inside pi), so the functions are cut out of the shipped source by brace
// matching and type-stripped — the same approach as mempalace-toolkit's tests.
import { test } from "node:test";
import assert from "node:assert/strict";
import { readFileSync, writeFileSync, mkdtempSync, mkdirSync } from "node:fs";
import { stripTypeScriptTypes } from "node:module";
import { tmpdir } from "node:os";
import path from "node:path";
const src = readFileSync(new URL("../extensions/task.ts", import.meta.url), "utf8");
function fn(name) {
const start = src.indexOf(`export function ${name}(`);
if (start < 0) throw new Error(`not found: ${name}`);
let depth = 0, seen = false;
for (let i = start; i < src.length; i++) {
if (src[i] === "{") { depth++; seen = true; }
else if (src[i] === "}") { depth--; if (seen && depth === 0) return src.slice(start, i + 1); }
}
throw new Error(`unterminated: ${name}`);
}
const work = mkdtempSync(path.join(tmpdir(), "task-test-"));
const modPath = path.join(work, "pure.mjs");
writeFileSync(modPath, stripTypeScriptTypes(
`import { existsSync } from "node:fs";\nimport path from "node:path";\n` +
[fn("rootsOverlap"), fn("anyOverlap"), fn("validateRoots")].join("\n\n"), { mode: "strip" }));
const { rootsOverlap, anyOverlap, validateRoots } = await import(modPath);
// Real directories, since validateRoots checks existence.
const repo = path.join(work, "repo"); const docs = path.join(repo, "docs"); const srcd = path.join(repo, "src"); const other = path.join(work, "other");
for (const d of [docs, srcd, other]) mkdirSync(d, { recursive: true });
test("rootsOverlap: identity, containment both ways, siblings, prefix-but-not-parent", () => {
assert.equal(rootsOverlap(repo, repo), true);
assert.equal(rootsOverlap(repo, docs), true);
assert.equal(rootsOverlap(docs, repo), true);
assert.equal(rootsOverlap(docs, srcd), false);
assert.equal(rootsOverlap(path.join(work, "repo"), path.join(work, "repo2")), false, "'/x/repo' is not a parent of '/x/repo2'");
assert.equal(anyOverlap([docs], [srcd, other]), false);
assert.equal(anyOverlap([docs, other], [srcd, other]), true);
});
test("validateRoots: read-only spec with existing absolute roots is accepted", () => {
assert.deepEqual(validateRoots([docs, srcd], undefined, true), []);
});
test("validateRoots: the migrate-0N shape (sibling roots, write_allowed subset) is accepted", () => {
assert.deepEqual(validateRoots([docs, srcd, other], [docs], false), []);
});
test("validateRoots: write task without write_allowed is rejected (violations undetectable)", () => {
const p = validateRoots([docs], undefined, false);
assert.equal(p.length, 1); assert.match(p[0], /requires write_allowed/);
});
test("validateRoots: write_allowed must be an exact root string", () => {
const p = validateRoots([repo], [docs], false);
assert.ok(p.some((x) => /not one of roots/.test(x)), p.join("\n"));
});
test("validateRoots: writable root nested in a watched-only root is rejected (certain false FAIL)", () => {
const p = validateRoots([repo, docs], [docs], false);
assert.ok(p.some((x) => /nested in/.test(x)), p.join("\n"));
});
test("validateRoots: read_only with write_allowed, relative root, missing root are each named", () => {
const p = validateRoots(["relative/path", path.join(work, "nope"), docs], [docs], true);
assert.ok(p.some((x) => /not absolute/.test(x)));
assert.ok(p.some((x) => /does not exist/.test(x)));
assert.ok(p.some((x) => /pick one/.test(x)));
});