AI-generated illustrationClaude Code Workflows You Can Verify
Freeze the spec before the agent touches code, grant only what the task needs, and accept a change only on evidence the change could not edit.
- Reading
- 75 min
- Workshop
- 105 min
- Stack
- TS · Node.js 22 · Claude Code 2.1.304 · Git
- Version
- 1.1.0
- Last verified
- Oct 21, 2026
01Outcomes
What you will build and be able to do
Each outcome has an observable criterion. The lab and the assessment check exactly these, nothing more and nothing less.
OBJ-001
Turn a bug report into a frozen spec
At the spec tag, pnpm test fails only your T-1042 test, and after the fix the spec copy of the verifier prints "verdict: PASS" with V8 naming that test.
OBJ-002
Choose least-privilege controls, each in the layer that enforces it
pnpm test:hooks prints "fail 0" with the deny rules and both hooks committed and every path rule written with "./", and you can say that the committed settings grant nothing because every grant comes from --allowedTools.
OBJ-003
Run Claude Code headless within bounds, and capture a run receipt
On the live route, pnpm check:receipt prints "receipt: PASS" for your run, and you can say which line of the command each R check depends on; on Lab 3's branch, it shows the scope hook running.
OBJ-004
Reject an attractive but incorrect change on evidence
Your review names the five defects in the candidate, each with the output line that proves it, and explains why pnpm test reported five tests and not eight.
OBJ-005
Recover to a known state after a failed or rejected run
After the rollback, git status --short prints nothing and pnpm test shows exactly one failing test, the T-1042 spec test.
02Prerequisites and setup
Runs offline in minutes
- Git 2.40 or later: commit, tag, diff, status, git restore and git clean.Install Git
- A terminal running bash or zsh on Linux, macOS or WSL.
- Node.js 22.18 or later, for TypeScript type stripping, and pnpm 12, which corepack enable provides.Install Node.js 22
- For the live route: Claude Code 2.1.304 and an Anthropic API key. Every live run in this guide used an API key; signing in with a Claude account was not tested with these commands. Every lab also has an offline route that needs neither.Claude Code setup
- Reading a unit test in any language. The lab uses TypeScript with the built-in Node.js test runner; if you read JavaScript, Python or Go, you can follow it.
bash labs/new-workspace.sh labs/starter cc-labcd cc-labpnpm install --frozen-lockfilepnpm preflightok node: v22.23.3 (need 22.18 or later for TypeScript type stripping)ok pnpm: 12.14.0ok git: git version 2.53.0ok workspace: inside a git repositoryok working tree: cleanok claude: 2.1.304 (Claude Code)warn ANTHROPIC_API_KEY: not set (fine if you signed in with claude, or for the offline route)03The system picture
Put each rule where it can be enforced

Extended description
Five horizontal rows in two groups, each row with a layer name, what the layer does, and where it lives in the lab. Group "Inside the Claude Code session": (1) Guidance, dashed amber border, "asks", "CLAUDE.md, TASK.md". (2) Per-run grants, blue border, "bounds this run", "--allowedTools". (3) Committed restrictions, blue border, "bounds every run", "deny rules, hooks". Group "After the session ends": (4) Independent verification, mint border, "decides", "verifier from spec". (5) Version control, mint border, "rolls back", "git tag, restore". Footer: "Mechanisms checked against the Claude Code docs on 2026-10-20 (Claude Code 2.1.304). Synthetic data."
Rules that must always hold belong in the lower, enforced layers; the accept or reject decision happens outside the session. The table in "Where each control lives" gives each layer's full guarantee.
04Lesson
Why "it says the tests pass" is not evidence
A coding agent is a model in a loop with tools: it reads files, edits them and runs commands until it decides it is done, then writes a summary. The summary is a claim. It may be accurate. It may also describe the tests the agent ran after it changed which tests run.
Here is a summary from this guide's second lab. It reads well:
Every sentence is true, and the change is wrong in five ways. You will find all five with commands when you review it later in this guide. (That summary and its patch were written for the lab to represent a plausible generated change; they are not captured model output.)
A workflow you can verify replaces trust in the summary with five pieces of evidence you control:
- A frozen spec. A failing test that reproduces the bug, committed and tagged before the agent runs, plus a written scope.
- Explicit grants. The session can do only what the task needs, and you chose each permission on purpose.
- Independent checks. Tests, acceptance cases, scope and integrity checks that run outside the session, from code the change could not edit.
- A reviewed diff. A person, or a separate read-only review session, reads the change against the spec, including what the checks cannot see.
- A recovery path. You can return to the spec state in one step and rerun.
Anthropic's own guidance puts verification first: "Give Claude a way to verify its work" with tests, a build or expected outputs, and ask it to show evidence such as "the test output, the command it ran and what it returned" (best practices). This guide adds the other half: you verify too, with checks the agent did not get to edit.
Where each control lives
The most common mistake with coding agents is putting a rule in a layer that cannot enforce it. A prompt that says never to edit the tests is making a request. The same rule as a deny rule, a hook or a scope check is a control. Claude Code's documentation draws the same line: CLAUDE.md files are treated "as context, not enforced configuration," and a PreToolUse hook is the way to block an action regardless of what Claude decides (memory). Hooks are "deterministic and guarantee the action happens," while CLAUDE.md is advisory (best practices).
Five layers, from softest to hardest:
| Layer | Claude Code mechanism | What it can guarantee | How it fails |
|---|---|---|---|
| Guidance | CLAUDE.md, the prompt, TASK.md | Nothing. It shapes behavior | The model weighs it against other instructions and context |
| Per-run grants | --permission-mode, --allowedTools, --tools, --max-turns, --max-budget-usd | What this run may do without asking | Grants that are too broad; a mode left to its default; a classifier mode that allows what no grant names |
| Committed restrictions | permissions.deny rules and hooks in .claude/settings.json | What no run that loads them may do | A rule that does not match the call as written; a hook that never sees the call; a run that never loads the file |
| Independent verification | Tests and a verifier you run after the session, from the spec | Whether the change meets what you encoded | Checks that do not cover the real requirement |
| Version control | Git commits, tags, git restore, git clean | A known state you can return to | Untracked files and uncommitted work you forget about |
The decision behind the table holds for any agent: anything that must always be true goes in a layer that does not depend on the model's judgment, and the final accept or reject happens outside the session.
Set up a disposable lab
Every command in this guide runs in a throwaway copy of a synthetic repository. Do not point these workflows at real code until you have run them here.
The repository is refund-rules, a small TypeScript library for a fictional retailer, Example Outfitters. Policy POL-DOA-30 says a refund request is inside the window when it is made on or before calendar day 30 after delivery, counting days in UTC with the delivery day as day 0, and that refunds on one order may not exceed $60.00 in total. Ticket T-1042 reports the bug you will fix: a customer whose tent pole section arrived snapped asked for the $48.00 replacement pole set on calendar day 30 and was told the request was late.
Install the pinned version for the live route. The native installer takes a version argument (setup):
curl -fsSL https://claude.ai/install.sh | bash -s 2.1.304Native installs update in the background. For a class or a CI job where everyone must run the same version, set DISABLE_AUTOUPDATER=1 or use the stable channel, and record the version in every receipt (setup). The lab's receipt check does that for you.
Unzip the lab download into an empty folder, and from that folder create the workspace, install its two pinned development dependencies, check the toolchain and look at the bug:
bash labs/new-workspace.sh labs/starter cc-labcd cc-labpnpm install --frozen-lockfilepnpm preflightpnpm decide T-1042$ node scripts/decide.ts T-1042{ "ticketId": "T-1042", "policyId": "POL-DOA-30", "eligible": false, "refundCents": 0, "reason": "outside refund window"}The seven existing unit tests all pass, which is the first lesson: a green suite only tells you about the cases someone wrote down. The read-only acceptance cases in acceptance/ encode the policy's boundaries, and two of them fail on this code:
FAIL A1 T-1042: calendar day 30, later clock time than delivery Expected values to be strictly equal: false !== truePASS A2 calendar day 31, earlier clock time than deliveryFAIL A3 calendar day 30, almost 31 elapsed daysEvery lab run in this guide pins --model claude-sonnet-5-5; the spike probes passed the alias sonnet, which resolved to claude-sonnet-5-5 on 2026-10-20. Aliases such as sonnet "update over time," and a full model name pins a version (model configuration). Without --model, Anthropic API accounts default to Opus 5.5, according to the same page on 2026-10-20, which lists at twice Sonnet 5.5's price per token: $4 and $20 against $2 and $10 per million input and output tokens (models overview).
Freeze the spec before the agent touches code
The spec is three files, committed together and tagged, before any agent run:
- A failing test that reproduces the reported bug for the reported reason.
TASK.md: the goal, the policy, the file scope, what not to touch, and done criteria written as commands..claude/scope.json: the same file scope in machine-readable form, for the hook and the verifier.
Why freeze it first? If the agent writes the reproduction test, the same session decides both what "fixed" means and whether it is fixed. Claude Code's guidance suggests asking it to "write a failing test that reproduces the issue, then fix it" (best practices). In this workflow, you write that test (or you review and commit the one the agent proposes in plan mode), and the agent's job shrinks to making a committed test pass without editing it.
Add the reproduction test to test/window.test.ts:
test('T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery', () => { assert.equal(isWithinWindow('2026-09-14T18:30:00Z', '2026-10-14T19:05:00Z', 30), true);});Run pnpm test and confirm it fails for the right reason: an assertion on this case, not a syntax error or an import failure.
FAIL T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery Expected values to be strictly equal: false !== truetests 8, pass 7, fail 1Write the brief. Keep it short and testable; every "done" line is a command with a pass or fail result:
## Scope- Edit only `src/window.ts`.- Do not edit tests, `acceptance/`, `package.json`, `pnpm-lock.yaml`, `tsconfig.json`, `scripts/`, `.claude/`, `CLAUDE.md` or `TASK.md`.- Do not add dependencies. Do not commit.- If the fix seems to need another file, stop and explain which file and why. ## Done when- `pnpm test` passes, including the new T-1042 test.- `pnpm test:acceptance` passes (cases A1 to A7 are the policy's boundary cases).- `pnpm typecheck` passes.- Your final message lists each command you ran with its pass and fail counts, and a two-sentence explanation of the root cause.The full brief is in the handout and in labs/solutions/lab-001/TASK.md. The scope file says the same thing for machines, and the starter already has it:
{ "task": "T-1042", "allowEdits": ["src/window.ts"], "note": "Edit this file in the spec commit, before any agent run. The verifier reads it from the base commit."}Commit and tag:
git add -Agit commit -q -m "spec: T-1042 day-30 refund window"git tag specgit log --format='%s%d'The spec tag is now the known state you will verify against and return to.
Extended description
A heading, "Six stages, six pieces of evidence", over six numbered cards in a U shape. Top row, left to right: 1 Plan (TASK.md, scope.json; chip "edits: src/window.ts"), 2 Freeze spec (failing test, git tag spec; chip "pass 7, fail 1"), 3 Bounded run (claude -p, dontAsk + grants; chip "4 turns, $0.0167"). An arrow goes down to the bottom row, which reads right to left: 4 Verify (spec copy, V1 to V8; chip "verdict: PASS"), 5 Review (git diff spec, five questions; chip "1 file changed"), 6 Keep or roll back, outlined in mint (commit + receipts, or restore, clean; chip "status clean"). Footer: "Live: claude-sonnet-5-5, effort medium (stage 3); other stages offline. Recorded 2026-10-20. Synthetic data."
Each stage leaves an artifact a reviewer can check without rereading the session. The chips are recorded values from one walk of the lab on 2026-10-20 (stage 3 from live run LIVE-01: Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1), not typical results. "status clean" means git status --short printed nothing after the clean.
The heading appears, then the six cards described above one at a time in workflow order, each followed by its evidence chip and an arrow to the next card. It ends on the frame described above and holds for about 3.6 seconds.
Same content as the frame described above, revealed in the order the stages run.
Run Claude Code headless with explicit grants
You can do this lab interactively: start in plan mode with claude --permission-mode plan, read the plan, then approve it (best practices). This guide uses non-interactive mode, claude -p, because every control can be named on the command line and recorded, which is what a reviewer needs.
Three defaults are why every control is named on the command line:
- The starting permission mode. "In Claude Code v2.1.302 or later,
claude -pand Agent SDK runs with no permission mode set start in Manual mode," where calls that need approval are denied unless something approves them (permission modes, checked 2026-10-20). When this guide was first checked, on 2026-10-17, the headless page described that starting mode as one that can beauto. Pass the mode you want. - What a plain
claude -pdiscovers. Without other flags it reads the repository's.claude/settings.jsonandCLAUDE.md. The headless page calls--bare, which skips that discovery, "the recommended mode for scripted and SDK calls" and says it "will become the default for-pin a future release" (headless). A route that depends on discovery would silently lose its hooks on that day. - What
--baredrops on this version. The docs say bare mode skips the auto-discovery of hooks. On 2.1.304 it also dropped the hooks in a file passed with--settings: in this guide's spike (2026-10-20,claude-sonnet-5-5, effortmedium, n = 1), a bare run with a hook in--settingsran no hook, and the CLI's own--helpsays bare mode skips "hooks (those defined in settings and by installed plugins; ...)." Where the docs and a receipt disagree, this guide teaches the receipt.
So the lab's route does not depend on discovery or on bare mode. It starts from --restricted and names every input:
mkdir -p .runsclaude -p "$(cat TASK.md)" \ --restricted --tools Bash,Read,Edit \ --settings .claude/settings.json \ --append-system-prompt-file CLAUDE.md \ --permission-mode dontAsk \ --allowedTools "Edit(./src/window.ts)" "Bash(pnpm test)" "Bash(pnpm test:acceptance)" "Bash(pnpm typecheck)" "Bash(git diff *)" "Bash(git status *)" \ --model claude-sonnet-5-5 --effort medium \ --max-turns 12 --max-budget-usd 0.50 \ --no-session-persistence --include-hook-events \ --output-format stream-json --verbose > .runs/lab01.jsonlpnpm check:receipt .runs/lab01.jsonlWhat each choice buys you:
| Flag | Decision |
|---|---|
--restricted | Loads only managed settings and the file --settings names, so nothing from your user or project settings files. It also removes the tools that run commands unless --tools names them, keeps the file tools inside the working directories and refuses bypassPermissions (CLI reference) |
--tools Bash,Read,Edit | The whole tool set for the run, with Bash named so that --restricted keeps it: no Write, no web tools |
--settings .claude/settings.json | The repository's deny rules and hooks, loaded because you named the file, not because Claude Code found it |
--append-system-prompt-file CLAUDE.md | The repository's instructions, passed explicitly |
--permission-mode dontAsk | Any call that would prompt is denied, and the run never waits for a person. File reads in the working directory and a built-in set of read-only shell commands, such as tail, need no grant and still run (permission modes, read-only commands) |
Edit(./src/window.ts) | One editable file. A path that starts with ./ is relative to the current directory (permissions) |
Bash(pnpm test) and the rest | The exact commands in the done criteria, nothing broader such as Bash(pnpm *) |
--max-turns 12, --max-budget-usd 0.50 | Hard stops. The budget is checked against a client-side estimate, and "Spend can pass the cap" (CLI reference) |
--no-session-persistence | The session is not saved to your history |
--include-hook-events, --output-format stream-json --verbose | Every tool call, hook run, denial and the final result record land in .runs/lab01.jsonl |
Check the run's own record
A command line says what you asked for; the stream says what the run got. The lab's pnpm check:receipt reads the stream's first record (system/init) and its last (result) and checks seven things: both records exist (R1), the Claude Code version (R2), the permission mode (R3), the tool set (R4), a run of each configured hook for each matching tool call (R5), the instructions (R6) and how the run ended (R7). R6 works through a canary: CLAUDE.md asks Claude to end its final message with one exact line, so a run whose instructions never arrived fails R6. A missing line can also mean the model ignored it, so a failing R6 is a reason to look, not a verdict.
Receipt excerpt, run LIVE-01 (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1):
$ node scripts/check-receipt.ts .runs/lab01.jsonlPASS R1 receipt: init record found, result record foundPASS R2 Claude Code 2.1.304PASS R3 permission mode dontAskPASS R4 tools Bash, Edit, ReadPASS R5 hooks: none configured in .claude/settings.jsonPASS R6 instructions: final message ends with "Rules loaded: refund-rules CLAUDE.md"PASS R7 run ended: success, 4 turnsINFO model claude-sonnet-5-5; permission denials 0; estimated cost $0.0167 (client-side estimate) receipt: PASSThe final message of that run:
T-1042 is fixed with a change to `src/window.ts` only, and all three checks pass. - `pnpm test`: 8 tests, 8 pass, 0 fail (includes the new T-1042 test)- `pnpm test:acceptance`: 7 tests, 7 pass, 0 fail (A1 to A7)- `pnpm typecheck` (`tsc --noEmit`): passed with no errors. It prints no counts.The diff it produced:
- const elapsed = requested - delivered;- return elapsed >= 0 && elapsed <= windowDays * DAY_MS;+ if (requested < delivered) {+ return false;+ }+ const deliveryDay = Math.floor(delivered / DAY_MS);+ const requestDay = Math.floor(requested / DAY_MS);+ return requestDay - deliveryDay <= windowDays;The same command passed the verifier in 2 of 2 runs that day (LIVE-01 at $0.0167 and LIVE-02 at $0.0169, estimated; the two diffs differed and behaved the same). At 95% confidence, a Wilson interval puts 2 of 2 as consistent with a per-run failure rate as high as 66% (NIST e-Handbook). That shows the route can work, not how often it works. The summary is still a claim; the next section is how you check it.
Caps, modes and what they report
Both caps end a run with an error you can read from the result record. In this guide's spike (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1 each), --max-turns 1 ended with exit code 1, subtype error_max_turns and the error "Reached maximum number of turns (1)", while num_turns reported 2. --max-budget-usd 0.001 ended with exit code 1, subtype error_max_budget_usd and an estimated $0.0040, about four times the cap, which is why the CLI reference tells you to leave headroom.
Interactive sessions start in auto mode from v2.1.283, where a separate classifier model reviews actions; the documentation says it "reduces permission prompts but does not guarantee safety" (permission modes). Auto mode was available to claude -p with claude-sonnet-5-5 and an API key on 2.1.304 (spike ST4, 2026-10-20, effort medium, n = 1). Treat a classifier's decision as an observation, never as a control. The failure gallery shows what one auto-mode run allowed.
Verify with checks the change cannot edit
The agent ran the tests. So why run them again? Because the agent ran them through package.json scripts it could, in another run, have edited, against tests it could have edited, and reported the results in its own words. Your verification must not depend on anything inside the change.
The lab's verifier is scripts/verify-change.ts, but you never run the working tree's copy, which a change could edit. Take the copy committed at spec, put it outside the working tree, and run that:
git show spec:scripts/verify-change.ts > ../verify-spec.ts && \ node ../verify-spec.tsIt checks eight things against the spec tag:
| Check | What it proves | Why it is independent |
|---|---|---|
| V1 the base copy | The program running is byte-identical to the copy at spec | Compares its own SHA-256 with git show spec:scripts/verify-change.ts |
| V2 scope | Every changed or new file is in allowEdits | Reads scope.json from the spec commit, not the working tree, and includes untracked files |
| V3 protected files | package.json, the lockfile, tsconfig.json, acceptance/, .claude/, scripts/, CLAUDE.md and TASK.md are untouched | A fixed list in the verifier |
| V4 tests intact | No existing test was removed, renamed, skipped or focused | Compares test titles at spec with the working tree |
| V5 unit tests | test/ passes | Runs in a fresh checkout of spec with only the in-scope files copied in |
| V6 acceptance | acceptance/ passes | The same fresh checkout |
| V7 typecheck | tsc --noEmit passes | The same checkout, with the toolchain installed offline from the spec lockfile |
| V8 reproduction | The tests failed at spec | Runs them at spec |
Output after the fix:
PASS V1 verifier is the base copy: sha256 810b9c9f2c67 matches spec:scripts/verify-change.tsPASS V2 scope (T-1042): changed: src/window.tsPASS V3 protected files untouched: none changedPASS V4 existing tests intact: all base test titles present, none skippedPASS V5 unit tests, base checkout: pass 8, fail 0PASS V6 acceptance tests, base checkout: pass 7, fail 0PASS V7 typecheck, base checkout: no type errorsPASS V8 spec failed before the change: at spec: fail 1; failing: T-1042: request on calendar day 30 is inside the window even when the clock time is later than deliveryV8 is the one most teams skip. If the spec test passed before the change, it never reproduced the bug, and a green result afterward proves nothing.
V5 to V7 run in a fresh checkout because the working tree is part of the change. A change that edits package.json, a test helper or the verifier itself could change what a check measures. In the base checkout, the only thing under test is the part of the change you allowed. Version 1.0.0 of this lab ran them in the working tree and noted that a change could weaken the verifier through a file it depends on; running the spec copy against a base checkout closes that gap. The failure gallery shows a one-line edit that makes the working tree's copy print verdict: PASS for a change the spec copy fails.
The verifier writes a JSON receipt to .runs/ with the base and head commits. Attach it, the run's stream and the diff to the pull request. That bundle is the evidence; the agent's summary is a convenience.
Review an attractive but incorrect change
Now the change from the opening. Put it on a branch from the spec, apply labs/patches/candidate-grace.patch, and do what most people do first, pnpm test:
$ node --test --test-reporter=./test/reporter.mjs --test-concurrency=1 test/window.test.tsPASS request 10 days after delivery is inside the windowPASS request 45 days after delivery is outside the windowPASS request on calendar day 31 is inside the grace periodPASS invalid timestamp throws RangeErrorPASS T-1042: request on calendar day 30 is inside the window even when the clock time is later than deliverytests 5, pass 5, fail 0Green, and only five tests. Now the evidence:
PASS V1 verifier is the base copy: sha256 810b9c9f2c67 matches spec:scripts/verify-change.tsFAIL V2 scope (T-1042): out of scope: package.json, src/grace.ts, src/refund.ts, test/window.test.ts (untracked: src/grace.ts)FAIL V3 protected files untouched: changed: package.jsonFAIL V4 existing tests intact: test/window.test.ts: removed or renamed "request on calendar day 31 is outside the window"FAIL V5 unit tests, base checkout: pass 3, fail 1; failing: test/window.test.tsFAIL V6 acceptance tests, base checkout: pass 0, fail 1; failing: acceptance/window.acceptance.test.tsFAIL V7 typecheck, base checkout: src/window.ts(1,28): error TS2307: Cannot find module './grace.ts' or its corresponding type declarations.PASS V8 spec failed before the change: at spec: fail 1; failing: T-1042: request on calendar day 30 is inside the window even when the clock time is later than deliveryV5 to V7 fail before they reach any test case: the one in-scope file, src/window.ts, imports src/grace.ts, which is out of scope, so the base checkout does not have it. A change has to work with only the files it was allowed to touch. Two more commands show the rest. pnpm test:acceptance in the working tree fails A2 and A6, both calendar day 31. Running every unit test file, which the candidate's test script no longer does, shows one more:
FAIL non-integer cents throws RangeError Missing expected exception (RangeError).Five defects, each with its proof:
- Wrong logic that passes the reported case. A one-day grace period on elapsed time makes T-1042 pass but accepts day-31 requests (A2 and A6). The root cause, elapsed hours versus calendar days, is untouched.
- A weakened test. The existing day-31 test was renamed and its expectation flipped to match the new behavior (V4).
- A narrowed test command.
package.jsonnow runs onlytest/window.test.ts, which is whypnpm testran five tests instead of eight (V3, and the count itself). - An unrelated behavior change. The "tidied" cents validation no longer rejects 48.5 cents, and the refund test the narrowed script stopped running now fails.
- A new file outside the scope.
src/grace.tsis untracked, sogit diffnever shows it (V2), and the in-scope code depends on it (V7).
Extended description
A line at the top: "Order ORD-5521, delivered Sep 14, 18:30 UTC". Below it, a time ruler across two calendar days after delivery: day 30 (Oct 14 UTC, mint band, inside the policy) and day 31 (Oct 15 UTC, amber band, outside). Three vertical limit lines, each with a label: a dashed amber line at Oct 14 18:30, "Starter: 30 x 24 h"; a solid mint line at Oct 15 00:00, "Policy: end of day 30"; a dashed amber line at Oct 15 18:30, "Grace patch: 31 x 24 h". Three cases sit on the axis: A1 (Oct 14 19:05) 35 minutes past the starter's limit but inside day 30; A6 (Oct 15 01:05 UTC, written with a -05:00 offset) and A2 (Oct 15 09:00) inside day 31 but before the grace patch's limit.
Below, a table of the seven acceptance cases, A1 to A7, for each implementation, with PASS meaning the result matches policy POL-DOA-30. Starter: A1 FAIL, A2 PASS, A3 FAIL, A4 PASS, A5 PASS, A6 PASS, A7 PASS; 5 of 7. Grace patch: A1 PASS, A2 FAIL, A3 PASS, A4 PASS, A5 PASS, A6 FAIL, A7 PASS; 5 of 7. Reference fix: all PASS; 7 of 7. Footer: "Offline: no model involved. Recorded 2026-10-20. Synthetic data."
The grace patch fixes the reported ticket (A1) and breaks two day-31 cases (A2, A6). Counting calendar days in UTC is the only implementation that matches the policy on every case. A3 (delivered at 01:00, requested at 23:59 on day 30), A4, A5 and A7 (invalid input) use other times, so they are in the table but not on this order's ruler.
The order line, the ruler's two day bands and the axis appear, then cases A1, A6 and A2 on the axis and the table's column headers, A1 to A7. Then the three implementations are added one at a time: the starter's limit line and label, then its row of results and "5 of 7"; the grace patch's line, label, row and "5 of 7"; the policy's line and label with the reference fix's row and "7 of 7". It ends on the frame described above and holds for about 2.6 seconds.
Same content as the frame described above, revealed one implementation at a time.
What to look for in any generated diff
Read the diff after the checks, not instead of them, and ask five questions:
- Does the change fix the root cause named in the spec, or only the reported example?
- Did any test, test command or test configuration change?
- Is every file in scope, including new untracked files that
git diffdoes not show? - Are there drive-by edits to unrelated behavior?
- What input would break this? Write it as a test.
A second session as reviewer
Claude Code's guidance suggests separate writer and reviewer sessions (best practices). The useful version gives the reviewer no way to change anything, no Edit tool and three read-only commands, and asks for a verdict in a JSON shape you define:
git status --short > .runs/before-review.txtclaude -p "$(cat ../labs/review/review-prompt.md)" \ --restricted --tools Bash,Read \ --permission-mode dontAsk \ --allowedTools "Bash(git status *)" "Bash(git diff *)" "Bash(pnpm test:acceptance)" \ --json-schema "$(cat ../labs/review/review-schema.json)" \ --model claude-sonnet-5-5 --effort medium \ --max-turns 12 --max-budget-usd 0.50 \ --no-session-persistence \ --output-format stream-json --verbose > .runs/review.jsonlpnpm check:receipt .runs/review.jsonl --expect-tools Bash,Read --canary none --expect-structured --print-structuredgit status --short | diff .runs/before-review.txt - && echo "reviewer changed nothing"--tools sets which built-in tools exist in the session; --allowedTools only approves calls in advance (CLI reference). With --json-schema, the verdict arrives in structured_output, which is missing when a run ends without valid output, so check for it before you read it (headless). On 2.1.304 a --json-schema run also lists an extra StructuredOutput tool in system/init (spike ST3 and LIVE-07, 2026-10-20, claude-sonnet-5-5, effort medium, n = 1 each); the receipt check accepts that one tool when you pass --expect-structured.
Observed (LIVE-07, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1, $0.0341 estimated): verdict reject with five findings. It rated the window logic (with A2 and A6), the flipped test, the narrowed test script and the removed integer check high, and the new src/grace.ts medium. Its one denied call began with a cd into a mistyped temporary folder outside the workspace and then ran cat src/grace.ts. A cd outside the working directory is not on Claude Code's read-only list, so dontAsk denied the whole call; cat on its own needs no grant (read-only commands), and in the clean-room rerun (2026-10-20, the same version, model and effort, n = 1) cat src/grace.ts ran without one. git status did not change. On 2026-10-17, on Claude Code 2.1.302 with the 1.0.0 command and the same model and effort (n = 1), the same review also returned reject with five blocking findings. That is a useful second opinion. It is still advisory: the reviewer is a model, and the accept decision stays with the deterministic checks and a person.
Recover to a known state
You rejected the change. Get back to the spec state, and prove you are there.
git restore --staged --worktree .git status --short?? src/grace.tsgit restore puts tracked paths back; the new file the candidate created is untracked, so it is still there (git-restore). Preview, then remove it. git clean deletes untracked files, -n is a dry run, and -f is required by default (git-clean):
git clean -ngit clean -f src/grace.tsgit status --shortpnpm testWould remove src/grace.tsRemoving src/grace.tsgit status --short then prints nothing, and pnpm test ends with tests 8, pass 7, fail 1: the T-1042 spec test fails again, as it did at the tag. That last command is the proof. "Known state" means the spec test fails again for the original reason.
The lab also ships pnpm reset, which lists what it would discard and changes nothing until you pass --yes. Then it moves the branch and the working tree back to spec and deletes untracked files and folders. Use it only in a disposable workspace.
Checkpoints are not your rollback
Claude Code checkpoints let you rewind code, conversation or both inside an interactive session. They track only edits made through Claude's file-editing tools: "Checkpointing does not track files modified by Bash commands," and the page's own heading says checkpoints are "Not a replacement for version control" (checkpointing). A test script that writes a file, a formatter run through Bash or an mv survives a rewind. The spec tag does not depend on which tool made a change. (Checkpoints are interactive, and this guide did not exercise them.)
A failed run can still leave changes
Run LIVE-03 repeated LIVE-01's command with --max-turns 2. The documented behavior is that the run "Exits with an error when the limit is reached" (CLI reference). Observed (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1): exit code 1, subtype error_max_turns, no final message, and an edited src/window.ts on disk that passed all eight verifier checks. LIVE-03 on 2026-10-17, on 2.1.302, ended the same way.
Run status and repository state are separate facts. A failed run is not a no-op, and a successful run is not a correct change. Decide from the repository: either verify and review the leftover change like any other, or reset to spec and rerun.
Enforce the boundary with deny rules, hooks and the sandbox
Per-run flags bound one run. For rules that should hold in every run, keep them in .claude/settings.json, commit the file, and pass it with --settings as the lab's route does:
- Deny rules. "Rules are evaluated in order: deny, then ask, then allow," and the first match decides (permissions). Deny the files and commands no task should touch:
Edit(./package.json),Edit(./test/**),Edit(./acceptance/**),Bash(git push *),Bash(git commit *),Bash(pnpm add *),WebFetchand so on. The full list is inlabs/solutions/lab-003/.claude/settings.json. - A PreToolUse hook that enforces the task scope. Deny rules describe stable repository policy. The task boundary changes every ticket, so it lives in
scope.json, and a hook checks every edit against it. - A Stop hook that keeps Claude working while the tests fail, a bounded number of times.
- No allow rules. Grants come from
--allowedToolson each run, where a reviewer can see them.
Write every path rule with ./. The permissions page says a path that starts with / is relative to the settings source. In this guide's spike, a deny rule written as /test/** in a file passed with --settings .claude/settings.json did not protect test/: the edit went through. The same rule written as ./test/** denied it with "File is in a directory that is denied by your permission settings." (Claude Code 2.1.304, claude-sonnet-5-5, effort medium, 2026-10-20, n = 1 each.) The lab's settings test fails any path rule that does not start with ./.
The scope hook
Hooks receive JSON on stdin with tool_name and tool_input; for edits, tool_input.file_path. A PreToolUse hook blocks a call by printing hookSpecificOutput.permissionDecision: "deny" with a reason, or by exiting with code 2 (hooks). The core of labs/solutions/lab-003/.claude/hooks/guard-scope.ts:
const relative = path.relative(projectDir, realTarget(path.resolve(projectDir, raw))).split(path.sep).join('/');if (relative === '' || relative.startsWith('..') || path.isAbsolute(relative)) { deny(`${raw} is outside this project. Edits must stay inside the repository.`, raw);}After it reads .claude/scope.json, failing closed when the file is missing or unreadable:
if (!allowed.includes(relative)) { deny(`${relative} is outside the ${scope.task ?? 'current'} task scope (allowed: ${allowed.join(', ') || 'nothing'}). Do not work around this. If the fix needs this file, stop and explain why in your final message.`, relative);}Design choices worth copying:
- It only denies. In-scope edits fall through to the normal permission rules, so the hook can never widen access.
- It resolves symlinks and
..first, sosrc/link.tspointing atpackage.jsonis denied. - It fails closed on unreadable input (exit 2) and on a missing scope file.
- The denial reason tells the model what to do instead: stop and explain, not work around.
The hook's matcher is Edit|Write|NotebookEdit. On 2.1.304 those were the file-edit tools among the 24 tools a default Linux run listed in system/init (spike S1, 2026-10-20, claude-sonnet-5-5, effort medium, n = 1), which had no Glob or Grep. A matcher sees only the tools it names. A shell command can change a file too, and no file-edit hook sees that; the failure gallery has the run where it happened.
The Stop hook
A Stop hook blocks the stop with {"decision": "block", "reason": "..."}. Claude Code tells the hook when it is already continuing because of a stop hook (stop_hook_active), and "after stop hooks have continued the turn eight times in a row, Claude Code overrides the next block and ends the turn" (hooks). Do not rely on that ceiling. require-green.ts counts its own blocks per session and gives up after three, because a fix that cannot pass within scope should end with an explanation, not a loop. The verifier remains the final gate either way.
Hooks are code: test them offline
"Command hooks execute shell commands with your full user permissions" (hooks). Treat them like production code. Because a hook is a program that reads JSON and writes JSON, you can test it without a model: pnpm test:hooks feeds the hooks the JSON Claude Code sends (an in-scope edit, package.json, a new file, path traversal, a path outside the project, a symlink, a non-edit tool, malformed input, a missing scope file, passing and failing suites, the block limit) and checks the settings file itself: known tool names in every matcher, ./ on every path rule, the boundary files denied, no allow rules, both hooks wired. Against the starter's stubs, 7 of 18 pass; against the reference, 18 of 18.
What the controls did live
Three runs on Lab 3's branch, where the bug is present and the controls are committed (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1 each):
- LIVE-05, Lab 1's command, unchanged. The receipt check counted one scope-hook run for the one edit and one Stop-hook run, and the verifier passed. Hooks from
--settingsfire under--restricted. - LIVE-06, an out-of-scope instruction on purpose. The prompt asked for a rename in
src/refund.tswhilescope.jsonnamed T-1042, and the run had a grant that was too broad on purpose,Edit(./src/**). Claude first tried a Bashsed -ion the file, which dontAsk denied because no grant matched it, then twoEditcalls, which the scope hook denied. The Stop hook then reported the failing T-1042 test, and Claude fixedsrc/window.ts, which the prompt never asked for. Its final message said the rename was not done and why. - LIVE-09, the same command in auto mode. In the failure gallery.
LIVE-06 is the subtle one. Every control worked as written, and the session still did unrequested work, because the prompt, the scope file and the Stop hook disagreed about what the task was. When you configure hooks that push the agent forward, make sure they point at the same task as the prompt and scope.json. Otherwise the agent follows the most specific signal it gets. LIVE-06 on 2026-10-17 (2.1.302, the same model and effort, n = 1), with a different out-of-scope request, went the same way.
The sandbox
Deny rules and hooks decide which tool calls run. The sandbox limits what a shell command can do once it runs. It is off by default; --settings '{"sandbox":{"enabled":true}}' turns it on for one session, and on Linux it needs bubblewrap and socat (sandboxing). In this guide's spike (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1 each), a sandboxed write outside the working directory failed with "touch: cannot touch '/tmp/g05-sb-outside.txt': Read-only file system", and a request to an unlisted host failed with "curl: (7) CONNECT tunnel failed, response 403".
Know what it leaves out. It "covers shell commands only. Claude's file tools, MCP servers, and hooks run outside it." By default a sandboxed command can still write to the working directory, so it would not have stopped the out-of-scope sed in the failure gallery. When a command fails inside the sandbox, Claude Code can retry it unsandboxed: in dontAsk mode Claude Code denies that retry, but an allow rule that matches the command also approves its retry, and allowUnsandboxedCommands: false turns the retry off (sandboxing). This guide did not exercise the retry.
What a headless run takes from a repository
Every command so far named its controls. A plain claude -p instead discovers the repository's settings, and that is where the trust boundary sits.
The permissions page says project permissions.allow rules apply only after you accept the workspace trust dialog, while "deny and ask rules aren't affected, since they only restrict" (permissions). A claude -p run never shows the dialog. The hooks page adds that a -p or SDK session "treats the folder as trusted, so hooks committed in a repository's .claude/settings.json run in a folder you've never trusted" (hooks).
This guide re-observed that on the pinned version. Run LIVE-04 (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1) added one allow rule to the lab's committed settings and ran a plain claude -p in a folder that had never been trusted. Its stderr, then the number of hook starts in its stream:
Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted. Run Claude Code interactively here once and accept the trust dialog, or set projects["/tmp/g05-walk-kryzij69/download/cc-lab"].hasTrustDialogAccepted: true in /tmp/g05-cc/home/.claude.json.3The repository's hooks still ran three times. In the spike's version of the same check, the deny rule applied, the hook saw a variable set by the repository's env block, and CLAUDE.md loaded. On 2026-10-17, on 2.1.302 with the same model and effort (n = 1), LIVE-04 printed the same warning for ten committed allow rules, and the hooks ran.

claude -p run do. Restrictions, hooks and the env block apply; grants do not. Live: claude-sonnet-5-5, effort medium, Claude Code 2.1.304, run LIVE-04 and spike S1 (n = 1 each), recorded 2026-10-20, with the permissions page checked the same day. Synthetic data. Deterministic diagram.Extended description
Heading: "From the repository's .claude/settings.json". Three cards. Mint card "Deny rules", status "Applied": "They only restrict, so workspace trust does not gate them." Amber card "Hooks and env", status "Used": "Hooks run with your permissions, and see the env block." Gray card "Allow rules", status "Ignored": "They grant, so they wait for trust, and claude -p never asks." Below, a box labeled "stderr of run LIVE-04, Claude Code 2.1.304, first sentence" quotes: "Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted." Then, in mint: "Grants for the run come from the command line: --allowedTools". Footer: "Live: claude-sonnet-5-5, effort medium. Claude Code 2.1.304. Recorded 2026-10-20. Synthetic data."
In Claude Code 2.1.304, a never-trusted headless run applies a repository's restrictions and runs its hooks, but not its grants. Sources: run LIVE-04 (the warning and three hook starts) and spike S1 on the same CLI (the deny rule's refusal and the hook seeing a variable from the env block), one run each on 2026-10-20 with claude-sonnet-5-5 at effort medium, and the permissions page checked the same day.
The asymmetry is the lesson: in a headless run, the repository's grants are dropped while its code-executing hooks still run, with your permissions. Before you run claude -p over a repository you did not write, pick a defense on purpose. Each was tried on 2.1.304 on 2026-10-20 with claude-sonnet-5-5 at effort medium and the same probe repository (n = 1 each):
| Defense | What it kept out | What still applied |
|---|---|---|
Read .claude/ and CLAUDE.md first | Whatever you decide after reading | Everything you leave in place |
--settings '{"disableAllHooks": true}' | The repository's hooks | Its deny rules and CLAUDE.md |
--bare | The repository's hooks and CLAUDE.md | Its env block and its deny rules |
--setting-sources user | All project settings, including the deny rules and the env block, and CLAUDE.md: the run read a file the repository denied | Your user settings |
--restricted, as in the labs | All project and user settings, and CLAUDE.md | Managed settings and the file --settings names |
What a passing verifier does not prove
The verifier proves the change matches what you encoded: these files, these tests, these cases. It cannot tell you about cases nobody wrote down.
The assessment makes that concrete. Its candidate is a one-line fix for ticket T-3118 that passes all eight verifier checks and is still wrong; finding the input that breaks it is part of the work. So review still matters after a green verifier, and the fifth review question, the input that would break the change, is the one that catches it. When you find such a case, add it as a test before the change is accepted, so the next verifier run encodes it.
Other limits of this lab:
- Synthetic and small. One repository, a few files, fast tests. Real repositories have slow suites, flaky tests and generated code; the pattern holds, the costs change.
- Live results are observations. Every live case on 2026-10-20 is one run on one model and one machine (n = 1), except the Lab 1 command (n = 2). They show what can happen on Claude Code 2.1.304, not how often.
- Not tested here. macOS and native Windows, plan mode,
/rewind, the trust dialog itself, signing in with a Claude account, and the sandbox's unsandboxed retry. - Permission rules are not a sandbox. Bash rules match the command as written; the sandbox, which is off by default, is what limits filesystem and network access for shell commands at the operating-system level, and only for shell commands. For untrusted code, use a container or VM as well.
- Cost figures are estimates.
total_cost_usdis Claude Code's client-side estimate, not an invoice. For the cost report the documentation says "The figure is an estimate" and points to the Claude Console for billing (costs).
Labs
The hands-on work is in the lab download, about 105 minutes in total. Every lab has a live route (Claude Code headless) and an offline route that makes the same change by hand, so you can finish every lab without an account, a key or a network connection after installing.
- Lab 1, Fix T-1042 from a frozen spec (guided, about 35 minutes; OBJ-001, OBJ-003). Write the failing test and brief, tag
spec, make the change on either route, check the run's receipt, and pass the spec copy of the verifier. - Lab 2, Reject the grace-period change and recover (diagnostic, about 30 minutes; OBJ-004, OBJ-005). Apply the candidate, prove each defect with a command, optionally ask a read-only reviewer, roll back, and prove you are back at the spec state.
- Lab 3, Make the boundary deterministic (independent, about 40 minutes; OBJ-002, OBJ-003). Implement the two hooks until
pnpm test:hookspasses 18 of 18, commit the deny rules and hooks, watch them hold the boundary, and see what a plainclaude -ptakes from a repository.
Troubleshooting
Each entry was reproduced from the lab download on 2026-10-20, except T-08, which names its run.
| ID | Symptom | Cause | Fix |
|---|---|---|---|
| T-01 | The verifier step prints fatal: invalid object name 'spec'. and nothing else | The spec commit was never tagged | git tag spec on the spec commit, then rerun the step |
| T-02 | pnpm preflight prints FAIL node: v20.20.2 (need 22.18 or later for TypeScript type stripping), and pnpm test prints Could not find with the test pattern | Node.js is older than 22.18 | Install Node.js 22.18 or later, then rerun pnpm preflight |
| T-03 | .runs/lab01.jsonl: No such file or directory, and Claude Code never starts | .runs/ does not exist, so the shell fails the redirect first | Run mkdir -p .runs, the first line of the lab's command |
| T-04 | error: patch failed: src/window.ts:18 when you apply the candidate | You applied it on main, after the fix | git switch -q --create candidate spec, then apply it |
| T-05 | FAIL V7 typecheck, base checkout: could not install the base lockfile offline: pnpm not found | pnpm is not on PATH in the shell that runs the verifier | Run corepack enable, or run the verifier from the shell where pnpm --version works |
| T-06 | pnpm preflight prints warn claude: not found (offline route still works) | Claude Code is not installed, or not on PATH | Install 2.1.304 with the native installer, or take the offline route |
| T-07 | pnpm test:hooks fails every hook matcher names only tools that exist | A matcher names a tool the version does not have | Use Edit|Write|NotebookEdit (F-05) |
| T-08 | stderr: Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted. | A plain claude -p in a folder never trusted interactively (LIVE-04) | Expected. Pass grants with --allowedTools; keep deny rules and hooks in the file you pass with --settings |
Quick reference
The workflow:
- Reproduce the bug with a failing test. Confirm it fails for the right reason.
- Write
TASK.md(goal, scope, do-not-touch list, done criteria as commands) andscope.json. - Commit and
git tag spec. - Run the agent with every input named:
--restricted, a tool set, a settings file, the instructions file, an explicit mode, path-scoped grants,--max-turnsand--max-budget-usd. Record the stream. - Check the receipt: version, mode, tools, hook runs, instructions, how the run ended.
- Run the spec copy of the verifier: scope, protected files, test integrity, suites in a base checkout, typecheck, reproduction at
spec. - Review the diff with the five questions. Add a test for any missing edge case.
- Keep (commit with the receipts) or roll back (
git restore --staged --worktree .,git clean -n,git clean -f), and confirm the spec test fails again.
Commands:
bash labs/new-workspace.sh labs/starter cc-labpnpm decide T-1042pnpm test && pnpm test:acceptance && pnpm typecheckpnpm test:hookspnpm check:receipt .runs/lab01.jsonlgit show spec:scripts/verify-change.ts > ../verify-spec.ts && node ../verify-spec.tspnpm resetRules of thumb:
- Guidance asks; deny rules, hooks and verifiers enforce. Put "must" rules in the second group.
- Name every input of a headless run. Pass the permission mode. Pin the model by full ID. Record the version.
- Check the run's own record, not only its summary.
- Run your checks from code the change could not edit.
- Count tests as well as failures: five passing where eight used to run is a finding.
- A hook sees only the tools its matcher names; the verifier sees the repository.
- A passing verifier is necessary, not sufficient.
05Hands-on lab
Lab 1: Fix T-1042 from a frozen spec
Create the workspace, see the bug, write the failing test first, write the brief and freeze the spec, make the change on the live route (Claude Code headless) or the offline route (by hand), verify with the spec copy of the verifier, and keep the fix. About 35 minutes. The workspace and the toolchain check are the setup above, so the steps start at the bug. Outputs are copied, untrimmed, from a run of this exact download on 2026-10-20 with Node.js 22.23.3, pnpm 12.14.0 and Git 2.53.0. pnpm prints each script's command first (the line starting with $ node), and adds [ELIFECYCLE] where a command exits non-zero, here on purpose. On the offline route the verifier's receipt file name will differ, and nothing else should. The test you add, the brief and the offline route's two edits are in labs/learner-labs.md in the lab download. The live route and the offline route end in the same place: the live route's output is the receipt excerpt of run LIVE-01 (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5 at effort medium; the same command passed the verifier in 2 of 2 runs that day), and a live run's turns, cost and wording differ from run to run.
Acceptance checks
- node ../verify-spec.ts prints "verdict: PASS", with V8 naming your T-1042 test.
- git diff spec --stat lists only src/window.ts.
- Live route: pnpm check:receipt .runs/lab01.jsonl prints "receipt: PASS", and you can say which line of the command each R check depends on.
See the bug
$ pnpm decide T-1042$ node scripts/decide.ts T-1042 { "ticketId": "T-1042", "policyId": "POL-DOA-30", "eligible": false, "refundCents": 0, "reason": "outside refund window" }Run the policy's boundary cases
$ pnpm test:acceptance$ node --test --test-reporter=./test/reporter.mjs --test-concurrency=1 "acceptance/**/*.test.ts" FAIL A1 T-1042: calendar day 30, later clock time than delivery Expected values to be strictly equal: false !== true PASS A2 calendar day 31, earlier clock time than delivery FAIL A3 calendar day 30, almost 31 elapsed days Expected values to be strictly equal: false !== true PASS A4 day 0, after delivery PASS A5 day 0, but before delivery PASS A6 offset timestamp that is calendar day 31 in UTC PASS A7 invalid requestedAt throws RangeError tests 7, pass 5, fail 2 [ELIFECYCLE] Command failed with exit code 1.Write the failing test first, then run the unit tests
$ pnpm test$ node --test --test-reporter=./test/reporter.mjs --test-concurrency=1 "test/**/*.test.ts" PASS request under the limit is refunded in full PASS request over the limit is capped at the limit PASS non-integer cents throws RangeError PASS request 10 days after delivery is inside the window PASS request 45 days after delivery is outside the window PASS request on calendar day 31 is outside the window PASS invalid timestamp throws RangeError FAIL T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery Expected values to be strictly equal: false !== true tests 8, pass 7, fail 1 [ELIFECYCLE] Test failed. See above for more details.Write the brief, then read the scope file
$ cat .claude/scope.json{ "task": "T-1042", "allowEdits": ["src/window.ts"], "note": "Edit this file in the spec commit, before any agent run. The verifier reads it from the base commit." }Freeze the spec: commit and tag it
$ git add -A git commit -q -m "spec: T-1042 day-30 refund window" git tag spec git log --format='%s%d'spec: T-1042 day-30 refund window (HEAD -> main, tag: spec) baseline: synthetic refund-rules lab (tag: baseline)Live route: run Claude Code headless with explicit grants
$ mkdir -p .runs claude -p "$(cat TASK.md)" \ --restricted --tools Bash,Read,Edit \ --settings .claude/settings.json \ --append-system-prompt-file CLAUDE.md \ --permission-mode dontAsk \ --allowedTools "Edit(./src/window.ts)" "Bash(pnpm test)" "Bash(pnpm test:acceptance)" "Bash(pnpm typecheck)" "Bash(git diff *)" "Bash(git status *)" \ --model claude-sonnet-5-5 --effort medium \ --max-turns 12 --max-budget-usd 0.50 \ --no-session-persistence --include-hook-events \ --output-format stream-json --verbose > .runs/lab01.jsonl pnpm check:receipt .runs/lab01.jsonl$ node scripts/check-receipt.ts .runs/lab01.jsonl PASS R1 receipt: init record found, result record found PASS R2 Claude Code 2.1.304 PASS R3 permission mode dontAsk PASS R4 tools Bash, Edit, Read PASS R5 hooks: none configured in .claude/settings.json PASS R6 instructions: final message ends with "Rules loaded: refund-rules CLAUDE.md" PASS R7 run ended: success, 4 turns INFO model claude-sonnet-5-5; permission denials 0; estimated cost $0.0167 (client-side estimate) receipt: PASSOffline route: make the change by hand, then check what changed
$ git status --short git diff --statM src/window.ts src/window.ts | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Run the unit tests again
$ pnpm test$ node --test --test-reporter=./test/reporter.mjs --test-concurrency=1 "test/**/*.test.ts" PASS request under the limit is refunded in full PASS request over the limit is capped at the limit PASS non-integer cents throws RangeError PASS request 10 days after delivery is inside the window PASS request 45 days after delivery is outside the window PASS request on calendar day 31 is outside the window PASS invalid timestamp throws RangeError PASS T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery tests 8, pass 8, fail 0Verify with the copy of the verifier the change cannot edit
$ git show spec:scripts/verify-change.ts > ../verify-spec.ts && \ node ../verify-spec.tsPASS V1 verifier is the base copy: sha256 810b9c9f2c67 matches spec:scripts/verify-change.ts PASS V2 scope (T-1042): changed: src/window.ts PASS V3 protected files untouched: none changed PASS V4 existing tests intact: all base test titles present, none skipped PASS V5 unit tests, base checkout: pass 8, fail 0 PASS V6 acceptance tests, base checkout: pass 7, fail 0 PASS V7 typecheck, base checkout: no type errors PASS V8 spec failed before the change: at spec: fail 1; failing: T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery verdict: PASS receipt: .runs/verify-2026-10-21T00-01-52-696Z.jsonKeep the fix
$ git add -A git commit -q -m "fix: T-1042 count calendar days in UTC" git tag lab-001 git log --format='%s%d'fix: T-1042 count calendar days in UTC (HEAD -> main, tag: lab-001) spec: T-1042 day-30 refund window (tag: spec) baseline: synthetic refund-rules lab (tag: baseline)Show the solution discussion
The reference spec and fix are labs/solutions/lab-001/ in the lab download: TASK.md, test/window.test.ts and src/window.ts. From the workspace, node ../labs/solutions/apply-checkpoint.mjs lab-001 . copies them over your files, so commit or copy your work first. Try the steps yourself before you compare.
More labs in this guide
Lab 2: Reject the grace-period change and recover
Put the candidate on a branch from the spec, run what its summary ran, verify from the spec, prove each defect with a command, optionally ask a read-only reviewer, then roll back and prove you are back at the spec. About 30 minutes, offline except the optional reviewer. The six steps, with every output, are in labs/learner-labs.md in the lab download.
Lab 3: Make the boundary deterministic
See the controls fail their tests, implement the scope hook, the Stop hook and the deny rules until pnpm test:hooks passes 18 of 18, put the controls on a spec that still has the bug, watch them hold the boundary, and see what a plain claude -p takes from a repository. About 40 minutes; every step has an offline route. The six steps, with every output, are in labs/learner-labs.md in the lab download.
06Failure gallery
11 ways this breaks, and how you will know
Five of these reproduce in the lab; the other six were observed in this guide's recorded runs. Learn the symptom first; it is what you will actually see.
- F-01
candidate-graceReproducible"All tests pass" and the verifier fails
- Symptom
- Reproduce it with Lab 2, steps 1 to 3: labs/patches/candidate-grace.patch on a branch from spec. pnpm test prints "tests 5, pass 5, fail 0".
- Diagnosis
- The spec copy of the verifier fails V2 to V7. The patch narrowed the test script, rewrote a test and added an out-of-scope file the in-scope code imports.
- Fix
- Reject the change. Count tests as well as failures: five where eight used to run is a finding.
- Verify
- On the fix from Lab 1, the same verifier prints "verdict: PASS".
- F-02Reproducible
git restore leaves a file behind
- Symptom
- Reproduce it with Lab 2, step 6. After git restore --staged --worktree ., git status --short prints "?? src/grace.ts".
- Diagnosis
- The candidate created an untracked file, and git restore restores tracked paths only.
- Fix
- git clean -n to preview, then git clean -f src/grace.ts.
- Verify
- git status --short prints nothing, and pnpm test fails only the T-1042 spec test.
- F-03Observed
Committed allow rules are ignored
- Symptom
- Observed in run LIVE-04, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1; spike S1 the same day. stderr: "Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted."
- Diagnosis
- Project allow rules need the workspace trust dialog, which claude -p never shows. The same run applied the repository's deny rules and ran its hooks.
- Fix
- Pass grants with --allowedTools on each run, and keep only deny rules and hooks in committed settings.
- Verify
- The lab's route (--restricted, --settings, --allowedTools) prints no such warning, and pnpm check:receipt passes.
- F-04Observed
A failed run leaves an edit on disk
- Symptom
- Observed in run LIVE-03, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, --max-turns 2, n = 1; the same on 2026-10-17 with 2.1.302 and the same model, effort and cap (n = 1). Exit code 1, subtype error_max_turns, no final message.
- Diagnosis
- git status --short shows " M src/window.ts". The turn cap stopped the run, not the edit it had already made.
- Fix
- Decide from the repository, not the exit code: verify and review the leftover change, or pnpm reset --yes back to spec.
- Verify
- On LIVE-03's leftover edit, the verifier printed "verdict: PASS"; after a reset, git status --short prints nothing.
- F-05Reproducible
A hook matcher names a tool that does not exist
- Symptom
- Reproduce it on the Lab 3 reference by changing the PreToolUse matcher to Edit|MultiEdit|Write, which is close to version 1.0.0's matcher. Nothing at run time: a name that matches no tool is never called, so nothing fails.
- Diagnosis
- pnpm test:hooks fails "every hook matcher names only tools that exist" with PreToolUse matcher names unknown tool "MultiEdit".
- Fix
- Name only tools the run's system/init record lists: Edit|Write|NotebookEdit on 2.1.304.
- Verify
- pnpm test:hooks prints "tests 18, pass 18, fail 0".
- F-06Reproducible
A test run that runs nothing, and exits 0
- Symptom
- Reproduce it at spec, where one test fails, with NODE_TEST_CONTEXT=child-v8 node --test --test-reporter=./test/reporter.mjs "test/**/*.test.ts". "Warning: node:test run() is being called recursively within a test file. skipping running files." and exit code 0.
- Diagnosis
- A process started from inside a Node.js test run inherits NODE_TEST_CONTEXT, and a child node --test that sees it skips its files. A hook or verifier tested from node --test would report green while a test fails.
- Fix
- Remove NODE_TEST_CONTEXT from the environment of every child test run, as the lab's Stop hook and verifier do.
- Verify
- The same command without the variable ends with "tests 8, pass 7, fail 1".
- F-07Observed
Every control holds, and the session does unrequested work
- Symptom
- Observed in run LIVE-06, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, dontAsk, n = 1. The prompt asked for a rename in src/refund.ts; git status --short shows " M src/window.ts".
- Diagnosis
- The scope hook denied both edits to src/refund.ts. The Stop hook's block named the failing T-1042 test, so Claude fixed T-1042 instead. The final message said the rename was not done.
- Fix
- Keep the prompt, scope.json and the Stop hook pointed at one task, for example by having the Stop hook name the task from scope.json.
- Verify
- The verifier against spec-drift passed, and the receipt check counted three scope-hook runs and two Stop-hook runs.
- F-08
weakened-verifierReproducibleA weakened verifier in the working tree prints PASS
- Symptom
- Reproduce it at spec with labs/patches/weakened-verifier.patch, a one-line change that records every check as passed. pnpm verify prints "verdict: PASS", though its own V1 line says the file "differs from spec:scripts/verify-change.ts".
- Diagnosis
- pnpm verify runs the working tree's copy, which the change edited.
- Fix
- Run the spec copy: git show spec:scripts/verify-change.ts > ../verify-spec.ts && node ../verify-spec.ts.
- Verify
- The spec copy fails V2 and V3 (scripts/verify-change.ts changed) and V5 and V6 (the bug is still there).
- F-09Observed
A "/" path rule protects the wrong folder
- Symptom
- Observed in spike S10 and S11, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, --settings .claude/settings.json, n = 1 each. With Edit(/test/) denied in a file passed as --settings .claude/settings.json, an edit to test/ went through.
- Diagnosis
- A leading "/" is relative to the settings source, here .claude/, so the rule covered .claude/test/.
- Fix
- Write path rules with "./", which anchors at the working directory in project settings and in --settings alike.
- Verify
- With Edit(./test/), the edit was denied: "File is in a directory that is denied by your permission settings."
- F-10Observed
--bare drops the hooks you passed
- Symptom
- Observed in spike S6, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, --bare with --settings, n = 1. A bare run with a PreToolUse hook in --settings shows no hook events, and pnpm check:receipt would fail R5.
- Diagnosis
- On 2.1.304, --bare skips "hooks (those defined in settings and by installed plugins; ...)," including a file named with --settings. The documentation describes skipping hook discovery only.
- Fix
- For hooks you choose, use --restricted with --settings, as the labs do.
- Verify
- Spike S8 and LIVE-05 show the same hooks running under --restricted.
- F-11Observed
Auto mode lets a shell command make an out-of-scope edit
- Symptom
- Observed in run LIVE-09, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, --permission-mode auto, n = 1. The receipt check passes, and the final message says "The day-30 bug is fixed and everything passes." git status --short lists " M src/refund.ts", the file the scope hook exists to protect.
- Diagnosis
- Claude made the rename with sed -i 's/assertCents/assertWholeCents/g' src/refund.ts, and auto mode ran it. The scope hook matches file-edit tools, so it never ran: R5 counted 0 runs for 0 matching calls. In dontAsk mode (F-07), the same command was denied because no grant matched it.
- Fix
- Use dontAsk with exact grants when the boundary must hold during the run. In any mode, let the verifier's scope check decide after the run.
- Verify
- The spec copy of the verifier failed V2: "out of scope: src/refund.ts".
07Assessment
Transfer task and rubric
Fix a refund-limit bug from a frozen spec with bounded grants, judge a candidate change that passes the verifier, and prove you returned to the spec.
| Criterion | 0 · Emerging | 1 · Developing | 2 · Proficient | 3 · Exemplary |
|---|---|---|---|---|
| C1 Frozen spec | No failing test at spec, or the test fails for an unrelated reason (syntax, import), or the brief has no runnable done criteria | The test fails at spec on the T-3118 assertion, but the brief leaves the editable files open or states some done criteria as prose instead of commands | The test fails at spec on the T-3118 assertion; TASK.md names src/refund.ts as the only editable file and lists done criteria as commands; verifier V8 passes | Proficient, and the brief states the policy rule completely, including every bound on the result, and the test name ties to the ticket |
| C2 Least-privilege controls | Mode left to its default, or a grant as broad as bypassPermissions, Bash(*) or an Edit with no path | Explicit mode and a path-scoped edit grant, but one grant is wider than the task needs (for example Edit(./src/) or "Bash(pnpm *)"), or the scope file still names T-1042 at spec | Explicit mode; edits limited to src/refund.ts by grant or by grant plus scope hook; deny rules and hooks in committed settings; scope file updated before spec | Proficient, and explains which layer enforces each rule, and why the grants came from the command line rather than the repository's settings |
| C3 Correct, bounded change | Hidden tests fail, or the diff touches files outside scope | All hidden tests pass and git diff --stat spec lists only src/refund.ts, but the verifier was not run from the spec copy, or it failed a check other than scope | All hidden tests pass; git diff --stat spec lists only src/refund.ts; the spec copy of the verifier passes all eight checks | Proficient, and adds a committed test for an edge case beyond the reported one (in a new spec commit), with reasoning |
| C4 Run receipt | Missing, or describes an offline edit as an agent run | Present, but a field is missing or disagrees with the stream (for example the turn count or the mode) | All receipt fields present and consistent with the stream or the offline statement | Proficient, and notes anything surprising in the receipt (denials, retries, cost) and what it implies |
| C5 Candidate review | Accepts the candidate, or rejects it without executed evidence | Rejects it with executed evidence that does not isolate the defect, such as a pnpm decide call or an existing test, instead of a new test that fails on the candidate | Rejects it with a new test that fails on the candidate, quoting its output | Proficient, and explains why every verifier check passed, and names the requirement that was not encoded |
| C6 Recovery | No proof of the known state, or recovery commands run on the wrong branch or workspace | Recovery commands shown and git status --short empty, but no rerun showing the spec test failing again | git status --short empty and the spec test failing again, with the commands that got there | Proficient, and explains what git restore and checkpoints would and would not have restored |
Proficiency standard: Proficient or higher on every criterion. Proposed local standard, not an official certification.
The full brief
Ticket T-3118 (synthetic): order ORD-7304 already received a $20.00 refund, and the customer asks for $48.00 more. Policy POL-DOA-30 caps the total refunded on one order at $60.00, so the correct refund is $40.00 (4000 cents). refundableCents in src/refund.ts ignores earlier refunds and returns 4800.
Start from the finished Lab 3 controls, in a new workspace next to labs/. Write a failing test for T-3118, a TASK.md and a .claude/scope.json that limit edits to src/refund.ts, then commit and tag spec. Produce the fix on either route: live, a claude -p run with an explicit permission mode, path-scoped grants, --max-turns, --max-budget-usd and a recorded stream; or offline, a fix you write yourself and say so. Run the spec copy of the verifier and keep its output.
Then judge a second change. On a branch from spec, apply labs/patches/asm-candidate.patch, a one-line fix written for this assessment to represent a plausible generated change (it is not captured model output). Decide whether to accept it, and support the decision with at least one new test you write and run. Finally, return that branch to the spec state and show that you are there.
Submit one folder or Markdown file with your spec, your run receipt, your diff and verifier output, your review notes and new test, and your recovery commands. Allow about 45 minutes. A reviewer runs hidden tests against your fix, reruns the verifier from your spec tag, and scores each criterion with the rubric.
Pass rule (a proposed local standard): Proficient (2) or higher on every criterion. Critical failure: a failing hidden test (C3), or a grant as broad as bypassPermissions, --dangerously-skip-permissions, Bash(*) or an Edit with no path (C2), fails the module regardless of every other score, as do editing tests or the verifier to make checks pass, running against real code or data, including a secret, and presenting a hand-written fix as an agent run. The rubric's shape follows the University of Hawaiʻi at Mānoa's guide to analytic rubrics, which scores each criterion separately and lists "Emerging, developing, proficient, exemplary" among its common level labels (UH Mānoa), and Carnegie Mellon's Eberly Center, which defines a rubric's criteria, descriptors and performance levels (Carnegie Mellon).
08Instructor kit
Teach it to a room
Facilitation guide and run of show
A run of show for a 12-person lab of about 170 minutes, hint ladders for the steps where learners get stuck, three demo scripts and an optional live demo, must-know misconceptions, live-room troubleshooting and accessibility notes.
Cohort sizes and fallbacks
An adaptation for a 100-person training day of about 4.5 hours, with breakouts of about 12 and the offline route as the default, a recovery decision tree, and two reference checkpoints, lab-001 and lab-003, so a stalled learner or breakout can rejoin.
Dry run and observation rubric
A train-the-trainer sequence, a dry-run protocol of about 70 minutes, an observation rubric, a proposed clearance rule and the triggers for observing an instructor again. It is a new procedure; no instructor has been observed with it.
Module clearance record
A pending template for recording that an instructor is cleared to teach this module version, filled in by an authorized person after an observed dry run. No instructor has been cleared with it.
Evaluation plan and pass-rate report
What is evaluated and when, data handling rules, module quality signals, a maintenance procedure and an issue log, and a blank monthly pass-rate report. No learner or instructor results exist yet.
10Sources and verification
- [1]Advanced setup · code.claude.comchecked Oct 20, 2026
- [2]CLI reference · code.claude.comchecked Oct 20, 2026
- [3]Run Claude Code programmatically · code.claude.comchecked Oct 20, 2026
- [4]Choose a permission mode · code.claude.comchecked Oct 20, 2026
- [5]Configure permissions · code.claude.comchecked Oct 20, 2026
- [6]Hooks reference · code.claude.comchecked Oct 20, 2026
- [7]Sandboxing · code.claude.comchecked Oct 20, 2026
- [8]How Claude remembers your project · code.claude.comchecked Oct 20, 2026
- [9]Best practices · code.claude.comchecked Oct 20, 2026
- [10]Checkpointing · code.claude.comchecked Oct 20, 2026
- [11]Model configuration · code.claude.comchecked Oct 20, 2026
- [12]Manage costs effectively · code.claude.comchecked Oct 20, 2026
- [13]Models overview · platform.claude.comchecked Oct 20, 2026
- [14]Claude Code changelog · github.comchecked Oct 20, 2026
- [15]Registry entry for @anthropic-ai/claude-code · registry.npmjs.orgchecked Oct 20, 2026
- [16]git-restore · git-scm.comchecked Oct 20, 2026
- [17]git-clean · git-scm.comchecked Oct 20, 2026
- [18]Modules: TypeScript (v22.x source) · github.comchecked Oct 20, 2026
- [19]Release schedule · github.comchecked Oct 20, 2026
- [20]Distribution index · nodejs.orgchecked Oct 20, 2026
- [21]Registry entry for pnpm · registry.npmjs.orgchecked Oct 20, 2026
- [22]Registry entry for typescript · registry.npmjs.orgchecked Oct 20, 2026
- [23]Registry entry for @types/node · registry.npmjs.orgchecked Oct 20, 2026
- [24]e-Handbook of Statistical Methods, 7.2.4.1 Confidence intervals · itl.nist.govchecked Oct 20, 2026
- [25]Creating and using rubrics · manoa.hawaii.educhecked Oct 20, 2026
- [26]Eberly Center, Grading and performance rubrics · cmu.educhecked Oct 20, 2026
Independent work. Not affiliated with or endorsed by any vendor named in this guide.
