Skip to content
Jordan Hudgens
Illustration: a small matte-white capsule robot lowers a machined mint part into a go/no-go gauge block on a dark workbench, while a rejected amber part sits on a tray beside it.AI-generated illustration
Guide 05IntermediateClaude Code and agentic codingEvaluation and testingSecurity and containment

Claude Code Workflows You Can Verify

Freeze the spec before the agent touches code, grant only what the task needs, and accept a change only on evidence the change could not edit.

Reading
75 min
Workshop
105 min
Stack
TS · Node.js 22 · Claude Code 2.1.304 · Git
Version
1.1.0
Last verified
Oct 21, 2026

What you will build and be able to do

Each outcome has an observable criterion. The lab and the assessment check exactly these, nothing more and nothing less.

  1. OBJ-001

    Turn a bug report into a frozen spec

    At the spec tag, pnpm test fails only your T-1042 test, and after the fix the spec copy of the verifier prints "verdict: PASS" with V8 naming that test.

  2. OBJ-002

    Choose least-privilege controls, each in the layer that enforces it

    pnpm test:hooks prints "fail 0" with the deny rules and both hooks committed and every path rule written with "./", and you can say that the committed settings grant nothing because every grant comes from --allowedTools.

  3. OBJ-003

    Run Claude Code headless within bounds, and capture a run receipt

    On the live route, pnpm check:receipt prints "receipt: PASS" for your run, and you can say which line of the command each R check depends on; on Lab 3's branch, it shows the scope hook running.

  4. OBJ-004

    Reject an attractive but incorrect change on evidence

    Your review names the five defects in the candidate, each with the output line that proves it, and explains why pnpm test reported five tests and not eight.

  5. OBJ-005

    Recover to a known state after a failed or rejected run

    After the rollback, git status --short prints nothing and pnpm test shows exactly one failing test, the T-1042 spec test.

Runs offline in minutes

  • Git 2.40 or later: commit, tag, diff, status, git restore and git clean.Install Git
  • A terminal running bash or zsh on Linux, macOS or WSL.
  • Node.js 22.18 or later, for TypeScript type stripping, and pnpm 12, which corepack enable provides.Install Node.js 22
  • For the live route: Claude Code 2.1.304 and an Anthropic API key. Every live run in this guide used an API key; signing in with a Claude account was not tested with these commands. Every lab also has an offline route that needs neither.Claude Code setup
  • Reading a unit test in any language. The lab uses TypeScript with the built-in Node.js test runner; if you read JavaScript, Python or Go, you can follow it.
Terminalbash
bash labs/new-workspace.sh labs/starter cc-labcd cc-labpnpm install --frozen-lockfilepnpm preflightok    node: v22.23.3 (need 22.18 or later for TypeScript type stripping)ok    pnpm: 12.14.0ok    git: git version 2.53.0ok    workspace: inside a git repositoryok    working tree: cleanok    claude: 2.1.304 (Claude Code)warn  ANTHROPIC_API_KEY: not set (fine if you signed in with claude, or for the offline route)
Lab starter, patches, checkpoints and reference solutions · ZIP · tested at v1.1.0

Put each rule where it can be enforced

Guidance shapes the session; grants and restrictions bound it; verification and Git decide what you keep. Synthetic data. Deterministic diagram drawn from the lab's controls.
Diagram of five layers in two groups. Inside the Claude Code session: guidance (CLAUDE.md, TASK.md), which asks; per-run grants (--allowedTools), which bound this run; and committed restrictions (deny rules, hooks), which bound every run. After the session ends: independent verification (the verifier from the spec), which decides, and version control (git tag, restore), which rolls back.
Extended description

Five horizontal rows in two groups, each row with a layer name, what the layer does, and where it lives in the lab. Group "Inside the Claude Code session": (1) Guidance, dashed amber border, "asks", "CLAUDE.md, TASK.md". (2) Per-run grants, blue border, "bounds this run", "--allowedTools". (3) Committed restrictions, blue border, "bounds every run", "deny rules, hooks". Group "After the session ends": (4) Independent verification, mint border, "decides", "verifier from spec". (5) Version control, mint border, "rolls back", "git tag, restore". Footer: "Mechanisms checked against the Claude Code docs on 2026-10-20 (Claude Code 2.1.304). Synthetic data."

Rules that must always hold belong in the lower, enforced layers; the accept or reject decision happens outside the session. The table in "Where each control lives" gives each layer's full guarantee.

Why "it says the tests pass" is not evidence

A coding agent is a model in a loop with tools: it reads files, edits them and runs commands until it decides it is done, then writes a summary. The summary is a claim. It may be accurate. It may also describe the tests the agent ran after it changed which tests run.

Here is a summary from this guide's second lab. It reads well:

Every sentence is true, and the change is wrong in five ways. You will find all five with commands when you review it later in this guide. (That summary and its patch were written for the lab to represent a plausible generated change; they are not captured model output.)

A workflow you can verify replaces trust in the summary with five pieces of evidence you control:

  1. A frozen spec. A failing test that reproduces the bug, committed and tagged before the agent runs, plus a written scope.
  2. Explicit grants. The session can do only what the task needs, and you chose each permission on purpose.
  3. Independent checks. Tests, acceptance cases, scope and integrity checks that run outside the session, from code the change could not edit.
  4. A reviewed diff. A person, or a separate read-only review session, reads the change against the spec, including what the checks cannot see.
  5. A recovery path. You can return to the spec state in one step and rerun.

Anthropic's own guidance puts verification first: "Give Claude a way to verify its work" with tests, a build or expected outputs, and ask it to show evidence such as "the test output, the command it ran and what it returned" (best practices). This guide adds the other half: you verify too, with checks the agent did not get to edit.

Where each control lives

The most common mistake with coding agents is putting a rule in a layer that cannot enforce it. A prompt that says never to edit the tests is making a request. The same rule as a deny rule, a hook or a scope check is a control. Claude Code's documentation draws the same line: CLAUDE.md files are treated "as context, not enforced configuration," and a PreToolUse hook is the way to block an action regardless of what Claude decides (memory). Hooks are "deterministic and guarantee the action happens," while CLAUDE.md is advisory (best practices).

Five layers, from softest to hardest:

The five layers and what each can guarantee
LayerClaude Code mechanismWhat it can guaranteeHow it fails
GuidanceCLAUDE.md, the prompt, TASK.mdNothing. It shapes behaviorThe model weighs it against other instructions and context
Per-run grants--permission-mode, --allowedTools, --tools, --max-turns, --max-budget-usdWhat this run may do without askingGrants that are too broad; a mode left to its default; a classifier mode that allows what no grant names
Committed restrictionspermissions.deny rules and hooks in .claude/settings.jsonWhat no run that loads them may doA rule that does not match the call as written; a hook that never sees the call; a run that never loads the file
Independent verificationTests and a verifier you run after the session, from the specWhether the change meets what you encodedChecks that do not cover the real requirement
Version controlGit commits, tags, git restore, git cleanA known state you can return toUntracked files and uncommitted work you forget about

The decision behind the table holds for any agent: anything that must always be true goes in a layer that does not depend on the model's judgment, and the final accept or reject happens outside the session.

Set up a disposable lab

Every command in this guide runs in a throwaway copy of a synthetic repository. Do not point these workflows at real code until you have run them here.

The repository is refund-rules, a small TypeScript library for a fictional retailer, Example Outfitters. Policy POL-DOA-30 says a refund request is inside the window when it is made on or before calendar day 30 after delivery, counting days in UTC with the delivery day as day 0, and that refunds on one order may not exceed $60.00 in total. Ticket T-1042 reports the bug you will fix: a customer whose tent pole section arrived snapped asked for the $48.00 replacement pole set on calendar day 30 and was told the request was late.

Install the pinned version for the live route. The native installer takes a version argument (setup):

Terminalbash
curl -fsSL https://claude.ai/install.sh | bash -s 2.1.304

Native installs update in the background. For a class or a CI job where everyone must run the same version, set DISABLE_AUTOUPDATER=1 or use the stable channel, and record the version in every receipt (setup). The lab's receipt check does that for you.

Unzip the lab download into an empty folder, and from that folder create the workspace, install its two pinned development dependencies, check the toolchain and look at the bug:

Terminalbash
bash labs/new-workspace.sh labs/starter cc-labcd cc-labpnpm install --frozen-lockfilepnpm preflightpnpm decide T-1042
Outputtext
$ node scripts/decide.ts T-1042{  "ticketId": "T-1042",  "policyId": "POL-DOA-30",  "eligible": false,  "refundCents": 0,  "reason": "outside refund window"}

The seven existing unit tests all pass, which is the first lesson: a green suite only tells you about the cases someone wrote down. The read-only acceptance cases in acceptance/ encode the policy's boundaries, and two of them fail on this code:

Output (excerpt)text
FAIL  A1 T-1042: calendar day 30, later clock time than delivery      Expected values to be strictly equal:      false !== truePASS  A2 calendar day 31, earlier clock time than deliveryFAIL  A3 calendar day 30, almost 31 elapsed days

Every lab run in this guide pins --model claude-sonnet-5-5; the spike probes passed the alias sonnet, which resolved to claude-sonnet-5-5 on 2026-10-20. Aliases such as sonnet "update over time," and a full model name pins a version (model configuration). Without --model, Anthropic API accounts default to Opus 5.5, according to the same page on 2026-10-20, which lists at twice Sonnet 5.5's price per token: $4 and $20 against $2 and $10 per million input and output tokens (models overview).

Freeze the spec before the agent touches code

The spec is three files, committed together and tagged, before any agent run:

  1. A failing test that reproduces the reported bug for the reported reason.
  2. TASK.md: the goal, the policy, the file scope, what not to touch, and done criteria written as commands.
  3. .claude/scope.json: the same file scope in machine-readable form, for the hook and the verifier.

Why freeze it first? If the agent writes the reproduction test, the same session decides both what "fixed" means and whether it is fixed. Claude Code's guidance suggests asking it to "write a failing test that reproduces the issue, then fix it" (best practices). In this workflow, you write that test (or you review and commit the one the agent proposes in plan mode), and the agent's job shrinks to making a committed test pass without editing it.

Add the reproduction test to test/window.test.ts:

window.test.tsts
test('T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery', () => {  assert.equal(isWithinWindow('2026-09-14T18:30:00Z', '2026-10-14T19:05:00Z', 30), true);});

Run pnpm test and confirm it fails for the right reason: an assertion on this case, not a syntax error or an import failure.

Output (excerpt)text
FAIL  T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery      Expected values to be strictly equal:      false !== truetests 8, pass 7, fail 1

Write the brief. Keep it short and testable; every "done" line is a command with a pass or fail result:

TASK.md (excerpt)markdown
## Scope- Edit only `src/window.ts`.- Do not edit tests, `acceptance/`, `package.json`, `pnpm-lock.yaml`, `tsconfig.json`, `scripts/`, `.claude/`, `CLAUDE.md` or `TASK.md`.- Do not add dependencies. Do not commit.- If the fix seems to need another file, stop and explain which file and why. ## Done when- `pnpm test` passes, including the new T-1042 test.- `pnpm test:acceptance` passes (cases A1 to A7 are the policy's boundary cases).- `pnpm typecheck` passes.- Your final message lists each command you ran with its pass and fail counts, and a two-sentence explanation of the root cause.

The full brief is in the handout and in labs/solutions/lab-001/TASK.md. The scope file says the same thing for machines, and the starter already has it:

scope.jsontext
{  "task": "T-1042",  "allowEdits": ["src/window.ts"],  "note": "Edit this file in the spec commit, before any agent run. The verifier reads it from the base commit."}

Commit and tag:

Terminalbash
git add -Agit commit -q -m "spec: T-1042 day-30 refund window"git tag specgit log --format='%s%d'

The spec tag is now the known state you will verify against and return to.

Plan, freeze, run, verify, review, keep or roll back. Each stage leaves an artifact someone else can check. Live: claude-sonnet-5-5, effort medium, Claude Code 2.1.304, n = 1, for the run's turns and estimated cost; the other values are from the lab's offline steps. Recorded 2026-10-20. Synthetic data. Deterministic diagram; its animation builds the six stages in order. Silent loop, 14 seconds.
Extended description

A heading, "Six stages, six pieces of evidence", over six numbered cards in a U shape. Top row, left to right: 1 Plan (TASK.md, scope.json; chip "edits: src/window.ts"), 2 Freeze spec (failing test, git tag spec; chip "pass 7, fail 1"), 3 Bounded run (claude -p, dontAsk + grants; chip "4 turns, $0.0167"). An arrow goes down to the bottom row, which reads right to left: 4 Verify (spec copy, V1 to V8; chip "verdict: PASS"), 5 Review (git diff spec, five questions; chip "1 file changed"), 6 Keep or roll back, outlined in mint (commit + receipts, or restore, clean; chip "status clean"). Footer: "Live: claude-sonnet-5-5, effort medium (stage 3); other stages offline. Recorded 2026-10-20. Synthetic data."

Each stage leaves an artifact a reviewer can check without rereading the session. The chips are recorded values from one walk of the lab on 2026-10-20 (stage 3 from live run LIVE-01: Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1), not typical results. "status clean" means git status --short printed nothing after the clean.

The heading appears, then the six cards described above one at a time in workflow order, each followed by its evidence chip and an arrow to the next card. It ends on the frame described above and holds for about 3.6 seconds.

Same content as the frame described above, revealed in the order the stages run.

Run Claude Code headless with explicit grants

You can do this lab interactively: start in plan mode with claude --permission-mode plan, read the plan, then approve it (best practices). This guide uses non-interactive mode, claude -p, because every control can be named on the command line and recorded, which is what a reviewer needs.

Three defaults are why every control is named on the command line:

  • The starting permission mode. "In Claude Code v2.1.302 or later, claude -p and Agent SDK runs with no permission mode set start in Manual mode," where calls that need approval are denied unless something approves them (permission modes, checked 2026-10-20). When this guide was first checked, on 2026-10-17, the headless page described that starting mode as one that can be auto. Pass the mode you want.
  • What a plain claude -p discovers. Without other flags it reads the repository's .claude/settings.json and CLAUDE.md. The headless page calls --bare, which skips that discovery, "the recommended mode for scripted and SDK calls" and says it "will become the default for -p in a future release" (headless). A route that depends on discovery would silently lose its hooks on that day.
  • What --bare drops on this version. The docs say bare mode skips the auto-discovery of hooks. On 2.1.304 it also dropped the hooks in a file passed with --settings: in this guide's spike (2026-10-20, claude-sonnet-5-5, effort medium, n = 1), a bare run with a hook in --settings ran no hook, and the CLI's own --help says bare mode skips "hooks (those defined in settings and by installed plugins; ...)." Where the docs and a receipt disagree, this guide teaches the receipt.

So the lab's route does not depend on discovery or on bare mode. It starts from --restricted and names every input:

Terminalbash
mkdir -p .runsclaude -p "$(cat TASK.md)" \  --restricted --tools Bash,Read,Edit \  --settings .claude/settings.json \  --append-system-prompt-file CLAUDE.md \  --permission-mode dontAsk \  --allowedTools "Edit(./src/window.ts)" "Bash(pnpm test)" "Bash(pnpm test:acceptance)" "Bash(pnpm typecheck)" "Bash(git diff *)" "Bash(git status *)" \  --model claude-sonnet-5-5 --effort medium \  --max-turns 12 --max-budget-usd 0.50 \  --no-session-persistence --include-hook-events \  --output-format stream-json --verbose > .runs/lab01.jsonlpnpm check:receipt .runs/lab01.jsonl

What each choice buys you:

The headless command, flag by flag
FlagDecision
--restrictedLoads only managed settings and the file --settings names, so nothing from your user or project settings files. It also removes the tools that run commands unless --tools names them, keeps the file tools inside the working directories and refuses bypassPermissions (CLI reference)
--tools Bash,Read,EditThe whole tool set for the run, with Bash named so that --restricted keeps it: no Write, no web tools
--settings .claude/settings.jsonThe repository's deny rules and hooks, loaded because you named the file, not because Claude Code found it
--append-system-prompt-file CLAUDE.mdThe repository's instructions, passed explicitly
--permission-mode dontAskAny call that would prompt is denied, and the run never waits for a person. File reads in the working directory and a built-in set of read-only shell commands, such as tail, need no grant and still run (permission modes, read-only commands)
Edit(./src/window.ts)One editable file. A path that starts with ./ is relative to the current directory (permissions)
Bash(pnpm test) and the restThe exact commands in the done criteria, nothing broader such as Bash(pnpm *)
--max-turns 12, --max-budget-usd 0.50Hard stops. The budget is checked against a client-side estimate, and "Spend can pass the cap" (CLI reference)
--no-session-persistenceThe session is not saved to your history
--include-hook-events, --output-format stream-json --verboseEvery tool call, hook run, denial and the final result record land in .runs/lab01.jsonl

Check the run's own record

A command line says what you asked for; the stream says what the run got. The lab's pnpm check:receipt reads the stream's first record (system/init) and its last (result) and checks seven things: both records exist (R1), the Claude Code version (R2), the permission mode (R3), the tool set (R4), a run of each configured hook for each matching tool call (R5), the instructions (R6) and how the run ended (R7). R6 works through a canary: CLAUDE.md asks Claude to end its final message with one exact line, so a run whose instructions never arrived fails R6. A missing line can also mean the model ignored it, so a failing R6 is a reason to look, not a verdict.

Receipt excerpt, run LIVE-01 (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1):

Outputtext
$ node scripts/check-receipt.ts .runs/lab01.jsonlPASS  R1 receipt: init record found, result record foundPASS  R2 Claude Code 2.1.304PASS  R3 permission mode dontAskPASS  R4 tools Bash, Edit, ReadPASS  R5 hooks: none configured in .claude/settings.jsonPASS  R6 instructions: final message ends with "Rules loaded: refund-rules CLAUDE.md"PASS  R7 run ended: success, 4 turnsINFO  model claude-sonnet-5-5; permission denials 0; estimated cost $0.0167 (client-side estimate) receipt: PASS

The final message of that run:

Final messagetext
T-1042 is fixed with a change to `src/window.ts` only, and all three checks pass. - `pnpm test`: 8 tests, 8 pass, 0 fail (includes the new T-1042 test)- `pnpm test:acceptance`: 7 tests, 7 pass, 0 fail (A1 to A7)- `pnpm typecheck` (`tsc --noEmit`): passed with no errors. It prints no counts.

The diff it produced:

window.ts (diff)diff
-  const elapsed = requested - delivered;-  return elapsed >= 0 && elapsed <= windowDays * DAY_MS;+  if (requested < delivered) {+    return false;+  }+  const deliveryDay = Math.floor(delivered / DAY_MS);+  const requestDay = Math.floor(requested / DAY_MS);+  return requestDay - deliveryDay <= windowDays;

The same command passed the verifier in 2 of 2 runs that day (LIVE-01 at $0.0167 and LIVE-02 at $0.0169, estimated; the two diffs differed and behaved the same). At 95% confidence, a Wilson interval puts 2 of 2 as consistent with a per-run failure rate as high as 66% (NIST e-Handbook). That shows the route can work, not how often it works. The summary is still a claim; the next section is how you check it.

Caps, modes and what they report

Both caps end a run with an error you can read from the result record. In this guide's spike (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1 each), --max-turns 1 ended with exit code 1, subtype error_max_turns and the error "Reached maximum number of turns (1)", while num_turns reported 2. --max-budget-usd 0.001 ended with exit code 1, subtype error_max_budget_usd and an estimated $0.0040, about four times the cap, which is why the CLI reference tells you to leave headroom.

Interactive sessions start in auto mode from v2.1.283, where a separate classifier model reviews actions; the documentation says it "reduces permission prompts but does not guarantee safety" (permission modes). Auto mode was available to claude -p with claude-sonnet-5-5 and an API key on 2.1.304 (spike ST4, 2026-10-20, effort medium, n = 1). Treat a classifier's decision as an observation, never as a control. The failure gallery shows what one auto-mode run allowed.

Verify with checks the change cannot edit

The agent ran the tests. So why run them again? Because the agent ran them through package.json scripts it could, in another run, have edited, against tests it could have edited, and reported the results in its own words. Your verification must not depend on anything inside the change.

The lab's verifier is scripts/verify-change.ts, but you never run the working tree's copy, which a change could edit. Take the copy committed at spec, put it outside the working tree, and run that:

Terminalbash
git show spec:scripts/verify-change.ts > ../verify-spec.ts && \  node ../verify-spec.ts

It checks eight things against the spec tag:

Verifier checks V1 to V8
CheckWhat it provesWhy it is independent
V1 the base copyThe program running is byte-identical to the copy at specCompares its own SHA-256 with git show spec:scripts/verify-change.ts
V2 scopeEvery changed or new file is in allowEditsReads scope.json from the spec commit, not the working tree, and includes untracked files
V3 protected filespackage.json, the lockfile, tsconfig.json, acceptance/, .claude/, scripts/, CLAUDE.md and TASK.md are untouchedA fixed list in the verifier
V4 tests intactNo existing test was removed, renamed, skipped or focusedCompares test titles at spec with the working tree
V5 unit teststest/ passesRuns in a fresh checkout of spec with only the in-scope files copied in
V6 acceptanceacceptance/ passesThe same fresh checkout
V7 typechecktsc --noEmit passesThe same checkout, with the toolchain installed offline from the spec lockfile
V8 reproductionThe tests failed at specRuns them at spec

Output after the fix:

Output (excerpt)text
PASS  V1 verifier is the base copy: sha256 810b9c9f2c67 matches spec:scripts/verify-change.tsPASS  V2 scope (T-1042): changed: src/window.tsPASS  V3 protected files untouched: none changedPASS  V4 existing tests intact: all base test titles present, none skippedPASS  V5 unit tests, base checkout: pass 8, fail 0PASS  V6 acceptance tests, base checkout: pass 7, fail 0PASS  V7 typecheck, base checkout: no type errorsPASS  V8 spec failed before the change: at spec: fail 1; failing: T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery

V8 is the one most teams skip. If the spec test passed before the change, it never reproduced the bug, and a green result afterward proves nothing.

V5 to V7 run in a fresh checkout because the working tree is part of the change. A change that edits package.json, a test helper or the verifier itself could change what a check measures. In the base checkout, the only thing under test is the part of the change you allowed. Version 1.0.0 of this lab ran them in the working tree and noted that a change could weaken the verifier through a file it depends on; running the spec copy against a base checkout closes that gap. The failure gallery shows a one-line edit that makes the working tree's copy print verdict: PASS for a change the spec copy fails.

The verifier writes a JSON receipt to .runs/ with the base and head commits. Attach it, the run's stream and the diff to the pull request. That bundle is the evidence; the agent's summary is a convenience.

Review an attractive but incorrect change

Now the change from the opening. Put it on a branch from the spec, apply labs/patches/candidate-grace.patch, and do what most people do first, pnpm test:

Outputtext
$ node --test --test-reporter=./test/reporter.mjs --test-concurrency=1 test/window.test.tsPASS  request 10 days after delivery is inside the windowPASS  request 45 days after delivery is outside the windowPASS  request on calendar day 31 is inside the grace periodPASS  invalid timestamp throws RangeErrorPASS  T-1042: request on calendar day 30 is inside the window even when the clock time is later than deliverytests 5, pass 5, fail 0

Green, and only five tests. Now the evidence:

Output (excerpt)text
PASS  V1 verifier is the base copy: sha256 810b9c9f2c67 matches spec:scripts/verify-change.tsFAIL  V2 scope (T-1042): out of scope: package.json, src/grace.ts, src/refund.ts, test/window.test.ts (untracked: src/grace.ts)FAIL  V3 protected files untouched: changed: package.jsonFAIL  V4 existing tests intact: test/window.test.ts: removed or renamed "request on calendar day 31 is outside the window"FAIL  V5 unit tests, base checkout: pass 3, fail 1; failing: test/window.test.tsFAIL  V6 acceptance tests, base checkout: pass 0, fail 1; failing: acceptance/window.acceptance.test.tsFAIL  V7 typecheck, base checkout: src/window.ts(1,28): error TS2307: Cannot find module './grace.ts' or its corresponding type declarations.PASS  V8 spec failed before the change: at spec: fail 1; failing: T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery

V5 to V7 fail before they reach any test case: the one in-scope file, src/window.ts, imports src/grace.ts, which is out of scope, so the base checkout does not have it. A change has to work with only the files it was allowed to touch. Two more commands show the rest. pnpm test:acceptance in the working tree fails A2 and A6, both calendar day 31. Running every unit test file, which the candidate's test script no longer does, shows one more:

Output (excerpt)text
FAIL  non-integer cents throws RangeError      Missing expected exception (RangeError).

Five defects, each with its proof:

  1. Wrong logic that passes the reported case. A one-day grace period on elapsed time makes T-1042 pass but accepts day-31 requests (A2 and A6). The root cause, elapsed hours versus calendar days, is untouched.
  2. A weakened test. The existing day-31 test was renamed and its expectation flipped to match the new behavior (V4).
  3. A narrowed test command. package.json now runs only test/window.test.ts, which is why pnpm test ran five tests instead of eight (V3, and the count itself).
  4. An unrelated behavior change. The "tidied" cents validation no longer rejects 48.5 cents, and the refund test the narrowed script stopped running now fails.
  5. A new file outside the scope. src/grace.ts is untracked, so git diff never shows it (V2), and the in-scope code depends on it (V7).
Passes the happy path, fails the edge case. The grace-period change fixes the reported ticket and breaks day 31. Offline: no model involved. Synthetic data. Deterministic diagram drawn from the lab's acceptance cases, recorded 2026-10-20; its animation adds the three implementations one at a time. Silent loop, 15 seconds.
Extended description

A line at the top: "Order ORD-5521, delivered Sep 14, 18:30 UTC". Below it, a time ruler across two calendar days after delivery: day 30 (Oct 14 UTC, mint band, inside the policy) and day 31 (Oct 15 UTC, amber band, outside). Three vertical limit lines, each with a label: a dashed amber line at Oct 14 18:30, "Starter: 30 x 24 h"; a solid mint line at Oct 15 00:00, "Policy: end of day 30"; a dashed amber line at Oct 15 18:30, "Grace patch: 31 x 24 h". Three cases sit on the axis: A1 (Oct 14 19:05) 35 minutes past the starter's limit but inside day 30; A6 (Oct 15 01:05 UTC, written with a -05:00 offset) and A2 (Oct 15 09:00) inside day 31 but before the grace patch's limit.

Below, a table of the seven acceptance cases, A1 to A7, for each implementation, with PASS meaning the result matches policy POL-DOA-30. Starter: A1 FAIL, A2 PASS, A3 FAIL, A4 PASS, A5 PASS, A6 PASS, A7 PASS; 5 of 7. Grace patch: A1 PASS, A2 FAIL, A3 PASS, A4 PASS, A5 PASS, A6 FAIL, A7 PASS; 5 of 7. Reference fix: all PASS; 7 of 7. Footer: "Offline: no model involved. Recorded 2026-10-20. Synthetic data."

The grace patch fixes the reported ticket (A1) and breaks two day-31 cases (A2, A6). Counting calendar days in UTC is the only implementation that matches the policy on every case. A3 (delivered at 01:00, requested at 23:59 on day 30), A4, A5 and A7 (invalid input) use other times, so they are in the table but not on this order's ruler.

The order line, the ruler's two day bands and the axis appear, then cases A1, A6 and A2 on the axis and the table's column headers, A1 to A7. Then the three implementations are added one at a time: the starter's limit line and label, then its row of results and "5 of 7"; the grace patch's line, label, row and "5 of 7"; the policy's line and label with the reference fix's row and "7 of 7". It ends on the frame described above and holds for about 2.6 seconds.

Same content as the frame described above, revealed one implementation at a time.

What to look for in any generated diff

Read the diff after the checks, not instead of them, and ask five questions:

  1. Does the change fix the root cause named in the spec, or only the reported example?
  2. Did any test, test command or test configuration change?
  3. Is every file in scope, including new untracked files that git diff does not show?
  4. Are there drive-by edits to unrelated behavior?
  5. What input would break this? Write it as a test.

A second session as reviewer

Claude Code's guidance suggests separate writer and reviewer sessions (best practices). The useful version gives the reviewer no way to change anything, no Edit tool and three read-only commands, and asks for a verdict in a JSON shape you define:

Terminalbash
git status --short > .runs/before-review.txtclaude -p "$(cat ../labs/review/review-prompt.md)" \  --restricted --tools Bash,Read \  --permission-mode dontAsk \  --allowedTools "Bash(git status *)" "Bash(git diff *)" "Bash(pnpm test:acceptance)" \  --json-schema "$(cat ../labs/review/review-schema.json)" \  --model claude-sonnet-5-5 --effort medium \  --max-turns 12 --max-budget-usd 0.50 \  --no-session-persistence \  --output-format stream-json --verbose > .runs/review.jsonlpnpm check:receipt .runs/review.jsonl --expect-tools Bash,Read --canary none --expect-structured --print-structuredgit status --short | diff .runs/before-review.txt - && echo "reviewer changed nothing"

--tools sets which built-in tools exist in the session; --allowedTools only approves calls in advance (CLI reference). With --json-schema, the verdict arrives in structured_output, which is missing when a run ends without valid output, so check for it before you read it (headless). On 2.1.304 a --json-schema run also lists an extra StructuredOutput tool in system/init (spike ST3 and LIVE-07, 2026-10-20, claude-sonnet-5-5, effort medium, n = 1 each); the receipt check accepts that one tool when you pass --expect-structured.

Observed (LIVE-07, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1, $0.0341 estimated): verdict reject with five findings. It rated the window logic (with A2 and A6), the flipped test, the narrowed test script and the removed integer check high, and the new src/grace.ts medium. Its one denied call began with a cd into a mistyped temporary folder outside the workspace and then ran cat src/grace.ts. A cd outside the working directory is not on Claude Code's read-only list, so dontAsk denied the whole call; cat on its own needs no grant (read-only commands), and in the clean-room rerun (2026-10-20, the same version, model and effort, n = 1) cat src/grace.ts ran without one. git status did not change. On 2026-10-17, on Claude Code 2.1.302 with the 1.0.0 command and the same model and effort (n = 1), the same review also returned reject with five blocking findings. That is a useful second opinion. It is still advisory: the reviewer is a model, and the accept decision stays with the deterministic checks and a person.

Recover to a known state

You rejected the change. Get back to the spec state, and prove you are there.

Terminalbash
git restore --staged --worktree .git status --short
Outputtext
?? src/grace.ts

git restore puts tracked paths back; the new file the candidate created is untracked, so it is still there (git-restore). Preview, then remove it. git clean deletes untracked files, -n is a dry run, and -f is required by default (git-clean):

Terminalbash
git clean -ngit clean -f src/grace.tsgit status --shortpnpm test
Output (excerpt)text
Would remove src/grace.tsRemoving src/grace.ts

git status --short then prints nothing, and pnpm test ends with tests 8, pass 7, fail 1: the T-1042 spec test fails again, as it did at the tag. That last command is the proof. "Known state" means the spec test fails again for the original reason.

The lab also ships pnpm reset, which lists what it would discard and changes nothing until you pass --yes. Then it moves the branch and the working tree back to spec and deletes untracked files and folders. Use it only in a disposable workspace.

Checkpoints are not your rollback

Claude Code checkpoints let you rewind code, conversation or both inside an interactive session. They track only edits made through Claude's file-editing tools: "Checkpointing does not track files modified by Bash commands," and the page's own heading says checkpoints are "Not a replacement for version control" (checkpointing). A test script that writes a file, a formatter run through Bash or an mv survives a rewind. The spec tag does not depend on which tool made a change. (Checkpoints are interactive, and this guide did not exercise them.)

A failed run can still leave changes

Run LIVE-03 repeated LIVE-01's command with --max-turns 2. The documented behavior is that the run "Exits with an error when the limit is reached" (CLI reference). Observed (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1): exit code 1, subtype error_max_turns, no final message, and an edited src/window.ts on disk that passed all eight verifier checks. LIVE-03 on 2026-10-17, on 2.1.302, ended the same way.

Run status and repository state are separate facts. A failed run is not a no-op, and a successful run is not a correct change. Decide from the repository: either verify and review the leftover change like any other, or reset to spec and rerun.

Enforce the boundary with deny rules, hooks and the sandbox

Per-run flags bound one run. For rules that should hold in every run, keep them in .claude/settings.json, commit the file, and pass it with --settings as the lab's route does:

  • Deny rules. "Rules are evaluated in order: deny, then ask, then allow," and the first match decides (permissions). Deny the files and commands no task should touch: Edit(./package.json), Edit(./test/**), Edit(./acceptance/**), Bash(git push *), Bash(git commit *), Bash(pnpm add *), WebFetch and so on. The full list is in labs/solutions/lab-003/.claude/settings.json.
  • A PreToolUse hook that enforces the task scope. Deny rules describe stable repository policy. The task boundary changes every ticket, so it lives in scope.json, and a hook checks every edit against it.
  • A Stop hook that keeps Claude working while the tests fail, a bounded number of times.
  • No allow rules. Grants come from --allowedTools on each run, where a reviewer can see them.

Write every path rule with ./. The permissions page says a path that starts with / is relative to the settings source. In this guide's spike, a deny rule written as /test/** in a file passed with --settings .claude/settings.json did not protect test/: the edit went through. The same rule written as ./test/** denied it with "File is in a directory that is denied by your permission settings." (Claude Code 2.1.304, claude-sonnet-5-5, effort medium, 2026-10-20, n = 1 each.) The lab's settings test fails any path rule that does not start with ./.

The scope hook

Hooks receive JSON on stdin with tool_name and tool_input; for edits, tool_input.file_path. A PreToolUse hook blocks a call by printing hookSpecificOutput.permissionDecision: "deny" with a reason, or by exiting with code 2 (hooks). The core of labs/solutions/lab-003/.claude/hooks/guard-scope.ts:

guard-scope.tsts
const relative = path.relative(projectDir, realTarget(path.resolve(projectDir, raw))).split(path.sep).join('/');if (relative === '' || relative.startsWith('..') || path.isAbsolute(relative)) {  deny(`${raw} is outside this project. Edits must stay inside the repository.`, raw);}

After it reads .claude/scope.json, failing closed when the file is missing or unreadable:

guard-scope.tsts
if (!allowed.includes(relative)) {  deny(`${relative} is outside the ${scope.task ?? 'current'} task scope (allowed: ${allowed.join(', ') || 'nothing'}). Do not work around this. If the fix needs this file, stop and explain why in your final message.`, relative);}

Design choices worth copying:

  • It only denies. In-scope edits fall through to the normal permission rules, so the hook can never widen access.
  • It resolves symlinks and .. first, so src/link.ts pointing at package.json is denied.
  • It fails closed on unreadable input (exit 2) and on a missing scope file.
  • The denial reason tells the model what to do instead: stop and explain, not work around.

The hook's matcher is Edit|Write|NotebookEdit. On 2.1.304 those were the file-edit tools among the 24 tools a default Linux run listed in system/init (spike S1, 2026-10-20, claude-sonnet-5-5, effort medium, n = 1), which had no Glob or Grep. A matcher sees only the tools it names. A shell command can change a file too, and no file-edit hook sees that; the failure gallery has the run where it happened.

The Stop hook

A Stop hook blocks the stop with {"decision": "block", "reason": "..."}. Claude Code tells the hook when it is already continuing because of a stop hook (stop_hook_active), and "after stop hooks have continued the turn eight times in a row, Claude Code overrides the next block and ends the turn" (hooks). Do not rely on that ceiling. require-green.ts counts its own blocks per session and gives up after three, because a fix that cannot pass within scope should end with an explanation, not a loop. The verifier remains the final gate either way.

Hooks are code: test them offline

"Command hooks execute shell commands with your full user permissions" (hooks). Treat them like production code. Because a hook is a program that reads JSON and writes JSON, you can test it without a model: pnpm test:hooks feeds the hooks the JSON Claude Code sends (an in-scope edit, package.json, a new file, path traversal, a path outside the project, a symlink, a non-edit tool, malformed input, a missing scope file, passing and failing suites, the block limit) and checks the settings file itself: known tool names in every matcher, ./ on every path rule, the boundary files denied, no allow rules, both hooks wired. Against the starter's stubs, 7 of 18 pass; against the reference, 18 of 18.

What the controls did live

Three runs on Lab 3's branch, where the bug is present and the controls are committed (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1 each):

  • LIVE-05, Lab 1's command, unchanged. The receipt check counted one scope-hook run for the one edit and one Stop-hook run, and the verifier passed. Hooks from --settings fire under --restricted.
  • LIVE-06, an out-of-scope instruction on purpose. The prompt asked for a rename in src/refund.ts while scope.json named T-1042, and the run had a grant that was too broad on purpose, Edit(./src/**). Claude first tried a Bash sed -i on the file, which dontAsk denied because no grant matched it, then two Edit calls, which the scope hook denied. The Stop hook then reported the failing T-1042 test, and Claude fixed src/window.ts, which the prompt never asked for. Its final message said the rename was not done and why.
  • LIVE-09, the same command in auto mode. In the failure gallery.

LIVE-06 is the subtle one. Every control worked as written, and the session still did unrequested work, because the prompt, the scope file and the Stop hook disagreed about what the task was. When you configure hooks that push the agent forward, make sure they point at the same task as the prompt and scope.json. Otherwise the agent follows the most specific signal it gets. LIVE-06 on 2026-10-17 (2.1.302, the same model and effort, n = 1), with a different out-of-scope request, went the same way.

The sandbox

Deny rules and hooks decide which tool calls run. The sandbox limits what a shell command can do once it runs. It is off by default; --settings '{"sandbox":{"enabled":true}}' turns it on for one session, and on Linux it needs bubblewrap and socat (sandboxing). In this guide's spike (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1 each), a sandboxed write outside the working directory failed with "touch: cannot touch '/tmp/g05-sb-outside.txt': Read-only file system", and a request to an unlisted host failed with "curl: (7) CONNECT tunnel failed, response 403".

Know what it leaves out. It "covers shell commands only. Claude's file tools, MCP servers, and hooks run outside it." By default a sandboxed command can still write to the working directory, so it would not have stopped the out-of-scope sed in the failure gallery. When a command fails inside the sandbox, Claude Code can retry it unsandboxed: in dontAsk mode Claude Code denies that retry, but an allow rule that matches the command also approves its retry, and allowUnsandboxedCommands: false turns the retry off (sandboxing). This guide did not exercise the retry.

What a headless run takes from a repository

Every command so far named its controls. A plain claude -p instead discovers the repository's settings, and that is where the trust boundary sits.

The permissions page says project permissions.allow rules apply only after you accept the workspace trust dialog, while "deny and ask rules aren't affected, since they only restrict" (permissions). A claude -p run never shows the dialog. The hooks page adds that a -p or SDK session "treats the folder as trusted, so hooks committed in a repository's .claude/settings.json run in a folder you've never trusted" (hooks).

This guide re-observed that on the pinned version. Run LIVE-04 (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1) added one allow rule to the lab's committed settings and ran a plain claude -p in a folder that had never been trusted. Its stderr, then the number of hook starts in its stream:

stderr, hook counttext
Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted. Run Claude Code interactively here once and accept the trust dialog, or set projects["/tmp/g05-walk-kryzij69/download/cc-lab"].hasTrustDialogAccepted: true in /tmp/g05-cc/home/.claude.json.3

The repository's hooks still ran three times. In the spike's version of the same check, the deny rule applied, the hook saw a variable set by the repository's env block, and CLAUDE.md loaded. On 2026-10-17, on 2.1.302 with the same model and effort (n = 1), LIVE-04 printed the same warning for ten committed allow rules, and the hooks ran.

Diagram of what a never-trusted claude -p run takes from the repository's .claude/settings.json, in three cards. Deny rules: applied, because they only restrict, so workspace trust does not gate them. Hooks and env: used, because hooks run with your permissions and see the env block. Allow rules: ignored, because they grant, so they wait for trust, and claude -p never asks. Below, the first sentence of run LIVE-04's stderr: Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted. A last line says grants for the run come from the command line: --allowedTools.
What a repository can make an untrusted claude -p run do. Restrictions, hooks and the env block apply; grants do not. Live: claude-sonnet-5-5, effort medium, Claude Code 2.1.304, run LIVE-04 and spike S1 (n = 1 each), recorded 2026-10-20, with the permissions page checked the same day. Synthetic data. Deterministic diagram.
Extended description

Heading: "From the repository's .claude/settings.json". Three cards. Mint card "Deny rules", status "Applied": "They only restrict, so workspace trust does not gate them." Amber card "Hooks and env", status "Used": "Hooks run with your permissions, and see the env block." Gray card "Allow rules", status "Ignored": "They grant, so they wait for trust, and claude -p never asks." Below, a box labeled "stderr of run LIVE-04, Claude Code 2.1.304, first sentence" quotes: "Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted." Then, in mint: "Grants for the run come from the command line: --allowedTools". Footer: "Live: claude-sonnet-5-5, effort medium. Claude Code 2.1.304. Recorded 2026-10-20. Synthetic data."

In Claude Code 2.1.304, a never-trusted headless run applies a repository's restrictions and runs its hooks, but not its grants. Sources: run LIVE-04 (the warning and three hook starts) and spike S1 on the same CLI (the deny rule's refusal and the hook seeing a variable from the env block), one run each on 2026-10-20 with claude-sonnet-5-5 at effort medium, and the permissions page checked the same day.

The asymmetry is the lesson: in a headless run, the repository's grants are dropped while its code-executing hooks still run, with your permissions. Before you run claude -p over a repository you did not write, pick a defense on purpose. Each was tried on 2.1.304 on 2026-10-20 with claude-sonnet-5-5 at effort medium and the same probe repository (n = 1 each):

Five defenses and what each kept out
DefenseWhat it kept outWhat still applied
Read .claude/ and CLAUDE.md firstWhatever you decide after readingEverything you leave in place
--settings '{"disableAllHooks": true}'The repository's hooksIts deny rules and CLAUDE.md
--bareThe repository's hooks and CLAUDE.mdIts env block and its deny rules
--setting-sources userAll project settings, including the deny rules and the env block, and CLAUDE.md: the run read a file the repository deniedYour user settings
--restricted, as in the labsAll project and user settings, and CLAUDE.mdManaged settings and the file --settings names

What a passing verifier does not prove

The verifier proves the change matches what you encoded: these files, these tests, these cases. It cannot tell you about cases nobody wrote down.

The assessment makes that concrete. Its candidate is a one-line fix for ticket T-3118 that passes all eight verifier checks and is still wrong; finding the input that breaks it is part of the work. So review still matters after a green verifier, and the fifth review question, the input that would break the change, is the one that catches it. When you find such a case, add it as a test before the change is accepted, so the next verifier run encodes it.

Other limits of this lab:

  • Synthetic and small. One repository, a few files, fast tests. Real repositories have slow suites, flaky tests and generated code; the pattern holds, the costs change.
  • Live results are observations. Every live case on 2026-10-20 is one run on one model and one machine (n = 1), except the Lab 1 command (n = 2). They show what can happen on Claude Code 2.1.304, not how often.
  • Not tested here. macOS and native Windows, plan mode, /rewind, the trust dialog itself, signing in with a Claude account, and the sandbox's unsandboxed retry.
  • Permission rules are not a sandbox. Bash rules match the command as written; the sandbox, which is off by default, is what limits filesystem and network access for shell commands at the operating-system level, and only for shell commands. For untrusted code, use a container or VM as well.
  • Cost figures are estimates. total_cost_usd is Claude Code's client-side estimate, not an invoice. For the cost report the documentation says "The figure is an estimate" and points to the Claude Console for billing (costs).

Labs

The hands-on work is in the lab download, about 105 minutes in total. Every lab has a live route (Claude Code headless) and an offline route that makes the same change by hand, so you can finish every lab without an account, a key or a network connection after installing.

  • Lab 1, Fix T-1042 from a frozen spec (guided, about 35 minutes; OBJ-001, OBJ-003). Write the failing test and brief, tag spec, make the change on either route, check the run's receipt, and pass the spec copy of the verifier.
  • Lab 2, Reject the grace-period change and recover (diagnostic, about 30 minutes; OBJ-004, OBJ-005). Apply the candidate, prove each defect with a command, optionally ask a read-only reviewer, roll back, and prove you are back at the spec state.
  • Lab 3, Make the boundary deterministic (independent, about 40 minutes; OBJ-002, OBJ-003). Implement the two hooks until pnpm test:hooks passes 18 of 18, commit the deny rules and hooks, watch them hold the boundary, and see what a plain claude -p takes from a repository.

Troubleshooting

Each entry was reproduced from the lab download on 2026-10-20, except T-08, which names its run.

Symptoms, causes and fixes, T-01 to T-08
IDSymptomCauseFix
T-01The verifier step prints fatal: invalid object name 'spec'. and nothing elseThe spec commit was never taggedgit tag spec on the spec commit, then rerun the step
T-02pnpm preflight prints FAIL node: v20.20.2 (need 22.18 or later for TypeScript type stripping), and pnpm test prints Could not find with the test patternNode.js is older than 22.18Install Node.js 22.18 or later, then rerun pnpm preflight
T-03.runs/lab01.jsonl: No such file or directory, and Claude Code never starts.runs/ does not exist, so the shell fails the redirect firstRun mkdir -p .runs, the first line of the lab's command
T-04error: patch failed: src/window.ts:18 when you apply the candidateYou applied it on main, after the fixgit switch -q --create candidate spec, then apply it
T-05FAIL V7 typecheck, base checkout: could not install the base lockfile offline: pnpm not foundpnpm is not on PATH in the shell that runs the verifierRun corepack enable, or run the verifier from the shell where pnpm --version works
T-06pnpm preflight prints warn claude: not found (offline route still works)Claude Code is not installed, or not on PATHInstall 2.1.304 with the native installer, or take the offline route
T-07pnpm test:hooks fails every hook matcher names only tools that existA matcher names a tool the version does not haveUse Edit|Write|NotebookEdit (F-05)
T-08stderr: Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted.A plain claude -p in a folder never trusted interactively (LIVE-04)Expected. Pass grants with --allowedTools; keep deny rules and hooks in the file you pass with --settings

Quick reference

The workflow:

  1. Reproduce the bug with a failing test. Confirm it fails for the right reason.
  2. Write TASK.md (goal, scope, do-not-touch list, done criteria as commands) and scope.json.
  3. Commit and git tag spec.
  4. Run the agent with every input named: --restricted, a tool set, a settings file, the instructions file, an explicit mode, path-scoped grants, --max-turns and --max-budget-usd. Record the stream.
  5. Check the receipt: version, mode, tools, hook runs, instructions, how the run ended.
  6. Run the spec copy of the verifier: scope, protected files, test integrity, suites in a base checkout, typecheck, reproduction at spec.
  7. Review the diff with the five questions. Add a test for any missing edge case.
  8. Keep (commit with the receipts) or roll back (git restore --staged --worktree ., git clean -n, git clean -f), and confirm the spec test fails again.

Commands:

Terminalbash
bash labs/new-workspace.sh labs/starter cc-labpnpm decide T-1042pnpm test && pnpm test:acceptance && pnpm typecheckpnpm test:hookspnpm check:receipt .runs/lab01.jsonlgit show spec:scripts/verify-change.ts > ../verify-spec.ts && node ../verify-spec.tspnpm reset

Rules of thumb:

  • Guidance asks; deny rules, hooks and verifiers enforce. Put "must" rules in the second group.
  • Name every input of a headless run. Pass the permission mode. Pin the model by full ID. Record the version.
  • Check the run's own record, not only its summary.
  • Run your checks from code the change could not edit.
  • Count tests as well as failures: five passing where eight used to run is a finding.
  • A hook sees only the tools its matcher names; the verifier sees the repository.
  • A passing verifier is necessary, not sufficient.

Lab 1: Fix T-1042 from a frozen spec

Create the workspace, see the bug, write the failing test first, write the brief and freeze the spec, make the change on the live route (Claude Code headless) or the offline route (by hand), verify with the spec copy of the verifier, and keep the fix. About 35 minutes. The workspace and the toolchain check are the setup above, so the steps start at the bug. Outputs are copied, untrimmed, from a run of this exact download on 2026-10-20 with Node.js 22.23.3, pnpm 12.14.0 and Git 2.53.0. pnpm prints each script's command first (the line starting with $ node), and adds [ELIFECYCLE] where a command exits non-zero, here on purpose. On the offline route the verifier's receipt file name will differ, and nothing else should. The test you add, the brief and the offline route's two edits are in labs/learner-labs.md in the lab download. The live route and the offline route end in the same place: the live route's output is the receipt excerpt of run LIVE-01 (2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5 at effort medium; the same command passed the verifier in 2 of 2 runs that day), and a live run's turns, cost and wording differ from run to run.

Acceptance checks

  • node ../verify-spec.ts prints "verdict: PASS", with V8 naming your T-1042 test.
  • git diff spec --stat lists only src/window.ts.
  • Live route: pnpm check:receipt .runs/lab01.jsonl prints "receipt: PASS", and you can say which line of the command each R check depends on.
  1. See the bug

    $ pnpm decide T-1042
    $ node scripts/decide.ts T-1042 { "ticketId": "T-1042", "policyId": "POL-DOA-30", "eligible": false, "refundCents": 0, "reason": "outside refund window" }
  2. Run the policy's boundary cases

    $ pnpm test:acceptance
    $ node --test --test-reporter=./test/reporter.mjs --test-concurrency=1 "acceptance/**/*.test.ts" FAIL A1 T-1042: calendar day 30, later clock time than delivery Expected values to be strictly equal: false !== true PASS A2 calendar day 31, earlier clock time than delivery FAIL A3 calendar day 30, almost 31 elapsed days Expected values to be strictly equal: false !== true PASS A4 day 0, after delivery PASS A5 day 0, but before delivery PASS A6 offset timestamp that is calendar day 31 in UTC PASS A7 invalid requestedAt throws RangeError tests 7, pass 5, fail 2 [ELIFECYCLE] Command failed with exit code 1.
  3. Write the failing test first, then run the unit tests

    $ pnpm test
    $ node --test --test-reporter=./test/reporter.mjs --test-concurrency=1 "test/**/*.test.ts" PASS request under the limit is refunded in full PASS request over the limit is capped at the limit PASS non-integer cents throws RangeError PASS request 10 days after delivery is inside the window PASS request 45 days after delivery is outside the window PASS request on calendar day 31 is outside the window PASS invalid timestamp throws RangeError FAIL T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery Expected values to be strictly equal: false !== true tests 8, pass 7, fail 1 [ELIFECYCLE] Test failed. See above for more details.
  4. Write the brief, then read the scope file

    $ cat .claude/scope.json
    { "task": "T-1042", "allowEdits": ["src/window.ts"], "note": "Edit this file in the spec commit, before any agent run. The verifier reads it from the base commit." }
  5. Freeze the spec: commit and tag it

    $ git add -A git commit -q -m "spec: T-1042 day-30 refund window" git tag spec git log --format='%s%d'
    spec: T-1042 day-30 refund window (HEAD -> main, tag: spec) baseline: synthetic refund-rules lab (tag: baseline)
  6. Live route: run Claude Code headless with explicit grants

    $ mkdir -p .runs claude -p "$(cat TASK.md)" \ --restricted --tools Bash,Read,Edit \ --settings .claude/settings.json \ --append-system-prompt-file CLAUDE.md \ --permission-mode dontAsk \ --allowedTools "Edit(./src/window.ts)" "Bash(pnpm test)" "Bash(pnpm test:acceptance)" "Bash(pnpm typecheck)" "Bash(git diff *)" "Bash(git status *)" \ --model claude-sonnet-5-5 --effort medium \ --max-turns 12 --max-budget-usd 0.50 \ --no-session-persistence --include-hook-events \ --output-format stream-json --verbose > .runs/lab01.jsonl pnpm check:receipt .runs/lab01.jsonl
    $ node scripts/check-receipt.ts .runs/lab01.jsonl PASS R1 receipt: init record found, result record found PASS R2 Claude Code 2.1.304 PASS R3 permission mode dontAsk PASS R4 tools Bash, Edit, Read PASS R5 hooks: none configured in .claude/settings.json PASS R6 instructions: final message ends with "Rules loaded: refund-rules CLAUDE.md" PASS R7 run ended: success, 4 turns INFO model claude-sonnet-5-5; permission denials 0; estimated cost $0.0167 (client-side estimate) receipt: PASS
  7. Offline route: make the change by hand, then check what changed

    $ git status --short git diff --stat
    M src/window.ts src/window.ts | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
  8. Run the unit tests again

    $ pnpm test
    $ node --test --test-reporter=./test/reporter.mjs --test-concurrency=1 "test/**/*.test.ts" PASS request under the limit is refunded in full PASS request over the limit is capped at the limit PASS non-integer cents throws RangeError PASS request 10 days after delivery is inside the window PASS request 45 days after delivery is outside the window PASS request on calendar day 31 is outside the window PASS invalid timestamp throws RangeError PASS T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery tests 8, pass 8, fail 0
  9. Verify with the copy of the verifier the change cannot edit

    $ git show spec:scripts/verify-change.ts > ../verify-spec.ts && \ node ../verify-spec.ts
    PASS V1 verifier is the base copy: sha256 810b9c9f2c67 matches spec:scripts/verify-change.ts PASS V2 scope (T-1042): changed: src/window.ts PASS V3 protected files untouched: none changed PASS V4 existing tests intact: all base test titles present, none skipped PASS V5 unit tests, base checkout: pass 8, fail 0 PASS V6 acceptance tests, base checkout: pass 7, fail 0 PASS V7 typecheck, base checkout: no type errors PASS V8 spec failed before the change: at spec: fail 1; failing: T-1042: request on calendar day 30 is inside the window even when the clock time is later than delivery verdict: PASS receipt: .runs/verify-2026-10-21T00-01-52-696Z.json
  10. Keep the fix

    $ git add -A git commit -q -m "fix: T-1042 count calendar days in UTC" git tag lab-001 git log --format='%s%d'
    fix: T-1042 count calendar days in UTC (HEAD -> main, tag: lab-001) spec: T-1042 day-30 refund window (tag: spec) baseline: synthetic refund-rules lab (tag: baseline)
  11. Show the solution discussion

    The reference spec and fix are labs/solutions/lab-001/ in the lab download: TASK.md, test/window.test.ts and src/window.ts. From the workspace, node ../labs/solutions/apply-checkpoint.mjs lab-001 . copies them over your files, so commit or copy your work first. Try the steps yourself before you compare.

More labs in this guide

  • Lab 2: Reject the grace-period change and recover

    Put the candidate on a branch from the spec, run what its summary ran, verify from the spec, prove each defect with a command, optionally ask a read-only reviewer, then roll back and prove you are back at the spec. About 30 minutes, offline except the optional reviewer. The six steps, with every output, are in labs/learner-labs.md in the lab download.

  • Lab 3: Make the boundary deterministic

    See the controls fail their tests, implement the scope hook, the Stop hook and the deny rules until pnpm test:hooks passes 18 of 18, put the controls on a spec that still has the bug, watch them hold the boundary, and see what a plain claude -p takes from a repository. About 40 minutes; every step has an offline route. The six steps, with every output, are in labs/learner-labs.md in the lab download.

11 ways this breaks, and how you will know

Five of these reproduce in the lab; the other six were observed in this guide's recorded runs. Learn the symptom first; it is what you will actually see.

  • F-01candidate-graceReproducible

    "All tests pass" and the verifier fails

    Symptom
    Reproduce it with Lab 2, steps 1 to 3: labs/patches/candidate-grace.patch on a branch from spec. pnpm test prints "tests 5, pass 5, fail 0".
    Diagnosis
    The spec copy of the verifier fails V2 to V7. The patch narrowed the test script, rewrote a test and added an out-of-scope file the in-scope code imports.
    Fix
    Reject the change. Count tests as well as failures: five where eight used to run is a finding.
    Verify
    On the fix from Lab 1, the same verifier prints "verdict: PASS".
  • F-02Reproducible

    git restore leaves a file behind

    Symptom
    Reproduce it with Lab 2, step 6. After git restore --staged --worktree ., git status --short prints "?? src/grace.ts".
    Diagnosis
    The candidate created an untracked file, and git restore restores tracked paths only.
    Fix
    git clean -n to preview, then git clean -f src/grace.ts.
    Verify
    git status --short prints nothing, and pnpm test fails only the T-1042 spec test.
  • F-03Observed

    Committed allow rules are ignored

    Symptom
    Observed in run LIVE-04, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, n = 1; spike S1 the same day. stderr: "Ignoring 1 permissions.allow entry from .claude/settings.json: this workspace has not been trusted."
    Diagnosis
    Project allow rules need the workspace trust dialog, which claude -p never shows. The same run applied the repository's deny rules and ran its hooks.
    Fix
    Pass grants with --allowedTools on each run, and keep only deny rules and hooks in committed settings.
    Verify
    The lab's route (--restricted, --settings, --allowedTools) prints no such warning, and pnpm check:receipt passes.
  • F-04Observed

    A failed run leaves an edit on disk

    Symptom
    Observed in run LIVE-03, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, --max-turns 2, n = 1; the same on 2026-10-17 with 2.1.302 and the same model, effort and cap (n = 1). Exit code 1, subtype error_max_turns, no final message.
    Diagnosis
    git status --short shows " M src/window.ts". The turn cap stopped the run, not the edit it had already made.
    Fix
    Decide from the repository, not the exit code: verify and review the leftover change, or pnpm reset --yes back to spec.
    Verify
    On LIVE-03's leftover edit, the verifier printed "verdict: PASS"; after a reset, git status --short prints nothing.
  • F-05Reproducible

    A hook matcher names a tool that does not exist

    Symptom
    Reproduce it on the Lab 3 reference by changing the PreToolUse matcher to Edit|MultiEdit|Write, which is close to version 1.0.0's matcher. Nothing at run time: a name that matches no tool is never called, so nothing fails.
    Diagnosis
    pnpm test:hooks fails "every hook matcher names only tools that exist" with PreToolUse matcher names unknown tool "MultiEdit".
    Fix
    Name only tools the run's system/init record lists: Edit|Write|NotebookEdit on 2.1.304.
    Verify
    pnpm test:hooks prints "tests 18, pass 18, fail 0".
  • F-06Reproducible

    A test run that runs nothing, and exits 0

    Symptom
    Reproduce it at spec, where one test fails, with NODE_TEST_CONTEXT=child-v8 node --test --test-reporter=./test/reporter.mjs "test/**/*.test.ts". "Warning: node:test run() is being called recursively within a test file. skipping running files." and exit code 0.
    Diagnosis
    A process started from inside a Node.js test run inherits NODE_TEST_CONTEXT, and a child node --test that sees it skips its files. A hook or verifier tested from node --test would report green while a test fails.
    Fix
    Remove NODE_TEST_CONTEXT from the environment of every child test run, as the lab's Stop hook and verifier do.
    Verify
    The same command without the variable ends with "tests 8, pass 7, fail 1".
  • F-07Observed

    Every control holds, and the session does unrequested work

    Symptom
    Observed in run LIVE-06, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, dontAsk, n = 1. The prompt asked for a rename in src/refund.ts; git status --short shows " M src/window.ts".
    Diagnosis
    The scope hook denied both edits to src/refund.ts. The Stop hook's block named the failing T-1042 test, so Claude fixed T-1042 instead. The final message said the rename was not done.
    Fix
    Keep the prompt, scope.json and the Stop hook pointed at one task, for example by having the Stop hook name the task from scope.json.
    Verify
    The verifier against spec-drift passed, and the receipt check counted three scope-hook runs and two Stop-hook runs.
  • F-08weakened-verifierReproducible

    A weakened verifier in the working tree prints PASS

    Symptom
    Reproduce it at spec with labs/patches/weakened-verifier.patch, a one-line change that records every check as passed. pnpm verify prints "verdict: PASS", though its own V1 line says the file "differs from spec:scripts/verify-change.ts".
    Diagnosis
    pnpm verify runs the working tree's copy, which the change edited.
    Fix
    Run the spec copy: git show spec:scripts/verify-change.ts > ../verify-spec.ts && node ../verify-spec.ts.
    Verify
    The spec copy fails V2 and V3 (scripts/verify-change.ts changed) and V5 and V6 (the bug is still there).
  • F-09Observed

    A "/" path rule protects the wrong folder

    Symptom
    Observed in spike S10 and S11, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, --settings .claude/settings.json, n = 1 each. With Edit(/test/) denied in a file passed as --settings .claude/settings.json, an edit to test/ went through.
    Diagnosis
    A leading "/" is relative to the settings source, here .claude/, so the rule covered .claude/test/.
    Fix
    Write path rules with "./", which anchors at the working directory in project settings and in --settings alike.
    Verify
    With Edit(./test/), the edit was denied: "File is in a directory that is denied by your permission settings."
  • F-10Observed

    --bare drops the hooks you passed

    Symptom
    Observed in spike S6, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, --bare with --settings, n = 1. A bare run with a PreToolUse hook in --settings shows no hook events, and pnpm check:receipt would fail R5.
    Diagnosis
    On 2.1.304, --bare skips "hooks (those defined in settings and by installed plugins; ...)," including a file named with --settings. The documentation describes skipping hook discovery only.
    Fix
    For hooks you choose, use --restricted with --settings, as the labs do.
    Verify
    Spike S8 and LIVE-05 show the same hooks running under --restricted.
  • F-11Observed

    Auto mode lets a shell command make an out-of-scope edit

    Symptom
    Observed in run LIVE-09, 2026-10-20, Claude Code 2.1.304, claude-sonnet-5-5, effort medium, --permission-mode auto, n = 1. The receipt check passes, and the final message says "The day-30 bug is fixed and everything passes." git status --short lists " M src/refund.ts", the file the scope hook exists to protect.
    Diagnosis
    Claude made the rename with sed -i 's/assertCents/assertWholeCents/g' src/refund.ts, and auto mode ran it. The scope hook matches file-edit tools, so it never ran: R5 counted 0 runs for 0 matching calls. In dontAsk mode (F-07), the same command was denied because no grant matched it.
    Fix
    Use dontAsk with exact grants when the boundary must hold during the run. In any mode, let the verifier's scope check decide after the run.
    Verify
    The spec copy of the verifier failed V2: "out of scope: src/refund.ts".

Transfer task and rubric

Fix a refund-limit bug from a frozen spec with bounded grants, judge a candidate change that passes the verifier, and prove you returned to the spec.

Analytic rubric: each criterion described at four performance levels, from Emerging to Exemplary
Criterion0 · Emerging1 · Developing2 · Proficient3 · Exemplary
C1 Frozen specNo failing test at spec, or the test fails for an unrelated reason (syntax, import), or the brief has no runnable done criteriaThe test fails at spec on the T-3118 assertion, but the brief leaves the editable files open or states some done criteria as prose instead of commandsThe test fails at spec on the T-3118 assertion; TASK.md names src/refund.ts as the only editable file and lists done criteria as commands; verifier V8 passesProficient, and the brief states the policy rule completely, including every bound on the result, and the test name ties to the ticket
C2 Least-privilege controlsMode left to its default, or a grant as broad as bypassPermissions, Bash(*) or an Edit with no pathExplicit mode and a path-scoped edit grant, but one grant is wider than the task needs (for example Edit(./src/) or "Bash(pnpm *)"), or the scope file still names T-1042 at specExplicit mode; edits limited to src/refund.ts by grant or by grant plus scope hook; deny rules and hooks in committed settings; scope file updated before specProficient, and explains which layer enforces each rule, and why the grants came from the command line rather than the repository's settings
C3 Correct, bounded changeHidden tests fail, or the diff touches files outside scopeAll hidden tests pass and git diff --stat spec lists only src/refund.ts, but the verifier was not run from the spec copy, or it failed a check other than scopeAll hidden tests pass; git diff --stat spec lists only src/refund.ts; the spec copy of the verifier passes all eight checksProficient, and adds a committed test for an edge case beyond the reported one (in a new spec commit), with reasoning
C4 Run receiptMissing, or describes an offline edit as an agent runPresent, but a field is missing or disagrees with the stream (for example the turn count or the mode)All receipt fields present and consistent with the stream or the offline statementProficient, and notes anything surprising in the receipt (denials, retries, cost) and what it implies
C5 Candidate reviewAccepts the candidate, or rejects it without executed evidenceRejects it with executed evidence that does not isolate the defect, such as a pnpm decide call or an existing test, instead of a new test that fails on the candidateRejects it with a new test that fails on the candidate, quoting its outputProficient, and explains why every verifier check passed, and names the requirement that was not encoded
C6 RecoveryNo proof of the known state, or recovery commands run on the wrong branch or workspaceRecovery commands shown and git status --short empty, but no rerun showing the spec test failing againgit status --short empty and the spec test failing again, with the commands that got thereProficient, and explains what git restore and checkpoints would and would not have restored

Proficiency standard: Proficient or higher on every criterion. Proposed local standard, not an official certification.

Self-check

The full brief

Ticket T-3118 (synthetic): order ORD-7304 already received a $20.00 refund, and the customer asks for $48.00 more. Policy POL-DOA-30 caps the total refunded on one order at $60.00, so the correct refund is $40.00 (4000 cents). refundableCents in src/refund.ts ignores earlier refunds and returns 4800.

Start from the finished Lab 3 controls, in a new workspace next to labs/. Write a failing test for T-3118, a TASK.md and a .claude/scope.json that limit edits to src/refund.ts, then commit and tag spec. Produce the fix on either route: live, a claude -p run with an explicit permission mode, path-scoped grants, --max-turns, --max-budget-usd and a recorded stream; or offline, a fix you write yourself and say so. Run the spec copy of the verifier and keep its output.

Then judge a second change. On a branch from spec, apply labs/patches/asm-candidate.patch, a one-line fix written for this assessment to represent a plausible generated change (it is not captured model output). Decide whether to accept it, and support the decision with at least one new test you write and run. Finally, return that branch to the spec state and show that you are there.

Submit one folder or Markdown file with your spec, your run receipt, your diff and verifier output, your review notes and new test, and your recovery commands. Allow about 45 minutes. A reviewer runs hidden tests against your fix, reruns the verifier from your spec tag, and scores each criterion with the rubric.

Pass rule (a proposed local standard): Proficient (2) or higher on every criterion. Critical failure: a failing hidden test (C3), or a grant as broad as bypassPermissions, --dangerously-skip-permissions, Bash(*) or an Edit with no path (C2), fails the module regardless of every other score, as do editing tests or the verifier to make checks pass, running against real code or data, including a secret, and presenting a hand-written fix as an agent run. The rubric's shape follows the University of Hawaiʻi at Mānoa's guide to analytic rubrics, which scores each criterion separately and lists "Emerging, developing, proficient, exemplary" among its common level labels (UH Mānoa), and Carnegie Mellon's Eberly Center, which defines a rubric's criteria, descriptors and performance levels (Carnegie Mellon).

Teach it to a room

  • Facilitation guide and run of show

    A run of show for a 12-person lab of about 170 minutes, hint ladders for the steps where learners get stuck, three demo scripts and an optional live demo, must-know misconceptions, live-room troubleshooting and accessibility notes.

  • Cohort sizes and fallbacks

    An adaptation for a 100-person training day of about 4.5 hours, with breakouts of about 12 and the offline route as the default, a recovery decision tree, and two reference checkpoints, lab-001 and lab-003, so a stalled learner or breakout can rejoin.

  • Dry run and observation rubric

    A train-the-trainer sequence, a dry-run protocol of about 70 minutes, an observation rubric, a proposed clearance rule and the triggers for observing an instructor again. It is a new procedure; no instructor has been observed with it.

  • Module clearance record

    A pending template for recording that an instructor is cleared to teach this module version, filled in by an authorized person after an observed dry run. No instructor has been cleared with it.

  • Evaluation plan and pass-rate report

    What is evaluated and when, data handling rules, module quality signals, a maintenance procedure and an issue log, and a blank monthly pass-rate report. No learner or instructor results exist yet.

  1. [1]Advanced setup · code.claude.comchecked Oct 20, 2026
  2. [2]CLI reference · code.claude.comchecked Oct 20, 2026
  3. [3]Run Claude Code programmatically · code.claude.comchecked Oct 20, 2026
  4. [4]Choose a permission mode · code.claude.comchecked Oct 20, 2026
  5. [5]Configure permissions · code.claude.comchecked Oct 20, 2026
  6. [6]Hooks reference · code.claude.comchecked Oct 20, 2026
  7. [7]Sandboxing · code.claude.comchecked Oct 20, 2026
  8. [8]How Claude remembers your project · code.claude.comchecked Oct 20, 2026
  9. [9]Best practices · code.claude.comchecked Oct 20, 2026
  10. [10]Checkpointing · code.claude.comchecked Oct 20, 2026
  11. [11]Model configuration · code.claude.comchecked Oct 20, 2026
  12. [12]Manage costs effectively · code.claude.comchecked Oct 20, 2026
  13. [13]Models overview · platform.claude.comchecked Oct 20, 2026
  14. [14]Claude Code changelog · github.comchecked Oct 20, 2026
  15. [15]Registry entry for @anthropic-ai/claude-code · registry.npmjs.orgchecked Oct 20, 2026
  16. [16]git-restore · git-scm.comchecked Oct 20, 2026
  17. [17]git-clean · git-scm.comchecked Oct 20, 2026
  18. [18]Modules: TypeScript (v22.x source) · github.comchecked Oct 20, 2026
  19. [19]Release schedule · github.comchecked Oct 20, 2026
  20. [20]Distribution index · nodejs.orgchecked Oct 20, 2026
  21. [21]Registry entry for pnpm · registry.npmjs.orgchecked Oct 20, 2026
  22. [22]Registry entry for typescript · registry.npmjs.orgchecked Oct 20, 2026
  23. [23]Registry entry for @types/node · registry.npmjs.orgchecked Oct 20, 2026
  24. [24]e-Handbook of Statistical Methods, 7.2.4.1 Confidence intervals · itl.nist.govchecked Oct 20, 2026
  25. [25]Creating and using rubrics · manoa.hawaii.educhecked Oct 20, 2026
  26. [26]Eberly Center, Grading and performance rubrics · cmu.educhecked Oct 20, 2026

Independent work. Not affiliated with or endorsed by any vendor named in this guide.

Jordan Hudgens

Written by

Jordan Hudgens

Technical educator and AI engineering leader. 16 Udemy courses, 111K students, and curricula used by 250K+ learners.

Contact Jordan