Route Claude Code Work Across Model Tiers and Prove the Savings

This guide reuses Guide 05's refund-rules project and three of its ideas. If you have not read it, these commands are enough to follow along.

Headless runs with explicit grants. A claude -p run with --permission-mode dontAsk performs only the actions you list in --allowedTools and rejects the rest without asking (CLI reference; permissions). Every measured run in this guide uses the same grants:

claude -p "Fix ticket T-4207 as TASK.md describes. Follow the routing rules in CLAUDE.md." \  --permission-mode dontAsk \  --allowedTools "Edit(./src/policy.ts)" "Bash(pnpm test)" "Bash(pnpm test:acceptance)" "Bash(pnpm typecheck)" "Bash(pnpm decide *)" "Bash(git diff *)" "Bash(git diff)" \  --max-turns 40 --max-budget-usd 2.00 --no-session-persistence \  --output-format stream-json --verbose

A verifier the change cannot edit. Guide 05's scripts/verify-change.ts checks a change from the tagged commit it started at: scope, protected files, typecheck, both test suites and the new test's failure before the change. This guide runs it from the configuration's own commit, so a change cannot weaken the check that judges it.

Workspace trust. A -p session shows no workspace trust dialog and runs a project's hooks and MCP servers even in a folder you have never trusted (headless docs). In recorded runs on 2026-10-25 with Claude Code 2.1.308, the same folder's committed model, modelSettings, advisorModel and env keys applied too. Read a repository's .claude/ folder before running it headless, and in these labs, run only the workspace the lab builds.

How Claude Code chooses a model and an effort

Three separate choices decide what a request costs:

• The model of the main session. It comes from the first of four places that sets it: the /model command during a session, the --model flag at startup, the ANTHROPIC_MODEL environment variable, or the model key in a settings file (model configuration). With none set, the account's default applies; on the Anthropic API on 2026-10-25 that was claude-opus-5-5.
• The model of each subagent. A subagent definition in .claude/agents/ can name its own model in its model field; otherwise a variable or the main session's model decides (subagents). The built-in Explore runs on the main conversation's model unless a user or project definition named Explore replaces it.
• The effort of each request. Effort levels control adaptive reasoning: lower is faster and cheaper, higher reasons more deeply (model configuration). Effort has its own ladder, per-model defaults (medium for Opus 5.5, Sonnet 5.5 and Haiku 5.5) and caps (maxEffortLevel) that limit every source above them.

An advisor adds a fourth choice. With advisorModel set and the advisor available, the main model can consult a stronger model "at key moments during a task" (advisor). The advisor's tokens are billed at the advisor model's rates and are part of the session's totals.

Settings themselves come from several files, and the most specific one wins: managed settings, then command-line settings, then .claude/settings.local.json, then .claude/settings.json, then ~/.claude/settings.json (settings).

The four precedence ladders, from recorded runs

Each ladder below comes from Anthropic's documentation and from a run recorded on 2026-10-25 with Claude Code 2.1.308, in which a higher rung overrides a lower one. The boxes are excerpts of the table table.mjs prints from each run's telemetry (Read a run from its telemetry), trimmed to the rows that matter. Where a run showed something the documentation does not state, the text says so.

The main session's model

From highest to lowest: /model in the session, --model at startup, ANTHROPIC_MODEL, the model key in settings, the account default (model configuration).

Recorded run LIVE-04 (2026-10-25, Claude Code 2.1.308, effort medium): project settings said "model": "claude-haiku-5-5" and ANTHROPIC_MODEL=claude-sonnet-5-5 was set. The variable won:

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-sonnet-5-5  main      -           medium         1       2       4       11352         5997     $0.0162total                                                   1       2       4       11352         5997     $0.0162

Recorded run LIVE-05 added --model claude-haiku-5-5 to the same setup. The flag won over the variable:

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-haiku-5-5   main      -           medium         1       2       4       11352         6773     $0.0010total                                                   1       2       4       11352         6773     $0.0010

Settings scopes

From highest to lowest: managed settings, command-line settings (--settings), .claude/settings.local.json, .claude/settings.json, ~/.claude/settings.json (settings).

Recorded run LIVE-06 (2026-10-25, Claude Code 2.1.308, effort medium): the user file said claude-sonnet-5-5, the project file claude-haiku-5-5 and the local file claude-sonnet-5-5. In the run before it, without the local file, the project file had beaten the user file and the session ran on Haiku. With the local file, it ran on Sonnet:

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-sonnet-5-5  main      -           medium         1       2       4       11352         5995     $0.0162total                                                   1       2       4       11352         5995     $0.0162

A subagent's model

From highest to lowest: a model named for one invocation, the definition's model field, CLAUDE_CODE_SUBAGENT_MODEL, the main session's model (subagents). model: inherit counts as set in the definition, so it beats the variable. Setting CLAUDE_CODE_SUBAGENT_MODEL_FORCE=1 (Claude Code 2.1.257 or later) applies the variable's model, or the main model, to every subagent and ignores the definitions (environment variables).

This order changed in Claude Code 2.1.251: before it, CLAUDE_CODE_SUBAGENT_MODEL outranked the definition. Notes and scripts written for older versions often assume the old order.

Recorded run LIVE-07 (2026-10-25, Claude Code 2.1.308, effort medium): the researcher's definition said model: claude-haiku-5-5 while CLAUDE_CODE_SUBAGENT_MODEL=claude-sonnet-5-5 was set. The main session was on Haiku too, so only the variable pointed at Sonnet. The definition won:

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-haiku-5-5   main      -           medium         3       6     249       48264         8047     $0.0016claude-haiku-5-5   subagent  researcher  medium         2       4     240        3790         4395     $0.0007total                                                   5      10     489       52054        12442     $0.0023

The built-in Explore is a different case. Recorded run LIVE-08 (same day and version) had no Explore definition, CLAUDE_CODE_SUBAGENT_MODEL=claude-haiku-5-5 and --model claude-sonnet-5-5. The built-in Explore ran on the main model, not on the variable's. The documentation says the variable alone does not change the built-in Explore or Plan; a definition named Explore does.

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-sonnet-5-5  main      -           medium         2       4     219       28858         6542     $0.0214claude-sonnet-5-5  subagent  Explore     medium         2       4      91        8596         8742     $0.0236total                                                   4       8     310       37454        15284     $0.0451

Effort, with its cap

From highest to lowest: an explicit choice (CLAUDE_CODE_EFFORT_LEVEL, --effort or /effort), then settings (the level saved for the model in modelSettings, or an effortLevel key), then the model's default (model configuration). The documentation groups the three explicit choices into one rung; in a recorded run, the variable beat the flag. For a subagent, the definition's effort field outranks the session's level, an effort Claude passes for one invocation outranks the field, and the variable outranks both (subagents). Then a maxEffortLevel cap limits the result, whatever set it, and the lowest cap across scopes applies. The cap can be a top-level key for every model or a field in one model's modelSettings entry (settings reference).

Recorded run LIVE-09 (2026-10-25, Claude Code 2.1.308, claude-haiku-5-5): project settings set Haiku's level to low, the command passed --effort xhigh, and CLAUDE_CODE_EFFORT_LEVEL=max was set. The variable won:

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-haiku-5-5   main      -           max            1       2     687           0        17961     $0.0026total                                                   1       2     687           0        17961     $0.0026

Recorded run LIVE-10 added a top-level "maxEffortLevel": "high" to the project settings. The cap held the variable to high:

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-haiku-5-5   main      -           high           1       2       4       11187         6774     $0.0010total                                                   1       2       4       11187         6774     $0.0010

Two behaviors in these runs are not spelled out in the documentation as of 2026-10-25, so treat them as observed on 2.1.308 only:

• A subagent without an effort field ran at the session's level when that level was set explicitly (--effort high gave the researcher high).
• With no explicit session level, a subagent ran at its own model's modelSettings level: a Haiku researcher got Haiku's entry, not the Sonnet main session's.

One more trap sits in the user settings file. A top-level effortLevel in ~/.claude/settings.json does not count for Opus 5.5, and models released after it start at their own default (model configuration). In recorded runs on 2026-10-25 it was ignored on Sonnet 5.5 and Haiku 5.5 and applied on Opus 5. Set effort per model in modelSettings instead.

Read a run from its telemetry

Claude Code exports usage as OpenTelemetry metrics and events when CLAUDE_CODE_ENABLE_TELEMETRY=1 and an exporter are set (monitoring). The labs send them over OTLP HTTP/JSON to a small collector the kit runs on 127.0.0.1, and labs/starter/telemetry/table.mjs turns one run's export into a table. In 2.1.308, the console exporter printed nothing for a one-second -p run and mixed object dumps into the run's own output for longer ones, so the kit does not use it.

The variables the kit's runner sets for every run:

CLAUDE_CODE_ENABLE_TELEMETRY=1OTEL_METRICS_EXPORTER=otlp OTEL_LOGS_EXPORTER=otlpOTEL_EXPORTER_OTLP_PROTOCOL=http/jsonOTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318OTEL_METRIC_EXPORT_INTERVAL=5000 OTEL_LOGS_EXPORT_INTERVAL=2000OTEL_LOG_TOOL_DETAILS=1

If port 4318 is already in use, the runner stops with a message saying so; stop whatever listens there, or set ROUTING_LAB_PORT to another port. The runner stops the collector when the run ends and checks that the port closed.

Each row of the table is one combination of four attributes of the api_request events and cost counters:

• model, the model that served the request (some models arrive with a date suffix, such as claude-haiku-4-5-20251001; the table strips it);
• query source, main for the main session and subagent for an agent's requests (the events say sdk, agent:custom or agent:builtin:Explore);
• agent, the subagent's name, which telemetry replaces with custom for agents you define unless OTEL_LOG_TOOL_DETAILS=1 is set (monitoring);
• effort, the level the request ran at, absent on models without effort.

Here is the median run of the five baseline runs of the first measurement (Measure the route), recorded as LIVE-01 (2026-10-25 19:11:27 UTC, Claude Code 2.1.308, every agent inheriting claude-opus-5-5, effort medium):

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-opus-5-5    main      -           medium         7      14    2377      137748        13163     $0.1410claude-opus-5-5    subagent  Explore     medium         5      10    3500       49592        14114     $0.1505claude-opus-5-5    subagent  researcher  medium         4       8    1572       20744         5390     $0.0626claude-opus-5-5    subagent  worker      medium         4       8    1022       19334         3359     $0.0411total                                                  20      40    8471      227418        36026     $0.3952cost estimates (none of these is an invoice)  telemetry cost.usage, summed:          $0.3952  total_cost_usd from the result line:   $0.3952  tokens at list price (2026-10-25):     $0.3952

The three cost lines under the table are estimates, and none is an invoice:

• the sum of the telemetry's cost.usage counter;
• total_cost_usd from the run's last result line in stream-json (a long run can print several cumulative result lines; the last one is the run's);
• the tokens recomputed at list prices read on 2026-10-25 (pricing).

In every recorded run of the fixed task, the three agreed to within a fraction of a cent, which says the client computes cost from the same token counts and prices. It does not say what you will be billed; cost management explains where the console and invoices come from.

Two columns explain most of the money. Cache read tokens are context the model has seen recently, charged at a fraction of the input price; a run that starts within a few minutes of a similar one reads most of its context from cache. Cache write tokens are context written to the cache, charged above the input price. Compare runs only when their caches started in the same state; the kit's comparison script reports a run whose first main request read nothing from cache as cold.

Configure the route

The routed configuration is two kinds of file, both committed to the repository: project settings and agent definitions.

.claude/settings.json keeps Guide 05's deny rules and adds three keys:

{  "model": "claude-sonnet-5-5",  "modelSettings": {    "claude-sonnet-5-5": { "effortLevel": "medium", "maxEffortLevel": "high" },    "claude-haiku-5-5": { "effortLevel": "medium", "maxEffortLevel": "medium" }  },  "advisorModel": "claude-opus-5-5"}

• model moves the main session from the account default to Sonnet 5.5. Anything higher on the model ladder still overrides it.
• modelSettings sets each model's effort and caps it. The caps are what keep a CLAUDE_CODE_EFFORT_LEVEL=max in someone's shell from running every request at max: in a recorded run with that variable set (Lab 3 case 3), the main session ran at high and the Haiku agents at medium. How much an uncapped run would cost was not measured.
• advisorModel names the advisor. It must rank at or above the main model, or the session runs without it (advisor).

Each agent definition in .claude/agents/ names a full model ID and only the tools the agent needs. The Explore override, which replaces the built-in agent of that name:

---name: Exploredescription: Read-only search of this repository. Use it first, to find the files, functions and data a ticket involves. It reports paths, function names and line numbers and never changes a file.tools: Read, Grep, GlobdisallowedTools: Write, Editmodel: claude-haiku-5-5maxTurns: 12---

The researcher is the same shape on claude-haiku-5-5, with omitClaudeMd: true so it does not carry the project's instructions into every data question. The worker runs on claude-sonnet-5-5 with tools: Read, Edit, Bash. A routing block in CLAUDE.md tells the main session which agent does what.

Route by task, not by price. Anthropic's Haiku 5.5 announcement reports Terminal-Bench 4.0 scores of 39.2% for Haiku 5.5 and 70.6% for Sonnet 5.5, and says "Sonnet 5.5 and Opus 5.5 remain better choices for complex agentic coding tasks like those measured by Terminal-Bench 4.0", while Haiku 5.5 "pairs well with Opus 5.5 and Sonnet 5.5 as a subagent on coding work" (Introducing Claude Haiku 5.5, 2026-10-07). So this route gives Haiku the search and the data questions and keeps planning and the edit on Sonnet. Putting every agent on the cheapest model is a different route, and it needs its own measurement.

Pin full model IDs. An alias such as haiku resolves through ANTHROPIC_DEFAULT_HAIKU_MODEL when that variable is set (model configuration). In a recorded run with that variable set to claude-haiku-4-5, an Explore agent with model: haiku ran on Haiku 4.5, with no effort attribute at all. On 2026-10-25, Haiku 4.5 listed at ten times Haiku 5.5's token prices for prompts up to 100,000 tokens, and at twice them for longer prompts (pricing). With model: claude-haiku-5-5, the same variable changed nothing. The full IDs in this guide are the 2026-10-25 names (models overview).

Allowlist the tools. On macOS, Linux and WSL, Claude Code leaves Glob and Grep out of the default tool set and searches through Bash; a subagent that lists them in tools and leaves out Bash gets them back (tools reference). The recorded runs on Linux showed exactly that. The definitions here list what each agent uses and nothing more.

The baseline the labs measure against is the same repository at its spec commit: the same agents, the same routing block and the same deny rules, with every model field set to inherit and no model keys in settings. Every agent then runs on the account default, so the only difference between the two configurations is the routing.

Watch the advisor

The advisor does not appear as a request of its own. In the stream-json output, a consultation is a server_tool_use block named advisor followed by an advisor_tool_result block whose content is encrypted. In telemetry, the advisor's tokens arrive as cost and token counters under the advisor model with no effort attribute and no api_request event; the table prints them on their own row marked (no request of its own). Its cost is included in total_cost_usd.

The advisor can be configured and still never run, with no error. In recorded runs on 2026-10-25 with Claude Code 2.1.308, it stayed off when:

• DISABLE_TELEMETRY was set, or CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC was set, even to 0; both turn off the feature-flag fetch the advisor depends on (advisor; environment variables);
• CLAUDE_CODE_DISABLE_ADVISOR_TOOL=1 was set;
• the advisor model ranked below the main model (a Haiku advisor for a Sonnet session).

When the advisor is off, a prompt that asks for it gets a reply such as "no advisor is available"; telemetry export is unaffected. The kit's advisor probe asks for exactly one consultation and prints whether an advisor row appeared.

Recorded run LIVE-03 (2026-10-25 19:16:46 UTC, Claude Code 2.1.308, main session on claude-sonnet-5-5, advisor claude-opus-5-5) used a prompt that asked for one consultation. The advisor's row has no request of its own:

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-sonnet-5-5  main      -           medium         8      18    1996      195064        16033     $0.0796claude-opus-5-5    main      -           -              0   27314    2036           0            0     $0.1500  (no request of its own)claude-haiku-5-5   subagent  Explore     medium         4       8    2695       33067        12945     $0.0033claude-haiku-5-5   subagent  researcher  medium         4       8    2299       31562        13545     $0.0032claude-sonnet-5-5  subagent  worker      medium         4       8     890       22997         4854     $0.0234total                                                  20   27356    9916      282690        47377     $0.2594cost estimates (none of these is an invoice)  telemetry cost.usage, summed:          $0.2594  total_cost_usd from the result line:   $0.2594  tokens at list price (2026-10-25):     $0.2594

Configured is not the same as used. In the 5 routed runs of the first measurement, the main session never consulted the advisor on its own; in the 5 of the second, one main session did, unasked, and the advisor's row cost $0.1133 (Measure the route). A variant prompt that ends "Consult the advisor once before your final message." produced 2, 2 and 1 consultations in its 3 runs, and every one passed the quality check. Each consultation reads the whole conversation again, and that read is not cached (advisor): the advisor's own tokens came to $0.1482 to $0.1716 of each run. Those runs cost $0.2594 at the median (range $0.2571 to $0.2811), against $0.0822 for the first measurement's routed runs, which did not consult it. The documentation says pairing a faster main model with a stronger advisor "typically costs less than running the stronger model throughout"; on this task, the consulted runs' median was below the first measurement's baseline median, $0.3952, from sessions that could not run their checks (Measure the route). The variant changes the prompt, so it is a different workload, and its runs are not part of the comparison in Measure the route.

Measure the route

A cost claim is only as good as the runs behind it, and a run only as good as the work it did. This section reports two measurements of the same task. The first found the routed configuration much cheaper, but none of its baseline sessions could run the task's checks. The second gave both configurations the same way to run them and measured again.

The first measurement

The protocol, fixed before the first measured run:

• Workload. Ticket T-4207 in the refund-rules project: a request under a new policy, POL-DMG-14, that the decision tool rejects as an unknown policy. The fix is one edit in src/policy.ts, but finding it means reading the ticket data, the decision code and the policy table. Same prompt, same fixture (Guide 05's download, version 1.1.0), same flags, Claude Code 2.1.308.
• Configurations. spec, the baseline, and routed, the configuration in Configure the route.
• Repetition and order. Five runs of each, alternating baseline and routed, so time of day and cache state do not favor one side.
• Quality. Every run is checked by labs/starter/checks/quality.mjs: Guide 05's verifier from the configuration's commit (V1 to V8), then six review checks the tests do not cover, including the new policy's last eligible day and the day after it. A policy copied with the wrong window passes the tests and the verifier, and fails this check.
• Statistics. The median and the full range of the client estimate, with n, over passing runs only. A failed run is reported and excluded.
• Rollback rule. Return to the baseline when any routed run fails the quality check, when either configuration has fewer than three passing runs, or when the two ranges overlap.
• Runs that stay open. The runner stops a run that has printed its result with no background task pending, 10 seconds later if a wake-up is still scheduled and 30 seconds later otherwise, and records it in run.json (F-11, T-12). None of this measurement's runs called a scheduling tool, and each ended within a second of its last telemetry event, so none would have been stopped.

Extended description

Title: "Estimated cost per run, ticket T-4207". Subtitle, two lines: "Claude Code 2.1.308, 5 runs each, 10 of 10 passed the quality check" and "First measurement: no baseline session could run its checks". A horizontal dollar axis labeled from $0.00 to $0.40 and running a little past it, with dotted gridlines every $0.10, and two rows. The upper row, "Baseline, n = 5", in white: five dots at $0.2935, $0.3670, $0.3952, $0.4064 and $0.4157, a gray line spanning the range, and a tall mark at the median labeled "median $0.3952". The lower row, "Routed, n = 5", in mint: five dots at $0.0802, $0.0819, $0.0822, $0.0847 and $0.0907, close together, a short range line and a median mark labeled "median $0.0822". Between the rows, an amber bracket from the routed median to the baseline median: "Difference of the medians: $0.3130 (79.2%)". Within a row, a dot's height only keeps close values apart; it carries no value. Footer: "Client estimates, measured 2026-10-25".

The difference of the medians is $0.3130, 79.2% of the baseline's median. The rollback rule is not triggered: every routed run passed, both configurations have at least three passing runs, and the ranges do not overlap.

The median routed run, recorded as LIVE-02 (2026-10-25 19:13:03 UTC, Claude Code 2.1.308, main and worker on claude-sonnet-5-5, Explore and researcher on claude-haiku-5-5, effort medium):

model              source    agent       effort  requests   input  output  cache read  cache write  cost.usageclaude-sonnet-5-5  main      -           medium         6      12    1302      117877        12439     $0.0559claude-haiku-5-5   subagent  Explore     medium         5      10    3075       52028        15420     $0.0040claude-sonnet-5-5  subagent  worker      medium         4       8     803       23032         4783     $0.0223total                                                  15      30    5180      192937        32642     $0.0822cost estimates (none of these is an invoice)  telemetry cost.usage, summed:          $0.0822  total_cost_usd from the result line:   $0.0822  tokens at list price (2026-10-25):     $0.0822

What the quality check did not see

The quality check runs pnpm test, pnpm test:acceptance and pnpm typecheck itself, after the session, on the code the session left. It does not look at what the session did. TASK.md and CLAUDE.md both tell the session to run those three checks and report each one's pass and fail counts, and no baseline session of the first measurement did. Claude Code rejected every command the 5 baseline sessions wrote for the checks, 17 in all. The run's grants name the checks as Bash(pnpm test), Bash(pnpm test:acceptance) and Bash(pnpm typecheck), and dontAsk mode denies whatever the grants do not cover (T-09). Every rejected command added $? or ${PIPESTATUS[0]} to print an exit status, or ran pnpm --dir <workspace> test, and no accepted command in the 10 runs had any of those. Each baseline session's final message said that the checks had not run. The routed workers wrote the commands in forms the grants cover and ran all three.

So the first measurement compared a baseline that could not run its checks with a route that did. Its numbers are what its protocol produced, and the protocol stands, but $0.3130 is not the price of the same work. The measurement's pilot baseline run showed the same rejections before the measured runs began, and they were missed: nothing in the protocol read what the sessions did.

Telemetry records every permission decision. Each rejected call is a tool_decision event with decision set to reject, and a safety_event whose reason code says what refused it usually follows; the documentation notes that some refused calls have none (monitoring). All 17 here had one, and each gave the reason code mode:dontAsk. The replays keep these events. In the folder where Step 3.2 wrote the ten measured runs, this counts each run's rejected calls:

for r in .runs/M*; do echo "$r $(grep -o '"stringValue":"reject"}},{"key":"source"' "$r/telemetry/logs.jsonl" | wc -l | tr -d ' ')"; done

.runs/M01-baseline 3.runs/M02-routed 0.runs/M03-baseline 5.runs/M04-routed 0.runs/M05-baseline 3.runs/M06-routed 0.runs/M07-baseline 3.runs/M08-routed 0.runs/M09-baseline 3.runs/M10-routed 0

On the live route, use your own run ids in place of M*. The pattern matches the tool_decision record, which Claude Code 2.1.308 writes with its source attribute right after decision, so each rejected call is counted once and not again for its safety_event.

The second measurement

The second measurement repeated the first with these changes, all fixed before its first run on 2026-10-26:

• Prompt. One sentence added for both configurations: "Run each check as a plain command, exactly as TASK.md writes it, in its own Bash call, with nothing added before or after it." A different prompt is a different workload, so the two measurements are not compared run for run.
• Session check. A run passes only if it passes the quality check and, in the session itself, each of the three checks ran in a call Claude Code did not reject, and the final message names all three.
• Pilot. One run of each configuration with the new prompt, which had to show the baseline session running its checks before the measured runs started. It did.

The task, the two configurations, the flags, the version, the quality check, the statistics and the rollback rule did not change. The runner was the download's own, which stops a run that stays open after its result (F-11, T-12); the first measurement's runner did not.

The difference of the medians is $0.2931, 76.9% of the baseline's median. The rollback rule is not triggered: every routed run passed both checks, both configurations have at least three passing runs, and the ranges do not overlap. Every session ran all three checks. Claude Code rejected 5 calls, each a git -C <workspace> diff or git -C <workspace> status, which the grants do not cover (T-09); none was a check.

Extended description

Title: "Estimated cost per run, ticket T-4207". Subtitle, two lines: "Claude Code 2.1.308, 5 runs each, 10 of 10 passed both checks" and "Second measurement: each session ran the task's three checks". The same axis as the first measurement's chart, labeled from $0.00 to $0.40 and running a little past it, with dotted gridlines every $0.10, and two rows. The upper row, "Baseline, n = 5", in white: five dots at $0.3554, $0.3582, $0.3813, $0.4077 and $0.4279, a gray line spanning the range, and a tall mark at the median labeled "median $0.3813". The lower row, "Routed, n = 5", in mint: four dots close together at $0.0857, $0.0875, $0.0882 and $0.0901, and one well to the right at $0.2067, the run whose main session consulted the advisor; a range line from $0.0857 to $0.2067 and a median mark labeled "median $0.0882". Between the rows, an amber bracket from the routed median to the baseline median: "Difference of the medians: $0.2931 (76.9%)". Within a row, a dot's height only keeps close values apart; it carries no value. Footer: "Client estimates, measured 2026-10-26".

Two routed runs did something no run of the first measurement did:

• One main session consulted the advisor without being asked. The advisor's row cost $0.1133 and made that run $0.2067, the top of the routed range; the other routed runs cost $0.0857 to $0.0901 (Watch the advisor).
• One main session scheduled a wake-up early in the run. The run printed its result with the wake-up still pending, and the runner stopped it 10 seconds later, before the wake-up could add a turn (F-11). It passed both checks.

What the two measurements show, and what they do not

• They show that on this task, on these two days and this version, the route ran each agent on its intended model and spent less. In the second measurement, both configurations also did the work the task asks, including its checks.
• They do not show a saving on other tasks, other repositories or other versions, and they say nothing about a month of use. A task with more exploration, longer context or harder decisions can move the result either way.
• All runs started with a warm cache, so the figures are for back-to-back work, not a first run of the day.
• In the first measurement, the routed main session skipped the researcher in 4 of its 5 runs, and the baseline's main session in 2 of 5; in the second, the routed main session skipped it in 5 of its 5 runs, and the baseline's in 0 of 5. Routing in CLAUDE.md is guidance the model weighs, not a rule it must follow; the telemetry table is how you find out what it did.
• A check that runs after the session cannot tell you whether the session did its own work. Count each run's rejected calls before you compare (F-12).

Labs

The labs build the measurement workspace from Guide 05's published download, so every learner works on the same files. Each lab has a live route, which calls the API and costs money, and an offline route, which replays a recorded run and costs nothing.

On the live route the labs cost roughly $2.88 in API usage at 2026-10-25 list prices, from the recorded runs; the offline route replays recorded runs and costs nothing.

• Lab 1: Read a baseline run (about 35 minutes). Build the workspace, run the fixed task on the baseline configuration with telemetry on, and read the table by model, query source and agent. You state the run's three cost estimates and the size and cause of any difference.
• Lab 2: Route in layers, predicting first (about 45 minutes). Apply the routing one layer at a time: the session model, the agents' models, effort settings and caps, the advisor, then full-ID pins. Before each probe you write down the model and effort you expect for every agent, then check the prediction against telemetry.
• Lab 3: Find the faults and write the report (about 40 minutes). Diagnose five seeded misconfigurations, fix each one and rerun its probe, then measure both configurations and write the measurement report with its rollback rule.

The download, model-routing-labs.zip, holds the handout (labs/learner-labs.md), the starter kit with Guide 05's fixture and its notice, the agent templates, the telemetry collector and table, the quality and report checks, the offline replay and the Lab 2 solution files. The code is MIT licensed.

Troubleshooting

Each entry quotes what the kit or Claude Code printed in a recorded run or a kit check on 2026-10-25, unless it gives another date.

T-01: Preflight fails on Node.js

FAIL Node.js: ... (needs 22.18 or later: the fixture runs TypeScript with Node.js's built-in type stripping). The fixture's scripts run .ts files directly. Install Node.js 22.18 or later (download Node.js).

T-02: pnpm is missing or not version 12

FAIL pnpm: not on PATH. On Node.js 22 or 24, run corepack enable (Corepack README); Corepack ships with Node.js only before 25.0.0, so on a later Node.js follow pnpm's installation page. The fixture pins pnpm 12.14.0 in its package.json.

T-03: The runner asks for an API key

claude-run: set ANTHROPIC_API_KEY first (this lab uses an Anthropic API key, not a sign-in). Export the key in the terminal that runs the labs, or take the offline route. The runner passes the key through to Claude Code and prints nothing about it.

T-04: The collector's port is in use

claude-run: port 4318 is in use; stop whatever listens there or set ROUTING_LAB_PORT. Another collector, often one left by an interrupted run, still listens. Stop it and confirm the port closed, or run with ROUTING_LAB_PORT=4319.

T-05: A run id is already taken

claude-run: .runs/<id> exists; pick a new run id. Every run keeps its own folder so nothing is overwritten. Choose a new id, such as lab2-L1b.

T-06: A measured run refuses to start

measure: the working tree has changes; commit, stash or move them first (git status lists them, untracked files included). measure.sh checks out a tag and resets the working tree afterward, so it will not start over uncommitted work. An untracked file counts too, and git stash alone leaves untracked files in place. Commit your routing (Lab 2 step 2.7), and write notes and the report after the measured runs or outside the workspace.

T-07: A run stops with a budget error

The last result line says error_max_budget_usd, and the exit code is 1 (LIVE-11). The run reached its --max-budget-usd limit. Count it as a failed run in any comparison; do not raise the limit inside a measured batch.

T-08: The table is empty or says to set a temporality

table: these counters use cumulative temporality; set OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta (the default) and run again. Something in your --env or --env-file changed the exporter settings. Run without it; the runner's own telemetry variables are listed in Read a run from its telemetry. Claude Code sends metrics and events when it exits, so a run killed early can leave the table empty.

T-09: Commands were rejected

The events line shows safety_event entries, and the result line may list permission_denials. In dontAsk mode, Claude Code denies every call that would otherwise ask for permission (permissions). A grant must match each part of a compound command on its own, read-only commands such as echo and tail need no grant, and a grant matches the command as written: Bash(pnpm typecheck) covers pnpm typecheck 2>&1 | tail -40, but Bash(pnpm test) does not cover pnpm --dir <workspace> test, and Bash(git diff) does not cover git -C <workspace> diff. In the recorded runs of 2026-10-25 and 2026-10-26, Claude Code accepted 57 calls that piped a check into tail and rejected 109 Bash calls. Each rejected call ran a command from another folder with --dir or -C, added $? or ${PIPESTATUS[0]}, ran a command held in a variable, or printed a variable with echo, though a plain echo needs no grant; no accepted call did any of these. Each rejection's safety_event gave the reason code mode:dontAsk (monitoring). A rejection is the grant working. It is a failure when it blocks what the task asks for: a session whose checks were rejected never ran them, even when the quality check passes afterward (F-12). Count each run's rejected calls before you compare (Measure the route).

T-10: A quality check fails on scope

FAIL Q1 verifier from <tag>: verdict FAIL; failing V2 together with FAIL Q2 only src/policy.ts changed. The run edited a file outside the task's scope. Report the run as failed and keep it out of every median.

T-11: Replay will not overwrite a run

replay: .runs/<id> already holds a run that was not replayed; move it before replaying <id>. A live run of yours has the same id as a recorded one. Move your folder; replayed folders carry a REPLAYED.txt.

T-12: The runner stopped a run after its result

claude-run: the run printed its result with a wake-up still scheduled and no background task pending; stopped it 10 s later, before the wake-up could add a turn. See <run-dir>/stopped-after-result.jsonclaude-run: the run printed its result and was still running 30 s later with no background task pending; stopped it. See <run-dir>/stopped-after-result.json

The first means the main session scheduled a wake-up (F-11); the second, that the process stayed open after its work for some other reason. The note lists every scheduling call and the rule that applied. run.json keeps Claude Code's own exit code as claude_exit_code and sets exit_code to 0 only when the last result has subtype success and is_error false; an API error prints a result with subtype success and is_error true. The quality check still decides whether the run passed.

Quick reference

The advisor runs only when advisorModel ranks at or above the main model and nothing turns off feature-flag fetching (DISABLE_TELEMETRY, CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC set to any value) or the tool (CLAUDE_CODE_DISABLE_ADVISOR_TOOL) (advisor).

Commands from the kit, run from the workspace:

bash <labs>/starter/run/probe.sh <run-id> [--env NAME=value] [--advisor]   # which model and effort each agent getsbash <labs>/starter/run/measure.sh <tag> <run-id>                          # one measured run with the quality checknode <labs>/starter/telemetry/table.mjs .runs/<run-id>                     # the table for one runnode <labs>/starter/telemetry/compare.mjs --baseline spec .runs/<ids>...   # medians, ranges and the rollback rulenode <labs>/starter/checks/report-check.mjs report.md                      # a report against the runs it citesnode <labs>/starter/replay/replay.mjs [<id>]                               # the offline route

Before you compare, count each run's rejected calls (Measure the route, F-12). This counts them for the ten recorded runs; for your own runs, put their ids in place of M*:

for r in .runs/M*; do echo "$r $(grep -o '"stringValue":"reject"}},{"key":"source"' "$r/telemetry/logs.jsonl" | wc -l | tr -d ' ')"; done