From Expert to Cleared-to-Teach: Instructor Calibration

A clearance decision answers a narrow question: has this person shown, under observation, that they can teach this segment of this module version, in this delivery format, to this audience, without supervision? It does not answer whether they are an expert, a good presenter or a good engineer. All of those can be true and the answer can still be "not yet".

Three engineering habits carry over directly:

1. Scope the claim. A test result applies to the commit it ran on. A clearance applies to the content that was observed. This guide binds it to a module id, a version and a SHA-256 fingerprint of the module's files.
2. Separate measurement from decision. Observers record evidence and score it against anchors. A rule, written down in advance, turns the reconciled scores into a decision. When the two are mixed, the loudest or most senior observer decides.
3. Distrust a single observer. In a three-year study of classroom observation, teachers' observation scores varied more from observer to observer than from lesson to lesson, and adding a second observer increased reliability significantly more than having the same observer score another lesson (MET project, 2013, p. 5). That is why every decision here starts with two independent scorecards.

Extended description

Title: "From expert to cleared". Seven numbered boxes in two columns. The left column runs down, 1 to 4; an arrow crosses to the bottom of the right column, which runs up, 5 to 7. Each box gives a step and the file the kit checks at that step:

1 Plan the segment: segment-plan.json; 2 Dry run: transcript.json; 3 Score twice: observer-*.json; 4 Calibrate: pnpm calibrate; 5 Reconcile: reconciled.json; 6 Coach one change: coaching-note.json; 7 The rule decides, mint border: clearance-record.json.

Above step 7, a dashed amber box: "Module changed", "invalid: back to a dry run". A dashed amber arrow runs down from it into step 7, and another runs left and down into step 2, Dry run. Footer: "Proposed local process. Synthetic data."

The rest of the guide walks that pipeline once on a worked example, CAL-01, and leaves the second sample, CAL-02, for the assessment.

Set up the lab

The lab download, instructor-calibration-labs.zip, holds one folder, labs/:

• labs/module-01/ is the module under clearance: the labs/ folder of Guide 01's published lab download, version 1.2.0, copied byte for byte, all 34 files, with its MIT license. The kit calls it MOD-01 1.2.0. Never edit it; the drill runs in a disposable copy.
• labs/starter/ is the calibration kit: the rubric, the clearance rule, the module card, two scripted teach-back samples, a checker, and acceptance tests for each lab.
• labs/solutions/ holds reference work for the four labs, and labs/faults/ three files that are wrong on purpose.

With pnpm 12 on your PATH, unzip it into an empty folder, then:

cd labs/starterpnpm install --frozen-lockfilepnpm preflightpnpm reset

pnpm install fetches TypeScript and the Node.js types for pnpm typecheck; the kit has no runtime dependencies. pnpm reset copies templates/ into work/, moving an existing work/ to .backup/ first. On a fresh extract, preflight prints:

PASS  Node.js: 22.23.4 (needs 22.18 or later for built-in TypeScript)PASS  pnpm: pnpm/12.14.0PASS  dev dependencies: installed (needed only for pnpm typecheck)PASS  module copy: ../module-01PASS  module fingerprint: MOD-01@1.2.0 matches the module cardINFO  work folder: run "pnpm reset" to create it Ready.

The fingerprint is the kit's own SHA-256 over every file's path and content in labs/module-01/. You can recompute it for your own copy of Guide 01's download with pnpm fingerprint --dir <your extract>/labs; on 2026-10-24 that printed sha256:6607ac8cda66123b3038172eae6119d42341d80e43d46cba7690759abf106457, the value every record in this guide carries.

Turn expertise into a teachable segment

An expert explaining a timeout bug tends to tell the story of the fix. A teachable segment makes learners commit to a prediction, watches the system contradict it, and ends with the learners reading the proof themselves. The shape is the same every time:

Extended description

Title: "The shape of a teachable segment". A mint label, "Learner evidence", over three mint tags, each joined by a dotted line to the step it belongs to: "a written prediction" over Predict, "who was wrong, and why" over Diagnose, and "a learner reads INV-03" over Verify. Five boxes left to right, joined by arrows: Predict, Break (amber), Diagnose, Repair, Verify (mint). A dashed amber arrow drops from Repair to an amber box, "Planned demo failure: Enter at the prompt means no", which leads right to "Recover: read the trace, reset, type y", and a mint arrow runs from Recover up to Verify. Footer: "Offline: scripted model double, not a live model. Synthetic data. Simulated services."

The segment: a timeout means unknown, not failed

The module is Guide 01's support desk. Its agent refunds $48.00 on ticket T-1042. With the refund-timeout fault injected, the payment service commits the refund and then loses the response. On checkpoint a, where the agent loop works and nothing is repaired yet, the agent treats the timeout as a failure and calls issue_refund again. Segment SEG-F02 teaches why that is wrong: a timeout means the outcome is unknown, not that the action failed.

Run it in a disposable copy of the module, from labs/starter:

cp -R ../module-01 ../drillcd ../drill/starterpnpm install --frozen-lockfilecp ../solutions/checkpoint-a/src/agent.ts src/pnpm resetpnpm agent --ticket T-1042 --inject refund-timeoutpnpm verify --ticket T-1042

pnpm reset here is the module's own command: it resets the desk's data. The agent claims success and the record disagrees (excerpt):

Agent said:   "Refunded $48.00 to order ORD-5521 and resolved ticket T-1042."Harness stop: end_turn after 6 model call(s)...  FAIL  INV-03  at most one payout per ticket: 2 payout(s): RF-0001 $48.00, RF-0002 $48.00...Outcome: failed

pnpm trace shows the cause: issue_refund with id toolu_scripted_004 returns POST /refunds timed out after 3000 ms, and the identical call toolu_scripted_005 succeeds. Checkpoint final, Guide 01's repair, makes the action's fingerprint (refund:<ticket>:<order>:<amount>) an idempotency key, a value the payment service uses to recognize a repeat of the same request, and after a timeout it reads back by that key before acting again. Apply it in the same copy and run again, typing y at the approval prompt:

cp ../solutions/final/src/*.ts src/pnpm resetpnpm agent --ticket T-1042 --inject refund-timeoutpnpm verify --ticket T-1042

  PASS  INV-03  at most one payout per ticket: 1 payout(s): RF-0001 $48.00...Outcome: verified_success

The run's log records the repair's own note: The first response timed out; this refund was confirmed by read-back.

The demo failure you should plan for

On checkpoint final the agent stops for approval, and pressing Enter without typing y is a denial. Two denials trip the repetition stop: the run ends with Harness stop: repeated_failure after 4 model call(s) and Outcome: safe_escalation, with no payout and the ticket set to needs_human. That is correct behavior, and in a live room it looks like your fix broke. A plan that names this failure, what it means and the recovery (pnpm reset, run again, type y) turns it into a teaching moment instead of a stall. CAL-01's candidate hits exactly this at line L20.

In Lab 1 you write the plan for this segment in work/segment-plan.json, and pnpm check compares it with the module card, so the fault, the failing check and the repair checkpoint must be the ones this module actually has. The checker also rejects the usual expert shortcuts: an objective with a verb nobody can observe ("understand", "know", "learn about"), phases out of order, no learner evidence before the fault or after the repair, rhetorical checks such as "Make sense?", no planned demo failure with a recovery, no pnpm reset in the reset steps, and honesty labels that do not say the model is the scripted double and the data is synthetic.

Write anchors observers can score

A rubric has criteria, descriptors and performance levels (Carnegie Mellon Eberly Center). The descriptors are where calibration lives or dies. "Explains clearly" invites each observer to use their own bar. An anchor names a behavior that either happened or did not, at a line you can point to. Rating scales anchored by examples of expected behavior go back at least to Smith and Kendall's 1963 procedure, which kept an example only when other judges assigned it to the same dimension (Smith and Kendall, 1963).

The kit's rubric (rubric/teachback-rubric.v1.json, a proposed local standard) has six dimensions scored 0 to 3. Four are critical, meaning a low score blocks clearance no matter how strong the others are:

Extended description

Title: "Six anchored dimensions", with "Mint meets the bar. Amber is below it." at the right. Six rows, each with an icon, its id, its name, its bar and four score cells, 0 to 3. Cells at or above the bar are mint, cells below it amber.

D1 Technical accuracy: bar 2, critical; mint cells 2, 3. D2 Goal and structure: bar 1; mint cells 1, 2, 3. D3 Learner evidence: bar 2, critical; mint cells 2, 3. D4 Failure recovery: bar 2, critical; mint cells 2, 3. D5 Responding to questions: bar 1; mint cells 1, 2, 3. D6 Honesty and safety: bar 2, critical; mint cells 2, 3.

Footer: "Newly authored work sample. Not a validated instrument."

Three design choices are worth copying, whatever your scale:

• Anchors are module-specific. D1's 0 anchor names the exact false claim this segment invites: "a timeout means the refund failed and a retry is safe." Generic anchors produce generic disagreement.
• The top level is "could model this behavior for other instructors", not "perfect". That keeps 3 rare and meaningful, and keeps the bar at 2.
• Scores need evidence. Every score above 0 cites at least one transcript line, and D4 must cite a line inside the transcript's drill window, the span where the live fault ran. A D4 score with no drill-window citation means the observer did not see the recovery.

Observe a dry run without scoring the person

A dry run is a rehearsal of the segment in front of observers and a few stand-in learners, with the live fault included. The protocol this guide proposes:

1. Two observers who score independently. They do not discuss the session until both scorecards are saved.
2. A transcript or recording with line ids. The kit's samples are scripted transcripts with ids L01 to L36 and scripted times, not recordings. In a real program you would record, with the candidate's consent.
3. Each observer writes what was said or done, then scores against the anchors. Notes describe behavior. "A polished speaker who owns the room" is about the person; "asked for predictions at L03, then at L12 asked Learner 1 to compare theirs with the INV-03 line" is evidence.

Why the note discipline matters: in a study of observations by school personnel, observers who formed a positive or negative impression of a teacher in the first several videos tended to carry it into later scoring (Ho and Kane, MET project). The kit's checker flags trait words (for example natural, confident, polished, energy) as a warning: a cheap heuristic that prompts the observer to rewrite the note as behavior.

To see the same view in your terminal, with each observer's cited dimensions next to each line:

pnpm replay CAL-01 samples/CAL-01/observer-a.json samples/CAL-01/observer-b.json

Score with evidence

In Lab 2 you read samples/CAL-01/transcript.json, a scripted teach-back of SEG-F02 by a synthetic candidate, CAND-SYN-01, and score it before you open anyone else's card. For each dimension: find the lines, find the anchor they meet, then write the note. A worked line of reasoning for D4:

• The fault runs live at L07, and the verifier shows two payouts at L11. The candidate diagnoses from the trace at L16.
• At L20 the candidate presses Enter at the approval prompt. That was not planned, and L21 shows safe_escalation.
• L22: "That is not what I expected, so I am going to read it before I touch anything." L24 explains what learners would see and what it means. L25 resets with a reason. L27 shows INV-03 passing.
• Anchor 3 asks for an unplanned problem turned into something learners could repeat alone. L22 to L25 do that. Whether the candidate caused the problem is not what D4 scores.

Then measure yourself. samples/CAL-01/reference.json holds reference scores for CAL-01 with cited lines and a rationale for each, so you can count how many of your six scores match exactly and how many are two or more points away. That comparison is the accuracy check of Calibrate two observers, applied to you.

Calibrate two observers

Two different questions hide behind "do the observers agree?", and published observation research keeps them apart. Inter-rater agreement is "the extent to which multiple observers assign the same score to the same lesson or teacher"; accuracy is "the extent to which observers are able to assign the correct score to a lesson using a particular rubric" (MET project, 2014, p. 14). Two observers can agree with each other and both be wrong. Programs measure accuracy against master-coded samples, scored by experts who "determine the correct scores, and the correct rationales for those scores" (p. 14), and they reassess observers over time, because observers drift: a drift is "a tendency to inflate or deflate scores over time" (p. 24), and the same guidance recommends re-assessing observers at least annually (p. 22).

This kit lets you run both checks on one sample. CAL-01's reference scores play the role of master-coded scores, with one honest difference: they were written for this guide from the anchors, not master-coded by a panel of experts.

Lab 3 starts here, with both checks in one command:

pnpm calibrate samples/CAL-01/observer-a.json samples/CAL-01/observer-b.json --reference samples/CAL-01/reference.json

Calibration CAL-01: OBS-SYN-A (A) and OBS-SYN-B (B) 1. Disagreements that change the decision  D3 Learner evidence (critical, bar 2): A 2, B 1  Two-point gaps: none  Settle each one from the cited lines and the anchors before any clearance decision. 2. Agreement between A and B, per dimension  dim  name                                  crit  bar  A  B  diff  status    shared evidence  D1   Technical accuracy                    yes   2    3  2  1     adjacent  25%  D2   Goal and structure                    no    1    3  2  1     adjacent  33%  D3   Learner evidence                      yes   2    2  1  1     adjacent  11%  D4   Live failure diagnosis and recovery   yes   2    3  2  1     adjacent  63%  D5   Responding to questions               no    1    2  1  1     adjacent  67%  D6   Honesty and safety boundaries         yes   2    3  3  0     exact     0%  Exact agreement: 1 of 6. Within one point: 6 of 6.  Same score, different evidence: D6 (0% shared). Check which cited lines meet the anchor before you treat it as settled. 3. Accuracy: each observer against the reference scores...  A: exact 4 of 6; discrepant (2 or more points away) 0 of 6; critical dimensions on the other side of the bar: none  B: exact 3 of 6; discrepant (2 or more points away) 0 of 6; critical dimensions on the other side of the bar: D3 4. Kappa between A and B, n = 6 paired ratings (too few for a stable estimate; decide from sections 1 to 3)  quadratic weighted 0.400; unweighted -0.154

Read the output in the order it prints:

1. Disagreements that change the decision. A disagreement matters when the two scores sit on different sides of the bar. D3 is 2 for A and 1 for B; 2 clears a critical dimension and 1 blocks it. The other four disagreements do not change anything. A two-point gap would come next: it means the observers read the anchors differently, not that they rounded differently, and the rule requires a calibration follow-up for it (R3).
2. Agreement, per dimension. Within one point on six of six sounds like strong agreement; exact agreement is one of six. Shared evidence says where to look first: on D3 the observers cited almost entirely different lines (11% shared), and B never cited L12 to L14, where the candidate compares a wrong prediction with INV-03. On D6 they agree exactly, from different lines: A cites L01 and L35, B cites L24 and L25. B's lines show the writes stayed in the lab, but not the label before the first run that the anchors ask for. Agreement on the number is not agreement on the evidence.
3. Accuracy. Against the reference, A matches four of six exactly and B three of six, and neither is two or more points away anywhere. For comparison only: MET's study of observation instruments required raters to show "At least 50% exact match with correct score. No more than 25% discrepant scores (2 or more points from the correct score)" on pre-scored videos before they rated lessons (MET project, 2014, p. 21). That is an example threshold for observers measured on many master-coded videos, not this guide's rule, and one sample of six scores cannot apply it. It still makes the point: a threshold like that would not catch B's one miss, D3, which is the one that changes the decision.
4. Kappa, last, beside its n. Cohen's kappa compares the observed agreement, p_o, with the agreement expected by chance, p_e, as kappa = (p_o - p_e) / (1 - p_e), and a weighted version penalizes larger gaps more (scikit-learn 1.9.1). On the same six pairs the quadratic weighted kappa is 0.400 and the unweighted kappa is -0.154: A is one point higher than B on five of the six dimensions, and unweighted kappa gives no credit for a near miss. The kit's numbers match scikit-learn's cohen_kappa_score with labels=[0, 1, 2, 3] on three test cases. Interpretation bands for kappa are themselves contested: one review argues that Cohen's suggested interpretation "may be too lenient for health related studies because it implies that a score as low as 0.41 might be acceptable" (McHugh, 2012). With six paired ratings from one session, no band applies. Use kappa across many calibration samples to watch for drift, and use sections 1 to 3 to make a decision.

Two scripted samples are not a reliability study, and none of these numbers came from real observers. They show what to compute and in what order.

Reconcile by evidence, not by averaging

Reconciliation produces one score per dimension that both observers can defend from the transcript. Guidance for master coders is to score independently and then reconcile in pairs, because "with more than two coders there's a temptation to reach consensus rather than agreement on what scores are best supported by the evidence" (MET project, 2014, p. 16).

Averaging looks fair and is the most common failure. Lab 3's third step makes it on purpose: copy the averaged card in labs/faults/ into place as your reconciled card and check it.

Reconciled card work/CAL-01.reconciled.json  FAIL  C2      D1: 2.5 is not a whole number. Averaging is not reconciliation: decide which anchor the evidence supports  FAIL  C2      D2: 2.5 is not a whole number. Averaging is not reconciliation: decide which anchor the evidence supports  FAIL  C2      D3: 1.5 is not a whole number. Averaging is not reconciliation: decide which anchor the evidence supports  FAIL  C2      D4: 2.5 is not a whole number. Averaging is not reconciliation: decide which anchor the evidence supports  FAIL  C2      D5: 1.5 is not a whole number. Averaging is not reconciliation: decide which anchor the evidence supports  WARN  C7      D6: both observers gave 3, from different lines (0% shared); write a resolution that says which cited lines meet the anchor

A 1.5 on D3 describes nothing that happened. Either the candidate collected and responded to learner evidence before and after the fault, or they did not. Averaging also hides the one disagreement that matters: rounded, D3's 1.5 becomes a 2 or a 1 by arithmetic, not by evidence.

Then write the reconciled card. For every dimension where your score differs from either observer, write a resolution that names the anchor and cites the lines. Example for D3:

"D3": {  "score": 2,  "evidence": ["L03", "L04", "L05", "L12", "L13", "L14", "L28", "L29"],  "resolution": "B cited only L03 and L06. L12 to L14 compare Learner 1's prediction with INV-03 and respond to the wrong answer; L28 and L29 collect a reading of the record after the repair. That is anchor 2. Not 3: L28 goes to Learner 3, not Learner 1 or 2, who predicted wrong at L04 and L05."}

The checker fails a resolution with no line ids, one that describes a compromise ("split the difference"), and any two-point gap without an entry in calibration_followups for the observer whose score the evidence did not support. It warns when the observers agree on a score from different lines and the card does not say which lines meet the anchor, as on D6. When the card is settled, the same check prints:

Reconciled card work/CAL-01.reconciled.json  PASS  C*      CAL-01 reconciled from OBS-SYN-A and OBS-SYN-B: every change has a cited resolution

Calibration follow-ups are optional when there is no two-point gap, and still useful. In CAL-01, observer A credited a D6 behavior (naming production changes at L35) under D1's top anchor, and observer B missed the learner lines for D3. Both are worth one sentence each.

Coach one change

A clearance decision says whether. A coaching note says what to do next, and the rule here is one change at a time: one dimension, the lines that show the behavior, the change, a practice task, and a re-observation target. If a dimension blocks clearance, coach that one first; the checker enforces it. Observation guidance describes the same shape for a post-observation conference: the "discussion is grounded in the rubric and objective evidence from the lesson, and the conference ends with agreement on one or two small changes the teacher will make before the next observation" (MET project, 2014, p. 19).

For CAL-01 nothing blocks clearance, and D5 is the weakest dimension. At L31 a learner asks whether the idempotency key would block a legitimate second identical refund. L32 answers a different question, about retries of model requests, and L33 ("Sort of.") shows it did not land. A useful note:

• Change: restate the question before answering ("You are asking whether a real second refund would be blocked"), then answer that question. In this module the key is refund:<ticket>:<order>:<amount> (solutions/final/src/tools.ts), so a second refund with the same ticket, order and amount returns the first payout. A legitimate second refund needs its own identifier in the key and a policy that allows it.
• Practice: rehearse three learner questions about the key with a peer; restate each in one sentence, answer from the code, and ask a checking question that needs an answer.
• Re-observe: D5 at 2.

"Be more confident and less defensive when learners ask questions" is a note about the person. The checker fails it (K4) and asks for a behavior instead.

Decide and record clearance for one module version

The proposed rule (rubric/clearance-rule.v1.json) has eight requirements. R1 to R4 make the evidence usable: two independent observers, scorecards that pass the check, two-point gaps settled with follow-ups, and resolutions that cite lines. R5 and R6 are the bar: every critical dimension at 2 or higher, the others at 1 or higher, and no open accuracy or safety flag. R7 binds the record to the module card and to the files on disk. R8 checks completeness, including an expiry no more than 12 months out and at least three review triggers, one of them a module change.

The rule gives one of three decisions:

In Lab 4 you write work/CAL-01.clearance-record.json, take the fingerprint from pnpm fingerprint, and let the kit recompute the decision from the evidence and compare it with yours. For the reference record:

  PASS  R5      critical dimensions at 2 or higher, the others at 1 or higher  PASS  R6      no open accuracy or safety flags  PASS  R7      MOD-01@1.2.0 SEG-F02, fingerprint sha256:6607ac8cda66... matches  PASS  R8      record is complete Decision computed from the evidence: cleared. Record says: cleared. They match.

"Cleared" here means cleared to teach SEG-F02 of MOD-01 1.2.0, live and virtual, to the stated audience, until the record expires or a trigger fires. It is not a certification, and nothing in the kit certifies anyone.

The failure path: the module changes

Now change the module by one comment line, in a copy, and check the same record against it:

pnpm simulate-updatepnpm clear work/CAL-01.clearance-record.json --module .sim/module-01

  FAIL  R7      module files in .sim/module-01 have fingerprint sha256:bbd798d3d1e4..., not the cleared sha256:6607ac8cda66...; the clearance does not transfer to changed content...Decision computed from the evidence: invalid. Record says: cleared. They do not match.

Extended description

Title: "A clearance names what was observed". Left card, mint border: "CLR-SYN-0001" with a mint "cleared" tag; candidate CAND-SYN-01, module MOD-01 1.2.0, segment SEG-F02, format live, virtual, expires 2027-10-24, fingerprint sha256:6607ac8cda66... in mint. A dashed amber arrow leads to the right card, dashed amber border: "One line changed"; module MOD-01 1.2.0; fingerprint sha256:bbd798d3d1e4... in amber; "same version, new content"; and an amber tag "R7 FAIL: invalid". Below both: "Review again when", then five tags: fingerprint changes, new minor or major version, 12 months pass, format or audience changes, learners miss the objective. Footer: "Synthetic record. Not a real clearance. Run 2026-10-24."

This is not hypothetical. This guide's own first draft copied Guide 01's lab on 2026-10-17. By 2026-10-24 Guide 01's published download differed from that copy in one drill file, starter/scripts/trace.ts, and the draft's reference record, checked against the current files, came out invalid (F-08).

Binding to the exact content is strict on purpose. In practice you would pair it with a light path: a reviewer reads the diff and decides whether the change needs a full dry run or a short re-check of the dimensions it touches. That review is itself a recorded decision. What the rule prevents is silent transfer, where a clearance for last quarter's module is assumed to cover this quarter's.

The record is about a person

A real clearance record is personnel data about the candidate. This is practice guidance, not legal advice; check your organization's own rules with whoever owns its personnel records.

• Where it lives. In the system your organization already uses for training or personnel records, not in a shared drive or a chat thread.
• Who may read it. The candidate, the two observers, the lead instructor and the manager who signs the decision. Nobody else needs the scores.
• How long it is kept. A retention period set with whoever owns personnel records. An expired clearance can be archived or deleted on that schedule; the expiry date in the record is not a retention rule.
• What the candidate sees. Their own record, both scorecards, the reconciled card and the transcript lines each score cites, so they can check the evidence and question it.
• Recordings and transcripts. Recorded with the candidate's consent, under the same access rules.

The lab's records are synthetic; none of them is about a real person.

What this guide simplifies

• Synthetic, scripted samples. Real teach-backs are messier, longer and recorded. A program needs consent and storage rules for recordings and transcripts.
• One segment, one module. Real clearance covers several segments, often with a different rubric emphasis per segment. The module card holds a list of segments for that reason.
• No validated rubric. The anchors here are a proposed local standard. Validating a rubric needs master-coded samples, many observers and learner outcome data, none of which exist for it.
• No observer certification. Some school systems "grade observers on a pass-fail basis depending on whether they can score within a certain range of a 'gold standard'" and require periodic re-certification (Gates Foundation, "Ensuring Accurate Feedback from Observations", p. 2). This guide calibrates two observers on one sample and measures them against reference scores that were not master-coded; it certifies nobody.
• A strict fingerprint. Any byte change invalidates the record. A production program would add the reviewed fast path described in Decide and record clearance for one module version.
• Assessment design and reporting are their own subjects. How to build a practical assessment and report honest pass rates, how to run this calibration in a large lab room, and how to keep a module and its clearances current are pointers for later guides, not covered here.

Troubleshooting

Each entry was reproduced from the lab files on 2026-10-24.

Quick reference

pnpm preflight                                  # environment and module fingerprintpnpm reset                                      # templates into work/pnpm check work/segment-plan.json               # LAB-001pnpm check work/CAL-01.observer-me.json         # LAB-002pnpm calibrate <card-a> <card-b> --reference <card>pnpm check <reconciled> --a <card-a> --b <card-b>pnpm check <coaching-note> --reconciled <reconciled>pnpm fingerprint                                # value for the recordpnpm clear <record> [--module <dir>]            # LAB-004 and ASM-001pnpm replay <sample-id> [cards...]              # annotated transcriptpnpm test:lab1                                  # also test:lab2 to test:lab4, test:asm

• Score before you see anyone else's card. Notes describe behavior, with line ids.
• Disagreements that change the decision first, agreement second, accuracy against the reference third, kappa last and beside its n.
• Reconcile to a whole-number anchor with cited lines. Never average.
• Coach one change: behavior, lines, change, practice, re-observation target.
• Clearance names a module id, version, fingerprint, segment, format and audience, and it expires.