Readiness
What Has Run, Where, and What Is Still Missing
This page says, module by module, what is known about whether the workshop can be taught as designed. It is generated from two sources, never written by hand: the readiness fields in _variables.yml and the run records in runs/, one record for each time a notebook was run and timed. A run time appears here only if a record exists.
As of 2026-10-09: not yet ready to teach. 0 of 15 labs have run end to end, with their current code, on the Colab runtime they are designed for. 15 have run end to end on their real path elsewhere, on another machine or on the CI runner, and 0 more in part. On the GitHub CPU runner (newest run of each notebook, 2026-10-06), 16 of 16 notebooks passed; 9 of the passing runs used test doubles, which check that the code runs, not what the models do. 8 of 11 pieces of blocking work are open.
| Module | Content | Designed for | Evidence | Without API keys | Accounts and cost |
|---|---|---|---|---|---|
| 0 · Coding Agents in the Terminal | Briefing drafted | The participant’s own laptop · estimate 90 min | No notebook. Not yet run on a fresh laptop | One path. Pair with a neighbor, or use Gemini CLI’s free tier | GitHub; optional: Claude Code, Codex, Gemini CLI. Claude Code and Codex need a paid plan; Gemini CLI has a free tier with a personal Google account |
| 1 · Text as Data | Briefing reviewed · lab written | Google Colab, CPU runtime · estimate 2 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 13 s CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 19 s, real path (no test doubles) |
One path. Runs fully on a CPU | Google. Free |
| 2 · Word Vectors and Neural Networks | Briefing reviewed · lab written | Google Colab, free T4 GPU · estimate 10 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 31 s CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 47 s, real path (no test doubles) |
One path. Runs fully; a CPU runtime is slower | Google. Free |
| 3 · Sequence Models | Briefing reviewed · lab written | Google Colab, free T4 GPU · estimate 10 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 4.4 min CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 79 s, QUICK settings, real path (no test doubles) |
One path. QUICK = True trains a quarter of the steps of the same model | Google. Free |
| 4 · Seq2seq and Attention | Briefing reviewed · lab written | Google Colab, free T4 GPU · estimate 10 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 13 min CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 96 s, QUICK settings, real path (no test doubles) |
One path. QUICK = True trains 500 steps and skips the accuracy checkpoints | Google. Free |
| 5 · The Transformer | Briefing reviewed · lab written | Google Colab, free T4 GPU · estimate 10 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 2.6 min CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 3.5 min, QUICK settings, real path (no test doubles) |
One path. On a CPU runtime the lab switches itself to QUICK | Google. Free |
| 6 · Pretraining and the Hugging Face Stack | Briefing reviewed · lab written | Google Colab, free T4 GPU · estimate 7 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 53 s CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 36 s, with test doubles |
Open model. On a CPU runtime, BERT-mini fine-tunes on 800 training texts: a real, smaller model. There is no path without the Hugging Face Hub | Google. Free |
| 7 · Fine-Tuning and LoRA | Briefing reviewed · lab written | Google Colab, free T4 GPU · estimate 7 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 9.8 min CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 10 s, with test doubles |
Open model. On a CPU runtime, FAST runs a shorter fine-tune of the same SmolLM2-135M. There is no path without the Hugging Face Hub | Google. Free |
| 8 · LLMs Through APIs | Briefing reviewed · lab written | Google Colab, free T4 GPU · estimate 3 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 2.3 min CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 9 s, with test doubles |
Open model. Qwen2.5-0.5B-Instruct answers in place of OpenAI and Claude; its numbers describe Qwen, not the commercial models | Google; optional: OpenAI, Anthropic. Free without keys. Keyed run: under 25 cents on Claude, under 5 cents on OpenAI (estimates) |
| 9 · Reinforcement and Preference Learning | Briefing drafted · lab written | Google Colab, CPU runtime · estimate 10 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 90 s CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 1.7 min, with test doubles |
One path. Part A runs anywhere; Part B loads the committed Lab 9 data files and downloads no model | Google. Free |
| 10 · RLHF | Briefing drafted · lab written | Google Colab, free T4 GPU · estimate 7 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 34 s CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 18 s, with test doubles |
Open model. On a CPU runtime, FAST runs short smoke trainings of the same GPT-2; the trained-policy checkpoints are skipped, so reward hacking is not shown | Google. Free |
| 11 · Calibration | Briefing drafted · lab written | Google Colab, CPU runtime · estimate 7 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 5.6 min CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 14 s, with test doubles |
Open model. Qwen2.5-0.5B-Instruct answers on a fixed subset; its numbers describe Qwen, not the commercial models | Google; optional: OpenAI, Anthropic. Free without keys. Keyed run: under 1 USD on Claude, under 25 cents on OpenAI (estimates) |
| 12 · Calibrated Decisions: RLCD and Jev | Briefing reviewed · lab written | Google Colab, CPU runtime · estimate 3 min | On its Colab runtime: not run Real path, other runs: 2026-10-08, Apple M1 Pro laptop, 16 GB (CPU/MPS), 16 s CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 14 s, real path (no test doubles) |
Toy model. A small decision model trained in the lab answers in Jev’s place. It is labeled ‘not Jev’, and its numbers say nothing about Jev | Google; optional: TypeSafe, OpenAI, Anthropic. Free without keys. Keyed run: under 5 cents on Jev; the stretch adds under 25 cents on Claude (estimates) |
| 13 · Retrieval-Augmented Generation | Briefing drafted · lab written | Google Colab, free T4 GPU · estimate 8 min | On its Colab runtime: not run Real path, other runs: 2026-10-09, Apple M1 Pro laptop, 16 GB (CPU/MPS), 2.0 min CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 21 s, with test doubles |
Open model. bge-small, a cross-encoder, an NLI model and Qwen replace the commercial models and Jev; their numbers describe those models | Google; optional: OpenAI, Anthropic, TypeSafe. Free without keys. Keyed run: under 50 cents on Claude, under 5 cents on OpenAI, under 5 cents on Jev (estimates) |
| 14 · Agents | Briefing drafted · lab written | Google Colab, free T4 GPU · estimate 6 min | On its Colab runtime: not run Real path, other runs: 2026-10-09, Apple M1 Pro laptop, 16 GB (CPU/MPS), 3.9 min CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 10 s, with test doubles |
Open model. Qwen2.5-0.5B-Instruct runs the agent and the guard, and Module 12’s toy decision model routes; their numbers describe those models, not Jev or the commercial models | Google; optional: OpenAI, Anthropic, TypeSafe. Free without keys. Keyed run: under 30 cents on Claude, under 5 cents on OpenAI, under 5 cents on Jev (estimates) |
| 15 · Capstone | Briefing drafted · lab written | Google Colab, free T4 GPU · estimate 15 min | On its Colab runtime: not run Real path, other runs: 2026-10-09, Apple M1 Pro laptop, 16 GB (CPU/MPS), 17 min CI: 2026-10-06, GitHub Actions runner (ubuntu-latest, CPU), 11 s, with test doubles |
Open model. Qwen, the open retrievers and the NLI judge replace the commercial models and Jev; their numbers describe those models | Google; optional: OpenAI, Anthropic, TypeSafe. Free without keys. Keyed, per pair for the capstone: under 2 USD on Claude, under 25 cents on OpenAI, under 10 cents on Jev (estimates) |
Blocking Work
- Open: Lab 13’s 80 questions, written and blind-checked by people (modules 13, 15;
data/rag_questions_v1.jsonldoes not exist) - Open: The capstone’s 45 new questions, written and blind-checked by people (module 15;
data/capstone_questions_v1.jsonldoes not exist) - Open: The briefing snapshot that Labs 13 to 15 quote is frozen (status final) (modules 13, 14, 15;
datasets.lectures.statusisprovisional) - Open: The decision set’s 80 hand-written items, written and labeled by two people (modules 11, 12, 14;
hand_itemsis 0 of 80) - Open: The audit of 60 template items in the decision set, by two people (modules 11, 12, 14; not done)
- Done: Lab 9’s preference data files, built on a machine with Hub access (modules 9, 10;
data/lab09_prompts.json,data/lab09_preferences.jsonl.gzexists) - Done: The reward model checkpoint that Lab 10 starts from (module 10;
data/lab09_reward_model.ptexists) - Done: Lab 6’s test-set logits, which Lab 11 calibrates (module 11;
data/lab06_logits.npzexists) - Open: TypeSafe’s own statements about Jev and RLCD, read from its documentation and signed off for Module 12 (module 12;
modules/12-rlcd-jev.qmdstill has its marker) - Open: Module 0 run end to end on a fresh laptop on macOS, Windows (WSL 2) and Linux (module 0; not done)
- Open: Each briefing given aloud once against its clock (modules 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15; not done)
Known Gaps by Module
- 0 · Coding Agents in the Terminal: Never run end to end on a fresh laptop; its section minutes and its estimate (which adds one-time setup and the Pages build) are plans, not measurements.
- 6 · Pretraining and the Hugging Face Stack: The Lab 6 logits file that Lab 11 reads has not been built.
- 9 · Reinforcement and Preference Learning: Part B and its Exercise 4 thresholds were measured on the build Mac’s CPU only (2026-10-06), not on Colab or a GPU.
- 10 · RLHF: The trained-policy checkpoints, including the reward-hacking signature, need a GPU; the seed protocol (seeds 0 to 4 on a T4) has not run, so their thresholds are provisional. On the build Mac the CPU FAST path ran, plus one exploratory full-settings run on its GPU (MPS) from a scratch copy (2026-10-06).
- 11 · Calibration: Until the Lab 6 logits are built, Part A analyzes the Lab 1 classifier instead of the fine-tuned encoder.
- 12 · Calibrated Decisions: RLCD and Jev: TypeSafe’s documentation was read on 2026-10-06 and quoted in Module 12; the quotations await sign-off, and no live Jev call has been made.
- 13 · Retrieval-Augmented Generation: The 80 human-written questions do not exist; runs use plumbing probes, which test the code, not retrieval.
- 15 · Capstone: The 125-question evaluation set does not exist; runs use plumbing probes, which test the code, not the system.
How to Read This Page
Evidence. Each line is the newest run of that kind, from the run records in runs/. A newer failure replaces an older pass, and on the same day a failure wins. On its Colab runtime means a worked run (solutions bound) of the whole notebook, on the path a participant without keys takes, with no QUICK shortcuts, on the runtime the module is designed for, recorded by a tool with the notebook’s code hash; it stops counting when the code changes. Real path, other runs is the newest other run without test doubles, on any machine except the CI runner; a laptop’s or a CPU runner’s time does not predict a T4’s. CI is the newest run on the GitHub CPU runner: with test doubles it shows that the code runs, not what a model does. A run marked before the notebook last changed was made against older code; backfilled runs carry no code hash, so they cannot be checked. An estimate is a planning figure from the lab’s brief, not a measurement. Dates are shown as recorded; the release check will also require a teaching run to be at most 21 days old.
Paths.
offline: Offline stand-ins and test doubles: checks that the code runs, not what a model does.open: Real open models from the Hugging Face Hub, no API keys.keyed: Commercial APIs with keys.
Without API keys.
- One path: One path: the lab always runs its real models.
- Open model: A real open model runs in place of the commercial one; its numbers are real, but they describe the smaller model.
- Toy model: A toy model trained in the lab shows the mechanism; its numbers illustrate, they do not measure the real system.