From Traditional NLP to Modern LLMs

Welcome: The Setup Check and the Week Ahead

Genial Labs · The Day 1 opening

One Line of Ideas

Modern language models did not appear from nowhere.

  • Each module fixes a specific failure of the one before it: from counting words to attention, to training on human preferences, to agents.
  • You build each step yourself, in a notebook, and see it work.
  • The same small datasets come back all week, so every improvement is measured against the module before it, not asserted.

The Five Days

Day Theme Modules
Day 1 Foundations: From Counts to Attention 1 Text as Data, 2 Word Vectors and Neural Networks, 3 Sequence Models, 4 Seq2seq and Attention
Day 2 Transformers and Pretrained Models 5 The Transformer, 6 Pretraining and the Hugging Face Stack, 7 Fine-Tuning and LoRA
Day 3 Using and Aligning LLMs 8 LLMs Through APIs, 9 Reinforcement and Preference Learning, 10 RLHF
Day 4 Calibration, Decisions and Retrieval 11 Calibration, 12 Calibrated Decisions: RLCD and Jev, 13 Retrieval-Augmented Generation
Day 5 Agents and the Capstone 14 Agents, 15 Capstone

Module 0 is optional pre-work on your own laptop. Nothing later depends on it.

Each Day Answers One Question

  1. Day 1. How do we turn text into something a model can learn from?
  2. Day 2. How does one architecture, pretrained at scale, become a general tool?
  3. Day 3. How do we use these models, and what are they trained to want?
  4. Day 4. When should a model’s answer be trusted, and how do we ground it in sources?
  5. Day 5. How do we build reliable systems out of these models?

How a Module Works

On Day 1, each module is 95 minutes: 45 minutes of briefing, then 50 of lab. On Days 2 to 5, each module is 120 minutes: 55 minutes of briefing, 55 of lab, then a 10-minute debrief. On Days 2 to 4 the second module breaks for lunch between its briefing and its lab. On Day 5, the capstone build runs on across lunch. The capstone (Module 15) has no briefing: it takes 11:30–12:25, 13:25–14:30 and 14:45–16:45 on Day 5.

Every lab follows one rhythm:

  1. Predict what a cell will output before you run it. Write the prediction down.
  2. Run it.
  3. Explain the result, especially when it differs from your prediction.
  4. Check your work against a checkpoint that passes or fails.

The Timetable

Day 1 · 95-minute modules

Time
08:00–09:00 Drop-In Clinic: Module 0 Pre-Work (optional)
09:00–09:10 Welcome, Setup Check
09:10–10:45 1 · Text as Data: briefing 45 · lab 50
10:45–11:00 Break
11:00–12:35 2 · Word Vectors and Neural Networks: briefing 45 · lab 50
12:35–13:35 Lunch
13:35–15:10 3 · Sequence Models: briefing 45 · lab 50
15:10–15:25 Break
15:25–17:00 4 · Seq2seq and Attention: briefing 45 · lab 50

Days 2–5 · 120-minute modules

Time
09:00–09:15 Warm-Up
09:15–11:15 Briefing 55 · Lab 55 · Debrief 10
11:15–11:30 Break
11:30–12:25 Briefing 55 (Day 5: hands-on)
12:25–13:25 Lunch
13:25–14:30 Lab 55 · Debrief 10 (Day 5: hands-on)
14:30–14:45 Break
14:45–16:45 Briefing 55 · Lab 55 · Debrief 10 (Day 5: hands-on)
16:45–17:00 Wrap-Up (Day 5: Wrap-Up and Close)

Breaks and lunch keep their length. From Day 2, each day opens with a warm-up and closes with a wrap-up of the day’s results. The full schedule is on the site.

Setup: Nothing to Install

  • Every lab runs in Google Colab, in the browser. You need a Google account.
  • When a lab asks for a GPU: Runtime → Change runtime type → T4 GPU. The free tier is enough.
  • API keys are optional. With none, the API labs run on a small open model and you still complete every exercise.
  • If you have keys, put them in Colab Secrets, never in a cell: OPENAI_API_KEY, ANTHROPIC_API_KEY, TYPESAFE_API_KEY.
  • Details are on the Setup page.

Setup Check: Do This Now

  1. Open 00-setup.ipynb in Colab.
  2. Choose Runtime → Run all.
  3. It prints four blocks: the runtime, the keys, the provider, and the checkpoint.
  4. The last line should read Checkpoint passed. You are ready for Day 1.

Tell us the provider line it printed: open, a keyed provider, or an error. If a cell stops with an error, raise a hand now, not in Lab 1.

The Same Data All Week

Data Used in
arXiv Topics v1 Modules 1, 2, 6 and 11
Tiny Shakespeare Modules 1, 3 and 5
Workshop Desk Decisions v1 Modules 11, 12 and 14
Workshop Lectures v1 Modules 13, 14 and 15

Lab 1 prints two numbers, an n-gram perplexity and a classifier accuracy. Every later module is measured against them.

Working in the Labs

  • Each exercise is a TODO stub with its solution folded directly beneath it. Try first; open the solution when you are stuck, run it, and go on.
  • A checkpoint after every exercise says whether it checked your code or the reference.
  • The core path fits the lab slot. The stretch section at the end is optional: fast groups may start it, the room as a whole never does.
  • In the debrief, your numbers go on the board. Keep them.

Where Everything Is

Next: Module 1 · Text as Data