Before Day 1

Check the Prerequisites, Set Up Colab, Then Start With Module 1

Do these five steps in order, at home, in the week before Day 1. For a short overview of the setup and the week’s schedule, see the intro slides, which open Day 1.

1. Take the entry check, about 15 minutes.
2. Read your score: for each area where you missed two or more questions, do the reading listed there.
3. Set up Colab, about 10 minutes.
4. Module 0, optional pre-work: 60 minutes of steps, about 90 with one-time setup.
5. Start with Module 1 on Day 1.

1. Take the Entry Check

The entry check tells you whether the prerequisites listed on the home page are in place. It is not graded, and nobody else sees your answers.

Fifteen questions in five areas, three per area, matching the prerequisites on the home page. Allow about 15 minutes. Work on paper or in a notebook, without searching the web, and write each answer down before you open its folded key.

Python

E1. What does this expression evaluate to?

[x * x for x in range(5) if x % 2 == 0]

Answer

[0, 4, 16]. The comprehension keeps 0, 2 and 4 and squares them.

E2. What does this print?

class Counter:
    def __init__(self):
        self.n = 0

    def add(self, k=1):
        self.n += k
        return self

c = Counter().add().add(3)
print(c.n)

Answer

4. Each call to add changes the same object and returns it, so the calls chain: 0 + 1 + 3.

E3. A loop over rows read from a CSV file stops with this traceback. What is wrong, and what is the fix?

  File "count.py", line 7, in <module>
    total = total + row["count"]
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Answer

row["count"] is a string (the CSV reader returns text), and Python will not add a string to an integer. Convert it: total = total + int(row["count"]). The last line of a traceback names the error; the line above it shows where it happened.

NumPy

E4. A has shape (3, 4) and b has shape (4,). What are the shapes of A + b and A @ b?

Answer

A + b is (3, 4): b is broadcast across the three rows. A @ b is (3,): a matrix-vector product.

E5. With x = np.arange(12).reshape(3, 4), what are x[1, :] and x[:, -1]?

Answer

x[1, :] is [4, 5, 6, 7] (the second row). x[:, -1] is [3, 7, 11] (the last column).

E6. X has shape (N, d). Write one line that divides each row by its Euclidean length, and say why keepdims=True matters.

Answer

X / np.linalg.norm(X, axis=1, keepdims=True). With keepdims=True the norms have shape (N, 1), which broadcasts across the d columns. Without it they have shape (N,), which NumPy tries to align with the columns and fails (or, if N == d, silently divides the wrong way).

Machine-Learning Basics

E7. Why are hyperparameters chosen on a validation split rather than on the test split?

Answer

Choosing on the test split makes the test score an optimistic estimate: the setting was picked because it did well on those examples. The test split is used once, after every choice is fixed. The workshop follows this rule in every lab.

E8. Logistic regression predicts p = \sigma(w^\top x + b) for a label y \in \{0, 1\}. Write the loss for one example, and its derivative with respect to the logit z = w^\top x + b.

Answer

The cross-entropy (log loss) \mathcal{L} = -\big[y \log p + (1 - y)\log(1 - p)\big]. Its derivative with respect to z is p - y: prediction minus target. The same form returns in Modules 1, 2 and 6.

E9. Gradient descent updates \theta \leftarrow \theta - \eta \nabla_\theta \mathcal{L}. What do you expect to see if the learning rate \eta is far too large?

Answer

The loss oscillates or grows instead of falling, because each step overshoots the minimum. It can also become nan.

PyTorch

E10. Put these five lines of one training step in order.

optimizer.step()
loss = F.cross_entropy(logits, y)
optimizer.zero_grad()
loss.backward()
logits = model(x)

Answer

optimizer.zero_grad(), logits = model(x), loss = F.cross_entropy(logits, y), loss.backward(), optimizer.step(). Putting zero_grad() right after step() instead is also correct; what matters is that gradients are cleared before the next backward(), because PyTorch adds to them.

E11. What does model.eval() change, and what does with torch.no_grad(): change?

Answer

model.eval() switches layers whose behavior differs between training and evaluation, such as dropout (turned off) and batch normalization (uses stored statistics). It does not stop gradients. torch.no_grad() stops autograd from recording operations, which saves memory and time when no gradient is needed. Several labs rely on the difference: Lab 5’s leak test runs in eval() mode.

E12. layer = nn.Linear(4, 3) is applied to a tensor of shape (8, 4). What is the output shape, and how many parameters does the layer have?

Answer

Output (8, 3). Parameters: a 3 \times 4 weight matrix and 3 biases, 12 + 3 = 15.

Mathematics

E13. Let y = \sigma(z) with z = wx. What is \dfrac{dy}{dw}?

Answer

By the chain rule, \dfrac{dy}{dw} = \sigma(z)\big(1 - \sigma(z)\big)\, x, using \sigma'(z) = \sigma(z)(1 - \sigma(z)).

E14. P(A) = 0.3 and P(B \mid A) = 0.5. What is P(A \text{ and } B)? And what is the expected value of one roll of a fair six-sided die?

Answer

P(A \text{ and } B) = P(B \mid A)\,P(A) = 0.15. The expected value is (1 + 2 + \dots + 6)/6 = 3.5.

E15. A model predicts the distribution (0.7, 0.2, 0.1) over three classes and the true class is the first. What is the cross-entropy, in nats? What is it for a uniform prediction over four classes? And what is the cosine similarity of the vectors (1, 0) and (1, 1)?

Answer

-\ln 0.7 \approx 0.357 nats. For a uniform prediction over four classes, \ln 4 \approx 1.386 nats: the starting loss of Lab 6’s four-class classifier. The cosine is 1 / \sqrt{2} \approx 0.707.

2. Read Your Score

Mark each answer right or wrong against its key, then count your misses in each area. The rule is the same for every area:

• Two or more of the three missed: do that area’s remediation below before Day 1.
• One missed: no action. Read the key of the question you missed and move on.

The workshop moves fast. Day 1 assumes all five areas, and there is no time in the room to learn them. Do the remediation before Day 1, not during it. If three or more areas need work, plan a full day for it, and tell the organizers before the workshop.

Every resource below is free and online. The section numbers are the sources’ own; the times are our estimates.

Area	Questions	What to read	Time
Python	E1–E3	The Python Tutorial, sections 3, An Informal Introduction to Python; 4, More Control Flow Tools; 5, Data Structures (5.1.3, List Comprehensions, is E1); and 9, Classes, sections 9.1 to 9.4 (9.3, A First Look at Classes, is E2). For E3, add 8, Errors and Exceptions, sections 8.1 to 8.3.	about 3 hours
NumPy	E4–E6	NumPy: the absolute basics for beginners: the sections Indexing and slicing (E5), Broadcasting (E4) and More useful array operations (the axis argument). Then the user guide’s Broadcasting page, General broadcasting rules: why E6 needs norms of shape (N, 1).	about 1 hour
Machine-learning basics	E7–E9	Dive into Deep Learning (Zhang, Lipton, Li and Smola), free at d2l.ai: 3.1, Linear Regression, sections 3.1.1.2 (Loss Function) and 3.1.1.4 (Minibatch Stochastic Gradient Descent); 3.6, Generalization, especially 3.6.3, Model Selection (E7); 4.1, Softmax Regression, section 4.1.2, Loss Function, where the gradient is prediction minus target (E8; logistic regression is softmax regression with two classes); and 12.3.1.1, Learning Rate (E9).	about 2.5 hours
PyTorch	E10–E12	Deep Learning with PyTorch: A 60 Minute Blitz, all four parts in order: Tensors, A Gentle Introduction to torch.autograd, Neural Networks (E10, E12) and Training a Classifier. Then Learn the Basics, Optimizing Model Parameters, Full Implementation: model.eval() against torch.no_grad() (E11). Last, skim the PyTorch Refresher in Module 2’s briefing: in the room it takes a few minutes, so it should read as review.	about 2 hours
Mathematics	E13–E15	Dive into Deep Learning: 2.4.4, Chain Rule (E13); 2.6, Probability and Statistics, sections 2.6.4, Multiple Random Variables and 2.6.6, Expectations (E14); 4.1.3, Information Theory Basics, for cross-entropy (E15); and 22.1.2, Dot Products and Angles, for cosine similarity (E15).	about 2 hours

After the reading, try the questions you missed again without opening the keys.

3. Set Up Colab

All labs run in Google Colab, so there is nothing to install. You need a Google account.

Open the setup notebook in Colab and choose Runtime → Run all. API keys are optional; the Setup page says how to add them and how to switch on a GPU.

What a working setup looks like. The notebook prints four short blocks, and every cell runs without an error:

• The runtime: the Python version, yes beside Colab, the PyTorch version, and a device line that names a GPU or says CPU only. A CPU is fine for this check; switch on a T4 GPU before the labs that need one.
• The keys: one line per key, found or not set. With no keys, all three say not set, and that is fine.
• The provider: with no keys, text generation is open and the decision model is toy, followed by No API keys found. The labs will use open models. That is fine.
• The checkpoint, the last line: Checkpoint passed. You are ready for Day 1.

If the last line is missing, or a cell stops with an error, check the Setup page, then send the error message to the organizers before Day 1.

4. Module 0: Optional Pre-Work

Module 0 · Coding Agents in the Terminal is optional, and it is not a prerequisite: nothing later in the workshop depends on it, and the entry check does not test it. It runs on your own laptop, not in Colab: you install a terminal coding agent, git and the GitHub CLI, so you need permission to install software and a GitHub account. If you get stuck, bring your laptop to the optional drop-in clinic on Day 1 from 08:00 to 09:00, before the welcome.

5. Then Start With Module 1

Day 1 opens at 09:00 with a welcome and a setup check, then Module 1 · Text as Data. The Day 1 page lists the day’s four modules, and the schedule has the times.