Day 1
Customers and churn
Who are our customers, and who is still with us?
Before Day 1: Module 0 · Pre-work: environment, warm-up and coding agents, optional pre-work. Run the environment check in Colab and the short SQL and data-frame warm-up before you arrive. The intro slides open the day.
The day’s modules
09:15–11:00
Module 1
Transaction logs to customer-base analysis
Turn raw transaction logs into a customer table with DuckDB window functions, draw cohort retention curves, and see why churn is never observed when customers can leave without telling you.
11:15–13:05
Module 2
Buy-till-you-die models: BG/NBD and Pareto/NBD
Model purchasing and silent dropout with BG/NBD and Pareto/NBD, check that the fit recovers known parameters, forecast a holdout period, and turn the probability a customer is alive into a retargeting rule.
14:00–15:50
Module 3
Covariates in CLVTools, and BTYD versus machine learning
Add the acquisition channel as a covariate to the Pareto/NBD in CLVTools and recover the true channel effects, then run a fair holdout comparison of BTYD against gradient boosting.
Warm-up: from the pre-work to Day 1 (15 minutes)
Answer alone on paper for 5 minutes, notes closed. Compare with a neighbor for 4 minutes. The facilitator takes the most-missed questions for 5 minutes, then says in one minute where today starts.
W1. You choose Run all in a lab and it stops at Exercise 2 with the message “TODO 2 is not written yet”. What are your two options, and how will the checkpoint tell them apart?
Answer
Write the function in the exercise cell and run it and its checkpoint again; or run workshop.use_reference(2) (in R, use_reference(2)) to load the reference solution and carry on. The checkpoint prints “passed on your code” in the first case and “passed on the REFERENCE solution” in the second, and the summary at the end of the notebook lists which checkpoints passed on your own code. Using the reference is not failing; it keeps you with the room.
W2. Write the SQL expression that gives, for every order, the number of days since the same customer’s previous order. What does it return for a customer’s first order?
Answer
order_date - LAG(order_date) OVER (PARTITION BY customer_id ORDER BY order_date). In DuckDB, subtracting two dates gives a number of days. The first order has no previous order, so the result is NULL.
W3. A customer buys at random times at an average rate of 0.5 purchases a month (a Poisson process). They have not bought for the last 4 months. What is the probability of that silence if they are still a customer? Does the silence prove they have left?
Answer
The count in 4 months is Poisson with mean 0.5 \times 4 = 2, so P(0) = e^{-2} \approx 0.135. About one active customer in seven with this rate would be silent this long, so silence is evidence of leaving, not proof. Module 2 turns this reasoning into P(alive), the probability that a customer is still active given their history.
W4. You build a customer table by left-joining all customers to orders and counting rows per customer. What goes wrong for customers who have never ordered, and how do you fix it?
Answer
They get a count of 1, from the single row the left join fills with missing values. Count non-missing order values instead: COUNT(order_id) rather than COUNT(*) in SQL, count() on an order column in pandas, sum(!is.na(order_id)) in dplyr.
Why today. Today’s question is Who are our customers, and who is still with us? Module 1 turns the transaction log into a table of customers with these same tools; Module 2 asks who is still a customer when nobody tells you they left; Module 3 asks whether how a customer was acquired changes that, and whether a machine-learning model does better.
Wrap-up: fixed and left open (20 minutes)
The facilitator keeps the results table on the board (or a shared sheet) all week; Days 2 to 5 add rows to it.
1. Fixed and left open (6 minutes: 3 in pairs, 3 with the room). For each module, say what it made possible that the step before could not, and what it still cannot do.
- Module 1 turned a transaction log into a customer table and cohorts, but describes only the past.
- Module 2 forecast each customer’s purchases and the probability that they are still active, with uncertainty, from frequency, recency and age alone.
- Module 3 added the acquisition channel and compared the model with machine learning on held-out data.
- Left open for Day 2: all of today’s forecasts count purchases, not money. What is a customer worth in dollars, and how sure are we?
2. The running results table (8 minutes). Fill in your own numbers from today’s labs, as each notebook’s Decision cell and last cell print them. Write the interval’s type and probability, and whether the run was FULL or QUICK and on your own code or the reference. On synthetic data, write the truth beside your estimate.
| Lab | Quantity | Your number | Interval (type, probability) | Truth (synthetic data) | Run (FULL or QUICK; yours or reference) |
|---|---|---|---|---|---|
| 1 | Customers in the base; share with no repeat purchase (CDNOW) | real data | |||
| 1 | Inactivity rule: your N, customers it drops, and how many of them are still alive | none (counts) | the alive count uses the truth | ||
| 2 | BG/NBD parameters r, α, a, b against the truth | ||||
| 2 | Holdout repeat purchases: forecast against actual (CDNOW) | actual: | |||
| 2 | Customers below the stop-retargeting threshold | ||||
| 3 | Effect of acquisition channel on the purchase rate | ||||
| 3 | Channels you would treat differently (rate ratio to search) | ||||
| 3 | Holdout error: BTYD against gradient boosting (same metric) | real data |
3. One decision claim (6 minutes: 3 writing, 3 with the room). Write one sentence you would defend to a manager, in the form number → rule → recommendation: “Based on [number and interval from Lab N], I would [action], because [rule, with its cost or margin], unless [the assumption that would change it].” Two or three people read theirs; the room asks one question of each: what would change your mind?
Run sheet
Times are on the schedule. Every lab opens from the notebooks page too.