Choose One Coding Agent

The foundations deck shows three installation routes:

• Install Claude Code: native installer, version check, sign-in.
• Install Codex: npm installation, version check, sign-in.
• Install Antigravity CLI: Google’s terminal agent for individual accounts.

Start claude, codex, or agy inside the lab folder. Use the same plain-English task prompts with the agent you choose.

Check: confirm the session’s working folder and record the tool version.

A Workspace for One Question

Question: how much additional sales does each email generate compared with sending no email?

quarto-promotions/
  context/
    hillstrom.csv       # original 64,000 customer rows
    notes.md            # design, dictionary, provenance
    sources.bib         # experiment and Quarto references
  brief.md              # current decision and deliverables
  AGENTS.md             # recurring project conventions

The unit of analysis is one assigned customer. A directory organizes context; tool permissions govern access.

Copying the Lab Inputs

Run from a clone of this workshop. Choose a fresh destination:

mkdir -p ~/code/quarto-promotions/context
cp examples/marketing-context/* ~/code/quarto-promotions/context/
cd ~/code/quarto-promotions
# Start one coding agent:
claude
# Or: codex

Check: hillstrom.csv, notes.md, and sources.bib are present. The agent session is using quarto-promotions.

Turn 1: Understand the Evidence

Ask your coding agent in the new workspace:

Read every file in context/. Summarize assignment, row meaning, outcomes, units, and observation window. Identify pre-campaign attributes and missing business information. Check the source citation before proposing an analysis.

Check against the files:

• segment is randomized campaign assignment; groups are independent.
• spend is dollars per customer over two weeks, including zeros.
• conversion marks any purchase; visit marks a website visit.
• Costs, margin, delivery/open status, and later sales are unavailable.

Define Success Before Looking for a Winner

Primary outcome: mean two-week sales per assigned customer, in dollars.

Secondary outcomes: purchase conversion and website visit rates.

For campaign \(a\) and no-email control \(0\):

\[\widehat{\Delta}_a = \overline{Y}_a - \overline{Y}_0\]

\(Y_i\) is customer \(i\)’s two-week spending, including zero purchases. Positive \(\widehat{\Delta}_a\) estimates additional sales per assigned customer.

Check: the denominator includes every customer assigned to that arm.

Turn 2: Have the Agent Write the Brief

Continue in the same conversation:

Write brief.md using the three context files. Require report.pdf, report.docx, and report.ipynb from report.qmd, plus exactly 8 slides including the title. Compare each email with no email on sales/customer and conversion in percentage points. Include uncertainty, provenance, business limitations, and completion checks. Label planning assumptions.

Check: review the question and outcomes before implementation.

Exercise

Add a criterion that catches a misleading claim, such as treating all campaign revenue as incremental sales.

Durable Instructions

Ask your coding agent:

Initialize Git if needed and work on a draft branch. Create AGENTS.md with rules to preserve raw inputs, compute claims, verify citations, keep execution enabled, and inspect rendered outputs. Connect these rules to the instruction file your tool reads. Record the build command.

brief.md describes this decision. AGENTS.md holds recurring project rules.

Check: the active tool reads the project’s instructions.

Turn 3: Create the Report

Ask your coding agent:

Implement brief.md as report.qmd. Read context/hillstrom.csv in Python and validate rows, assignments, and outcomes. Compute arm sizes, sales means, conversion, and contrasts with control. State uncertainty assumptions. Add labeled figures, captions, alt text, and citations. Export PDF, DOCX, and IPYNB. Prepare dependencies as needed and record working build commands.

Check: the source reads the CSV and computes each result. The next slides show how to export and inspect it.

One Source, Three Report Formats

In report.qmd, the coding agent selects PDF and Word:

format:
  pdf: default
  docx: default
bibliography: context/sources.bib
jupyter: python3

The notebook conversion preserves Markdown and executable Python cells. Ask the agent to include readable reference links in the notebook.

Render and Compare the Formats

quarto render report.qmd --to pdf
quarto render report.qmd --to docx
quarto convert report.qmd --output report.ipynb
jupyter nbconvert --to notebook --execute --inplace report.ipynb

PDF: inspect page breaks, captions, and references. DOCX: open in Word or LibreOffice and inspect tables and figures. IPYNB: inspect Markdown, Python cells, and saved results. Restart the kernel and run all cells from the lab root to check reproducibility.

What to Inspect in the Source

Ask your coding agent:

Show where report.qmd reads the raw CSV, computes each numerical claim, and sets execution options. Explain these parts briefly.

Expected configuration pattern:

execute:
  echo: true
  error: false
  cache: false
  freeze: false

Check: errors stop the build; results recompute from local inputs.

A Calculation to Inspect

One possible cell in the student’s report:

```{python}
import pandas as pd

df = pd.read_csv("context/hillstrom.csv")
summary = df.groupby("segment").agg(
    n=("spend", "size"), sales=("spend", "mean"),
    conversion=("conversion", "mean"))
summary["incremental_sales"] = (
    summary["sales"] - summary.loc["No E-Mail", "sales"])
print(summary)
```

Check: use every assigned customer, including nonbuyers. Conversion differences become percentage points when multiplied by 100.

An Independent Numerical Check

Two-week outcomes computed from the original customer data:

Check: calculate each campaign’s mean minus the no-email mean. Rounded display values can differ slightly from full-precision calculations.

Additional Sales Versus No Email

Men’s email minus women’s email: +$0.35 per assigned customer (95% interval $0.03 to $0.66).

pp = percentage points. The head-to-head interval barely excludes zero: the ranking is less certain than the point estimates suggest.

Check: the entire treatment-arm sales mean includes purchases that might have happened without an email.

See the Sales Effect

Check: both intervals exclude zero, and they overlap. Overlap does not settle the ranking; the direct contrast on the previous slide does.

Turn 4: Correct a Marketing Overclaim

Suppose the draft says: “Every dollar of email-arm revenue was caused by the promotion.”

Ask your coding agent:

Check that claim against the no-email control. Replace it with the computed difference in sales per assigned customer. Audit denominators and distinguish percentage points from percent change. Re-render and show the values that support the corrected interpretation.

Check: both observed sales and additional sales are clearly labeled.

A Tempting but Misleading Winner

Ask your coding agent:

Compare average spending among buyers with sales per assigned customer. Explain why the buyer-only ranking cannot identify the campaign with the largest causal sales effect. Keep the primary analysis on all assigned customers.

Check: purchasing is an outcome of the promotion. Selecting only buyers breaks the original randomized comparison.

Exercise

Explain why more visits, more purchases, and more revenue are different outcomes.

Turn 5: Improve the Figure

If the figure shows only total campaign revenue, ask your coding agent:

Plot each campaign’s additional sales per assigned customer versus no email. Show an uncertainty interval and a zero reference line. Label the two-week window and dollars/customer. Put conversion differences in a separate panel with percentage-point units. Update the caption and alt text, then render again.

Check: arm size does not determine the ranking; units and reference are explicit.

Exercise

Request one visual revision and confirm the underlying estimates do not change.

What Does Uncertainty Mean Here?

Ask your coding agent:

Explain the assumptions behind the intervals. For independent arms, use customer-level sampling and preserve assignment groups. Compare men’s and women’s emails directly before declaring a winner. State how multiple comparisons and skewed spending affect interpretation.

Check: most customers spend zero; a few spend much more. Intervals describe estimation uncertainty under stated assumptions.

From a Sales Effect to a Business Decision

Ask your coding agent:

Draft a campaign recommendation supported by the experiment. State which assumptions would be needed to project sales to a future audience. List the margin, campaign cost, and later outcomes needed to assess profit. Propose a follow-up randomized test.

Check: a recommendation distinguishes measured sales from projected sales and profit. Any new audience size or cost is a labeled planning assumption.

Exercise

Choose one unresolved business question and define an outcome for the next test.

Turn 6: Adapt the Report into Slides

Ask your coding agent:

Create slides.qmd with exactly 8 slides including the title. Reuse the report’s verified computation. Cover the decision, experiment, outcome definitions, sales results, uncertainty, recommendation, and follow-up test. Use a dark Reveal.js theme and speaker notes. Render and inspect every slide.

quarto render slides.qmd --to revealjs

Check: count eight slides and compare every number with the report.

Turn 7: Fix an Observed Layout Defect

If a figure and explanation are crowded, ask your coding agent:

On the Sales Results slide, the caption falls below the visible area. Move the detailed interpretation to speaker notes and shorten the caption. Keep the figure readable and the total at 8 slides. Re-render at desktop and laptop presentation sizes.

Check: the entire caption fits; notes retain the necessary interpretation.

If the agent lacks a browser tool, inspect the output yourself and describe the defect. Ask it to distinguish completed checks from checks still needed.

Turn 8: Review the Whole Deliverable

Ask your coding agent:

Rebuild all three report formats and the slides. Run the notebook from a fresh kernel in the lab root. Recompute claims from the CSV. Audit citations, figures, denominators, and business claims. Review the diff and report commands, limitations, and files ready for review. Keep this work local.

Use the report export commands above, then rebuild the slides:

quarto render slides.qmd --to revealjs
git diff --check

Check: source, raw inputs, build instructions, and rendered outputs are included.

Optional: A Campaign Evidence Website

From a fresh workspace with the report and context, ask your coding agent:

Create a Quarto website in campaign-site with Decision, Experiment, and Results pages. Adapt the verified report and copy its inputs. Preserve the outcome definitions and limitations. Render the site and check all local links and its desktop layout.

quarto create project website campaign-site --no-open --no-prompt

Check: navigation works; results recompute from the copied inputs.

Optional: A Marketing Analytics Blog

In another fresh workspace, ask your coding agent:

Create a Quarto blog in marketing-blog. Adapt the report into posts/promotion-sales/index.qmd with title, date, categories, executable analysis, experiment citation, and limitations. Copy the inputs and fix relative paths. Render and check the listing’s post link.

quarto create project blog marketing-blog --no-open --no-prompt

Check: data paths work from the post’s new location.

Different Outputs Need Different Reviews

Keep the computation consistent while adapting the explanation.

Requests for a Larger Team

For a larger project, you can ask for bounded roles:

Have an evidence reviewer inspect sources read-only, an author edit only report.qmd, and a render reviewer report visual defects. One lead should integrate changes and run the final build.

Use your tool’s delegation support, or perform these stages sequentially.

Partner Review

Exercise — 15 minutes

Exchange all three report formats and slides. Each partner checks:

1. One additional-sales estimate against the CSV.
2. One causal or business assumption against the notes.
3. One figure or slide in the browser.

Use the feedback for a plain-English revision request.

Done: show the defect, the request, and evidence of correction; or a check that passed.

Lab Submission

Submit report.pdf, report.docx, report.ipynb, and the eight-slide slides.html, plus both .qmd sources, unchanged input copies, and dependency/build instructions.

If no defect is found, report a check that passed. Do not invent an agent mistake.

Evaluation Rubric

Which business conclusions could a successful build never validate?

References and Questions

• Hillstrom’s original promotion experiment
• Claude Code setup and project memory
• Codex CLI and repository instructions
• Quarto Reveal.js and execution
• Report formats: PDF, Word, notebook conversion

Case and installation sources checked 2026-10-06. Report format documentation checked 2026-10-07.

Back to the three-hour Quarto workshop