15 Pull Requests and Builds You Can Trust
Quarto, reproducible publishing, coding agents, Markdown, Python, EPUB, data analysis, k-means clustering
Chapter 3 reviewed an agent’s change as a pull request and rebuilt it from a fresh clone by hand. That last check is the one a reviewer most often skips, because it takes time and a clean machine, and it is the one an agent’s fluent summary cannot replace. Continuous integration, CI, makes it automatic: on every pull request, a machine that has never seen the project clones it, installs the tools, computes the results, runs the tests and renders every page, and the pull request shows a green tick or a red cross. A guard that the results no longer support turns it red, naming the sentence.
This chapter adds that to the Acme project, with the rules a team needs for agents’ changes: one change per pull request, nothing generated committed, and nothing called done until it has been rebuilt. It then publishes the site from main and renders the monthly reports on a schedule, both by the same build.
- Write the rules a team’s agents follow, and a pull request template a reviewer can use.
- Ask for a CI workflow that runs the project’s own rebuild commands, with versions pinned from their sources.
- Read a red build: which step failed, and which sentence or test it names.
- Publish a site and render recurring reports from CI, and say what each costs on GitHub’s plans.
15.1 What CI is made of
| Ask for | GitHub Actions calls it | Check |
|---|---|---|
| a build, written down | a workflow, a YAML file in .github/workflows/ |
it appears under the repository’s Actions tab |
| when it runs | on: a trigger: pull_request, push, schedule, workflow_dispatch (on demand) |
a run starts when the event happens |
| a clean machine | a runner, runs-on: ubuntu-24.04 |
the log shows it installing everything |
| one thing it does | a step: an action (uses:) or a command (run:) |
each step is green or red in the log |
| a result on the pull request | a status check | a tick or a cross beside the commit |
| a file the run made, kept | an artifact | a download at the foot of the run’s page |
“nothing reaches main unbuilt” |
a protected branch with a required check | the merge button waits for the tick |
A workflow should run the commands you run, not new ones. The project’s AGENTS.md already lists its rebuild: compute the results, run the tests, render the site, check the pages. CI runs the same four, so a green tick means what “it builds on my machine” was meant to mean.
15.2 Rules for a team
A team’s agents need rules a lone author can keep in their head. Each agent session starts fresh, reads AGENTS.md, and does what it says, so the rules go there; each reviewer opens a pull request, so the checklist goes in its template.
Add a section "Team rules" to AGENTS.md for working in a team: work on a branch, never push to `main` or merge; one coherent change per pull request; never commit outputs/, reports/, _site/ or rendered files; before saying a change is done, run the four rebuild commands (evaluate, tests, render, check_site) and say which passed; never weaken a test or a `claim` guard to make a step pass; report what you could not verify. Then create .github/pull_request_template.md: the three checks (scope, correctness, rebuild) as a checklist, saying what to look at for each in this project, and a line asking which agent made the change and what it did not verify. Do this on a new branch `team-rules` and commit it.git switch -c team-rules · edits AGENTS.md · writes .github/pull_request_template.md · git commit
Read the diff as the reviewer of a rule book. Each rule should name this project’s files and commands, not general good practice: “never commit outputs/” is a rule an agent can check, “keep the repository clean” is not. The rule against weakening a guard matters most. An agent told to make a failing step pass has two routes, fixing the sentence or loosening its guard, and only one is right.
15.3 A build on every pull request
Create .github/workflows/check.yml: on every pull request and every push to `main`, on `ubuntu-24.04` with read-only permissions, install Quarto 1.10.20 and uv 0.11.32 with its cache on, then run the four rebuild commands from AGENTS.md in order. Pin each action to its latest release, the major tag where it has one, looking the versions up rather than remembering them, and tell me what you chose and where you checked. Commit it on a new branch `ci` from `team-rules`.The finished workflows pin actions/checkout@v7, actions/deploy-pages@v5, actions/upload-artifact@v7, actions/upload-pages-artifact@v5, astral-sh/setup-uv@v10.3.3 and quarto-dev/quarto-actions/setup@v2, the latest releases on 2026-11-18, and Quarto 1.10.20. Ask for the versions to be looked up: an agent’s memory of an action’s version is as old as its training, and an old major version can stop working when GitHub retires what it runs on. Name the versions of the tools as well: in the test run, a first version of this prompt that said only “uv” got whatever uv was newest that day, two minor versions ahead of the one the project was built with. It worked, but a build that installs whatever is newest is not the same build next month.
git push -u origin team-rules
gh pr create --base main --fill
gh pr merge team-rules --merge
git push -u origin ci
gh pr create --base main --fillReview the first pull request against its own template’s three checks before you merge it. Merge it with a merge commit, --merge, as above: ci starts from team-rules, so once team-rules is on main unchanged, the second pull request shows only the workflow, one change as the rules ask. (Squashing the first would rewrite its commits, and the second would carry them again.) The second pull request runs the check; its Checks tab shows each step as it runs. The first run downloads Quarto and every package, which takes a few minutes; later runs restore uv’s downloads from a cache. Merge it when the check is green.
What it costs: GitHub Actions’ standard runners are free in public repositories, and a free account’s private repositories get 12 US dollars of compute a month (checked 2026-11-18; Appendix K). A private repository’s check uses that allowance; a public one does not.
15.4 A red build
A check that has never failed has not been tested. Break a guard on purpose, on a branch you will throw away:
This breaks a guard on purpose, to see CI fail. On a new branch `try-red` from `ci`, change the condition of one `claim` guard in index.qmd so that it no longer holds, without changing its sentence, commit it, and tell me which sentence CI should name.git push -u origin try-red
gh pr create --base main --fill --draftThe check goes red at its render step, and the log names the sentence: in the test run, “ClaimError: The results no longer support this sentence”, followed by the home page’s sentence about the best release. That is what a teammate sees when a change to the data or the model makes a typed claim untrue: not a report that quietly says the wrong thing, but a pull request that cannot be merged with it. Close the pull request and delete the branch (gh pr close --delete-branch).
To make “cannot be merged” literal, protect main: under the repository’s Settings, add a branch protection rule, or a ruleset, that requires a pull request and the Check job before merging. One limit: on a free GitHub account, protected branches and rulesets work in public repositories only; private repositories need GitHub Pro, Team or Enterprise (checked 2026-11-18). On a free private repository the check still runs and still shows red; it is the team’s rule, not GitHub’s, that nothing red is merged.
15.5 Publish from main
Create .github/workflows/publish.yml: on every push to `main`, and on demand, run the same build, then deploy _site/ to GitHub Pages with GitHub's own Pages actions (no `gh-pages` branch), with the permissions and environment they need, given only to the job that deploys. Tell me the one repository setting I must change before it first runs.The setting is the Pages source: Settings, Pages, Build and deployment, Source: GitHub Actions. Set the project’s site-url to the address it gives, so that the social cards of Chapter 12 work. A page whose guard fails is never published, because publishing runs the same build first. The permission to deploy belongs to the deploy job alone: in the test run, a version of the prompt that did not say so gave it to the whole workflow, build included.
Where it may go comes first (Section 12.6): a free GitHub account publishes Pages from public repositories only; private repositories need a paid plan, and a site only your organisation can see needs GitHub Enterprise Cloud (checked 2026-11-18). The Acme evaluation is of public data and could be public; a client’s evaluation could not, and publishing it needs a host your organisation approves, or no site at all, only the PDFs.
15.6 Reports on a schedule
Create .github/workflows/monthly.yml: on the 2nd of every month at 06:17 UTC, and on demand, run the same build, then `uv run python scripts/render_params.py --formats pdf`, keep reports/ as a downloadable artifact for 90 days, failing if it is empty, and run scripts/check_refs.py last. Explain the schedule line in a comment, including its time zone and why the minute is not 0.The finished schedule is the cron line “17 6 2 * *”: minute 17, hour 6, day 2 of every month. GitHub reads it in UTC unless a timezone: line beside it names one, such as timezone: "Europe/London"; it runs on the latest commit of main, and may start late, or in the worst case not at all, when Actions is busy, which it is at the top of each hour; hence minute 17, not 0 (all checked 2026-11-18). The PDFs appear as an artifact at the foot of the run’s page, kept 90 days. The reference check runs last because it needs the network and can fail for reasons outside the project; placed last, it fails the run without losing the reports. Run workflow, on the workflow’s page under Actions, runs it now, which is how to test it without waiting a month.
Check any time arithmetic an agent writes. In the test run, a first comment said 06:00 UTC was 07:00 in London in winter; it is 06:00, and 07:00 in summer. The agent caught it, and a reader skimming a comment might not. And GitHub turns scheduled workflows off in a public repository after sixty days with no activity (checked 2026-11-18). A monthly report that matters needs someone who notices when it does not arrive.
What was not tested
The check workflow was run locally with act, in Docker, green and then red as above. Pushing, pull requests, the Publish and Monthly reports runs, the uv cache and branch protection need a GitHub repository and were not run for this edition; the actions and settings they use were checked against GitHub’s documentation on 2026-11-18.
Stuck for two minutes? Paste this into your agent. It backs up your work and brings the project to this point; then carry on from here.
Catch me up: follow ../quarto-workshop/checkpoints/20-eval-ci/README.mdSummary
- Rules for agents go in
AGENTS.md, which every session reads; the reviewer’s checklist goes in the pull request template, which every reviewer sees. - CI runs the project’s own rebuild commands on a clean machine for every pull request, so a green tick means it builds anywhere.
- Break a guard on purpose once: a check that has never failed has not been tested.
- Publishing and scheduled reports run the same build first, so nothing unchecked goes out.
- Pin actions to versions looked up at their source, and know what each step costs on your plan.
Further reading
- GitHub, workflow syntax, events that trigger workflows and custom workflows for Pages (checked 2026-11-18).
- GitHub, about protected branches (checked 2026-11-18).
- Quarto, GitHub Actions (checked 2026-11-18).
- nektos, act, to run a workflow on your own computer (checked 2026-11-18).