The classic guides to reading a research paper were written before an AI assistant could summarize, explain or critique one. Their advice still holds: read in layers, with a goal, and verify by reconstruction. What has changed is that AI can now do the first layer for you, tutor you through the second, and argue with you after the third.
The field-specific guides (mathematics, statistics, CS, computational biology, neuroscience) change what you read closely, not the layers. The evidence on AI tools, on how researchers use them, and on where they fail points to one protocol: AI before and after your own reading, not instead of it.
A 20-minute slide version of this post, for a lab meeting or journal club, steps through the passes on a 3D model of a paper.
1 The classics: general guides
Every good guide agrees on one thing: a paper is not read front to back, once. You read it in layers, with a goal set before you start.
| Guide | Year | Core idea | Best for |
|---|---|---|---|
| Keshav, “How to Read a Paper” | 2007 | Three passes. Pass 1 (5–10 min): title, abstract, intro, headings, conclusions; answer the “five Cs” (category, context, correctness, contributions, clarity). Pass 2 (~1 h): figures and argument, skip proofs. Pass 3: virtually re-implement the work. Also a recipe for a literature survey. | CS, systems, any conference/arXiv paper |
| Carey, Steiner & Petri, “Ten simple rules for reading a scientific paper” (PLOS Comp Bio) | 2020 | Pick a reading goal first; read the figures before the text; question each panel; afterwards ask “now what?” and connect it to your own science. | Biology, comp bio, students and mentors |
| Mensh & Kording, “Ten simple rules for structuring papers” (PLOS Comp Bio) | 2017 | A writing guide, but its context–content–conclusion model tells a reader where each claim lives in a well-built paper. | Neuroscience, comp bio |
| Leek group guide to reading papers | 2016 | A working statistician’s own process, kept as an open GitHub guide; pairs with his guides to reviewing papers. | Statistics, genomics, data science |
Keshav’s method is the shared skeleton. The later guides mostly change what you read on each pass (figures in biology, theorems in mathematics, data and design in statistics), not the layered structure.
2 Field-specific guides
Each field shifts the unit of attention: theorems in mathematics, design and inference in statistics, claims-versus-evidence in CS, figures and data in biology, analysis pitfalls in neuroscience.
2.1 Mathematics
- Simonson & Gouvêa, “How to Read Mathematics”: mathematics has its own “reading protocol”; read slowly, keep the big picture, test every claim with an example, make the idea your own.
- Petra Schwer, “How to Read Mathematics?” (arXiv 2607.24136, 2026): a recent study guide. Read in hours per page, not pages per hour; for proofs, skim for the idea, go line by line, then summarize. It also warns against leaning on AI to do the reasoning for you.
- Tanya Khovanova, “How to Read a Math Paper” (2026): match the reading to the goal. To learn a paper deeply, iterate up to six reads: structure, definitions, statements with small examples, proof ideas, the references, then reproduce the proofs yourself.
- MathComm’s reading-mathematics list: links Lara Alcock’s research on how experts and novices read proofs, Reiter’s question lists, and Matt Baker’s “How to Read a Research Paper”.
2.2 Statistics and biostatistics
- Greenhalgh, How to Read a Paper (BMJ series 1997; book now in its 6th edition): the standard critical-appraisal text. Its two “Statistics for the non-statistician” chapters (part II on significance pitfalls) are a checklist for any applied-statistics paper.
- Leek group guide to reading papers and the Leek genomics paper list: how a statistical genomicist reads, plus a curated canon to read.
- Reader’s checklist for statistical claims: what is the estimand (the quantity the study sets out to estimate); is the design causal or observational; how many comparisons were made; is the effect size reported with uncertainty; are code and data available.
2.3 Computer science and machine learning
- Keshav’s three passes remain the default for systems and conference papers.
- Andrew Ng’s Stanford CS230 talk on reading papers (2019): read a field as a list of papers, skimming many and going deep on a few; about 5–20 papers for working knowledge, 50–100 for depth; re-derive the math and re-run the code.
- For ML papers specifically: check baselines, compute budgets, ablations, seeds, and whether the benchmark is contaminated.
2.4 Computational biology
- Carey et al. (2020): read the figures first and interrogate each panel.
- Add the computational layer: is the code public and runnable; which reference versions, parameters and thresholds were used; would the result survive a different pipeline.
2.5 Neuroscience
There is no single canonical neuroscience guide; the field’s best reading aids are its methods-critique papers, used as checklists:
- Mensh & Kording (2017) on where claims live in a paper.
- Known traps to check for: low statistical power, “double dipping” (selecting and testing on the same data), and comparing two significance tests instead of testing the interaction directly.
- Practical course guides such as Case Western’s NeuroWiki and this cognitive-neuroscience primer.
3 The AI tool landscape for reading papers
AI tools now cover every stage Keshav describes, from finding papers to interrogating a single equation. They are strongest at triage and navigation, weakest at judging whether a result is right.
| Job | What the tools do | Examples | Main caveat |
|---|---|---|---|
| Discover and map a field | Semantic search, citation graphs, “papers like this one” | Semantic Scholar, Connected Papers, ResearchRabbit, Undermind | Coverage gaps; ranking favors highly cited work |
| Synthesize many papers | Retrieval-augmented answers with citations; “deep research” agents | Elicit, Consensus, OpenScholar, PaperQA2, ChatGPT/Claude/Gemini research modes | Fabricated or mismatched citations; overgeneralized claims |
| Read one paper with help | Chat with the PDF, explain a figure, define a term inline, summarize a section | alphaXiv, Semantic Reader, SciSpace, NotebookLM, general chat assistants | Summaries flatten caveats and hedges |
| Check citation context | Shows whether later work supports or disputes a claim | scite | Classifier errors on nuanced citations |
| Verify math and code | Re-derive steps, check a proof sketch, re-run or port code | General assistants with code execution; proof assistants (Lean) for formal checks | Confident but wrong derivations unless formally checked |
The most rigorous evidence so far is for retrieval-grounded systems. OpenScholar, published in Nature in February 2026, answers questions from a store of 45 million open-access papers. In its tests a general model (GPT-4o) fabricated citations roughly 78–90% of the time, while OpenScholar’s citation accuracy approached that of human experts, and experts preferred its answers over expert-written ones about 50–70% of the time.
The practical lesson: use tools that retrieve and cite real documents, and click through every citation that matters. Product rankings in this space change monthly; independent roundups like this 2026 comparison are a starting point, not a verdict.
4 How researchers use AI to read
AI-assisted reading is now mainstream, and it is moving fastest in peer review, where reading closely is the job.
- Peer reviewers. A Frontiers survey of about 1,600 researchers (reported January 2026) found over half now use AI when reviewing, up from 24% in 2024. Use runs higher among early-career researchers (61%) than senior ones (45%), and varies by region (77% in China, 31% in North America). The top uses were writing the report (59%), summarizing the article’s findings (29%) and flagging possible misconduct (28%).
- Researchers broadly. Wiley’s ExplanAItions study of almost 5,000 researchers (2025) found use still fairly narrow in tools and tasks, with 63% citing unclear guidelines or lack of training as barriers.
- Common workflows that have emerged:
- Triage at scale: ask an assistant for the five Cs of each of 30 candidate papers, then pick five to read yourself.
- Tutor on demand: paste an unfamiliar method section and ask for the prerequisites, a worked toy example, or the notation decoded.
- Adversarial second reader: after your own pass, ask the model to list the weakest assumption, missing controls or alternative explanations, then judge those yourself.
- Reproduction partner: have it help re-implement an analysis or re-derive an equation, which turns Keshav’s “virtual re-implementation” into a literal one.
The pattern that holds up: AI before and after your own reading, not instead of it.
5 Risks and pitfalls
The three failure modes to plan around are distorted summaries, invented sources, and the slow loss of your own reading skill.
| Risk | Evidence | What to do |
|---|---|---|
| Summaries overstate findings | Peters & Chin-Yee (Royal Society Open Science, 2025) tested 10 models on 4,900 summaries. Model summaries of medical papers were nearly 5 times likelier to overgeneralize than expert-written ones; some newer models were worse, and asking to “avoid inaccuracies” made it worse still. | Read the abstract’s hedges and the limitations section yourself. Ask the model to quote the exact sentence behind each claim. |
| Fabricated or mismatched citations | General models invented most citations in the OpenScholar tests. | Use retrieval-grounded tools; open every citation you will rely on or repeat. |
| Deskilling and shallow understanding | MIT’s “Your Brain on ChatGPT” preprint (small sample, essay writing) found lower neural engagement and weaker recall in the LLM-assisted group. Schwer’s 2026 math reading guide warns of the same trade-off. | Do the first pass and the hard parts (proofs, derivations, figure reading) unaided; use AI to check, not to replace, that work. |
| Confidentiality | Uploading an unpublished manuscript you are reviewing may breach journal policy. | Check the journal’s AI policy before using any tool on a manuscript under review. |
6 A protocol: the three passes, updated for AI
Keep Keshav’s structure, add an AI-assisted pass zero and a verification pass, and keep the judgment passes yours.
- Pass 0, triage (AI-led, ~2 min per paper). Use a retrieval-grounded tool to find candidates and get each one’s five Cs. Set your reading goal (Carey et al., Rule 1) before going further.
- Pass 1, the bird’s-eye view (you, 5–10 min). Title, abstract, intro, headings, conclusion, and every figure. Write your own one-sentence summary before reading any AI summary, then compare the two.
- Pass 2, the argument (you, with AI as tutor, ~1 h). Read the main text and figures. Ask AI to unpack unfamiliar methods, notation or prerequisites, with toy examples. Note each claim and its supporting evidence.
- Pass 3, the field-specific deep read (you, hours).
- Math: work small examples; re-prove the key lemma; formalize one step if it matters.
- Statistics: identify the estimand, design and multiplicity; re-run the analysis if data exist.
- CS and ML: check baselines, ablations and compute; re-implement the core method.
- Comp bio: run the pipeline, or at least audit versions and parameters.
- Neuroscience: check power, circular analysis and interaction tests.
- Pass 4, adversarial check (AI as skeptic, then you). Ask for the weakest assumption, missing controls and alternative explanations; demand quoted evidence from the paper. Verify any citation you will rely on.
- Integrate (you). Write a short note in your own words: what it claims, how far you believe it, and what you would do next.
Read. In. Layers. AI. Triages. Tutors. Argues. You. Judge.
References
Reading guides
- Keshav, How to Read a Paper (2007)
- Carey, Steiner & Petri, Ten simple rules for reading a scientific paper (2020)
- Mensh & Kording, Ten simple rules for structuring papers (2017)
- Leek group guide to reading papers
- Schwer, How to Read Mathematics? (2026)
- Khovanova, How to Read a Math Paper (2026)
- MathComm: Reading mathematics resources
- Greenhalgh, How to Read a Paper (6th ed.)
- Andrew Ng on reading research papers (KDnuggets)
AI and reading