Skill can be bought; generalization has to be earned
Watch what a person does with a problem nobody prepared them for — a jammed mechanism, an unfamiliar script, a social situation with no rules — and you are watching the thing worth calling intelligence. Not the competence itself, but the exchange rate: how little experience it took to get there. Skill at a known task is purchasable: enough repetitions, enough data, and a system plays the game well while learning nothing it could redeploy. François Chollet’s On the Measure of Intelligence makes the distinction sharp: measure the conversion of experience into competence outside the training distribution, and memorization stops counting.
Which raises a question, since that capacity is expensive: what was worth paying for it?
An unstable niche made the general-purpose engine worth its price
Humans never had a fixed task to specialize into: terrain, climate, food supply and social arrangement all shifted faster than genomes could track. An animal in that position has two options. It can compile behaviour into instinct — cheap to run, superbly tuned, catastrophically brittle the moment the environment moves. Or it can pay for a general inference engine that works things out inside a single lifetime: a metabolically ruinous brain, a childhood long enough to be a liability. Humans went hard on the second.
Tools are that engine’s visible output. Making one requires modelling a cause you cannot see, decoupling means from ends, planning for a need that has not arrived, and carrying a technique to a material it was never developed on — the same operations a novel problem demands, in physical form. But tools also drive the thing they express. Better calories helped fund the tissue doing the thinking, dexterity and planning circuits plausibly grew into each other, and once a technique could be taught rather than reinvented, cumulative culture ratcheted: each generation starting from the last one’s best version, which put its own pressure on language and social cognition.
So the capacity is not one mechanism but three locked together: world-modelling, transmission, practice — which is where each machine learning paradigm alone falls short.
No single paradigm reproduces it; the stack roughly does
Self-supervised learning is childhood. A toddler is not labelled; they run raw experience through a predictor and come out holding intuitive physics, object permanence and a causal map. Supervised learning is culture — teaching, language, imitation — the high-fidelity channel that hands over a technique instead of making a child rediscover it. Reinforcement learning is the consequence loop: the tool works or it doesn’t — and at species scale, evolution itself, reward paid in survival.
Any one alone falls short: an unguided predictor, a mimic, or a specialist that burns a million trials on one task. Breadth comes from the integration — world model first, culture on top, practice sharpening the result. That this is roughly pretraining → supervised fine-tuning → RLHF is no coincidence; it is the best-known recipe for the same target.
The rhyme stops short of identity. A child’s world model is built by intervening, not by reading descriptions, and their culture arrives from people who correct them in real time. Still, the framing survives: the benchmark that matters was never the task you trained for. It’s the one you’d never seen.