This post separates skill from generalization, argues that the human niche selected for the second, and shows where the large-language-model training recipe rhymes with it.
Skill at a known task can be bought with data. Generalization is converting little experience into competence on a problem outside the training distribution.
Chollet’s On the Measure of Intelligence scores a system on that conversion efficiency. Memorization of the training task does not count.
1 Why a general engine was worth the cost
Human environments (terrain, climate, food, social arrangement) shifted faster than genomes could specialize.
Two options:
- Compile behaviour into instinct: cheap to run, brittle when the environment moves.
- Pay for a general inference engine inside one lifetime: expensive brain, long childhood.
Humans took the second.
Tools are the visible output of that engine: model an unseen cause, separate means from end, plan for a future need, transfer a technique to a new material.
Once a technique can be taught, each generation starts from the last one’s best version (cumulative culture). That ratchet pressured language and social cognition.
Three capacities lock together:
- Build a model of the world.
- Pass what you learn to someone else.
- Practise against consequences.
Machine learning has a paradigm for each. None of them alone is the whole system.
2 Three paradigms
| Capacity | Human analogue | ML analogue |
|---|---|---|
| World model from unlabelled experience | Childhood prediction (“what happens if I let go”) | Self-supervised learning |
| High-fidelity transfer of a technique | Teaching, correction, vocabulary | Supervised learning |
| Adjustment from consequences | Trial and error; at species scale, evolution | Reinforcement learning |
Alone, each falls short: an unguided predictor, a mimic, or a specialist that burns many trials on one task. Breadth comes from stacking them.
The large-language-model recipe is the same stack:
- Pretrain on unlabelled text.
- Fine-tune on curated good answers.
- RLHF: preferences between responses as the consequence.
The rhyme is not identity. A child’s world model is built by intervening in the world, not by reading descriptions of it. Culture arrives from people who correct in real time, not from a fixed pile of examples.
The relevant benchmark is a task the system was not trained on.
Skill. Buys. Nothing. New. Generalization. Does. Novel. Tasks. Are. The. Benchmark.
3 References
- Chollet, F. (2019). On the Measure of Intelligence.