I first heard it on Inteligencia Artificial Semanal (Spotify, YouTube, iVoox). I did not go looking for the paper. The host often jokes that he is a Seventh-day Suttonist: aligned with Sutton, with The Bitter Lesson, and with reinforcement learning. The podcast kept returning to the essay the way you name a tool you already take for granted, and in some episode the sentence clicked: human knowledge, the thing that feels like the most valuable ingredient we can put into a system, can be both the shortcut and the ceiling.
In March 2019, Richard Sutton compressed seventy years of research into a few pages. The title, The Bitter Lesson, warns that the conclusion is unwelcome. The claim is easy to state and hard to accept: in the long run, general methods that leverage computation (search and learning) win. Human knowledge of the domain helps at first, satisfies the person who put it there, and later flattens the curve. Sometimes it gets in the way.
That flattening is what I care about. Not the slogan “more GPUs”. The shape of the curve: climb faster with what humans already know, then approach, asymptotically, a ceiling that has a human shape.
What the essay says
Sutton starts from a historical observation, not a lab prediction. Most AI research has been done as if the computation available to the agent were constant. If the compute budget will not grow, leveraging human knowledge is almost the only lever. Over a slightly longer horizon than a typical project, the cost per unit of computation falls and vastly more calculation appears. Then the lever that matters is a different one.
The two strategies do not have to fight. In practice they do. Time spent formalizing how you think you think is time not spent on methods that scale when ten times more compute shows up tomorrow. There are psychological commitments: you have invested in an approach and you want it to win. And the human-knowledge approach tends to complicate methods in ways that make them worse at absorbing more calculation.
Sutton boils the lesson down to four observations:
- Researchers have tried, again and again, to build knowledge into their agents.
- This always helps in the short term, and it is personally satisfying.
- In the long run it plateaus and even inhibits further progress.
- Breakthrough progress arrives by the opposite route: scaling computation through search and learning.
The later success tastes bitter because it wins against an approach the field preferred: the human-centric one, the one that “understands” the domain the way we do.
The sentence the essay leaves behind, and the one the podcast left running in my head, is this: we want AI agents that can discover like we can, not ones that contain what we have already discovered.
Why it is bitter
The bitterness is not technical. It is professional, and a little bit about identity.
If your craft is putting into the system how chess, go, speech, or vision works (the openings, the phonemes, the generalized cylinders, the features that “make sense”), the system is an extension of your understanding. When a cruder method, with more search or more data, overtakes you, the loss is not just a paper. It is an idea of what counts as intelligence.
Sutton tells it without varnish in chess. In 1997, Deep Blue beat Kasparov with massive, deep search. A large part of the community that had worked on understanding the special structure of the game were not good losers. They said brute force had won this time, that it was not a general strategy, and that anyway it was not how people played chess. They wanted methods based on human input to win.
That “we wanted ours to win” is the bitter part. The field is slow to digest that progress arrived by the path that least resembled us.
The examples in the essay
Sutton does not invent a pattern. He points to it in four places where the field had already paid for the same cycle.
| Domain | Human shortcut | What ended up scaling |
|---|---|---|
| Chess | Understanding the structure of the game, less search | Deep search at scale (Deep Blue, 1997) |
| Go | Avoiding search with knowledge and the board’s peculiarities | Search plus learning by self-play, twenty years later |
| Speech | Words, phonemes, the vocal tract (DARPA competition, 1970s) | Statistical methods (HMMs) and, later, deep learning |
| Vision | Edges, generalized cylinders, SIFT features | Networks that use little more than convolution and invariances |
Speech recognition is the cleanest cycle. One side carried human knowledge of language and the vocal apparatus. The other spent more computation on statistical models. The statistical side won. Over decades, natural language processing drifted in that direction. Deep learning was the next step of the same movement: still less embedded knowledge, still more calculation and more data.
Vision did the same thing with a different vocabulary. For years people designed primitives that imitated how we thought we saw. Today those primitives are off the critical path. What remains is a minimal inductive bias (convolution, some invariance); training finds the rest.
Go, in the essay, is chess with a twenty-year delay. The field spent enormous effort on not searching, on using what humans already knew about the game. Once search and self-play were applied at scale, that effort became irrelevant or worse. Sutton does not walk through the AlphaGo versions. The versions are what make the asymptote concrete.
The curve: shortcut and ceiling
Human knowledge is not noise. It is a prior: the belief you start with, before you see new data. In statistics you update that belief with what you observe. In AI the mechanism is the same, more informal: what the system already assumes conditions what it can learn next.
A good prior puts you in a decent place sooner. If you feed the network professional games, it does not start from random moves: it already “believes”, in some sense, that human openings and good shape are the reasonable place to be. A prior that describes how humans play, speak, or see also pulls you toward that place when the system could have gone further.
The shape, in crude form:
| Embedded human knowledge | Search + learning | |
|---|---|---|
| Start | You climb fast: you imitate the experts | You start near random play |
| Asymptote | A ceiling with a human shape (styles, features, biases) | Set by compute and the problem space |
| Side effect | More complex methods, worse at absorbing more calculation | Cost, and less control over what it found |
“A ceiling with a human shape” does not mean “a professional’s Elo”. AlphaGo, the one that did use human games, was already superhuman: it beat Lee Sedol. The ceiling is not “you cannot surpass people”. It is the basin of attraction of what people already explored and of how they described the problem. The system learns to be a very good human, or a very good human with a little extra search. Leaving that basin is hard.
That is the asymptotic part. The shortcut gets you earlier to a local optimum that looks like us. The method that starts with less (rules, search, learning) takes longer to look intelligent and can overshoot that optimum.
AlphaGo and AlphaGo Zero
In 2016, AlphaGo became the first program to beat a world champion at go. The recipe mixed the two strategies. Neural networks trained first with supervised learning on amateur and professional games, then with reinforcement learning; and, at playing time, a tree search (MCTS) guided by those networks. Human knowledge was the ramp: thousands of games that already existed, a style, a repertoire of openings and of “good shape”.
It worked. Fan Hui, Lee Sedol, later Ke Jie. The shortcut delivered. In a few years go went from “too large for search” to a problem solved above the human ceiling.
In 2017, DeepMind published AlphaGo Zero: same game, different ramp. No human games. No domain features beyond the stones on the board. No separate policy and value networks. A single network, a simpler search, and a loop in which the program is its own teacher. It starts near random play. It plays against itself. It updates the network to predict its own moves and the winner. It searches again. It repeats.
The Nature paper is titled, without disguise, Mastering the game of Go without human knowledge. The team’s own numbers:
- After three days of self-play, AlphaGo Zero won 100-0 against the published version that had beaten Lee Sedol (AlphaGo Lee).
- After 21 days it reached the level of AlphaGo Master, the one that had swept the best humans.
- After 40 days it surpassed every previous version. In a 100-game match, 40-day Zero beat Master 89-11.
Zero did not only go higher. In three days of self-play it had left behind the AlphaGo trained on people, and then it kept climbing. The paper uses the word that matters here: better asymptotic performance than training on expert data. Demis Hassabis put it later: Zero was no longer constrained by the limits of human knowledge.
You could see it on the board. Zero rediscovered known joseki and then skipped them. It found moves that looked ugly to a professional and that, once you had the result, were better. Human go is a library of centuries. It turned out to be an incomplete library. Training against that library puts you inside it. Training against yourself lets you leave.
One caveat is worth not hiding. Zero did not start from metaphysical nothing. It started from the rules of go, which are human knowledge, and from a search algorithm that is human knowledge too. “Without human knowledge” in the paper means no games, no style advice, and no domain features. People still wrote the environment. They did not write the contents of the agent’s mind.
That caveat does not weaken the lesson. It sharpens it. What Sutton wants left out is the catalog of what we already found (the good moves, the shapes, the phonemes, the edges). What he wants left in are the meta-methods with which an agent can find that catalog, and a larger one, when more compute arrives.
Reinforcement learning
Sutton is not talking about just any kind of learning. The kind that scales alongside search is, in practice, reinforcement learning: the system tries, gets a signal for how it went (won, lost, got it right), and adjusts what it will do next time. Nobody dictates the correct move. Sooner or later they tell it whether the result was worth it.
Imitating a human is a different thing. Supervised learning starts from labels we already brought: this position is played this way because a professional did it. It is a prior in the shape of a demonstration. Reinforcement starts from a reward. It can discover a line no professional played, if that line wins.
AlphaGo used both in sequence: first imitate games, then reinforcement. Zero skipped the imitation. That is why the paper can talk about better asymptotic performance: the human prior was not there to cap how it plays.
In 2026 the same gesture shows up under other names. Pretraining a model on internet text is imitation at scale (a huge human prior). Then a reinforcement phase often follows: the model generates, the result is evaluated (a problem with a checkable answer, a judge, a preference signal), and it updates. The bitter lesson does not ask for a specific algorithm. It asks us not to confuse the library of human examples with the ability to improve from what happens.
Emergence
On Inteligencia Artificial Semanal you hear the verb emerge a lot. As you add computational power (more parameters, more data, more self-play, more search at inference) capabilities appear that nobody wrote as a rule. Sometimes you expected them: play go better. Sometimes you did not: a move that looks ugly; a skill that was not the training objective.
Sutton describes it without using the word. If you put into the agent how you think you think, the ceiling is that theory. If you put in methods that leverage more compute, the system can find complexity you had not named. The new behavior is not added by hand. It shows up.
Zero rediscovering joseki and then skipping them is emergence on a measurable board. In large models the pattern looks similar: as compute grows, the system does something the version ten times smaller did not do, or does it in a way nobody anticipated. There is a debate about whether the jump is sudden or a smooth curve we were measuring badly. The useful point does not change. The surprise is the consequence of not having built the catalog in.
Emerge is the name the podcast gives to Sutton’s bet: we want agents that discover, and discovery, seen from outside, looks as if it came out of nowhere.
Two methods that scale
Sutton names two families that remain useful when available calculation becomes enormous: search and learning.
Search spends compute at decision time: more depth, more simulations, more candidates. The learning he has in mind is experience turned into better decisions, the reinforcement loop, more than copying labels. Both turn “tomorrow there will be more calculation” into “tomorrow the same method will be better”, without you having to rewrite the theory of the domain by hand. And if the method really scales, what shows up afterwards is what the podcast calls emerging.
Embedded knowledge does not scale that way. A well-designed go feature from 2014 does not become ten times better because the cluster is ten times larger. Often it becomes a hindrance: the system has to negotiate with it instead of spending the extra budget on searching or learning.
The essay’s second conclusion is drier. The actual contents of minds are irredeemably complex. We should stop looking for simple ways to think about that content (space, objects, multiple agents, symmetries) in order to build them into the agent as if they were the essence. That is the world, arbitrary and bottomless. What we should build are methods that can find and approximate that complexity. The search for good approximations should be done by our methods, not by us.
Why it matters now
In 2019 the essay read as a recap: chess, speech, vision, go. In 2026 it reads as the working assumption of the labs.
The cost per unit of computation has kept falling, and the computation a training run or an inference pass can swallow has kept rising. Imitating text at scale, then reinforcement, and spending more calculation per answer are, under other names, Sutton’s two families. Every time a model “reasons” longer and improves, you are looking at search. Every time a long training run displaces a system full of rules, you are looking at learning. And every time, as parameters or compute go up, a capability appears that nobody put in the design, the podcast would say it has emerged.
The cycle in the essay has not closed. It is still tempting to build how we think we think: hand-crafted cognitive architectures, curricula that imitate school, features that “make sense”, expert recipes injected as if they were the system’s destination rather than its first step. It helps. It is satisfying. It publishes well. And it draws an asymptote.
Go left it measured. AlphaGo, with human games, got earlier to a place that was already superhuman. AlphaGo Zero, without those games, took a while to look like someone and then overshot the place people (and the system trained on people) had reached. Human knowledge was an accelerator and a magnet. The magnet is what puts the ceiling on.
Sutton is not asking us to discard what we know. He is asking us not to confuse it with the content the agent should carry inside. The rules, the evaluation, the judgment of whether something matters: that can still be ours. The catalog of solutions, the “this is how you play”, the “this is how you see”, the “this is how you speak”, is a local optimum that feels like arrival.
The bitter part is about identity: we like being the source. The essay’s bet is a different one. Put into the system the capacity to discover, not the library of what has already been discovered. The podcast left that said in passing, between the Seventh-day Suttonist joke and the verb emerge. Zero’s board, 100-0, left it written in the score.