Why is learning possible? The answer lives in the same place as "why is the universe coarse-grainable?"
Bonus ③ showed statistics, and ④ information theory, looking at the same thing as the renormalization group. Last comes AI. This is a field flooded with "it's kind of like…", so this episode begins by sorting claims by how confident we can be: rigorous / promising but disputed / metaphor only. The first tier has real content: take a neural network's width to infinity and you get a free field theory; 1/width acts as an interaction; and a renormalization group flow runs along the depth. That is a calculation, not a poem. Then we close the whole series with one claim ── 〈why is learning possible〉 and 〈why is the universe coarse-grainable〉 are the same question.
Machine learning has a blunt theorem.
Averaged over all possible target functions, every learning algorithm performs exactly as well as guessing.
In other words ── there is no universally clever learner. If a learner works, it is because the world has some bias and the learner carries assumptions (an inductive bias) matching it.
So what is our world's bias? Episode 4's answer transfers directly ── this world is layered, with very few directions that matter (relevant) and overwhelmingly many that don't (irrelevant). In Bonus ④'s language, the rate–distortion curve has a knee. Learning is possible because it bets on that bias.
"Discard or die" is not a figure of speech. Collect data in a space with \(D\) features and distances between points progressively lose meaning as \(D\) grows (the curse of dimensionality). Even if only 2 features actually matter, a learner that dutifully looks at the other \(D-2\) is weaker by exactly that much.
In the figure below two learners compete. Same data, same algorithm (nearest neighbours). One difference only ── whether the irrelevant features are discarded.
Push \(D\) upward. The blue one (which discards) barely notices, while the red one (which doesn't) degrades until it approaches "no learning at all." Same data, same algorithm. The only thing making the difference is what it decided not to look at.
That is the meaning of this episode's title ── learning is finding out what may be discarded. Episode 4 said discarding determines the answer; in machine learning it literally determines the performance.
Now the sorting. First, the things that are not metaphors.
Let \(n\) be the width of each layer (its number of neurons). By the central limit theorem (Bonus ③!), as \(n\to\infty\) the output distribution becomes Gaussian ── in field-theory language, a free (non-interacting) theory. At finite \(n\), \(1/n\) corrections appear and behave as interactions. And stacking layers (depth) makes the parameters flow as under a renormalization group.
Roberts–Yaida–Hanin's The Principles of Deep Learning Theory (2022) builds a perturbation theory on exactly this. Not "deep learning resembles the renormalization group" but "the tools of the renormalization group actually compute deep learning" ── the direction is reversed, and this one is real.
What is delightful is that the starting point is the central limit theorem. Bonus ③ called the CLT the simplest example of the renormalization group; that simplest example is now the foundation of the theory of our most complicated machines.
Another non-metaphor. Koch-Janusz and Ringel (2018) did the following ── they had a machine search for a coarse-graining rule that maximises the mutual information between what is inside a block and what is outside it.
The machine then found, with nobody telling it, the block-spin transformation of the Ising model (the very thing we did in Episode 4). This is Bonus ④'s information bottleneck working as an actual tool ── the single instruction "keep only what helps predict the outside/future" produces a renormalization transformation.
Meanwhile this area is full of things that merely sound right. Honest sorting:
| Tier | Content | Status |
|---|---|---|
| rigorous | infinite width = Gaussian process, 1/width = interaction, flow along depth (Roberts–Yaida–Hanin) | computable; functions as a perturbation theory |
| rigorous | discovering renormalization transformations by maximising mutual information (Koch-Janusz–Ringel 2018) | rediscovered Ising block spins; works as a tool |
| rigorous | wavelet scattering transforms (Mallat) | multiscale, with provable invariance and stability |
| promising, disputed | diffusion models ≈ the renormalization group. Adding noise in stages = coarse-graining; reversing it = generation; the noise schedule corresponds to scale | rigorous correspondences are being attempted. Attractive but unsettled |
| promising, disputed | explaining neural scaling laws (loss falling as a power law) from data-manifold dimension or spectra | power laws smell of criticality, but explanations are only partial |
| metaphor only | the general claim "deep learning = the renormalization group" (generalising Mehta–Schwab 2014) | a correspondence in a specific setting; strong objections to generalising it |
| metaphor only | the information-bottleneck explanation of deep learning (the "compression phase") | criticised on reproducibility; unsettled |
When Bonus ② said "the correspondence between deep learning and the renormalization group is a suggestive resemblance, not a theorem," it meant tier 3. Tier 1 can be stated without hedging.
From here, the opening question gets an answer from a new angle.
By the no-free-lunch theorem, the very possibility of learning is a strong claim about the world. If learners work, the world must be the kind that splits into a few relevant and a great many irrelevant directions.
why learning is possible
=
why the universe is coarse-grainable
One question. Which means, read backwards ── the fact that learning actually works is empirical evidence that the universe is coarse-grainable. Bonus ② said "a universe that cannot be coarse-grained has nobody in it to ask." We can now put it more sharply: the fact that we can learn anything at all is itself proof that this universe has a knee.
Established: the no-free-lunch theorem (Wolpert 1996); the curse of dimensionality (degradation of nearest-neighbour methods in high dimensions); that neural networks become Gaussian processes in the infinite-width limit and that the \(1/n\) expansion can be treated as a perturbative effective theory (Roberts–Yaida–Hanin 2022 and others); learning renormalization transformations via mutual information (Koch-Janusz & Ringel, Nature Physics 2018); and the invariance and stability of the scattering transform (Mallat).
Limited or unsettled: (1) The infinite-width correspondence describes a particular (lazy / perturbative) regime and does not fully explain feature learning itself. (2) The diffusion-model / renormalization-group correspondence and the explanation of scaling laws are active but unsettled. (3) Generalisations of the Mehta–Schwab correspondence and the information-bottleneck explanation of deep learning both face strong objections and remain undecided. (4) "Why learning is possible = why the universe is coarse-grainable" is this series' claim, not a theorem; the practical import of the no-free-lunch theorem is itself debated (the premise of averaging uniformly over all target functions). (5) The figure is an idealised demonstration with a simple nearest-neighbour algorithm and does not represent the performance of any real learner.
By the no-free-lunch theorem, there is no universally clever learner. If learning works, it is betting on the world's bias ── and that bias is Episode 4's "few relevant, many irrelevant," Bonus ④'s "knee in the rate–distortion curve." As the figure shows, with the same data and the same algorithm, whether or not you discard the irrelevant decides everything. Learning is finding out what may be discarded.
The connection to AI splits into three tiers. Rigorous: infinite width = free field (starting from the central limit theorem!), 1/width = interaction, depth = renormalization flow. And set machine learning to maximise mutual information and it rediscovers the Ising block-spin transformation on its own. Promising but unsettled: diffusion models ≈ renormalization group; scaling laws as criticality. Metaphor only: the general "deep learning = renormalization group" claim, and the information-bottleneck explanation. Traffic already moves in both directions, physics → AI and AI → physics.
And the close of the series. 〈Why is learning possible〉 and 〈why is the universe coarse-grainable〉 are the same question. Bonus ② said "a universe that cannot be coarse-grained has nobody in it to ask." Now we can go one step further ── the fact that we can actually learn anything is itself the evidence that this universe has a knee. The story that began with the neutron's 880 seconds ends here. The world is built so that things can be thrown away, and that is why we are here.
Print / make a PDF: ⌘+P (Ctrl+P on Windows). On screen, raising the total number of features D breaks only the learner that refuses to coarse-grain. "Resample the data" regenerates the random numbers. "See the answer" opens each solution.