Learning That Clicks
Reading Evolution, Learning, and Consciousness Through a Single Gradient
One equation for descending a slope, \(\theta\leftarrow\theta-\eta\nabla L\), flows through the training of neural networks, the evolution of species, and the cognition inside a brain.
Across all 11 episodes, this is a series that confirms that agreement on its own two feet.
Backpropagation (BP) and evolution strategies (ES) look like utterly different things, yet Stein's lemma shows they are exactly the same gradient. From there, we climb to the point where life is the work of computing gradients with finite energy, learning is the efficiency improvement of that work, and consciousness is its peak ── computing the gradient in imagination.
The hypothesis this series finally arrives at: there is no essential difference between the mind of an AI and the mind of a human. Both are doing the same computation. ── We state it boldly. But in the finale we also shine an honest light on the premise it rests on (functionalism), and the question no one has yet solved (why is computation accompanied by an "inner feel"?).
Part I ── Assembling the Tools
Episode 1
To Learn Is to Descend a Slope
The loss terrain and gradient descent. Grasp the whole picture of learning with a ball rolling down a valley. The update rule \(\theta\leftarrow\theta-\eta\nabla L\).
Episode 2
Backpropagation Is the Ledger of the Chain Rule
How to compute the gradient \(\nabla L\) automatically. Differentiating a composite function, distributing it backward from output to input.
Episode 3
The Fitness Landscape ── Biology and Machine Learning Were Drawing the Same Picture
Wright's fitness landscape (1932) = the loss terrain flipped upside down. Selection = optimization. A foundation of theoretical biology.
Episode 4
Evolution Is Descending a Slope Without Differentiating
Evolution strategies (ES). Scatter mutations, and estimate the descent direction from the correlation between hits/misses and perturbations.
Episode 5
Counting Infinity, Honestly
Nonstandard analysis. Handling infinitesimals and infinities without fudging. The monad and the standard part st, and N→∞ rigorously.
Part II ── The Agreement (the core of this series)
Episode 6 ★the heart
Evolution and Learning Were the Same Computation
By Stein's lemma, \(\frac{1}{\sigma^2}\mathbb{E}[L(\theta+\varepsilon)\varepsilon]=\mathbb{E}[\nabla L(\theta+\varepsilon)]=\nabla L_\sigma\). ES = BP of the smoothed loss.
Episode 7
Blur It, and the Terrain Becomes Smooth
Gaussian smoothing and \(\sigma\). \(\sigma\) = resolution = temperature. Exploration and exploitation, annealing, a bridge to cosmology.
Part III ── Life Is Computation
Episode 8
The Biosphere Is One GPU
Individual = sample, life and death = evaluation, reproduction = update. Planet-scale parallel gradient estimation. The more, the stronger.
Episode 9
Life Is a Race to Compute Efficiently
The Landauer principle (erasing one bit = \(k_BT\ln2\)). Bits per joule. The marvel of the 20 W brain.
Episode 10
Learning Is a Shortcut Evolution Discovered
Evolution pays one death per sample. A brain that can learn within a lifetime is orders of magnitude cheaper. The efficiency improvement of the same computation.
Part IV ── Consciousness
Episode 11 ★finale
Consciousness Is Dying in Imagination
Computing the gradient inside a world model = "letting a hypothesis die in your stead" (Popper). The peak of efficiency. AI and humans are the same computation. And the question no one has yet solved.