Evolution, learning, and consciousness read through a single gradient
A machine "learns"; a living thing "evolves." We tend to speak of these two as though they belonged to entirely separate worlds. One is a story of mathematics and computers, the other a story of life stretching back to the dawn of time. And yet the loss landscape we drew in Episode 1 ── that scenery with the parameter \(\theta\) along the horizontal axis and the badness \(L(\theta)\) as height ── was not, in fact, an invention of machine learning. It is the very same picture, merely flipped upside down, as the adaptive landscape that population geneticist Sewall Wright drew back in 1932 to explain evolution. Biology aims for "high peaks = well-adapted genotypes"; machine learning aims for "low valleys = weights with small loss." Only the name and the direction differ. In this episode we'll see that natural selection is, on paper, gradient ascent up the landscape itself, and that the "local-minimum trap" we met in Episode 1 comes back here wearing exactly the same face ── as getting stuck on a local peak. Across a gap of eighty years, evolution and learning turn out to be standing on the same terrain.
In Episode 1 we defined learning in a single line ── moving the parameters \(\theta\) so as to lower the loss \(L(\theta)\). The update rule is \(\theta\leftarrow\theta-\eta\nabla L\). In Episode 2 we saw the mechanism by which a neural network computes that gradient \(\nabla L\) automatically, via the chain rule (backpropagation, BP). So far, this is all a story of "machines learning."
So what about the evolution of living things? There is no god who kindly differentiates DNA for us. And yet life has kept adapting to its environment for billions of years. If evolution ran on a principle utterly different from learning, the backbone of this series would snap. Conversely, if it performs the same operation on the same terrain ── then "learning machines" and "evolving life" are of one continuous piece. Let me put the answer up front. They are the same. To see it, let's first set Wright's landscape right next to Episode 1's.
Wright put the genotype (or trait) \(\theta\) on the horizontal axis, and took as height how successfully an individual carrying that \(\theta\) survives and leaves offspring. This height is called fitness, \(F(\theta)\). The higher the peak, the "better-adapted \(\theta\)"; the lower the valley, the "harder-to-survive \(\theta\)." Evolution is the process of a population climbing toward the peaks of this landscape ── that is Wright's picture.
Compare it with Episode 1's loss landscape. There, "lower valleys are better"; here, "higher peaks are better." Only up and down are reversed; the undulations of the terrain themselves are identical. In equations, the two are joined by a single identity.
Lowering the loss (descending the valley) is exactly the same operation as raising the fitness (climbing the hill). In Episode 1 we moved in the direction of \(-\nabla L\), which was steepest descent; but if \(L=-F\), then \(-\nabla L=+\nabla F\). In other words, it is nothing but gradient ascent in the direction of the fitness gradient \(\nabla F\). Only the direction is turned around: it's the same slope, descended by one and climbed by the other.
Let's turn "a population climbs the peaks" into proper equations. Suppose the population contains types (genotypes / strategies) \(i=1,2,\dots,n\), and let the frequency of type \(i\) in the population be \(x_i\) (with \(x_i\ge0,\ \sum_i x_i=1\)). If the fitness of type \(i\) is \(f_i\) (its average reproductive success per individual), then the mean fitness of the whole population is
What is natural selection? Types above average increase, types below average decrease ── that's all. The differential equation that captures exactly that one sentence is the replicator equation.
Reading it is easy. The quantity in the parentheses, \(f_i-\bar f\), is how far type \(i\) sits "above average." If this is positive (above the mean), then \(\dot x_i>0\) and the frequency grows; if negative (below the mean), it shrinks. The \(x_i\) out front is an amplification factor set by how many of that type there currently are ── so a type not currently present in the population (\(x_i=0\)) stays at \(\dot x_i=0\) no matter how superior it would be; it never springs up on its own. This single point is the setup for the "local-peak trap" to come.
First, let's verify that this equation preserves the total of the frequencies (a sanity check that it remains a probability distribution).
All we use is \(\sum_i x_i f_i=\bar f\) and \(\sum_i x_i=1\). Since the rate of change of the total is \(0\), \(\sum_i x_i\) stays at \(1\) forever. Selection merely redistributes the slices of the pie; the pie itself neither grows nor shrinks.
The most beautiful consequence of the replicator equation is this ── as long as the fitnesses are held fixed, the population's mean fitness \(\bar f\) never decreases. What's more, the amount by which it rises is exactly equal to the spread (variance) within the population. Let's derive it by hand.
Since \(f_i\) is constant, differentiating the mean \(\bar f=\sum_j x_j f_j\) in time gives
$$\frac{d\bar f}{dt}=\sum_i \dot x_i\,f_i=\sum_i x_i(f_i-\bar f)\,f_i$$Expand the parentheses and use Σx_i f_i = f̄
$$=\sum_i x_i f_i^2-\bar f\sum_i x_i f_i=\langle f^2\rangle-\bar f^{\,2}$$This is precisely the definition of variance
$$\frac{d\bar f}{dt}=\mathrm{Var}_x(f)=\sum_i x_i\,(f_i-\bar f)^2\;\ge\;0$$Variance is a sum of squares, so it is always at least \(0\). Hence \(\bar f\) increases monotonically, and it halts only when everyone has the same fitness (variance \(=0\)). This is the core of Fisher's fundamental theorem of natural selection ── the mean rises by exactly the amount of the spread. Diversity itself was the fuel that drives evolution forward.
This relationship holds cleanly even for a single discrete step. In the small numerical experiment below, let's confirm with a calculator that \(\bar f\) really does rise, and that the amount it rises matches the variance exactly.
Three types, with fitnesses \(f=(1.0,\ 1.2,\ 0.9)\). The initial frequencies are all equal, \(x=(\tfrac13,\tfrac13,\tfrac13)\). We run the discrete update rule \(x_i\leftarrow x_i\bigl(1+(f_i-\bar f)\bigr)\) just once.
First, the mean fitness
$$\bar f=\tfrac13(1.0+1.2+0.9)=\tfrac{3.1}{3}=1.0333$$Update each type (multiply by f_i−f̄)
$$x_1'=\tfrac13(1-0.0333)=0.3222,\quad x_2'=\tfrac13(1+0.1667)=0.3889,\quad x_3'=\tfrac13(1-0.1333)=0.2889$$The total is \(0.3222+0.3889+0.2889=1.0000\) ── the sum is preserved exactly (just as we checked in STEP 03, there is no need even to renormalize). The superior type 2 grew, and the inferior type 3 shrank. The new mean is
$$\bar f'=0.3222\times1.0+0.3889\times1.2+0.2889\times0.9=1.0489$$Compare the rise against the variance
$$\Delta\bar f=1.0489-1.0333=0.0156,\qquad \mathrm{Var}_x(f)=\tfrac13\bigl[(-0.0333)^2+(0.1667)^2+(-0.1333)^2\bigr]=0.0156$$A perfect match. \(\Delta\bar f=\mathrm{Var}_x(f)\) ── the mean rises by exactly the amount of the spread. The numbers moved just as the theory predicted.
To sum up, natural selection is the sign-flipped twin of Episode 1's gradient descent. One descends the valley of \(L\) via \(-\nabla L\); the other climbs the peaks of \(F\) via \(+\nabla F\). The population crawls its way up toward the peaks of the landscape.
The landscape below is the fitness \(F(\theta)\), and it has two peaks. On the left is a low peak (a local optimum), and on the right a high peak (the global optimum). Between them lies a deep valley. The cluster of orange dots is the individuals of the population, and the red marker is their mean \(\bar\theta\). The population starts at the foot of the low peak on the left. When you press "Advance a generation," each generation repeats ① selection according to fitness (fitter individuals leave more offspring) and ② mutation (fluctuation added to the offspring's trait), and the population climbs the slope.
First run it with the initial setting (mutation \(\sigma=0.08\)) unchanged. The population glides up the low peak on the left, sticks there, and stops moving. The mean fitness rose, yet even though there's a higher peak right next door, it can't reach it. Next, before you press "Advance a generation" again, try raising the mutation width \(\sigma\) to around \(0.35\) and running it. The individuals' spread reaches across the valley to the right slope, and once selection acts there, the population eventually hops over to the high peak. Raising the selection pressure \(\beta\) makes the climb fast but narrows the field of view; lowering it makes the climb slow but more exploratory.
The "sticking to the low peak" we just saw is exactly the same thing as Episode 1's local-minimum trap. There, the ball got stuck in a shallow valley and couldn't get out. Here, the population climbs all the way up a low peak and can't come back down. Flip up and down with \(L=-F\), and a local minimum becomes, unchanged, a local maximum (a local peak). Same landscape, same weakness.
Recall that the replicator equation was \(\dot x_i=x_i(f_i-\bar f)\). There's an \(x_i\) out front ── types that aren't present don't increase. So by pure selection alone, the population can't take a single step off the peak it's standing on. To cross the valley to another peak, you need a mechanism that tries out places you're not currently in ── namely mutation (= exploration of the landscape). When we raised \(\sigma\) in the simulation and the population managed to cross the valley, it was because mutation played the role of "trying the neighbor." Diversity is the fuel that raises the mean, and at the same time the lifeline for escaping a local peak.
Every episode, this series honestly judges the powers and the limits of its tools. For the fitness landscape and natural selection, let's answer three questions.
| Question | What the fitness landscape + natural selection says | Verdict |
|---|---|---|
| Is natural selection the same operation as Episode 1's gradient descent? | The same. Under \(L=-F\), selection = gradient ascent on the fitness landscape = gradient descent on the loss landscape. Isomorphic, differing only by a sign | Isomorphic |
| Does the population's mean fitness always rise? | It never falls. \(d\bar f/dt=\mathrm{Var}_x(f)\ge0\) (monotonically non-decreasing if fitnesses are fixed) | Non-decreasing |
| Does it always reach the highest peak (the global optimum)? | Not necessarily. It sticks to a nearby local peak ── exactly the same weakness as Episode 1 | No guarantee |
The fitness landscape is the finest picture we have for surveying evolution, but it carries several idealizations. First, real landscapes are not stationary ── under the coevolution of predator and prey, or frequency-dependent selection (where whether a type is favored changes as it becomes more common), the landscape itself ripples as the population moves. In a world where "the footing you're climbing on shifts," the peak metaphor is too naive. Second, the picture we saw here had a one-dimensional trait, but intuition changes dramatically in high dimensions, where local peaks have far more "escape routes" than you'd expect. Third, the replicator equation is an infinite-population, noise-free approximation; in finite populations, chance fluctuations (genetic drift) come into play.
Even so ── the structural identity "selection = optimization on a landscape" is the real thing. Learning and evolution are doing the same computation on the same terrain, differing only in sign. And here one sharp question remains. Both Episode 1's gradient descent and Episode 2's BP assumed that \(\nabla L\) can be computed. But nature has no god of differentiation. You cannot differentiate DNA. So how does non-differentiable nature come to know the gradient of the slope? Next time, that answer ── evolution strategies (ES).
Wright's fitness landscape \(F(\theta)\) is the same picture as Episode 1's loss landscape, joined by \(L(\theta)=-F(\theta)\) (STEP 02). Natural selection is the replicator equation \(\dot x_i=x_i(f_i-\bar f)\) ── types above average increase, types below average decrease (STEP 03). If fitnesses are fixed, mean fitness is monotonically non-decreasing, and the amount it rises equals the population's variance, \(d\bar f/dt=\mathrm{Var}_x(f)\ge0\) (STEP 04, Fisher's fundamental theorem). So selection is gradient ascent, differing from gradient descent on the loss landscape only by a sign. But Episode 1's local minimum returns here as the local-peak trap, and pure selection cannot cross the valley ── mutation, the exploration, is the lifeline (STEP 06).
Learning and evolution were doing the same computation on the same terrain. Just one question remains ── how does non-differentiable nature come to know that gradient? Next time, on to a way of descending the slope without any differentiation at all.
Print / PDF: ⌘+P (Ctrl+P on Windows). On screen, change the "selection pressure" and the "mutation width," then press "Advance a generation" to watch the population climb the peaks. Try seeing how small mutation sticks to the low peak while large mutation moves to the high peak. Click "See the answer" to open the solutions.