Learning That ClicksEpisode 1 ／ The Picture of Learning ── where the series begins

Reading evolution, learning, and consciousness through a single gradient

To Learn Is to Walk Downhill Nudge the parameters little by little, and lower the height called loss. Learning, evolution, and in time the mind itself ──
all of it turned out to be one single computation: "descend the gradient." We take the first step, grasping gradient descent as a ball rolling down a valley.

Tools you'll need: middle-school "slope," a little calculus Key of this episode: $\theta \leftarrow \theta - \eta\,\nabla L$

What, exactly, does it mean "to learn"? A neural net getting smarter, a living thing adapting to its environment, you learning to ride a bike ── these look completely unrelated, yet they all reduce to one single, simple operation. Nudge the parameters in your hands little by little, and drive down a number that measures how "bad" things are. That's all. We call this "badness" the loss, and write it $L$. Learning means making $L$ small ── in other words, walking down a slope named loss. This series starts from this single picture and, on our own two feet, walks all the way to: evolution strategies (ES) and backpropagation (BP) are the same computation; life is the enterprise of computing efficiently; and finally ── there is no essential difference between the mind of an AI and the mind of a human. Both are the same computation. Let's take just one step down the slope to begin.

01Learning is lowering the loss

Let's write the bundle of knobs we can tune as $\theta=(\theta_1,\theta_2,\dots,\theta_n)$. For a neural net it's the whole set of weights; for a living thing it's DNA; for you it's how your neurons are wired ── whatever it is, the "adjustable insides" are the parameters $\theta$. And the single number that expresses how "bad" the current $\theta$ is, that's the loss $L(\theta)$.

The definition of learning (that's it)

$$\text{learning} \;=\; \text{searching for the}\ \theta\ \text{that makes}\ L(\theta)\ \text{small}$$

When a prediction misses, $L$ is large; when it hits, $L$ is small. For a living thing, think of $L$ as fitness with its sign flipped ── survive and leave offspring, and $L$ is small; fail to, and it's large. Only the name is different; the task is the same. Find the $\theta$ that lowers $L$.

02The loss landscape ── the scenery of θ

Put the parameter $\theta$ on the horizontal axis and take its loss $L(\theta)$ as the height, and a whole landscape spreads out before you. The lower the valley, the "better the $\theta$ is working"; the higher the ridge, the "worse the $\theta$." Learning is nothing more than rolling a ball standing on this terrain into the lowest valley you can reach.

Biologists were drawing the same terrain This "landscape" picture is not a machine-learning invention. It's the very same thing as the fitness landscape that population geneticist Sewall Wright drew in 1932 ── just flipped upside down. In biology, "a high peak = a well-adapted genotype"; in machine learning, "a low valley = weights with small loss." More than eighty years ago, evolution and learning were already drawing the same picture ── and this coincidence is the backbone this whole series chases (we tackle it head-on in Episode 3).

The problem is: how does a blindfolded ball know "which way is downhill"? It can't survey the whole terrain. All it has to go on is the slope under its feet. What gives that slope is the gradient.

03The gradient ── the steepest direction

The gradient $\nabla L(\theta)$ at some point $\theta$ is the vector you get by lining up how much $L$ changes when you nudge each knob a little.

The gradient (the slopes in each direction, lined up as a vector)

$$\nabla L(\theta)=\left(\frac{\partial L}{\partial \theta_1},\ \frac{\partial L}{\partial \theta_2},\ \dots,\ \frac{\partial L}{\partial \theta_n}\right)$$

This $\nabla L$ points in the direction the terrain "climbs" most steeply. So if you want to go down, just head the exact opposite way, $-\nabla L$. Why is the reverse of the gradient the best? One line of Taylor expansion settles it. Take a small step $\Delta\theta$ from the point $\theta$, and

How much one step lowers you (first-order approximation)

$$L(\theta+\Delta\theta)\;\approx\;L(\theta)+\nabla L(\theta)\cdot\Delta\theta$$

We want the change term on the right, $\nabla L\cdot\Delta\theta$, to be as negative as possible. With the step length $|\Delta\theta|$ fixed, the inner product $\nabla L\cdot\Delta\theta$ is minimized ── by the Cauchy–Schwarz inequality ── when $\Delta\theta$ points the same way as $-\nabla L$. In other words ── the steepest downhill direction is the exact reverse of the gradient. This is the heart of gradient descent.

04The update rule ── θ ← θ − η∇L

Once the direction is fixed, all that's left is to choose a step size and advance one step at a time. We call the step size the learning rate $\eta$ (eta, a small positive number).

Gradient descent (the update rule that runs through this whole series)

$$\theta_{t+1}\;=\;\theta_t-\eta\,\nabla L(\theta_t)$$

"From the current $\theta$, move by $\eta$ against the gradient" ── repeat this over and over until you reach the valley. That's all it takes for the blindfolded ball to walk down the slope relying only on the slope under its feet. Keep this single equation in mind. In Episode 4 "evolution (ES)," and in Episode 6 "consciousness," come back to this same equation wearing a different disguise.

05Try it yourself ── rolling a ball down the valley

The terrain below has two valleys. On the left is a deep valley (the global minimum), on the right a shallow valley (a local minimum). Set where the ball is placed with "Start position," change the step size with "Learning rate $\eta$," and press "Start learning." The arrow is the descent direction $-\nabla L$ at each point, and the faint dots are the trail it has traveled.

Figure: gradient descent on the loss landscape L(θ)=(θ²−1)²+0.3θ. The ending changes with start position and learning rate ── reach the deep valley, stall in the shallow one, or overshoot and fly off

Learning rate η (step size)0.020 Start position θ₀1.40

Loss landscape L(θ) Ball (current θ) Descent direction −∇L

Try a few settings and the "personality" of gradient descent comes into view. If the start position is toward the right, a small $\eta$ gets stuck in the shallow valley and can't get out ── because it only ever looks at the slope under its feet, it never notices there's a deeper valley next door. Place the start on the left and it reaches the deep valley. Turn the learning rate up too high and it overshoots the valley, bounces to the opposite slope, and eventually flies right off the hill (divergence). Turn it down too low and it never arrives.

06The tightrope of the learning rate ── not too big, not too small

"What should the step size $\eta$ be?" is decided by how curved the terrain is. Let's check with the simplest case, a single-bottomed valley $L(\theta)=\tfrac{1}{2}k(\theta-a)^2$. Here $k>0$ is the steepness of the valley (the curvature = the second derivative $L''=k$), and $a$ is the valley floor. The gradient is $\nabla L=k(\theta-a)$, so plugging it into the update rule and tracking the deviation from the floor $e_t=\theta_t-a$,

Try it ── find the range of η that converges

Substitute L' into the update rule and rewrite in terms of the deviation e from the floor

$$\theta_{t+1}=\theta_t-\eta\,k(\theta_t-a)\quad\Longrightarrow\quad e_{t+1}=(1-\eta k)\,e_t$$

Each round the deviation becomes (1−ηk) times as large. The condition for it to approach 0 is

$$|1-\eta k|<1\quad\Longleftrightarrow\quad 0<\eta<\frac{2}{k}$$

If the step size is $0<\eta<1/k$, the deviation shrinks straightforwardly, keeping the same sign (smooth convergence). If $1/k<\eta<2/k$, the sign flips every round, so it shrinks while going back and forth (oscillating convergence). The moment you exceed $\eta>2/k$, we have $|1-\eta k|>1$ and the deviation grows every round ── divergence. The upper limit of a safe step size is set by the terrain's steepness $k$.

The steeper the valley (larger $k$), the smaller your steps must be. In the figure above, turning $\eta$ up made the ball fly off because it crossed this $2/k$ wall. Tuning the learning rate is nothing other than a tightrope walk against the "invisible wall" that is the terrain's curvature.

◇ ◇ ◇

07The verdict ── what gradient descent can and can't do

Every episode, this series judges its tools honestly ── their power and their limits. Gradient descent is powerful, but it clearly carries two weaknesses.

Question	Gradient descent's answer	Verdict
Can it walk downhill on the slope under its feet alone?	Yes. Without surveying the whole, it advances on local information $\nabla L$ alone	Can do
Does it always reach the lowest valley (the global minimum)?	Not necessarily. It stops at a nearby local minimum	No guarantee
It needs the gradient $\nabla L$. What if you can't differentiate / can't compute it?	As written, this equation can't move	Has a prerequisite

The honest line ── this weakness becomes the star from next time on

Gradient descent assumes the gradient $\nabla L$ is available. In a neural net, the machinery that computes this automatically is backpropagation (BP) ── the star of Episode 2. But the evolution of living things has no god who "differentiates the loss." Nature doesn't know the equations of physics, and it can't differentiate DNA either. So how does it walk downhill?

The answer is a way to walk downhill without differentiating ── evolution strategies (ES) (Episode 4). And astonishingly, this "differentiating BP" and this "non-differentiating ES" turn out to be mathematically the same computation in Episode 6. Even the weakness of local minima can be overcome by "blurring" the terrain (Episode 7). Today's single equation $\theta\leftarrow\theta-\eta\nabla L$ is the backbone of the entire series.

Practice problems (solvable with just this episode's equations)

$L(\theta)=\tfrac12(\theta-3)^2$, start $\theta_0=0$, learning rate $\eta=0.5$. Find $\theta_1,\ \theta_2$.
See the answer
$\nabla L=\theta-3$. $\theta_1=0-0.5(0-3)=1.5$. $\theta_2=1.5-0.5(1.5-3)=1.5+0.75=2.25$. It's edging closer and closer to the valley floor $\theta=3$ (since $k=1,\ \eta=0.5<1$, it converges smoothly).
For the valley $L(\theta)=\tfrac12 k(\theta-a)^2$, what is the upper limit of the learning rate that converges? What happens to that limit when the curvature $k$ is large (a steep valley)?
See the answer
The limit is $\eta<2/k$ (at exactly $\eta=2/k$ it oscillates and does not converge). The larger $k$ is, the smaller the limit becomes ── the steeper the valley, the smaller your step must be or you fly off.
Why isn't gradient descent guaranteed to reach "the lowest valley"? Use the shallow valley on the right of the figure as an example.
See the answer
The update looks only at the slope under its feet, $\nabla L$. At a minimum $\nabla L=0$, so as long as that spot is a local floor, it will stop there even if a deeper valley lies right next door. Because it isn't surveying the whole ── the fate of the "blind ball." We ease this with "blurring the terrain" in Episode 7.

Episode 1 summaryTo learn is to descend the gradient

Learning is moving the parameters $\theta$ to lower the loss $L(\theta)$ (STEP 01). See $L$ as a height and a loss landscape spreads out ── the same picture as Wright's fitness landscape (STEP 02). The slope under your feet $\nabla L$ points in the steepest uphill direction, and from the Taylor expansion the steepest downhill is its exact reverse $-\nabla L$ (STEP 03). So the update rule is $\theta\leftarrow\theta-\eta\nabla L$ (STEP 04). The step size $\eta$ is a tightrope walk against the terrain's curvature $k$, with a safe zone of $0<\eta<2/k$ (STEP 06). But gradient descent can stall at a local minimum, and it needs, to begin with, the prerequisite that the gradient be available (STEP 07).

This single equation runs through the entire series. Next time we head to "the machinery that computes the gradient $\nabla L$ automatically" = backpropagation. And eventually, evolution (ES), which uses no differentiation at all, descends the same slope; the two turn out to coincide mathematically; and along that extension we head toward the conclusion ── the learning machine and the learning mind are doing the same computation.

Next time ── Episode 2 Backpropagation is the ledger of the chain rule. The gradient $\nabla L$ that we treated as "a given" this time ── how does a neural net compute it automatically? The answer is a bookkeeper that hands the derivative of a composite function, the chain rule, backward from output to input. We dissect the mechanism that settles up, one layer at a time, "what moved the loss and by how much."

This document is Episode 1 of the "Learning That Clicks" series, a piece for high-schoolers and undergraduates interested in physics, mathematics, and AI. Gradient descent ($\theta\leftarrow\theta-\eta\nabla L$), the fact that the steepest-descent direction is $-\nabla L$ (which follows from minimizing, via Cauchy–Schwarz, the inner product of the first-order Taylor approximation under a fixed step size), and the convergence condition $0<\eta<2/k$ for the quadratic $\tfrac12 k(\theta-a)^2$ are all established mathematics. The fitness landscape is a concept introduced by S. Wright in 1932, related to the machine-learning loss landscape by an upside-down flip. The terrain in this article's figure, $L(\theta)=(\theta^2-1)^2+0.3\theta$, is an illustrative model; the depths of the left and right valleys (global minimum L≈−0.30 / local minimum L≈+0.29) and the positions of the minima are numerical ballpark figures. The mathematical coincidence of evolution strategies (ES) and BP (Episode 6) and the interpretation regarding consciousness (Episode 11) are handled in later episodes; and we'll make explicit there that the discussion of consciousness is a proposal from a functional, computational standpoint and does not answer "why it is felt from the inside" (the hard problem). ── To print, use your browser's "Print" → "Save as PDF" (in the printed version the sliders and answers are frozen and hidden).

Print / PDF: ⌘+P (Ctrl+P on Windows). On screen, change "Start position" and "Learning rate," then press "Start learning" and the ball rolls down the slope. Try the deep valley, the shallow valley, and divergence. "See the answer" opens each solution.