Learning That ClicksEpisode 9 / The Landauer principle and bits per joule

Reading evolution, learning, and consciousness through a single gradient

Life Is a Race to Compute Efficiently Erase a single bit of information, and the universe charges you at least $k_BT\ln 2$ in heat. Evaluation, selection, the firing of a neuron ── all are computation, and
computation demands energy. So life is exactly this: a contest over how many bits of adaptation you can squeeze out of one joule.

Tools you'll need: logarithms and exponentials, a taste of thermodynamics ($k_BT$) Key of this episode: $E_{\min}=k_BT\ln 2$

Last time, we looked at the biosphere as one enormous computer ── a GPU that has been running for billions of years. But is that computer running for free? No. Physics is harsher than that. Processing information always costs energy. And this cost has a minimum price set by the universe itself ── the Landauer principle. Erasing one bit of information dumps at least $k_BT\ln 2$ of energy away as heat. At room temperature that is a mere $2.9\times10^{-21}$ joules, but it is not zero. And with that, the face of life changes completely. Where to hunt for food, which genes to pass on, how to take the next step ── every one of these is "computation that measures a gradient," and because it is computation, it must pay a thermodynamic cost. Which is why natural selection has quietly judged organisms along one more axis. Whoever can squeeze more "information that matters for adaptation" out of the same one joule survives. Today we reread life as a contest in computational efficiency, using this yardstick of bits per joule ── information per joule.

01Computation Is Not Free

We tend to think of computation as an abstract event "inside the head." An equation is complete on paper; thought lies outside the laws of physics. And yet, in the mid-twentieth century, physicists stumbled onto a strange question ── physically, how much does computation actually "cost"? Flipping a switch, overwriting a bit, forgetting information you no longer need ── these are all physical processes within this universe. Atoms move, electrons flow, heat is generated. So there must be a price tag attached.

The answer came in 1961, from Rolf Landauer at IBM. What he found was the part of computation's price tag that can never be reduced to zero. No matter how cleverly you design it, no matter what future technology you use, there is a floor below which it cannot get cheaper. And its source, surprisingly, lay hidden in the operation of "erasing."

02The Landauer Principle ── Forgetting Requires Heat

Landauer's insight was this. The one thing in computation whose energy cost is truly unavoidable is a logically irreversible operation ── above all, the erasure of information. Take a bit you didn't know was 0 or 1, and crush it so that it "is guaranteed to become 0." This operation folds two possible states into one. Information decreases by one bit. And then the second law of thermodynamics demands that the entropy you just lost be expelled somewhere. The only place it can go is the surrounding heat bath ── in other words, it must be thrown out as heat. The minimum price is the following equation.

The Landauer principle (minimum energy to erase one bit)

$$E_{\min}\;=\;k_B\,T\,\ln 2$$

Here $k_B=1.38\times10^{-23}\ \mathrm{J/K}$ is Boltzmann's constant, $T$ is the absolute temperature, and $\ln 2\approx0.693$ is the coefficient that translates "two possibilities → one," a factor-of-two count of cases, into information. The higher the temperature, and the more information you erase, the more heat you pay. Let's plug in numbers at room temperature, $T=300\,\mathrm{K}$.

Let's try it ── how much for one bit at room temperature

(1) Compute k_B T ln2 numerically at T=300 K

$$E_{\min}=(1.38\times10^{-23})\times 300 \times 0.693 \;\approx\; 2.9\times10^{-21}\ \mathrm{J}$$

Converting to electron volts (1 eV = 1.602×10⁻¹⁹ J)

$$2.9\times10^{-21}\ \mathrm{J}\;\div\;1.602\times10^{-19}\ \mathrm{J/eV}\;\approx\;0.018\ \mathrm{eV/bit}$$

Forgetting one bit costs, at room temperature, about $2.9\times10^{-21}$ joules ── dizzyingly small. But it is by no means zero. This is the "floor of computation." Put the other way: erase $10^{21}$ bits and you finally reach a few joules ── enough heat to warm a glass of water just a touch. Little by little, it adds up.

A bridge to the sister series ── Cosmology That Clicks, Episode 10 The very same equation $k_BT\ln 2$ also appears in "Cosmology That Clicks," Episode 10 of our sister series. Its theme there is "why can we describe black holes and the whole universe in terms of information?" ── and its starting point was exactly this: "erasing one bit = $k_BT\ln 2$." That information and thermodynamics form one continuous landscape is one of the greatest discoveries of twentieth-century physics. Cosmology uses this equation at the scale of the universe; the theory of learning uses it at the scale of life. We are crossing one and the same bridge from both sides ── the universe's law that erasing information always produces heat applies equally to stars and to cells.

03Life Processes Information ── So It Pays the Cost

Here we connect to the view that runs through this whole series. Since Episode 1 we have seen both learning and evolution as computation that "descends the slope of a loss = moves by measuring a gradient." If so, then the life that carries out that computation cannot escape Landauer's price tag. This is not an abstraction. The everyday activities of a living thing are, quite literally, physical information processing.

Evaluating ── Is this food safe? Is this rival strong? Folding sensory input into "good / bad" is computation that involves compressing information = erasure.
Selecting ── Natural selection itself is a vast information process that narrows down survivors from a "bundle of candidates," the diverse individuals. The death of an individual is an operation that rewrites the information held by the population.
Computing with neurons ── The firing of neurons and the weighting of synapses is, literally, electrochemical computation. And it comes with measurable energy consumption.

And this cost is not armchair theory; it has left its mark on the design of living things. The proofreading machinery of DNA replication detects copying errors, erases them, and replaces them with the correct base ── a device that erases and rewrites information, burning extra energy (ATP) for the sake of accuracy. The same goes for the energy-thrifty design of neural spikes. In carrying information, the brain has been honed over evolution to move only the bare minimum of ions. Precisely because computation is costly, the tricks for computing cheaply were the ones selected.

04Computational Efficiency Under Selection ── Bits per Joule

Now that we have come this far, we can write down one more of natural selection's yardsticks. To survive, an organism must obtain gradient information about its environment ── "which way should I move to raise my fitness?" But acquiring and processing information costs energy. And energy is finite ── there's a limit to how much you can eat. So the deciding factor can be written like this.

The efficiency yardstick ── information per joule

$$\eta_{\text{info}}\;=\;\frac{\text{information gained that matters for adaptation [bit]}}{\text{energy spent [J]}}\quad(\text{bits per joule})$$

From the same amount of food (the same joules), an individual that can squeeze out more "information that matters for adaptation" can act correctly on fewer resources. That difference piles up across generations. So natural selection quietly judges not only how high your fitness is, but also at what price you computed that fitness. It is not the individual that runs fast, but the one that runs the same distance on fewer calories, that gets selected. The brain that can reach the same decision with less neural activity gets selected. Computational efficiency, too, is a target of selection.

This viewpoint reminds us of the weakness of evolution strategies (ES) we saw a couple of episodes back. ES could estimate a gradient only by "letting one sample = one individual actually live, then measuring its fitness." The cost of one sample is the energy of one individual's entire lifetime, and one death. To gain a single bit of gradient, life burns a staggering quantity of joules. This "computation that is too expensive" is exactly the setup for next time.

05Let's Run It ── Comparing the Cost to Arrive

The figure below shows how many joules a strategy burns cumulatively to close in on a certain goal of adaptation (the "progress toward the goal" on the horizontal axis). The vertical axis takes cumulative energy in units of $k_BT$, on a logarithmic scale (biological costs and the Landauer limit are many orders of magnitude apart, so without a log scale they wouldn't fit in one figure). It is a simple model: the "adaptive information" required to reach progress $p$ is $I(p)=-\log_2(1-p)$ bits, and the cost per bit differs from strategy to strategy.

Figure: cumulative energy needed to reach the goal (90% progress). Green = an efficient strategy (brain-like computation, ~10⁶ k_BT/bit), amber = the sample-inefficient ES (varies with the slider). Gray dashed = the Landauer limit (k_BT ln2/bit, the floor of physics). The more efficient the strategy, the lower (less energy) it achieves the same arrival.

ES inefficiency (cost per bit)1.0×10¹² k_BT/bit

Efficient strategy (learning-like) ES (evolution, sample-inefficient) Landauer limit (floor)

Drag the slider for ES inefficiency (cost per bit) and the amber curve moves up and down. No matter how efficient you make it, no curve comes anywhere near the gray floor (the Landauer limit) ── this is the heart of today's verdict. At the same time, green (the efficient strategy) runs far below amber (ES). It reaches the same level of progress with orders of magnitude less energy. This "gap between green and amber" is exactly the room natural selection takes aim at, and it is the true nature of next time's theme ── learning, the cheap shortcut.

06The Marvel of the Brain ── Far from the Limit, Yet Wildly Smart

So where does the finest computer in existence ── the human brain ── operate, relative to this floor? The brain consumes roughly $20\ \mathrm{W}$, that is, twenty joules per second. If it could spend all of that energy on "bit erasure" right at the Landauer limit, how many bits could it erase per second at most? Let's estimate.

Let's try it ── the brain's Landauer ceiling, and its distance from the limit

(2) Upper bound on bits per second if all 20 W went into erasure computation

$$\frac{20\ \mathrm{J/s}}{2.9\times10^{-21}\ \mathrm{J/bit}}\;\approx\;6.9\times10^{21}\ \mathrm{bit/s}$$

Compared with the brain's actual operation count (roughly ~10¹⁶ ops/second)

$$\frac{20\ \mathrm{J/s}}{10^{16}\ \text{ops/s}}\;=\;2\times10^{-15}\ \mathrm{J/op}\;=\;\frac{2\times10^{-15}}{4.14\times10^{-21}}\,k_BT\;\approx\;5\times10^{5}\,k_BT/\text{op}$$

The physical floor permits an upper bound of $6.9\times10^{21}$ bits per second. Yet the brain actually performs only about $\sim10^{16}$ operations per second ── less than a millionth of that ceiling. Turned around: per operation, the brain uses not $k_BT\ln 2$ (= the floor for one bit) but $10^4$ to $10^6$ times that much energy. It sits many orders of magnitude above the Landauer limit. There is no contradiction ── the fact that the ceiling is $6.9\times10^{21}$ bit/s while actual operations stay at $10^{16}$/s is just another way of saying "it uses a large amount of $k_BT$ per operation = it is far from the limit."

Even so, the brain is a marvel. That figure of $\sim10^5\,k_BT$ per operation is often orders of magnitude better than the energy today's digital computers spend on comparable operations. As a living thing it is unbelievably energy-thrifty, while from the standpoint of the physical floor it is unbelievably wasteful ── the two are perfectly compatible. Life has not saturated the limit. But it has, without doubt, been honed in the direction of the limit. The proofreading machinery, the energy-thrifty neurons ── these are the marks of that honing.

◇ ◇ ◇

07The Verdict ── Where Physics Ends and Story Begins

Today we judge the established physics and the interpretation laid on top of it clearly apart. Mix them, and a beautiful story becomes a lie.

Claim	Substance	Verdict
Is the Landauer principle correct?	Logically irreversible erasure of one bit requires at least $k_BT\ln 2$ of energy. Established physics, both theoretically and experimentally.	◯ Established
Is computational efficiency under selection pressure?	Metabolic costs are real, and there are marks of honed efficiency such as proofreading machinery and energy-thrifty neurons. The direction "efficiency, too, is selected for" is sound.	◯ Sound
Does life operate near the Landauer limit?	It does not. Even the brain uses $10^4$ to $10^6\,k_BT$ per operation, many orders of magnitude from the limit. "Saturates the limit" does not hold.	× Does not hold

The honest line ── "efficiency is selected" and "saturates the limit" are two different things

The Landauer principle is solid physics. And the claim about the direction ── that "computational efficiency is selected for" ── is sound too. Metabolic costs are real, and there are concrete examples of efficiency being selected, such as DNA proofreading and energy-thrifty neurons. Up to here, we can say it with our heads held high.

But to step one pace further and declare outright that "life operates near the computational and thermodynamic optimum" is not yet established ── indeed, as we saw in the main text, organisms are many orders of magnitude away from the Landauer limit. So we draw the line honestly. The claim that is interesting and defensible is "efficiency is selected for (honed in the direction of the limit)" ── not "it saturates the limit." Not confusing these two is where integrity divides. ── And that story about "efficiency" leads straight into next time. ES (evolution) pays one lifetime and one death for a single bit of gradient. Is there no invention that computes the same gradient more cheaply? There is. It is called learning.

Practice problems (solvable with today's equations and numbers alone)

At a temperature of $T=310\ \mathrm{K}$ (about human body temperature), what is the minimum energy $k_BT\ln 2$ to erase one bit? How does it compare with the $2.9\times10^{-21}\ \mathrm{J}$ at room temperature, $300\ \mathrm{K}$?
Show the answer
$E_{\min}=1.38\times10^{-23}\times310\times0.693\approx2.97\times10^{-21}\ \mathrm{J}$. That is about $3.0\times10^{-21}\ \mathrm{J}$, a factor of $310/300\approx1.03$ larger than at $300\ \mathrm{K}$ ── roughly 3% more. Because it is proportional to temperature, the warmer it is, the higher the minimum cost to erase one bit.
Raising the temperature of one gram of water by one degree takes about $4.2\ \mathrm{J}$. If you were to generate that $4.2\ \mathrm{J}$ of heat "by bit erasure alone," how many bits at least must you erase at room temperature?
Show the answer
$4.2\ \mathrm{J}\div 2.9\times10^{-21}\ \mathrm{J/bit}\approx1.4\times10^{21}\ \mathrm{bit}$. Even if the floor for one bit is tiny, pile up $10^{21}$ bits and it becomes an everyday amount of energy ── a felt sense that information erasure really is physical heat. Note too that it is close in order of magnitude to the brain's ceiling from Section 5, $6.9\times10^{21}\ \mathrm{bit/s}$.
Suppose the brain uses about $5\times10^{5}\,k_BT$ per operation. How many times the Landauer limit (one-bit erasure $=\ln 2\,k_BT\approx0.69\,k_BT$) is this? Does this number support or refute the claim that "life operates near the limit"?
Show the answer
$5\times10^{5}/0.69\approx7\times10^{5}$, i.e. about 700,000 times. That is 5 to 6 orders of magnitude above the limit. So the strong claim "operates near the limit" is refuted. What we can support is the weaker (and correct) directional claim: "efficiency has been honed in the direction of the limit."

Episode 9 SummaryLife is a race to squeeze information out of a joule

Computation is not free (STEP 01). By the Landauer principle, erasing one bit requires at least $E_{\min}=k_BT\ln 2$, which at room temperature is $2.9\times10^{-21}\ \mathrm{J}\approx0.018\ \mathrm{eV}$ (STEP 02, the same bridge as Episode 10 of the sister series "Cosmology That Clicks"). Evaluation, selection, and neural computation are all information processing, and life bears this cost ── DNA proofreading and energy-thrifty neurons are the marks of it (STEP 03). So selection also judges organisms along the efficiency axis of bits per joule (STEP 04). As the figure showed, an efficient strategy achieves the same arrival with less energy (STEP 05). The brain runs on $20\ \mathrm{W}$; at the Landauer ceiling it should be able to erase $6.9\times10^{21}$ bits per second, yet in reality it uses $10^4$ to $10^6\,k_BT$ per operation ── many orders of magnitude from the limit (STEP 06). Hence the verdict: the limit is established (◯), the direction "efficiency is selected for" is sound (◯), but saturation of the limit does not hold (×) (STEP 07).

Life does not reach the physical floor. But it has, without doubt, been descending the slope toward that floor. Next time is the story of the invention that descends that slope the most cheaply of all ── evolution pays one lifetime and one death for a single bit of gradient. A computation method orders of magnitude cheaper than that exists in this universe.

Next time ── Episode 10 Learning is a shortcut evolution discovered. To gain a single bit of gradient, evolution strategies (ES) pay the energy of one individual's whole lifetime, and one death ── an all-too-expensive computation. What if you could test "what happens if I fail" in a model inside your head, without actually dying? What if you could squeeze a large amount of gradient information out of a single experience? That is the invention called learning. We dissect the overwhelmingly cheap shortcut for computing gradients that evolution took billions of years to reach.

This document is Episode 9 of the "Learning That Clicks" series, a piece of reading for high-school and university students interested in physics, mathematics, and AI. The Landauer principle (logically irreversible erasure of one bit requires at least $E_{\min}=k_BT\ln 2$ of energy, dissipated as heat) is established physics, derived by R. Landauer in 1961; at room temperature $T=300\ \mathrm{K}$, $k_BT\ln 2=(1.38\times10^{-23})\times300\times0.693\approx2.9\times10^{-21}\ \mathrm{J}\approx0.018\ \mathrm{eV/bit}$ ($k_B$ is Boltzmann's constant; 1 eV = 1.602×10⁻¹⁹ J). The estimates for the brain ── consumption $\approx20\ \mathrm{W}$ and $\sim10^{16}$ operations/second, roughly $10^4$ to $10^6\,k_BT$ per operation, and the Landauer ceiling $20/2.9\times10^{-21}\approx6.9\times10^{21}\ \mathrm{bit/s}$ ── are all order-of-magnitude guides; the figures shift depending on how one defines a brain "operation" and how one measures power consumption. The figure in this article is an illustrative model that simplifies the required information to reach progress $p$ as $I(p)=-\log_2(1-p)$ bits and a strategy's cost as "a constant number of $k_BT$ per bit"; values such as efficient strategy ~10⁶ k_BT/bit and ES 10⁸–10¹⁸ k_BT/bit are examples meant to convey the concept. The directional claim that "computational efficiency is under selection pressure (honed in the direction of the limit)" is sound, but the strong claim that "life operates near the computational and thermodynamic optimum (saturates the limit)" is not established, and organisms are in fact many orders of magnitude from the Landauer limit. This line was made explicit in the verdict table and the "honest line" in the main text. Episode 10 of the sister series "Cosmology That Clicks" treats the same $k_BT\ln 2$ at the scale of the universe. The relationship between evolution strategies (ES) and learning is taken up head-on next time (Episode 10). ── To print, use your browser's "Print" and "Save as PDF" (in the print version, the sliders and answers are frozen and hidden).

Print / save as PDF: ⌘+P (on Windows, Ctrl+P). On screen, drag the "ES inefficiency" slider and the amber curve (the ES cumulative energy) moves up and down. Confirm that no matter how much you improve efficiency, it never reaches the gray floor (the Landauer limit), and that green (the efficient strategy) always runs below. "Show the answer" opens each solution.