Learning That ClicksEpisode 5 / The infinitesimal as a tool ── making the sleight of hand of limits rigorous

Reading evolution, learning, and consciousness through a single gradient

Counting Infinity Honestly "Let it approach 0 without bound" ── have you ever felt a twinge of guilt at that turn of phrase?
Accept the infinitesimal ε ── a number that is neither 0 nor finite ── as a genuine number you can hold in your hand. Then differentiation returns to division itself, with no ritual of taking limits.

Tools you'll need: the "infinite population N→∞" from Episode 4, and the definition of the derivative Key idea this episode: $f'(x)=\operatorname{st}\!\big(\tfrac{f(x+\varepsilon)-f(x)}{\varepsilon}\big)$

Last time (Episode 4), when we talked about evolution strategies (ES), we used ── hesitantly ── phrases like "an infinite population $N\to\infty$" and "probing the infinitesimal neighborhood, extremely close to $\theta$." Convenient, yes, but with a faintly illicit ring to them ── does a "population" of $N\to\infty$ really exist? An "infinitesimal neighborhood" is, in the end, just the limit $\varepsilon\to 0$, so shouldn't we simply write the limit sign honestly? This time we answer that guilty conscience head-on, from the mathematics side. The tool that lets you genuinely pick up the infinitesimal $\varepsilon$ as a single "number" that is neither 0 nor finite ── that is non-standard analysis. Once you have it, differentiation is no longer the time-consuming ritual of "approaching without bound"; it becomes a single-step operation: divide by an infinitesimal, then read off only the real part of the answer. The "infinite population" of Episode 4, and the "average of $\infty$ samples = differentiation" that we'll shake hands with in Episode 6, can finally be spoken of here without any sleight of hand.

01The guilt hidden in "approaching without bound"

When you first learned differentiation, this is surely how you were taught. If you want to know a slope, build the slope of the secant joining two points, $\dfrac{f(x+h)-f(x)}{h}$, and let $h$ "approach 0 without bound." Then the secant morphs into the tangent.

The textbook derivative (definition via limits)

$$f'(x)\;=\;\lim_{h\to 0}\frac{f(x+h)-f(x)}{h}$$

There is a small dishonesty here. During the calculation, $h$ must not be 0 ── because you cannot divide by 0. Yet at the very last moment, when you take the limit, you want $h$ to be 0 itself. You are quietly using a can't-decide-which entity: "an $h$ that is not 0, but as close to 0 as you like." In the 18th century, Leibniz and Euler called this an infinitesimal and calculated with it as a "number" without hesitation. In 1734 the philosopher Berkeley skewered it as "the ghosts of departed quantities." In the 19th century, Weierstrass and others invented the $\varepsilon\text{-}\delta$ method to banish this ghost, demoting the infinitesimal to a mere figure of speech and thereby making analysis rigorous. The limit sign $\lim$ is precisely the convention that lets us "do without infinitesimals."

Can't we make the ghost real, then? Weierstrass's road bought rigor by "erasing the infinitesimal." But a question remains ── can we accept the infinitesimal itself, consistently, as a number? The logician Abraham Robinson answered yes. In 1960, using mathematical logic (model theory), he extended the real field $\mathbb{R}$ and constructed ── without a single contradiction ── a new number system that genuinely contains infinitesimals and infinities. Leibniz's intuition, 300 years on, was granted rigorous citizenship.

02The hyperreal field *ℝ ── holding infinitesimals and infinities as numbers

The number system Robinson built is called the hyperreal numbers ${}^{*}\mathbb{R}$. It contains the entire real field $\mathbb{R}$, and onto it welcomes new residents of two kinds ── a larger number line. Addition, subtraction, multiplication, division, and comparisons all work freely by the same rules as the reals (it keeps its properties as a field). The new residents are ──

Definition of the infinitesimal ε (a positive number smaller than every positive real)

$$\varepsilon>0\quad\text{and}\quad \varepsilon<\frac{1}{n}\ \ \text{(for every natural number } n=1,2,3,\dots\text{)}$$

For an ordinary real $r>0$, however small it is, there is always some larger $1/n$ to be found (the Archimedean property). But $\varepsilon$ is smaller than all of $1,\ \tfrac12,\ \tfrac13,\ \dots$. It is not 0 ($\varepsilon>0$), yet it is too small to be measured by any finite ruler ── this is the infinitesimal. And taking its reciprocal,

The infinity H ── the reciprocal of the infinitesimal

$$H=\frac{1}{\varepsilon}\quad\Longrightarrow\quad H>n\ \ \text{(larger than every natural number } n \text{)}$$

If $\varepsilon$ is smaller than $1/n$, then taking reciprocals, $H=1/\varepsilon$ is larger than every $n$ ── an infinite hyperreal. This is the decisive difference. $\infty$ is a symbol (a motion, "growing without bound") and not a number, whereas $H$ is one number that actually exists inside ${}^{*}\mathbb{R}$, and you can carry out calculations like $H+1$, $2H$, $H^2$, $\sqrt{H}$, all of them. The "population of $N\to\infty$" we wrote so haltingly in Episode 4 can from now on be named as a single, motionless number: $N=H$, a population of infinite size.

03The standard part st(x) ── the one and only real infinitely close

If a hyperreal $x$ is finite (not infinite), then there exists exactly one real number infinitely close to it. The operation that extracts it is called the standard part, written $\operatorname{st}(x)$. First let us define "infinitely close."

Infinitely close (≈) and the standard part st

$$x\approx y\ \ \overset{\text{def}}{\Longleftrightarrow}\ \ x-y\ \text{is infinitesimal}\qquad\qquad \operatorname{st}(x)=\big(\,\text{the one and only real infinitely close to }x\,\big)$$

For example, $3+\varepsilon$ differs from the real $3$ by only an infinitesimal. So $3+\varepsilon\approx 3$, and $\operatorname{st}(3+\varepsilon)=3$. Think of the standard part as the operation that brushes the "dust of infinitesimals" off a hyperreal and rounds it to the nearest real. $\operatorname{st}(7-2\varepsilon)=7$, $\operatorname{st}(\varepsilon)=0$, $\operatorname{st}(5)=5$. Every finite hyperreal splits uniquely into the form $\big(\text{real}\big)+\big(\text{infinitesimal}\big)$, and $\operatorname{st}$ pulls out that real part.

monad ── the infinitesimal mist around a point The cloud of all hyperreals infinitely close to a single point $x$ is called the monad (the monad of $x$). $$\operatorname{monad}(x)=\{\,y\in{}^{*}\mathbb{R} : y\approx x\,\}=\{\,x+(\text{infinitesimal})\,\}$$ Picture, directly "above" each point of the real line, a mist of infinitesimals so thin it is invisible. Wherever in the mist you stand, $\operatorname{st}$ points to the one and only real at the mist's center. The "neighborhood" from Episode 4, where we wrote "probe the infinitesimal neighborhood of $\theta$," is exactly this $\operatorname{monad}(\theta)$. ES was, in effect, scattering samples across the monad of $\theta$ and reading the slope of the hill from their responses.

04Returning differentiation to honest division

The tools are in place. We may divide by the infinitesimal $\varepsilon$ (a nonzero number!), and at the end round to a real with $\operatorname{st}$. Then, from the definition of the derivative, the "approaching motion" of $\lim$ vanishes, and it becomes a static two-step calculation: divide, then round.

The honest definition of the derivative (non-standard version)

$$f'(x)\;=\;\operatorname{st}\!\left(\frac{f(x+\varepsilon)-f(x)}{\varepsilon}\right)\qquad(\varepsilon\ \text{is any nonzero infinitesimal})$$

Here $\varepsilon\ne 0$, so the division $\dfrac{f(x+\varepsilon)-f(x)}{\varepsilon}$ is a bona fide division ── not "close to 0," but dividing by a genuinely nonzero number. Its result is a hyperreal of the form "$f'(x)$ + an infinitesimal," and at the end $\operatorname{st}$ sweeps away that infinitesimal and returns the real slope. The "ratio of infinitesimals" that Leibniz wanted to express with the symbol $\dfrac{dy}{dx}$ holds here as a literal fraction. As long as $f$ is differentiable (in the standard sense), the value of $\operatorname{st}$ is the same whichever infinitesimal $\varepsilon$ you choose ── the 300-year-old intuition and the modern $\varepsilon\text{-}\delta$ coincide perfectly here. In the next section, let's actually divide out $x^2$.

Let's try it ── dividing f(x)=x² by the infinitesimal ε

First, the secant slope (expand the numerator)

$$\frac{f(x+\varepsilon)-f(x)}{\varepsilon}=\frac{(x+\varepsilon)^2-x^2}{\varepsilon}=\frac{x^2+2x\varepsilon+\varepsilon^2-x^2}{\varepsilon}=\frac{2x\varepsilon+\varepsilon^2}{\varepsilon}$$

Since ε≠0, we can boldly cancel by ε

$$=\ 2x+\varepsilon$$

Finally take the standard part ── the infinitesimal ε drops out

$$f'(x)=\operatorname{st}(2x+\varepsilon)=2x$$

Two things to notice. First, the quadratic term $\varepsilon^2$ is an infinitesimal one order smaller than the others even before canceling; after dividing it appears as $\varepsilon$ and is dropped by $\operatorname{st}$ as a higher-order infinitesimal. This is the true identity of Leibniz's rule of thumb, "the square of an infinitesimal is negligible." Second, $\lim$ never appears anywhere along the way. Divide by a nonzero $\varepsilon$, then round with $\operatorname{st}$ at the end ── with just this, $(x^2)'=2x$ came out rigorously. The ritual of the limit has been absorbed into the single step of $\operatorname{st}$.

05Set it in motion ── the moment the secant morphs into the tangent

Let's see that calculation with our own eyes. On the parabola $f(x)=x^2$, fix a point $P=(x,\,x^2)$, and draw the secant $PQ$ joining it to the point $Q=(x+\varepsilon,\,(x+\varepsilon)^2)$ a distance $\varepsilon$ to the right. Its slope, as we just computed, is exactly $2x+\varepsilon$. As you shrink $\varepsilon$ with the slider (toward the "infinitesimal" side on the logarithmic scale), you can watch the secant's slope $2x+\varepsilon$ get sucked into the tangent's slope $2x$ ── that is, into $\operatorname{st}(2x+\varepsilon)=2x$.

Figure: on f(θ)=θ², fix a point P and draw the secant to a point Q at distance ε. The secant slope is exactly 2x+ε. Shrink ε toward the "infinitesimal" side, and the slope coincides with the tangent slope st=2x

Position of the point of tangency x1.00 Infinitesimal ε (log scale; the further left, the closer to 0)0.60000

parabola f(x)=x² point of tangency P (x fixed) nearby point Q (x+ε) secant PQ (slope 2x+ε) tangent (slope 2x = st)

Push $\varepsilon$ to the far left, and the blue point $Q$ sinks into the "monad" of the red point $P$ ── into the infinitesimal mist ── until the brown secant and the green tangent can no longer be told apart. The read-out slope $2x+\varepsilon$ is the standard part $2x$ offset by "just an infinitesimal." Move the $x$ slider and you can confirm that at every point of tangency, "secant slope $=2x+\varepsilon\ \to\ \operatorname{st}=2x$" holds just as before. The $\varepsilon$ on screen is, of course, a finite real number; but the single fact that the division never breaks down no matter how small you make it is a scale model of the proof that the infinitesimal $\varepsilon$ can behave as a number.

06The transfer principle ── why calculating with infinitesimals is correct

A natural anxiety arises here. We computed an answer using infinitesimals in the other world ${}^{*}\mathbb{R}$ ── so why may we trust it in our world of reals? What guarantees the crossing is the heart of non-standard analysis ── the transfer principle.

The transfer principle

$$\text{for a first-order statement }\varphi\quad \mathbb{R}\models\varphi\ \ \Longleftrightarrow\ \ {}^{*}\mathbb{R}\models\varphi$$

Put plainly ── a "first-order" statement that is true over the reals $\mathbb{R}$ is true, exactly as it stands, over the hyperreals ${}^{*}\mathbb{R}$, and vice versa. "First-order" means a statement that says "for all" or "there exists" about elements (numbers) (statements that quantify over whole sets as a unit are excluded). So the calculational laws that hold for the reals ── the commutative law, the distributive law, rules like "if $\varepsilon\ne 0$ then $\varepsilon/\varepsilon=1$" ── may be used as-is for the infinitesimal $\varepsilon$ too. The reason we could expand $(x+\varepsilon)^2$ and cancel by $\varepsilon$ back in Section 4 is exactly that this principle guarantees "the same algebraic rules as the reals are valid in ${}^{*}\mathbb{R}$."

A round-trip ticket ── this is why the conclusion is correct for the reals What matters is that the transfer principle is bidirectional (⟺). Take a problem about the reals, carry it into the convenient ${}^{*}\mathbb{R}$ where infinitesimals and infinities are available (the outbound trip), calculate there intuitively and easily, round back to the reals with $\operatorname{st}$ (the return trip). If the equality obtained is a first-order statement, the transfer principle guarantees it is true in the world of reals too. The infinitesimal is "a working scaffold that makes the calculation easy," and even after you remove the scaffold, the building (the theorem about the reals) still stands firm ── this is the trick behind "calculating with infinitesimals is correct in the standard world."

◇ ◇ ◇

07The verdict ── what non-standard analysis can and cannot do

Every episode of this series honestly judges a tool's power and its limits. The infinitesimal is alluring, but it is not magic. Let us make clear what it newly makes possible, and what it does not.

Question	Non-standard analysis's answer	Verdict
Can infinitesimals and infinities be handled rigorously as "numbers" without contradiction?	They can. ${}^{*}\mathbb{R}$ holds $\varepsilon,\,H=1/\varepsilon$ as numbers that actually exist, and the transfer principle guarantees the algebraic rules	yes
Can it give a rigorous footing to the intuitions "differentiation is a ratio of infinitesimals" and "integration is an infinite sum of infinitesimals"?	It can. $f'=\operatorname{st}\!\big(\tfrac{\Delta f}{\varepsilon}\big)$ and $\int=\operatorname{st}\!\big(\sum(\text{infinitesimals})\big)$ become theorems	yes
Can it prove new theorems that standard analysis ($\varepsilon\text{-}\delta$) cannot?	No. Since the transfer principle lets us go back and forth, the range of provable theorems about the reals is exactly the same ── all that differs is the choice of expression	equivalent; a difference of expression

The honest line ── not magic that births new physics

Let us be clear. Non-standard analysis neither replaces standard analysis ($\varepsilon\text{-}\delta$) nor is stronger than it. Because there is the round-trip ticket that is the transfer principle, the two are two correct foundations that prove the same theorems about the reals. Accepting infinitesimals does not suddenly make new physical laws spring forth ── that kind of magic does not happen. What non-standard analysis gives is not an increase in power but a lens for how we tell the story ── a lens that replaces the dynamic ritual of "approaching without bound" with the static, intuitive single step of "divide by an infinitesimal and take the standard part."

But for this series, that is enough. What we wanted was not new power but words for speaking honestly, without sleight of hand, about the "infinite population $N\to\infty$" and the "infinitesimal neighborhood" that we stumbled over in Episode 4. The population $N=H$, the search over $\operatorname{monad}(\theta)$, the $\operatorname{st}$ of an average ── all of these are now the language of rigorous mathematics. On this footing, next time, ES (the integral over $\infty$ samples) and BP (differentiation) will shake hands.

Practice problems (solvable with only this episode's definitions)

Using an infinitesimal $\varepsilon>0$, find each of $\operatorname{st}(5+3\varepsilon)$, $\operatorname{st}(2-\varepsilon^2)$, and $\operatorname{st}\!\big(\tfrac{4\varepsilon}{\varepsilon}\big)$.
Show the answer
$\operatorname{st}(5+3\varepsilon)=5$ ($3\varepsilon$ is infinitesimal, so it drops). $\operatorname{st}(2-\varepsilon^2)=2$ ($\varepsilon^2$ is also infinitesimal). $\tfrac{4\varepsilon}{\varepsilon}$ can be canceled since $\varepsilon\ne0$, giving $=4$, so $\operatorname{st}(4)=4$. The last example shows that "a ratio of infinitesimals is not necessarily infinitesimal" ── this is why the derivative comes out finite.
Find the derivative of $f(x)=x^3$ using the non-standard definition $f'(x)=\operatorname{st}\!\big(\tfrac{f(x+\varepsilon)-f(x)}{\varepsilon}\big)$.
Show the answer
$(x+\varepsilon)^3-x^3=3x^2\varepsilon+3x\varepsilon^2+\varepsilon^3$. Dividing by $\varepsilon$ gives $3x^2+3x\varepsilon+\varepsilon^2$. Taking the standard part, $3x\varepsilon$ and $\varepsilon^2$ are infinitesimal and drop, so $f'(x)=\operatorname{st}(3x^2+3x\varepsilon+\varepsilon^2)=3x^2$. The pattern of higher-order infinitesimals vanishing under $\operatorname{st}$ is the same as for $x^2$.
For the infinity $H=1/\varepsilon$, why can we say that $H$ is an element of the hyperreals ${}^{*}\mathbb{R}$ and differs from the symbol $\infty$? Also, can $\operatorname{st}(H)$ be defined?
Show the answer
$H=1/\varepsilon$ is a single number that actually exists inside ${}^{*}\mathbb{R}$, and calculations like $H+1,\ 2H,\ H^2$ can be carried out. $\infty$, by contrast, is a symbol representing the motion "growing without bound"; it is not a number and cannot be an object of calculation. $\operatorname{st}(H)$ cannot be defined ── $\operatorname{st}$ applies only to "finite" hyperreals, and the infinity $H$ is not infinitely close to any real. With this, the "population of $N\to\infty$" from Episode 4 can be named as the static number $N=H$ (a size of infinity).

Episode 5 summaryCounting Infinity Honestly

The limit ritual of "approaching 0 without bound" carried the guilt of secretly using a quantity that is neither 0 nor finite (STEP 01). Robinson's (1960) hyperreal field ${}^{*}\mathbb{R}$ contains the reals $\mathbb{R}$ and holds, as numbers that actually exist, an infinitesimal $\varepsilon$ smaller than every $1/n$ and its reciprocal, the infinity $H=1/\varepsilon$ (STEP 02). From a finite hyperreal, the standard part $\operatorname{st}$ returns the one and only real infinitely close to it, and the infinitesimal mist around a point is $\operatorname{monad}(x)$ (STEP 03). Then differentiation is $f'(x)=\operatorname{st}\!\big(\tfrac{f(x+\varepsilon)-f(x)}{\varepsilon}\big)$ ── a two-step operation of honestly dividing by a nonzero $\varepsilon$ and rounding with $\operatorname{st}$ ── and $(x^2)'=\operatorname{st}(2x+\varepsilon)=2x$ came out without $\lim$ (STEPs 04 and 05). What guarantees that calculating with infinitesimals is correct for the reals is the bidirectional transfer principle $\mathbb{R}\models\varphi\Leftrightarrow{}^{*}\mathbb{R}\models\varphi$ (STEP 06). It has, however, no power to birth new theorems; it is a lens, equivalent to standard analysis, that makes intuition rigorous (STEP 07).

With this, the homework from Episode 4 is settled. "The infinite population $N\to\infty$" is $N=H$; "the infinitesimal neighborhood of $\theta$" is $\operatorname{monad}(\theta)$; "the average of $\infty$ samples reconstructs the gradient" can now be written rigorously as the claim that the standard part of the average equals the derivative. Integration (a sum of $\infty$ infinitesimals) and differentiation (a ratio of infinitesimals) shake hands under the infinitesimal ── that foreshadowing has been laid. Next time, on this footing, ES and BP will be revealed to be one and the same calculation mathematically.

Next time ── Episode 6 Evolution and learning were the same calculation. Evolution strategies (ES), which do not differentiate, descend the hill by looking at $\infty$ samples scattered across the monad of $\theta$ ── the average of the loss with noise $\varepsilon$ mixed in. Backpropagation (BP), which does differentiate, computes the gradient $\nabla L$ directly. These two, seemingly entirely different things, are joined by a single equation ── Stein's lemma. The standard part of ES's "integral over $\infty$ samples" coincides exactly with BP's "differentiation" itself. On the footing of the infinitesimal and the standard part we forged this time, we will witness the moment evolution and learning dissolve into one formula.

This document is Episode 5 of the "Learning That Clicks" series, a reading piece for high-school and university students interested in physics, mathematics, and AI. The hyperreal field ${}^{*}\mathbb{R}$, infinitesimals and infinities, the standard part $\operatorname{st}$, and the non-standard definition of the derivative $f'(x)=\operatorname{st}\!\big(\tfrac{f(x+\varepsilon)-f(x)}{\varepsilon}\big)$ together with the transfer principle are the standard content of non-standard analysis, which A. Robinson constructed rigorously around 1960 using mathematical logic (model theory). A rigorous construction requires tools such as an ultrapower via an ultrafilter; we do not enter into that here and state only the results. Note that the transfer principle applies to first-order statements (about elements), that second-order statements quantifying over whole sets do not transfer as-is, and that $\operatorname{st}$ is defined only for finite hyperreals. The point that "it does not birth new theorems about the reals but has power equivalent to standard analysis" refers to the two systems having the same first-order consequences (being a conservative extension). The figure in this piece is a model for explaining the relationship between the secant and tangent of $f(x)=x^2$, and the $\varepsilon$ on screen is a finite real value. The correspondences with the infinite population and infinitesimal neighborhood of Episode 4, and the ES=BP coincidence via Stein's lemma in Episode 6, are stated as the structural framing of this series; their rigorous formulation is treated in each episode. ── To print, use your browser's "Print" and "Save as PDF" (in the print version, the sliders and answers are static and hidden).

Print / save as PDF: ⌘+P (Ctrl+P on Windows). On screen, changing "Position of the point of tangency x" and "Infinitesimal ε" lets you watch the secant slope 2x+ε get sucked into the tangent slope 2x (= st). "Show the answer" opens each solution.

Question	Non-standard analysis's answer	Verdict
Can infinitesimals and infinities be handled rigorously as "numbers" without contradiction?	They can. \({}^{*}\mathbb{R}\) holds \(\varepsilon,\,H=1/\varepsilon\) as numbers that actually exist, and the transfer principle guarantees the algebraic rules	yes
Can it give a rigorous footing to the intuitions "differentiation is a ratio of infinitesimals" and "integration is an infinite sum of infinitesimals"?	It can. \(f'=\operatorname{st}\!\big(\tfrac{\Delta f}{\varepsilon}\big)\) and \(\int=\operatorname{st}\!\big(\sum(\text{infinitesimals})\big)\) become theorems	yes
Can it prove new theorems that standard analysis (\(\varepsilon\text{-}\delta\)) cannot?	No. Since the transfer principle lets us go back and forth, the range of provable theorems about the reals is exactly the same ── all that differs is the choice of expression	equivalent; a difference of expression