Thinking in Waves Episode 21 ／ Part IX · Information as waves

The limits on information were all the bandwidth theorem in disguise

Integrate a bit rate and you get information ──
and information lives on the imaginary axis Integrate a speed and you get a distance. Integrate a bit rate and you get an amount of information.
Exactly the same shape ── information has a ladder of orders too.
★ And Shannon's formula is "bandwidth × time" ── Episode 9, unchanged.

Prerequisites: Episodes 1, 3, 7, 9, 10, 19, 20Computation: kenshou/joho.js

This series has been looking at physical quantities as waves throughout. Information can be looked at that way too.
We tried, and the limits of information theory and the limits of physics joined up in one formula.

01Information has a ladder of orders too

Order	Quantity	Units	Content
0	information $I$	bit	★ the thing that looks constant (Episode 3)
1	bit rate $dI/dt$	bit/s	communication speed, entropy production rate
2	$d^2I/dt^2$	bit/s²	acceleration of information production (§06)

§01 ── the constant of integration applies too

Episode 3 said the constant of integration is the $\omega=0$ component and cannot be determined in principle.
The same holds here ── the constant of integration of information is the initial information.

★ "How many bits did the universe start with?" cannot be settled by integrating a bit rate, however long you integrate.

02Shannon's formula was "bandwidth × time"

Integrate information theory's basic formula $$C=B\log_2\Bigl(1+\frac{S}{N}\Bigr)\ \text{[bit/s]}\quad\Longrightarrow\quad I=C\,T=\underbrace{B\,T}_{\text{band}\times\text{time}}\cdot\log_2\Bigl(1+\frac{S}{N}\Bigr)$$

The heart of §02

information ＝ (bandwidth × time) × (bits per sample)

★ Episode 9's bandwidth theorem $\Delta t\,\Delta\omega\ge1/2$ drops straight out.
By Nyquist's sampling theorem, a signal of bandwidth $B$ and duration $T$ has $2BT$ degrees of freedom ── that is the number of "modes".

★ "To send one bit you need time × bandwidth."

Channel	Bandwidth $B$	S/N [dB]	Capacity $C$	Real-world value
Telephone line (voice band)	3.1 kHz	30	30.9 kbps	★ the dial-up wall
4G LTE (20 MHz)	20 MHz	20	133 Mbps	150 Mbps class
Wi-Fi (80 MHz)	80 MHz	35	930 Mbps	a few hundred Mbps in practice
Optical fibre, C+L band	10 THz	20	67 Tbps	10–100 Tbps in service

The telephone line's 30.9 kbps ── why 56k modems stopped where they did. Raising S/N only helps logarithmically, so you must widen the band. ★ Hence optical fibre is nine orders faster.

03The exchange rate between information and energy ── Landauer

The energy needed to erase one bit $$E\ge k_BT\ln2$$

★ Episode 7 called $\hbar$, $c$, $k_B$ exchange rates. Here $k_B$ is the information ↔ energy rate

Temperature	Price of one bit [J]	[eV]
CMB, 2.725 K	$2.61\times10^{-23}$	$1.63\times10^{-4}$
Liquid nitrogen, 77 K	$7.37\times10^{-22}$	$4.60\times10^{-3}$
Room temperature, 300 K	$2.87\times10^{-21}$	0.0179
Solar core, $1.5\times10^7$ K	$1.43\times10^{-16}$	896

§03 ── semiconductors still have four orders to give

One transistor switching $\approx\frac12CV^2=3.2\times10^{-17}$ J (with $C=0.1$ fF and $V=0.8$ V as rough figures)
Landauer's limit at 300 K $=2.87\times10^{-21}$ J

★ a ratio of $1.1\times10^{4}$

Today's semiconductors still have ten thousand times of headroom ── and in 2012 Landauer's limit itself was confirmed experimentally.

04Limits on bit rate ── the bandwidth theorem again

Energy sets the speed $$\nu\le\frac{2E}{\pi\hbar}\ \text{(Margolus–Levitin)},\qquad \text{bit rate}\le\frac{mc^2}{h}=1.356\times10^{50}\ \text{bit/(s·kg)}\ \text{(Bremermann)}$$

System	Mass	Bound on operations [/s]
one electron	$9.1\times10^{-31}$ kg	$4.9\times10^{20}$
one proton	$1.7\times10^{-27}$ kg	$9.1\times10^{23}$
1 kg (the ultimate laptop)	1 kg	$5.4\times10^{50}$
the Earth	$6.0\times10^{24}$ kg	$3.2\times10^{75}$

§04 ── Episode 9, unchanged

A kilogram of matter can change state $5.4\times10^{50}$ times a second. Today's CPUs manage $10^{10}$ ── forty orders of headroom.

★ The logic is Episode 9's exactly ── "to change fast you need a wide band; a wide band needs high energy".
The uncertainty principle and the limit on computation speed are one theorem.

05Limits on stored information ── Bekenstein

There is a bound on what can be held $$I\le\frac{2\pi ER}{\hbar c\ln2}$$

Object	Mass	Radius	Bound on information [bit]
USB stick (10 g)	0.01 kg	2 cm	$5.2\times10^{39}$
A human	70 kg	0.5 m	$9.0\times10^{44}$
The Earth	$6.0\times10^{24}$ kg	6371 km	$9.8\times10^{74}$
The observable universe (holographic)	──	$1.3\times10^{26}$ m	$3.0\times10^{122}$

06★ Differentiating the universe's bit rate

The holographic bound is proportional to area, and area goes as $(ct)^2$. So it can be differentiated.

Differentiate the information with respect to time $$I(t)=\frac{\pi c^2t^2}{l_P^2\ln2}\quad\Longrightarrow\quad \frac{dI}{dt}=\frac{2I}{t}=1.36\times10^{105}\ \text{bit/s}$$ $$\frac{d^2I}{dt^2}=\frac{2\pi c^2}{l_P^2\ln2}=3.12\times10^{87}\ \text{bit/s}^2\qquad\textbf{★ constant!}$$

Epoch	$ct$ [m]	Information $I$ [bit]	Bit rate [bit/s]
Planck time	$1.6\times10^{-35}$	4.53	$1.7\times10^{44}$
1 second	$3.0\times10^{8}$	$1.6\times10^{87}$	$3.1\times10^{87}$
Nucleosynthesis, 3 min	$5.4\times10^{10}$	$5.1\times10^{91}$	$5.6\times10^{89}$
Recombination, 380 kyr	$3.6\times10^{21}$	$2.2\times10^{113}$	$3.7\times10^{100}$
Today	$1.3\times10^{26}$	$3.0\times10^{122}$	$1.4\times10^{105}$

The heart of §06 ── it joins up with Episode 20

At the Planck time there were 4.5 bits ($\pi/\ln2$). Today there are $3\times10^{122}$ ── 122 orders of growth.

And "the universe's information acceleration is constant" ── the same shape as Episode 1's "uniform acceleration means $x\propto t^2$".

★★ Here it meets Episode 20. Since $I\propto(ct)^2$:

The $c\cdot t=$const coordinate system is also
"the system that holds the universe's information constant".

Episode 20 found it to be "time measured logarithmically". In the language of information ── it is using the information itself as the ruler.

07★ Information lives on the imaginary axis

Episode 19 separated the real axis (oscillation) from the imaginary axis (decay). Which one do information and entropy belong to?

Kind	Axis	Examples	Character
Oscillating quantities	real $\omega$	energy, phase, amplitude	reversible; returns under time reversal
Monotonically increasing	imaginary $i\lambda$	★ entropy, information	★ irreversible; the arrow of time
Monotonically decreasing	imaginary $i\lambda$	numbers of radioactive nuclei, free energy	likewise

The heart of §07 ── three irreversibilities converge

Entropy does not oscillate; it only increases ── so it is on the imaginary axis. That is the second law of thermodynamics, said as a wave.

And the three "irreversibilities" of this series meet here:

・Episode 10: odd-order derivatives → sign flips under time reversal → irreversible
・Episode 19: imaginary frequencies → monotone change → irreversible
・this episode: information increase → the second law → irreversible

★ "The arrow of time" was the imaginary axis of the complex frequency plane.

08Counting cells in phase space ── how big is one bit?

Two statements of "no finer than this" $$\text{phase-space cell}:\ \Delta x\,\Delta p\ge\frac{\hbar}{2},\qquad \text{time × band}:\ \Delta t\,\Delta\omega\ge\frac12$$

Signal	Band [Hz]	Time [s]	Modes $2BT$	Information [bit]
One minute of telephone	3100	60	$3.7\times10^5$	$1.9\times10^6$
One CD track (stereo)	$4.0\times10^4$	240	$1.9\times10^7$	$3.1\times10^8$
One second of 4K video	$1.0\times10^7$	1	$2.0\times10^7$	$1.3\times10^8$

Conclusion of §08

information ＝ number of modes × bits per mode

And the number of modes is fixed by the bandwidth theorem (Episode 9).
★ The Gabor limit, the uncertainty principle and Shannon's formula are one theorem about the size of a single mode.

Exercises

Is "raise S/N without limit and any narrow band carries infinite information" correct?
Show answer
Correct on paper, wrong physically.
$C=B\log_2(1+S/N)$ diverges as $S/N\to\infty$. But raising $S$ costs energy, and raising energy runs into §04's bound ($\nu\le2E/\pi\hbar$).
★ Moreover $N$ has a floor ── quantum noise. Each mode carries at least $\hbar\omega/2$ of zero-point fluctuation, which cannot be removed (Episode 9's bandwidth theorem again).
★ So "you must widen the band" is physically right as well.
What does "information lives on the imaginary axis" mean in practice?
Show answer
The same as Episode 19 ── you need not wait a full period to measure it.
An entropy production rate is fixed by a short observation (being monotone, you just look at the slope).
★ Conversely it does not come back. Real-axis quantities conserve phase and can be rewound; imaginary-axis quantities have thrown information away and cannot.
★ "Erasing a computation costs energy" (Landauer) is exactly this ── erasure is an imaginary-axis operation, so it must be paid for in heat.
If $c\cdot t=$const and "constant information" are the same thing, what is cosmic expansion in the language of information?
Show answer
The ruler is gaining tick marks.
Since $I\propto(ct)^2$, a growing horizon means more distinguishable states.
★ The $c\cdot t=$const system absorbs that growth into the units ── so nothing grows there, and time becomes logarithmic instead (Episode 20).
★ Verdict: a correspondence, not a claim about cosmology. Note also that the holographic bound is an upper bound, not the number of bits actually in use.

Summary　The limits on information were all the bandwidth theorem

Integrate a bit rate [bit/s] and you get information [bit] ── the same shape as speed and distance. And the constant of integration is the initial information, undeterminable in principle (Episode 3).

Shannon's formula is $I=BT\log_2(1+S/N)$, that is "bandwidth × time" ── Episode 9's bandwidth theorem itself. The telephone line's 30.9 kbps was the dial-up wall, and optical fibre is nine orders faster because its band is nine orders wider.

Landauer's limit is 0.0179 eV/bit at room temperature, with semiconductors still ten thousand times above it. The speed bound ($5.4\times10^{50}$ /s·kg) and the capacity bound ($9\times10^{44}$ bit for a human) are all restatements of one theorem: energy sets bandwidth.

The universe's information goes as $I\propto(ct)^2$, so it can be differentiated ── $1.4\times10^{105}$ bit/s today, with a constant information acceleration of $3.1\times10^{87}$ bit/s². At the Planck time there were 4.5 bits.

★ And ── the $c\cdot t=$const coordinate system is also the one that holds the universe's information constant (Episode 20 called it logarithmic time).

★ Information lives on the imaginary axis, and so does not come back.
"The arrow of time" was the imaginary axis of the complex frequency plane.

Next time Part IX continues a little further.
What appears when measurement itself is seen as a wave ── and which component of a wave does "observing" extract?

This document is Episode 21 of the series "Thinking in Waves". The computation is in kenshou/joho.js. On the numbers: the Shannon capacities are theoretical values for assumed bandwidths and S/N; real systems differ through coding, multiplexing and MIMO (optical fibre exceeds the computed value because wavelength- and polarisation-division multiplexing effectively widen the band). The switching energy $\frac12CV^2$ varies by orders with process and design and is not a figure for any particular product. The Bekenstein and holographic bounds are upper bounds, not information actually in use ── a human does not hold $9\times10^{44}$ bits. "Constant information acceleration" follows from writing the holographic bound as $I=\pi(ct)^2/(l_P^2\ln2)$, and both the coefficient and the time dependence change with the choice of horizon (Hubble, particle, event) ── we took the most naive choice, $ct$. "Information lives on the imaginary axis" is our framing, a restatement of existing theorems, not a proof of the second law. ── To print, use your browser's Print → Save as PDF.

Epoch	\(ct\) [m]	Information \(I\) [bit]	Bit rate [bit/s]
Planck time	\(1.6\times10^{-35}\)	4.53	\(1.7\times10^{44}\)
1 second	\(3.0\times10^{8}\)	\(1.6\times10^{87}\)	\(3.1\times10^{87}\)
Nucleosynthesis, 3 min	\(5.4\times10^{10}\)	\(5.1\times10^{91}\)	\(5.6\times10^{89}\)
Recombination, 380 kyr	\(3.6\times10^{21}\)	\(2.2\times10^{113}\)	\(3.7\times10^{100}\)
Today	\(1.3\times10^{26}\)	\(3.0\times10^{122}\)	\(1.4\times10^{105}\)