Entropy

Pour milk into a cup of coffee. Within seconds it swirls, spreads, and settles into a uniform brown. Now wait for the reverse: for the milk to gather itself back into a white cloud and leave the coffee black. Nothing in the laws of motion forbids it. Every collision between molecules could, in principle, be run backwards. Yet no one has ever seen it happen.

The same asymmetry appears everywhere. Hot things cool down; cold things do not spontaneously heat up. A dropped glass shatters; the pieces never jump back into a glass. We remember the past and not the future.

Entropy is the name physics gives to what increases in all these processes. It began as an engineering quantity for steam engines, became a way of counting molecular arrangements, and ended up as a measure of information. That last step came from an unusual source: a thought experiment about a small, intelligent being that seemed able to break the laws of physics.

Why Engineers Needed It

In the early nineteenth century, steam engines were transforming industry, and no one knew their limits. In 1824 the French engineer Sadi Carnot showed that an engine producing work from heat must take heat from a hot body and dump some into a cold one, and that there is a maximum efficiency set only by the two temperatures.

Rudolf Clausius turned this into a general principle. In 1865 he defined a new quantity, which he named entropy from a Greek word for transformation, and stated the two laws of thermodynamics in their famous form: the energy of the universe is constant; the entropy of the universe tends to a maximum. ^[1]

In Clausius's definition, when a small amount of heat $\delta Q$ flows reversibly into a system at absolute temperature $T$, the entropy changes by

Here $S$ is entropy, and $T$ is measured in kelvin. The formula explains why heat flows from hot to cold. The same amount of heat raises the entropy of a cold body more than it lowers the entropy of a hot one, because $T$ is in the denominator. So the transfer raises the total. The reverse would lower it, and the second law of thermodynamics says that in an isolated system this does not happen.

What Entropy Counts

A box of coins

Clausius's entropy worked, but it did not say what entropy is. Ludwig Boltzmann answered that in the 1870s by looking at the molecules. ^[2]

Imagine 100 coins in a box. Shake it. The macrostate is what you can see at a glance: how many heads there are. The microstate is the full detail: which particular coins are heads.

There is exactly one microstate with all 100 coins showing heads. There are about $10^{29}$ microstates with exactly 50 heads. Shake the box and you will almost certainly land near 50–50, not because anything pushes the coins there, but because almost all the possibilities are there.

That is the second law in miniature. Systems drift towards macrostates that can be realised in the most ways, simply because there are overwhelmingly more of them.

Boltzmann's formula

Boltzmann expressed this as

where $W$ is the number of microstates consistent with the macrostate, $\ln$ is the natural logarithm, and $k_B \approx 1.38 \times 10^{-23}\ \text{J/K}$ is Boltzmann's constant, which converts the count into thermodynamic units. The equation is carved on Boltzmann's tombstone in Vienna.

On this view, entropy measures how many ways the hidden details could be arranged without changing what we observe. Low entropy means the macrostate pins down the details; high entropy means it hides almost all of them.

A law of probability

This changes the status of the second law. It is not an absolute prohibition but a statistical one. The milk could unmix; the probability is just so small that it would not be expected to happen in many lifetimes of the universe. For small systems of a few molecules, fluctuations that briefly lower entropy do happen, and are measured.

It also creates a puzzle. If the microscopic laws run equally well forwards and backwards, why does entropy increase towards the future and not towards the past? The standard answer is that the universe began in a state of extraordinarily low entropy, and everything since has been running downhill from it. ^[3] Why the beginning was like that is unknown. It is one of the central problems for any theory of the early universe, including the Big Bounce, and it is closely tied to the question of whether time "flows" at all, discussed in Block Universe.

Maxwell's Demon

The thought experiment

In 1867, James Clerk Maxwell, whose equations had already unified electricity, magnetism and light (see Maxwell's Equations), described a challenge to the second law in a letter, and later published it in his book Theory of Heat (1871). ^[4]

Take a box of gas at uniform temperature, divided in two by a wall with a tiny door. The molecules are not all moving at the same speed: some are fast, some slow. A small being stands by the door. When a fast molecule approaches from the left, it opens the door and lets it through. When a slow molecule approaches from the right, it lets that one through the other way. It never pushes anything; it only opens and closes a frictionless door.

After a while, the right side is full of fast molecules and is hot. The left side is full of slow ones and is cold. A temperature difference has appeared from nothing, and it could be used to run an engine. Entropy has decreased, apparently with no cost.

William Thomson (Lord Kelvin) later named this being Maxwell's demon. Maxwell did not believe it could beat the second law. His point was that the law is statistical: it holds because we cannot see and sort individual molecules, not because sorting them is impossible in principle.

Szilard: the demon needs to know

For more than half a century, the demon resisted a clean answer. In 1929 Leo Szilard stripped it down to a single molecule in a box. ^[5] The demon learns which half the molecule is in, inserts a partition, and lets the molecule push the partition like a piston. One bit of knowledge, "left or right", produces a definite amount of work:

where $T$ is the temperature of the gas and $\ln 2$ reflects the two equally likely possibilities.

Szilard's key insight was that information has a thermodynamic value. The demon converts knowledge into energy. So where does the entropy go?

Landauer and Bennett: forgetting costs heat

The final answer came from computing. In 1961 Rolf Landauer at IBM argued that erasing information is physically irreversible. ^[6] Resetting a memory bit to a fixed state, whatever it held before, must release at least

of heat into the surroundings. At room temperature this is about $3 \times 10^{-21}$ joules per bit: tiny, but not zero.

In 1982 Charles Bennett applied this to the demon. ^[7] Measuring a molecule can, in principle, be done without cost. But the demon has a finite memory. To keep working, it must eventually erase what it learned about earlier molecules. That erasure produces at least as much entropy as the sorting removed. The demon does not break the second law; it pays with forgetting.

Experimental confirmation

Both halves of the story have now been tested. In 2010 Shoichi Toyabe and colleagues used feedback control on a microscopic bead to convert information about its position into free energy, a working version of Szilard's engine. ^[8] In 2012 Antoine Bérut and colleagues erased single bits stored in a colloidal particle held by lasers, and measured the heat released approaching the Landauer limit. ^[9]

Status: the link between information and thermodynamic entropy is established experimentally, not merely theoretically. Philosophers still debate how general Landauer's argument is and whether it has been derived without circularity. ^[4]

Information Entropy

In 1948 Claude Shannon, working on telephone signals at Bell Labs, needed a measure of how much information a message carries. ^[10] He arrived at a formula of the same shape as Boltzmann's:

where $p_i$ is the probability of the $i$-th possible message and $H$ is measured in bits. A fair coin toss carries 1 bit. A coin that always lands heads carries 0 bits, because you learn nothing from seeing it.

Shannon's entropy measures missing information: how uncertain you are before the message arrives. Boltzmann's entropy measures the missing information about which microstate a system is in. The demon story shows that the resemblance is not a coincidence. Thermodynamic entropy is, in a precise sense, information we do not have about the molecules. ^[4,11]

This also connects to modern cryptography. A hash function such as SHA-256 maps many inputs to one output, so it discards information and cannot be run backwards. In a physical computer, Landauer's principle says that the discarded information does not vanish: it leaves as heat.

Is Information Ever Lost?

What the laws say

Throw a book into a fire. The words seem gone. But the fundamental laws of physics, as currently understood, suggest they are not. Classical mechanics preserves the volume of possible states over time (Liouville's theorem), and quantum mechanics evolves states by unitary transformations, which can always be undone. In both cases, the exact present fixes the exact past as well as the future.

In principle, then, the smoke, ash, heat and light from the burning book still encode every word. Reading them back is hopelessly impractical, but the information is scrambled, not destroyed. This principle, sometimes called conservation of information, is really a statement about the reversibility of the microscopic laws. It is closely related to determinism, which asks whether the present fixes the future. Conservation of information asks, in addition, whether the present fixes the past.

There is one caveat. In interpretations of quantum mechanics where measurement causes a genuine, random collapse, that collapse is not reversible. Whether it happens is an open question of interpretation.

Black holes have entropy

Black holes seemed to offer a way around the second law: drop a box of hot gas into one, and its entropy disappears from the universe. In 1973 Jacob Bekenstein argued that this could not be the whole story, and that a black hole must itself have entropy, proportional to the area of its event horizon. ^[12] Stephen Hawking made this precise and, in 1974–75, showed that quantum effects make black holes emit thermal radiation. ^[13]

Here $A$ is the area of the horizon, $c$ is the speed of light, $G$ is Newton's gravitational constant, and $\hbar$ is the reduced Planck constant. The formula is unusual: it contains the constants of thermodynamics, relativity, gravity and quantum mechanics at once. The entropy grows with the area of the horizon, not the volume inside, a clue that has shaped much of modern theoretical physics.

The temperature of this Hawking radiation is inversely proportional to the mass:

where $M$ is the black hole's mass. For a black hole the mass of the Sun, $T_H$ is about $6 \times 10^{-8}$ kelvin, far colder than the cosmic microwave background. Such a black hole would take roughly $10^{67}$ years to evaporate.

The paradox

In 1976 Hawking drew the uncomfortable conclusion. ^[14] If a black hole radiates away its mass, it eventually disappears. The radiation, in his calculation, is exactly thermal: it depends only on the black hole's mass, charge and spin, not on what fell in. A black hole made from an encyclopaedia and one made from the same mass of rock would leave behind identical radiation.

If that is right, the information about what fell in is destroyed. A pure quantum state would evolve into a mixed one, something unitary quantum mechanics forbids. Hawking called this a "breakdown of predictability". That is the black hole information paradox: two of our best theories, applied together, disagree about whether information can be lost.

The main proposals

The positions differ on where the information ends up. Here, the physics column describes the proposal and the status column describes how it is regarded; none has been tested by observation.

A common intuition is that the information is simply kept at the singularity, the point at the centre where, in general relativity, infalling matter ends up. The difficulty is that this only postpones the problem. In classical general relativity the singularity is where the theory stops giving answers, and once the black hole has evaporated there is no interior left to keep anything in. Information stored inside must either come out, remain in a leftover object (a remnant), or leave our universe altogether. So "kept in the singularity" becomes one of the rows in the table rather than a separate answer.

The escape view gained strong support from the AdS/CFT correspondence, proposed by Juan Maldacena in 1997, in which a theory with gravity is exactly equivalent to an ordinary quantum theory without gravity on its boundary. ^[15] Since the boundary theory is unitary, black holes in that setting cannot destroy information. In 2004 Hawking publicly changed his position, and argued in 2005 that information is preserved. ^[16]

In 1993 Don Page showed what information escape would look like: the entanglement between the radiation and the remaining black hole should rise and then fall, following what is now called the Page curve. ^[17] In 2019, several groups showed how to derive this curve from the gravitational calculation itself, using new techniques involving so-called "islands". ^[18] Many physicists regard this as major progress, but it is established only in simplified models, and exactly how the information gets out remains unclear.

The difficulty sharpened in 2012, when Almheiri, Marolf, Polchinski and Sully argued that if information escapes, an observer falling into an old black hole might meet a wall of high-energy particles, a firewall, at the horizon, contradicting general relativity's prediction that nothing special happens there. ^[19] There is still no agreed resolution.

Not everyone accepts that information must be preserved. In 2017 William Unruh and Robert Wald argued that information loss is a natural consequence of black hole formation and is not in conflict with any established principle. ^[20]

Evidence

• The second law is among the best-tested principles in physics, confirmed across chemistry, engineering and biology.
• Maxwell's demon and Landauer's bound have been realised with microscopic particles in the laboratory. ^[8,9]
• Black hole horizon area behaves as the entropy analogy predicts: an analysis of the first detected black hole merger, GW150914, found the final horizon area larger than the sum of the initial ones, consistent with Hawking's area theorem. ^[21]
• Hawking radiation has never been observed from a real black hole; it is far too faint. Laboratory analogues, such as sound waves in a Bose–Einstein condensate that behave like a horizon, have produced radiation with the predicted features. ^[22] These test the mathematics, not black holes themselves.

Limitations and Open Problems

• The low-entropy beginning is unexplained. The second law depends on it, but no accepted theory says why the early universe was so ordered.
• The meaning of entropy in cosmology is unclear. Defining the entropy of the whole universe, including gravity, is not straightforward.
• The information paradox is not solved. The Page curve calculations are strong evidence in simplified models, not a complete mechanism, and they do not describe real astrophysical black holes directly.
• Quantum gravity is missing. Any final answer about the singularity requires a theory that no one yet has.

Order on Borrowed Time

Life seems to defy entropy. A seed becomes a sunflower whose seeds pack into precise spirals, a pattern connected to the golden ratio. Cells build intricate molecules from simple ones. In 1944 Erwin Schrödinger asked how this was possible, and answered that living things survive by exporting disorder to their surroundings: they take in low-entropy energy and give back heat. ^[23] The same principle constrains every theory of the origin of life. Order is not forbidden; it is paid for.

Entropy therefore connects things that seem unrelated: the efficiency of engines, the direction of time, the price of forgetting, the security of cryptography, and the fate of whatever falls into a black hole. Maxwell's demon, which looked like a loophole, turned out to reveal that information is physical. Laplace imagined a demon who knew everything and so could predict everything, a figure behind The Illusion of Free Will. Maxwell's demon teaches the complementary lesson: even perfect knowledge would have to be stored, and eventually erased, and that has a cost.

What we know is that entropy rises and that information carries a thermodynamic price. What physics suggests, though it has not been confirmed, is that information is never truly destroyed, not even by black holes. What remains unresolved is how a black hole gives it back, and why the universe started with so little entropy that all of this could happen at all.

References

[1] Clausius, R. (1865). “Ueber verschiedene für die Anwendung bequeme Formen der Hauptgleichungen der mechanischen Wärmetheorie.” Annalen der Physik, 201(7), 353–400. https://doi.org/10.1002/andp.18652010702

[2] Uffink, J. “Boltzmann’s Work in Statistical Physics.” Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/statphys-Boltzmann/

[3] Callender, C. “Thermodynamic Asymmetry in Time.” Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/time-thermo/

[4] Maroney, O. “Information Processing and Thermodynamic Entropy.” Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/information-entropy/

[5] Szilard, L. (1929). “Über die Entropieverminderung in einem thermodynamischen System bei Eingriffen intelligenter Wesen.” Zeitschrift für Physik, 53, 840–856. https://doi.org/10.1007/BF01341281

[6] Landauer, R. (1961). “Irreversibility and Heat Generation in the Computing Process.” IBM Journal of Research and Development, 5(3), 183–191. https://doi.org/10.1147/rd.53.0183

[7] Bennett, C. H. (1982). “The thermodynamics of computation—a review.” International Journal of Theoretical Physics, 21, 905–940. https://doi.org/10.1007/BF02084158

[8] Toyabe, S., Sagawa, T., Ueda, M., Muneyuki, E., & Sano, M. (2010). “Experimental demonstration of information-to-energy conversion and validation of the generalized Jarzynski equality.” Nature Physics, 6, 988–992. https://doi.org/10.1038/nphys1821

[9] Bérut, A., Arakelyan, A., Petrosyan, A., Ciliberto, S., Dillenschneider, R., & Lutz, E. (2012). “Experimental verification of Landauer’s principle linking information and thermodynamics.” Nature, 483, 187–189. https://doi.org/10.1038/nature10872

[10] Shannon, C. E. (1948). “A Mathematical Theory of Communication.” Bell System Technical Journal, 27(3), 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x

[11] Adriaans, P. “Information.” Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/information/

[12] Bekenstein, J. D. (1973). “Black Holes and Entropy.” Physical Review D, 7(8), 2333–2346. https://doi.org/10.1103/PhysRevD.7.2333

[13] Hawking, S. W. (1975). “Particle creation by black holes.” Communications in Mathematical Physics, 43, 199–220. https://doi.org/10.1007/BF02345020

[14] Hawking, S. W. (1976). “Breakdown of predictability in gravitational collapse.” Physical Review D, 14(10), 2460–2473. https://doi.org/10.1103/PhysRevD.14.2460

[15] Maldacena, J. (1998). “The large N limit of superconformal field theories and supergravity.” Advances in Theoretical and Mathematical Physics, 2, 231–252. https://doi.org/10.4310/ATMP.1998.v2.n2.a1

[16] Hawking, S. W. (2005). “Information loss in black holes.” Physical Review D, 72, 084013. https://doi.org/10.1103/PhysRevD.72.084013

[17] Page, D. N. (1993). “Information in black hole radiation.” Physical Review Letters, 71, 3743–3746. https://doi.org/10.1103/PhysRevLett.71.3743

[18] Almheiri, A., Hartman, T., Maldacena, J., Shaghoulian, E., & Tajdini, A. (2021). “The entropy of Hawking radiation.” Reviews of Modern Physics, 93, 035002. https://doi.org/10.1103/RevModPhys.93.035002

[19] Almheiri, A., Marolf, D., Polchinski, J., & Sully, J. (2013). “Black holes: complementarity or firewalls?” Journal of High Energy Physics, 2013, 62. https://doi.org/10.1007/JHEP02(2013)062

[20] Unruh, W. G., & Wald, R. M. (2017). “Information loss.” Reports on Progress in Physics, 80, 092002. https://doi.org/10.1088/1361-6633/aa778e

[21] Isi, M., Farr, W. M., Giesler, M., Scheel, M. A., & Teukolsky, S. A. (2021). “Testing the Black-Hole Area Law with GW150914.” Physical Review Letters, 127, 011103. https://doi.org/10.1103/PhysRevLett.127.011103

[22] Steinhauer, J. (2016). “Observation of quantum Hawking radiation and its entanglement in an analogue black hole.” Nature Physics, 12, 959–965. https://doi.org/10.1038/nphys3863

[23] Schrödinger, E. (1944; Canto edition 1992). What Is Life? Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139644129