Statistical Foundations of Thermodynamics §

Why the logarithm? Entropy must be extensive: if you combine two independent systems A and B, the total number of microstates is $W_{\text{tot}} = W_A \cdot W_B$ (product, by the multiplication rule). Extensivity requires $S_{\text{tot}} = S_A + S_B$. The only function satisfying $f(W_A W_B) = f(W_A) + f(W_B)$ is $f(W) = k_B \ln W$ (up to a constant).

Concrete combinatorial example: $N$ coins. Toss $N$ fair coins. A macrostate is specified by the number of heads $k$ (from $0$ to $N$); a microstate is a specific sequence of heads and tails. The number of microstates for a given $k$ is the binomial coefficient:

For $N = 10$, the macrostate $k = 5$ (five heads) has $W = \binom{10}{5} = 252$ microstates, while $k = 0$ or $k = 10$ has only $W = 1$. The total number of microstates is $\sum_k W(k) = 2^N$. The most probable macrostate is overwhelmingly the one with $k \approx N/2$, and its multiplicity grows as $\binom{N}{N/2} \sim 2^N / \sqrt{\pi N/2}$ — exponentially in $N$. Taking the logarithm, $S = k_B \ln W$, converts this exponential growth into a quantity that is linear in $N$ (extensive): $\ln \binom{N}{N/2} \approx N \ln 2 - \tfrac{1}{2}\ln(\pi N/2) \approx N \ln 2$ for large $N$.

Why does entropy increase? The number of microstates corresponding to a macrostate with energy spread over a larger volume (or more mixed composition) is overwhelmingly larger. The system evolves toward the macrostate with the most microstates, as formalized by the Second Law and Entropy.

Gibbs/von Neumann entropy. The Boltzmann formula $S = k_B \ln W$ assumes every microstate is equally probable — true in the microcanonical ensemble. When microstates are not equally probable (e.g., the canonical ensemble, where higher-energy states are less likely), the correct generalization is the Gibbs entropy:

For a quantum system: $S = -k_B \, \text{Tr}(\rho \ln \rho)$ where $\rho$ is the density matrix. The two forms agree when all $p_i = 1/W$: $-k_B \sum_i (1/W)\ln(1/W) = k_B \ln W$. Use the Boltzmann form for an isolated system (equal a priori probabilities) and the Gibbs form for a system in contact with a reservoir (unequal probabilities); the full derivation is in CanonicalEnsemble.

Authoritative sources

• Boltzmann entropy — Wikipedia
• Gibbs entropy — Wikipedia

Derivation. Consider two systems A and B in thermal contact, with total energy $U = U_A + U_B$ fixed. The most probable energy division maximizes $S_{\text{tot}} = S_A(U_A) + S_B(U_B)$:

This gives $(\partial S_A/\partial U_A) = (\partial S_B/\partial U_B)$, i.e., the two systems have the same value of $1/T$. This is the statistical derivation of the zeroth law: thermal equilibrium equalizes temperature.

Reading the slope. The slope of the $S(U)$ curve at any energy is the inverse temperature. Because $S(U)$ is concave (its slope decreases as $U$ grows), a system with more energy has a lower temperature. For a concrete model, take $S(U) = \sqrt{U}$ (in units of $k_B$): at $U = 25$, $1/T = 1/(2\sqrt{25}) = 0.1$, so $T = 10$. Energy flows from the steeper-slope (hotter) system to the flatter-slope (cooler) one until the slopes match.

Equilibrium of two systems. The total entropy $S_{\text{tot}}(U_A) = S_A(U_A) + S_B(U - U_A)$ is a concave function of $U_A$ with a single maximum. At the maximum the slope vanishes, which is exactly the condition $\partial S_A/\partial U_A = \partial S_B/\partial U_B$. For two identical systems the maximum sits at $U_A = U/2$.

Negative temperature. For systems with an upper bound on energy (e.g., spin systems in a magnetic field), $S(U)$ can decrease with $U$, giving $T < 0$. Such a system is hotter than any positive-temperature system: heat flows from $T = -1$ K$to$T = +\infty K$ to T = +1 K$.

Authoritative sources

• Temperature — Wikipedia
• Schroeder, Thermal Physics, Ch. 2

Step 1 — from a shell to a density. The microcanonical count $W(E) = \Omega(E)\,\Delta E$ is only meaningful for a thin shell. The density of states $\Omega(E)$ strips off the arbitrary shell width $\Delta E$ and gives the number of states per unit energy. For a system with a discrete spectrum, $\Omega(E)$ is a comb of delta peaks at each level; for a macroscopic system the levels are so dense that $\Omega(E)$ is a smooth function.

Step 2 — why a Laplace transform. The canonical ensemble weights each energy by the Boltzmann factor $e^{-\beta E}$ (derived in CanonicalEnsemble). Summing (integrating) the weighted density of states over all energies gives the partition function:

This is exactly the Laplace transform of $\Omega(E)$ with transform variable $\beta$. The Laplace transform is the tool that connects the two ensembles: the microcanonical ensemble is characterized by $\Omega(E)$ (a function of energy), and the canonical ensemble by $Z(\beta)$ (a function of inverse temperature). The transform is invertible, so no information is lost — the two descriptions are equivalent.

Step 3 — where the integral is dominated. The integrand $\Omega(E)\, e^{-\beta E}$ is a product of a growing function ($\Omega(E)$, more states at higher energy) and a decaying function ($e^{-\beta E}$, Boltzmann suppression). The product is a peaked function, and the peak location $E^*$ is the mean energy $\langle E \rangle$. At low temperature ($\beta$ large) the peak sits near the ground state; at high temperature ($\beta$ small) it moves to higher energy.

Worked example (single 3D harmonic oscillator). The energy levels are $E_n = \hbar\omega(n + \tfrac{1}{2})$, so the density of states is a comb of delta peaks. The partition function is a geometric series:

At high temperature ($\beta\hbar\omega \ll 1$), $Z \approx 1/(\beta\hbar\omega)$, and the mean energy $\langle E \rangle = -\partial \ln Z/\partial \beta \approx k_B T$ — the classical equipartition result. The full treatment, including the free energy and entropy, is in CanonicalEnsemble.

Authoritative sources

• Laplace transform — Wikipedia
• Partition function — Wikipedia

Ideal gas (key result). For $N$ non-interacting, indistinguishable particles in volume $V$:

where $\lambda_T$ is the thermal de Broglie wavelength and the $1/N!$ (Gibbs factor) corrects for indistinguishability. From this single expression: $PV = Nk_B T$ (ideal gas law), $U = \tfrac{3}{2}Nk_B T$ (monatomic), $C_V = \tfrac{3}{2}Nk_B$.

Quantum correction. When $n\lambda_T^3 \gtrsim 1$ (high density or low temperature), quantum statistics (Fermi–Dirac or Bose–Einstein) replace the classical Boltzmann factor.

The full ensemble formalism — the Boltzmann-factor derivation from a reservoir, additivity of partition functions, pressure, and the density-matrix formulation — is developed in CanonicalEnsemble.

Authoritative sources

• Partition function — Wikipedia
• CanonicalEnsemble

Why thermodynamics works. The energy distribution $P(E) = \Omega(E)e^{-\beta E}/Z$ is a narrow peak around $\langle E \rangle$ with width $\sigma_E = \sqrt{k_B T^2 C_V}$. In the thermodynamic limit ($N \to \infty$, $V \to \infty$, $N/V$ fixed), the relative width $\sigma_E/\langle E \rangle \sim 1/\sqrt{N} \to 0$, so the peak becomes a delta function. This justifies treating $U, S, T$ as sharp, deterministic quantities: the first and second laws are not violated, they are the overwhelmingly probable behavior.

Small systems. For nanoparticles, molecular motors, or single-molecule experiments ($N \sim 10^2$–$10^6$), the relative fluctuations are significant. The fluctuation-dissipation theorem and Jarzynski equality ($\langle e^{-\beta W} \rangle = e^{-\beta \Delta F}$) replace the deterministic second law with statistical inequalities. These are treated in Nonequilibrium statistical mechanics.

The full derivation of the energy variance, and the related pressure and volume fluctuations (including critical opalescence), are in CanonicalEnsemble.

Authoritative sources

• Fluctuation–dissipation theorem — Wikipedia
• CanonicalEnsemble

Statistical statement of the third law. In the canonical ensemble, the probability of a state of energy $E$ is $e^{-\beta E}/Z$. As $T \to 0$ ($\beta \to \infty$), the Boltzmann factor $e^{-\beta E}$ suppresses every state except the ground state. If the ground state is unique (non-degenerate, $g_0 = 1$), then $Z \to 1$ and the system is certainly in that one state: $W \to 1$, so $S = k_B \ln W \to 0$.

Role of ground-state degeneracy. If the ground state is $g_0$-fold degenerate, then $Z \to g_0$ and $S \to k_B \ln g_0 \neq 0$. The residual entropy $S_0 = k_B \ln g_0$ is the statistical content of the third law’s “perfect crystal” caveat: a perfectly ordered crystal has a unique ground state ($g_0 = 1$), while a disordered solid (e.g., ice, with many equivalent proton arrangements) retains a finite residual entropy. This connects to the thermodynamic statement in Second Law and Entropy.

Connection to the heat capacity. The third law is consistent with the low-temperature behavior of the heat capacity: for a non-degenerate system, $C \to 0$ as $T \to 0$ (as seen in the two-level Schottky anomaly and the harmonic oscillator), so no entropy is “left behind” at zero temperature. For the two-level system, at $k_B T = \varepsilon$ ($\beta\varepsilon = 1$), $S/k_B = \ln(1+e^{-1}) + 1/(e+1) \approx 0.582$ — already well below the high-temperature limit $k_B \ln 2 \approx 0.693\, k_B$.

Authoritative sources

• Third law of thermodynamics — Wikipedia
• Second Law and Entropy

A three-dimensional harmonic oscillator has energy levels $E_{n_x n_y n_z} = \hbar\omega\left(n_x + n_y + n_z + \tfrac{3}{2}\right)$, $n_i = 0, 1, 2, \ldots$. It is in contact with a heat bath at temperature $T$.

(a) Write the partition function $Z$ as a product of three identical 1D sums and evaluate it. (b) Find the mean energy $\langle E \rangle$. (c) Find the heat capacity $C_V$ and show that $C_V \to 3 k_B$ as $T \to \infty$ and $C_V \to 0$ as $T \to 0$.