Continuing from the discussion on entropy in Chapter 7, a significant outcome from Boltzmann’s theoretical analysis of entropy was his discovery of the most probable distribution based on energy, which naturally bacame known as the famous Boltzmann distribution. He worked a great example to show his work. I introduce the origins of the Boltzmann distribution here as I will be referring to it later chapters.

Boltzmann and probability

In his wonderful paper of 1877 [1], Ludwig Boltzmann employed a great balls-in-buckets example to illustrate the probabilistic basis for entropy. This paper, which, if you are involved in statistical mechanics I strongly encourage you to read, shows how the maximum number of ways to organize a system of molecules occurs when lower energy states are more populated than higher energy states. In the illustration below from my book, I show how Boltzmann’s work leads to the famous Maxwell-Boltzmann distribution of molecule population as a function of kinetic energy.

In the following exerpt from Chapter 42 of my book, Block by Block – The Historical and Theoretical Foundations of Thermodynamics, I share the balls-in-buckets example that Boltzmann created to illustrate his probabilistic approach.

How many ways can you place 7 balls in 8 buckets?

There’s a great illustrative example that Boltzmann used in this 1877 paper to explain the mathematics involved in these calculations (Figure 42.1 from my book).  In this example he showed how 7 balls[2] can be arranged into 8 energy buckets, ranging in equal increments from zero to the final and highest-energy bucket while keeping total energy constant at an amount equal to a single ball being in the final bucket.  This naturally becomes one possible distribution of these balls:  place one ball in the final bucket, achieve the necessary total energy of the system, and then place all other balls into the zero-energy bucket.  By rotating which of the 7 balls goes into the final bucket one finds that there are 7 different ways to create this specific distribution, each being a unique complexion of the same identical distribution.

It turns out that there are 1716 different complexions possible for this example and these can be grouped into any of 15 possible distributions.  Permutation theory can be used to write an equation quantifying the number of complexions (P) in each of the 15 distributions

            P  =  n! / [n₀! n₁! n₂! n₃! n₄! n₅! n₆! n₇!]                                                             [42.1]

where the n_i terms represent the number of molecules in the bucket having energy i and n the total number of balls.  Each set of n_i values characterizes a specific distribution that fulfills the fixed energy constraint.  So for the example used above in which n = 7, setting n₇ = 1 and n_o = 6 results in P = 7.  In another example, you could put all the balls in bucket n₁.  Since n₁ has 1/7^th the energy of n₇, then this distribution meets the fixed total energy requirement.  Plug these numbers into the above equation and you’ll see that there’s just one single way to put all balls into one bucket.  Remember, it’s only whether or not a given ball goes into a given bucket that matters; the order in which they are placed there doesn’t.

Continuing with Equation 42.1, since n is fixed at 7, the goal of finding the distribution with the most number of complexions is to minimize the denominator.  In this example, n is rather small which makes the mathematics somewhat complex, but for the tremendously large values of n for a system of molecules in a given system, the Law of Large Numbers comes into play and makes the mathematics somewhat easier since the limit of x! is x^x for large values of x, thus leaving the optimization process to become the minimization of the product of the n_iⁿⁱ terms or, in the more popular logarithmic version, the sum of n_i ln (n_i) terms.[3]  The caveat of fixed total energy adds complexity to the optimization process, but Boltzmann shows the reader how to work through the mathematics.[4]  Again, this is a very nice mathematics tutorial for those who are interested.  As Boltzmann noted, when the goal is the minimization of these terms without constraint, the solution is an equiprobable distribution of n_i terms.  This occurs when looking at how molecules spread through space in the absence of any external force.  The solution simply says that molecular density is spread equally throughout the constrained volume V.  With the energy constraint though, the n_i terms occur as equiprobable only for infinite energy; when energy is constrained, the Gaussian shape results.

After many pages of extensive mathematics, Boltzmann arrived at the solution.  The distribution that encompasses the maximum number of complexions is none other than Maxwell’s distribution.  Starting from an entirely new direction in physics, that of pure probability and having nothing to do with collisions or time, Boltzmann demonstrated that Maxwell’s is indeed the most frequently occurring and thus the most probable distribution.

Of course Boltzmann did not stop there.  He went on to demonstrate how the mathematics could be performed for both discrete and continuous energy distributions and then additionally went on to demonstrate how the mathematics could be performed for the distribution involving both location (three coordinates) and momentum (three velocity components), resulting in location-momentum buckets of six dimensions.  This opened the door to understanding the impact of such events as fluid flow and external forces such as gravity on the spatial and momentum coordinates of a collection of molecules, including dissimilar ones.  The inclusion of location also opened the door to incorporating the variable of volume into his mathematics, which enabled the dependency of entropy on both energy and volume to be captured, leading to state function relationships:  S (U,V).[5]

References

[1] [Note: I highly recommend this paper if you want to understand Boltzmann transformation to probability Very well written by Boltzmann and very well summarized by Kim Sharp.] Sharp, Kim, and Franz Matschinsky. 2015. “Translation of Ludwig Boltzmann’s Paper ‘On the Relationship between the Second Fundamental Theorem of the Mechanical Theory of Heat and Probability Calculations Regarding the Conditions for Thermal Equilibrium’ Sitzungberichte Der Kaiserlichen Akademie Der Wissenschaften. Mathematisch-Naturwissen Classe. Abt. II, LXXVI 1877, Pp 373-435 (Wien. Ber. 1877, 76:373-435). Reprinted in Wiss. Abhandlungen, Vol. II, Reprint 42, p. 164-223, Barth, Leipzig, 1909.” Entropy 17 (4): 1971–2009.

[2] Boltzmann actually used molecules in his example.  I chose to use balls.

[3]When converted into the world of logarithms, the sequence of n_iⁿⁱ  terms transforms into the familiar form that some have used ∑ n_i ln (n_i) and the minimization process thus focuses on this term.  Since for large numbers, frequency of occurrence can be transformed to probability to yield another familiar form of this term, ∑ p_i ln (p_i), which will re-appear in the discussion around Claude Shannon and Information Theory.

[4] Dill, Ken A., and Sarina Bromberg. 2011. Molecular Driving Forces: Statistical Thermodynamics in Biology, Chemistry, Physics, and Nanoscience. 2nd ed. London ; New York: Garland Science. Dill demonstrates the need of Lagrange multipliers to solve the mathematics involved.  Although Boltzmann did not specifically reference as such, this is the approach he used.

[5] (Sharp and Matschinsky, 2015) See commentary regarding the importance of including volume together with energy—S(U,V)—in Boltzmann’s statistical mechanics.

END