Histograms and free energies ChE210D - UCSB College of ...Shell/Che210d/Histograms.pdfthe free energy minimum • the free energy change upon binding, from the difference in free energy

© M. S. Shell 2009 1/29 last modified 11/7/2019

Histograms and free energies ChE210D

Today's lecture: basic, general methods for computing entropies and free ener-

gies from histograms taken in molecular simulation, with applications to phase

equilibria.

Overview of free energies Free energies drive many important processes and are one of the most challenging kinds of quan-

tities to compute in simulation. Free energies involve sampling at constant temperature, and

ultimately are tied to summations involving partition functions. There are many kinds of free

energies that we might compute.

Macroscopic free energies

We may be concerned with the Helmholtz free energy or Gibbs free energy. We might compute

changes in these as a function of their natural variables. For single-component systems:

𝐴(𝑇, 𝑉, 𝑁)

𝐺(𝑇, 𝑃, 𝑁)

For multicomponent systems,

𝐴(𝑇, 𝑉, 𝑁1, … , 𝑁𝑀)

𝐺(𝑇, 𝑃, 𝑁1, … , 𝑁𝑀)

Typically we are only interested in the dependence of these free energies along a single param-

eter, e.g.,

𝐴(𝑉), 𝐺(𝑃), 𝐺(𝑇), etc.

for constant values of the other independent variables.

Free energies for changes in the interaction potential

It is also possible to define a free energy change associated with a change in the interaction po-

tential. Initially the energy function is 𝑈0(𝐫𝑁) and we perturb it to 𝑈1(𝐫𝑁). If this change hap-

pens in the canonical ensemble, we are interested in the free energy associated with this pertur-

bation:

Δ𝐴 = 𝐴1(𝑇, 𝑉, 𝑁) − 𝐴0(𝑇, 𝑉, 𝑁)


= −𝑘𝐵𝑇 ln (∫ 𝑒−𝛽𝑈1(𝐫𝑁)𝑑𝐫𝑁

∫ 𝑒−𝛽𝑈0(𝐫𝑁)𝑑𝐫𝑁)

What kinds of states 1 and 0 might we use to evaluate this expression? Here is a small number

of sample applications:

• electrostatic free energy – charging of an atom or atoms in a molecule, in which state 0

has zero partial charges and state 1 has finite values

• dipolar free energy – adding a point dipole to an atom between states 0 and 1

• solvation free energy – one can “turn on” interactions between a solvent and a solute as

a way to determine the free energy of solvation

• free energy associated with a field – states 0 and 1 correspond to the absence and pres-

ence, respectively, of a field, such as an electrostatic field

• restraint free energy – turning on some kind of restraint, such as confining a molecule to

have a particular conformation or location in space. Such restraints would correspond to

energetic penalties for deviations from the restrained space in state 1.

• free energies of alchemical transforms – we convert one kind of molecule (e.g., CH4) to

another kind (e.g., CF4). This gives the relative free energies of these two kinds of mole-

cules in the system of interest (e.g., solvation free energies in solution).

Potentials of mean force (PMFs)

Oftentimes we would like to compute the free energy along some order parameter or reaction

coordinate of interest. These are broadly termed potentials of mean force, for reasons we will

see shortly. This perspective enables us to understand free-energetic driving forces in many pro-

cesses. For the purposes of this discussion, we will notate a PMF by 𝐹(𝜉) where 𝜉 is the reaction

coordinate of interest. This coordinate might be, for example:

• an intra- or intermolecular distance (or combination of distances)

• a bond or torsion angle

• a structural order parameter (e.g., degree of crystallinity, number of hydrogen bonds)

Consider the example of a protein in aqueous solution interacting with a surface. The reaction

coordinate might be the distance between the center of mass of the protein and the surface:


The PMF along 𝑧, 𝐹(𝑧) would give the free energy of the system as a function of the protein-

surface distance. It might look something like:

This curve would show us:

• the preferred distance at which the protein binds to the surface, from the value of 𝑧 at

the free energy minimum

• the free energy change upon binding, from the difference in free energy between the

minimum and large values of 𝑧

• the barrier in free energy for binding and unbinding, from the height of the hump

Importantly, the free energy function does not just include the direct potential energy interac-

tions between atoms in the molecule with atoms in the surface. It also includes the effects of all

of the interactions in the solvent molecules. This may be crucial to the behavior of the system.

𝑧

𝐹(𝑧)

𝑧


For example, the direct pairwise interactions of an alkane with a silica surface will be the same

regardless of whether the solvent is water or octanol. However, the net interaction of the alkane

and surface will be very different in the two cases due to solvent energies and entropies, and this

effect is exactly determined by the PMF.

Definition

Formally, a potential of mean force (the free energy) along some reaction coordinate 𝜉 is given

by a partial integration of the partition function. In the canonical ensemble, we begin with the

configurational part of the Helmholtz free energy,

𝐹(𝜉) = 𝐴𝑐(𝑇, 𝑉, 𝑁, 𝜉)

= −𝑘𝐵𝑇 ln 𝑍(𝑇, 𝑉, 𝑁, 𝜉)

= −𝑘𝐵𝑇 ln ∫ 𝑒−𝛽𝑈(𝐫𝑁)𝛿[𝜉 − 𝜉(𝐫𝑁)]𝑑𝐫𝑁

Here, 𝜉(𝐫𝑁) is a function that returns the value of the order parameter for a particular configu-

ration 𝐫𝑁. The integral in this expression entails a delta function that filters for only those Boltz-

mann factors for configurations with the specified 𝜉.

One can think of the PMF as the free energy when the system is constrained to a given value of

𝜉. Notice that we have the identity

∫ 𝑒−𝛽𝐹(𝜉)𝑑𝜉 = 𝑒−𝛽𝐴𝑐

The potential of mean force is so-named because its derivative gives the average force along the

direction of 𝜉 at equilibrium. We proceed to find the derivative of the PMF:

𝑑𝐹(𝜉)

𝑑𝜉= −𝑘𝐵𝑇

𝑑

𝑑𝜉ln ∫ 𝑒−𝛽𝑈(𝐫𝑁)𝛿[𝜉 − 𝜉(𝐫𝑁)]𝑑𝐫𝑁

= −𝑘𝐵𝑇

𝑑𝑑𝜉 ∫ 𝑒−𝛽𝑈(𝐫𝑁)𝛿[𝜉 − 𝜉(𝐫𝑁)]𝑑𝐫𝑁

∫ 𝑒−𝛽𝑈(𝐫𝑁)𝛿[𝜉 − 𝜉(𝐫𝑁)]𝑑𝐫𝑁

To make progress, we need the mathematical identity

𝑑

𝑑𝑎∫ 𝑔(𝑥)𝛿(𝑥 − 𝑎)𝑑𝑥 = ∫

𝑑𝑔(𝑥)

𝑑𝑥𝛿(𝑥 − 𝑎)𝑑𝑥

This allows us to pull the derivative inside the integral:

𝑑𝐹(𝜉)

𝑑𝜉= −𝑘𝐵𝑇

∫ (𝑑

𝑑𝜉𝑒−𝛽𝑈(𝐫𝑁)) 𝛿[𝜉 − 𝜉(𝐫𝑁)]𝑑𝐫𝑁



= −𝑘𝐵𝑇∫ (−𝛽

𝑑𝑈𝑑𝜉 𝑒−𝛽𝑈(𝐫𝑁)) 𝛿[𝜉 − 𝜉(𝐫𝑁)]𝑑𝐫𝑁


= −∫ 𝑓𝜉𝑒−𝛽𝑈(𝐫𝑁)𝛿[𝜉 − 𝜉(𝐫𝑁)]𝑑𝐫𝑁


Here, the term 𝑓𝜉 gives the force along the direction of 𝜉,

𝑓𝜉 = −𝑑𝑈

𝑑𝜉

= −𝑑𝐫𝑁

𝑑𝜉⋅ ∇𝑈

=𝑑𝐫𝑁

𝑑𝜉⋅ 𝐟𝑁

The remainder of the terms in the PMF equation serve to average the force for a specified value

of 𝜉. Thus,

𝑑𝐹(𝜉)

𝑑𝜉= −⟨𝑓𝜉(𝜉)⟩

Paths

Keep in mind that free energies are state functions. That is, if we are to compute a change in any

free energy between two conditions, we are free to pick an arbitrary path of interest between

them. This ultimately lends flexibility to the kinds of simulation approaches that we can take to

compute free energies.

Overview of histograms in simulation Until now, we have focused mainly on computing property averages in simulation. Histograms,

on the other hand, are concerned with computing property distributions. These distributions can

be used to compute averages, but they contain much more information. Importantly, they relate

to the fluctuations in the ensemble of interest, and ultimately can be tied to statistical-mechan-

ical partition functions. It is through this connection that histograms enable us to compute free

energies and entropies.

Definitions and measurement in simulation

For the purposes of illustration, we will consider a histogram in potential energy. In our simula-

tion, we might measure the distribution of the variable 𝑈 using a long simulation run and many

observations of the instantaneous value of 𝑈.


In classical systems, the potential energy is a continuously-varying variable. Therefore, the un-

derlying ℘(𝑈) is a continuous probability distribution. However, in the computer we must meas-

ure a discretized version of this distribution.

• We specify a minimum and maximum value of the energy that defines a range of energies

in which we are interested. Let these be 𝑈min and 𝑈max.

• We define a set of 𝑚 bins into which the energy range is discretized. Each bin has a bin

width of

𝛿𝑈 =𝑈max − 𝑈min

𝑚

• Let the variable 𝑘 be the bin index. It varies from 0 to 𝑚 − 1. The average energy of bin

𝑘 is then given by

𝑈𝑘 = 𝑈min + (𝑘 +1

2) 𝛿𝑈

• We create a histogram along the energy bins. This is simply an array in the computer that

measures counts of observations:

𝑐𝑘 = counts of 𝑈 observations in the range [𝑈𝑘 − 𝛿𝑈 2⁄ , 𝑈𝑘 + 𝛿𝑈 2⁄ )

For the sake of simplicity, we will often write the histogram array using the energy, rather than

the bin index,

𝑐(𝑈) = 𝑐𝑘 where 𝑘 = int (𝑈 − 𝑈min

𝛿𝑈)

Here the int function returns the integer part of its argument. For example, int(2.6) = 2.

To determine a histogram in simulation, we perform a very large number of observations 𝑛 from

a long, equilibrated molecular simulation. At each observation, we update:

𝑐(𝑈) ← 𝑐(𝑈) + 1

This update is only performed if 𝑈min ≤ 𝑈 < 𝑈max. Otherwise, the energy would be outside of

the finite range of energies of interest. However, we still need to keep count of all energies,

whether or not inside the range, in order to properly normalize our histogram.

We can normalize the histogram to determine a discretized approximation to the true underlying

distribution ℘(𝑈) in the energy range of interest:


℘̃(𝑈)𝛿𝑈 =𝑐(𝑈)

𝑛

where 𝑛 is the total number of observations, including those energies outside of the defined

range. On the LHS we include the bin width so as to approximate the continuous probability

differential ℘(𝑈)𝑑𝑈. Thus,

℘̃(𝑈) =𝑐(𝑈)

𝑛𝛿𝑈

In the limit of an infinite number of observations from an infinitely long, equilibrated simulation,

this approximation converges to the true one in the following manner:

℘̃(𝑈𝑘)𝛿𝑈 = ∫ ℘(𝑈)𝑑𝑈𝑈𝑘+𝛿𝑈 2⁄

𝑈𝑘−𝛿𝑈 2⁄

This equation simply says that we sum up all of the underlying probabilities for the continuous

energies within a bin to obtain the observed, computed probabilities. As the bin width goes to

zero, we have

lim𝛿𝑈→0,𝑛→∞

℘̃(𝑈) = ℘(𝑈)

Notice that there are two components to this limit:

• We need an infinite number of observations.

• We need an infinite number of bins.

These two limits “compete” with each other: as we increase the number of bins, we need more

observations so that we have enough counts in each bin to have good statistical accuracy. Prac-

tically speaking, we must choose a finite bin width that enables us to balance the length of the

run with statistical accuracy in each bin. Typically, for the energy example above,

• the bin width is chosen to be of the order of a basic energy scale in the force field. For a

Lennard-Jones system, this might be 𝜖.

• the simulation is performed long enough to achieve on the order of ~1000 or more aver-

age counts per bin, that is, 𝑛 ≥ 𝑚 × 1000.

Statistical considerations

Keep in mind that the computation of a histogram is subject to the same statistical considerations

as simple simulation averages. That is, the histogram needs to be performed for many correlation


times of the energy 𝑈 in order to reach good statistical accuracy. It can be shown that the ex-

pected squared error in the histogram bin 𝑐(𝑈) goes as

𝜎𝑐(𝑈)2 ∝ 𝑐(𝑈)

This implies that the error goes as the square root as the number of counts. Similarly, the ex-

pected squared error in the corresponding estimate ℘̃(𝑈) goes as:

𝜎℘̃(𝑈)2 ∝

℘(𝑈)

𝑛

The relative error in ℘̃(𝑈) is given by:

𝜎℘̃(𝑈)

℘̃(𝑈)∝

1

√𝑛℘(𝑈)

Notice that the relative error is higher for lower values of ℘(𝑈), i.e., at the tails of the distribu-

tion.

Multidimensional histograms

In this example, we considered only a histogram of potential energy. However, it is possible to

construct histograms of many kinds of simulation observables. In other ensembles, we might

compute distributions of other fluctuating macroscopic quantities like 𝑁 in GCMC simulations

and 𝑉 in 𝑁𝑃𝑇 simulations. We can also view histograms of arbitrary parameters of interest, such

as the end-to-end distance of a polymer chain or that number of hydrogen bonds a water mole-

cule makes.

We can also compute joint distributions using multidimensional histogram arrays. For example,

℘̃(𝑈, 𝑉)𝛿𝑈𝛿𝑉 =𝑐(𝑈, 𝑉)

𝑛

Note that any continuous variables will require discretization and specification of a bin width.

Discrete variables, on the other hand, do not require such a definition because the underlying

distribution itself is discrete:

℘̃(𝑁) =𝑐(𝑁)

𝑛

Many kinds of distributions, however, require the specification of a minimum and maximum ob-

servable value.


Connection to statistical mechanics

The true power of histograms is that they allow us to measure fluctuations in the simulation that

can be used to extract underlying partition functions. That is, we measure ℘(𝑈) in simulation

and then we post-process this discretized function to make connections to free energies and en-

tropies.

Consider the energy distribution ℘(𝑈). Its form in the canonical ensemble is given by the ex-

pression:

℘(𝑈) =Ωc(𝑈, 𝑉, 𝑁)𝑒−𝛽𝑈

𝑍(𝑇, 𝑉, 𝑁)

where 𝑍(𝑇, 𝑉, 𝑁) is the configurational partition function,

𝑍(𝑇, 𝑉, 𝑁) = ∫ Ωc(𝑈, 𝑉, 𝑁)𝑒−𝛽𝑈𝑑𝑈

Here, Ω𝑐 is the configurational density of states. It is the part of the microcanonical partition

function that corresponds to the potential energy and configurational coordinates. It is defined

by the equation

Ω𝑐(𝑈, 𝑉, 𝑁) = ∫ 𝛿[𝑈(𝐫𝑁) − 𝑈]𝑑𝐫𝑁

The reason we are concerned only with the configurational energy distribution is that the kinetic

component can always be treated analytically, within the context of equilibrium simulations.

We can rewrite both using the dimensionless configurational entropy:

𝑆𝑐(𝑈, 𝑉, 𝑁) ≡ ln Ω𝑐(𝑈, 𝑉, 𝑁)

℘(𝑈) =𝑒𝑆𝑐(𝑈,𝑉,𝑁)−𝛽𝑈

𝑍(𝑇, 𝑉, 𝑁) 𝑍(𝑇, 𝑉, 𝑁) = ∫ 𝑒𝑆𝑐(𝑈,𝑉,𝑁)−𝛽𝑈𝑑𝑈

Basic histogram analysis and reweighting For the remainder of this section we will drop the subscript “c” from the configurational proper-

ties, for simplicity.

The above formalism shows that if we were able to compute the function 𝑆(𝑈, 𝑉, 𝑁), we would

be able to predict the complete energy distribution ℘(𝑈) at any temperature. In fact, things are

a bit simpler than this: we only need the energy-dependence of 𝑆, since the volume and number

of particles stay fixed.


For example, we can compute the average potential energy at any temperature using the expres-

sion

⟨𝑈⟩(𝑇) = ∫ 𝑈℘(𝑈; 𝑇)𝑑𝑈

= ∫𝑈𝑒𝑆(𝑈)−𝛽𝑈𝑑𝑈

𝑍(𝑇)

=∫ 𝑈𝑒𝑆(𝑈)−𝛽𝑈𝑑𝑈

∫ 𝑒𝑆(𝑈)−𝛽𝑈𝑑𝑈

Here, we use the notation ℘(𝑈; 𝑇) to signify a distribution at a given specified temperature 𝑇.

Extracting the entropy and density of states from a histogram

We might extract an estimate for 𝑆𝑐 by measuring ℘(𝑈; 𝑇) from a histogram in a canonical sim-

ulation at specified temperature 𝑇. Inverting the above relationship,

𝑆(𝑈) = ln [𝑍(𝑇)℘(𝑈; 𝑇)

𝑒−𝛽𝑈]

= ln ℘(𝑈; 𝑇) + 𝛽𝑈 + ln 𝑍(𝑇)

= ln ℘(𝑈; 𝑇) + 𝛽𝑈 − 𝛽𝐴(𝑇)

Here, 𝐴(𝑇) technically denotes the configurational part of the Helmholtz free energy, which is

𝑘𝐵𝑇 ln[𝑁! Λ(𝑇)3𝑁] less than the total free energy in a single component system.

We can measure ℘(𝑈; 𝑇) using a histogram along a set of discrete energies. Post-simulation, we

can then take this measured distribution and use it to compute a discrete approximation to the

dimensionless entropy function at the same energies,

𝑆(𝑈𝑘) = ln ℘(𝑈𝑘; 𝑇) + 𝛽𝑈𝑘 − 𝛽𝐴(𝑇)

Notice that the partition function part of this expression is a temperature-dependent constant

that is independent of potential energy 𝑈. Thus, we can compute the entropy function 𝑆 to within

an additive constant using this approach. This also says that we can compute the configurational

density of states to within a multiplicative constant. This basic inversion of the probability distri-

bution function underlies many histogram-based methods.

Keep in mind that 𝑆(𝑈) is fundamentally independent of temperature since it stems from the

microcanonical partition function. That is, any temperature-dependencies on the RHS of this

equation should exactly cancel to leave a 𝑇-independent function. Another way of putting this

is that, in principle, it does not matter at which temperature we measure ℘(𝑈; 𝑇). At equilib-

rium, the ℘(𝑈; 𝑇) from every temperature should give the same 𝑆(𝑈) function.


Free energy differences from histograms

Imagine that we measure ℘(𝑈; 𝑇1) and ℘(𝑈; 𝑇2) from two simulations at different tempera-

tures. We can use the relationship of both with the density of states to compute a free energy

difference. We construct

ln℘(𝑈; 𝑇2)

℘(𝑈; 𝑇1)= [𝑆(𝑈) − 𝛽2𝑈 + 𝛽2𝐴(𝑇2)] − [𝑆(𝑈) − 𝛽1𝑈 + 𝛽1𝐴(𝑇1)]

= −(𝛽2 − 𝛽1)𝑈 + 𝛽2𝐴(𝑇2) − 𝛽1𝐴(𝑇1)

Rearranging,

𝛽2𝐴(𝑇2) − 𝛽1𝐴(𝑇1) = ln℘(𝑈; 𝑇2)

℘(𝑈; 𝑇1)+ (𝛽2 − 𝛽1)𝑈

This expression provides us with a way to determine the configurational free energy difference

between temperatures 1 and 2 using measured histograms of the energy distribution.

You may find it interesting that the RHS is dependent on 𝑈, whereas the LHS is not. In fact, the

free energies should have no 𝑈-dependence. In principle, any value of 𝑈 could be plugged into

the RHS and the same free energy would be returned. In practice, statistical errors in the meas-

ured ℘(𝑈) mean that the estimate for the free energy can have different accuracies depending

on the value of 𝑈 chosen. More on this below.

Reweighting

With a computed a discrete approximation to the underlying energy distribution, we can also use

it to predict ℘(𝑈) at temperatures other than the original simulation temperature. This basic

procedure is called reweighting, since we use a distribution measured at one temperature to

predict that at another. The scheme is as follows.

Imagine 𝑆(𝑈) is computed from ℘(𝑈; 𝑇1) in a canonical simulation at 𝑇1. We have

𝑆(𝑈) = ln ℘(𝑈; 𝑇1) + 𝛽1𝑈 − 𝛽1𝐴(𝑇1)

We want to predict ℘(𝑈; 𝑇2) at another temperature 𝑇2. We have,

℘(𝑈; 𝑇2) =𝑒𝑆(𝑈)−𝛽2𝑈

𝑍(𝑇2)

Plugging in the above expression for 𝑆,

℘(𝑈; 𝑇2) = ℘(𝑈; 𝑇1)𝑒−(𝛽2−𝛽1)𝑈𝑍(𝑇1)

𝑍(𝑇2)


However, the last term on the RHS involving the ratio of partition functions is independent of 𝑈.

We can find it using the probability normalization condition,

∫ ℘(𝑈; 𝑇2)𝑑𝑈 = 1

Thus,

℘(𝑈; 𝑇2) =℘(𝑈; 𝑇1)𝑒−(𝛽2−𝛽1)𝑈

∫ ℘(𝑈; 𝑇1)𝑒−(𝛽2−𝛽1)𝑈 𝑑𝑈

This equation is an important one. It states that a distribution measured at 𝑇1 can be used to

predict a distribution at a different 𝑇2. Thus, in principle, we would only need to perform a single

simulation, measure ℘(𝑈) once, and use this expression to examine the energy distribution at

any other temperature of interest. For example, we might compute the average energy as a

function of temperature using the equation

⟨𝑈⟩(𝑇2) = ∫ 𝑈℘(𝑈; 𝑇2)𝑑𝑈

=∫ 𝑈℘(𝑈; 𝑇1)𝑒−(𝛽2−𝛽1)𝑈 𝑑𝑈

∫ ℘(𝑈; 𝑇1)𝑒−(𝛽2−𝛽1)𝑈 𝑑𝑈

Similar expressions could be found for other moments of the potential energy, such as ⟨𝑈2⟩.

These could be used to compute the temperature dependence of the heat capacity, using the

relationship 𝑘𝐵𝑇2𝐶𝑉 = ⟨𝑈2⟩ − ⟨𝑈⟩2.

Statistical analysis

Unfortunately, there are practical limits to the histogram reweighting procedure just described.

The main problem is the measurement of ℘(𝑈) to good statistical accuracy in the tails of its

distribution. Consider that a typical ℘(𝑈) is very sharply peaked:

℘(𝑈)

𝑈


The width of the distribution relative to the mean goes roughly as 𝑁−1

2 so that, for macroscopic

systems, the distribution is infinitely peaked.

The implication of this result is that it is very hard to measure the distribution at it tails, where

we typically only have a few counts in each bin. If we reweight a measured distribution to a

temperature where the tails change to high probability, the error can be magnified to be very

large.

Here, the mean of the distribution at the new temperature is well within the tail region of the

distribution at the original temperature. If we reweight ℘(𝑈; 𝑇1) to 𝑇2, errors in the tails will be

magnified. The error in the new distributions can be written approximately as

𝜎℘(𝑈;𝑇2)2 ≈ 𝜎℘(𝑈;𝑇1)

2 (℘(𝑈; 𝑇2)

℘(𝑈; 𝑇1))

2

This formula is derived using standard error propagation rules and the reweighting expression.

Notice that if 𝑇2 = 𝑇1, the error is the same at each energy as from the original measurement.

Otherwise, we must weigh the error by a ratio involving the two probability distributions. In the

above picture, the ratio at the mean energy

℘(𝑈∗; 𝑇2)

℘(𝑈∗; 𝑇1)

is very large due to the small probability of this energy at 𝑇1. Therefore, the error is greatly mag-

nified in the reweighting procedure according to the above equation.

In general,

• Distributions are subject to statistical inaccuracies at their tails, owing to the finite num-

ber of counts in each bin.

℘(𝑈)

𝑈

𝑇2 > 𝑇1

𝑇1

𝑈∗


• If the important parts of the energy distribution at 𝑇2 correspond to the tails of a meas-

ured distribution at 𝑇1, a reweighting procedure will fail due to large statistical errors.

• If computed from a measured ℘(𝑈; 𝑇), the quality of an estimated 𝑆𝑐(𝑈) function is only

good around the frequently-sampled energies, i.e., for energies where the histogram has

many entries.

These techniques limit the determination of free energies and entropies using measurements

from single histograms. The solution here, as we will see next, it to incorporate multiple histo-

gram estimates of these quantities.

Multiple histogram reweighting Imagine that we were able to take histograms from multiple simulations 𝑖, each at a different

temperature 𝑇𝑖. For each, we could get a different estimate of the underlying dimensionless

configurational entropy function 𝑆:

𝑆(𝑈) = ln ℘(𝑈; 𝑇𝑖) + 𝛽𝑖𝑈 − 𝛽𝑖𝐴(𝑇𝑖)

We should get the same function every time. We don’t know the free energies, but they are

independent of 𝑈 and thus can be treated as additive constants. We can therefore shift the

different estimates for the entropy function up or down in value until they all match each other:

℘(𝑈)

𝑈

𝑇2 > 𝑇1

𝑇1

𝑇3 > 𝑇2

ln ℘(𝑈) + 𝛽𝑈

+ const

𝑈


This is the basic idea behind multiple-histogram methods. Notice that by finding the amounts to

shift each curve to achieve overlap, we are essentially determining free energy differences be-

tween the adjacent state points.

The trick in these calculations is to determine the shift amounts. The best ways of doing this take

into account the different errors for each energy in each measured ℘(𝑈). Below we describe

methods for computing free energy differences so as to maximize the statistical quality of the

predicted results.

For this approach to work, we must have good overlap between the histograms. That is, there

must be ranges of the histograms that overlap with a reasonable number of counts in each, for

statistical efficiencies. Otherwise, it will be challenging to determine the shift amounts to good

statistical accuracy.

Ferrenberg-Swendsen reweighting and WHAM

Ferrenberg and Swendsen in 1989 proposed an optimal way to stitch together different histo-

grams by minimizing the statistical error in the computed density of states and entropy function.

It was later generalized by [Kumar et al., 1992] and was named the Weighted Histogram Analysis

Method (WHAM). Although the original derivation relied on certain forms for the error propaga-

tion, the same result can be derived using a more elegant maximum likelihood approach, which

is what we discuss here.

The maximum likelihood approach is a statistical method for parameterizing probability models.

It simply says the following: posit some form of the underlying probability distribution function

for a random process. Then, given an observed set of events, the best parameters for that dis-

tribution are those that maximize the probability (likelihood) of the observed events.

In this context, imagine we are trying to determine S(𝑈) from 𝐽 simulations at different temper-

atures 𝑇𝑗. We know that the underlying S(𝑈) for each should be the same, but that the measured

histograms 𝑐𝑗(𝑈) will be different because the simulations are performed at different tempera-

tures. The observed events are the energies tabulated in the histograms.

Say that we make 𝑛 independent observations of the energy 𝑈 in each temperature. Here, inde-

pendent implies that we are beyond twice the correlation time. In order to make a connection

with the histogram, we will discretize all of energy space into discrete values 𝑈𝑘 separated by

intervals 𝛿𝑈. Before we start, let’s define some notation:

• 𝑛 – number of observations at each temperature

• 𝑖 – index of observations (𝑖 = 1, … , 𝑛)


• 𝑗 – index of temperature (𝑗 = 1, … , 𝐽)

• 𝑘 – index of discrete energy values

• 𝑈𝑘 – discrete energy values

• Ω𝑘, 𝑆𝑘 – density of states and entropy at the values 𝑈𝑘

• 𝑈𝑖𝑗 – energy of the 𝑖th observation at temperature 𝑗

• 𝑐𝑗𝑘– count of histogram entries for temperature 𝑗 and energy bin 𝑘

Derivation

With these notations, we construct the total probability or likelihood 𝐿 of making the 𝑛𝐽 obser-

vations from the simulations in terms of a yet-unknown unknown density of states function:

𝐿 = ∏ ∏ ℘(𝑈𝑖𝑗; 𝑇𝑗)

𝐽

𝑗=1

𝑛

𝑖=1

= ∏ ∏ (𝑒𝑆(𝑈𝑖𝑗)−𝛽𝑗𝑈𝑖𝑗

𝑍𝑗)

𝐽

𝑗=1

𝑛

𝑖=1

Here, the 𝑍𝑗 are formally given by

𝑍𝑗 = ∫ 𝑒𝑆(𝑈)−𝛽𝑗𝑈𝑑𝑈

In terms of our discrete energies,

𝑍𝑗 = ∑ 𝑒𝑆𝑘−𝛽𝑗𝑈𝑘

𝑘

Note that

𝑍𝑗 = 𝑒−𝛽𝐴𝑗

where the 𝐴𝑗 are the free energies at each temperature.

With all of these considerations, we can rewrite the above probability in terms of histograms:

𝐿 = ∏ ∏ (𝑒𝑆𝑘−𝛽𝑗𝑈𝑘

𝑍𝑗)

𝑐𝑗𝑘𝐽

𝑗=1𝑘


= ∏ ∏(𝑒𝑆𝑘−𝛽𝑗𝑈𝑘+𝛽𝑗𝐴𝑗 )𝑐𝑗𝑘

𝐽

𝑗=1𝑘

Taking the logarithm,

ln 𝐿 = ∑ ∑ 𝑐𝑗𝑘[𝑆𝑘 − 𝛽𝑗𝑈𝑘 + 𝛽𝑗𝐴𝑗]

𝐽

𝑗=1𝑘

= ∑ ∑ 𝑐𝑗𝑘[𝑆𝑘 − 𝛽𝑗𝑈𝑘]

𝐽

𝑗=1𝑘

+ 𝑛 ∑ 𝛽𝑗𝐴𝑗

𝐽

𝑗=1

According to the maximum likelihood approach, we want to maximize this probability with re-

spect to any adjustable parameters in our model and the given observations. The former are the

yet unknown values 𝑆𝑘 while the latter are the histograms 𝑐𝑗𝑘 . Therefore, we take the derivative

with respect to 𝑆𝑘 and set it equal to zero:

0 =𝜕 ln 𝐿

𝜕𝑆𝑘 for all 𝑘

Evaluating,

0 = ∑ (𝑐𝑗𝑘 + 𝑛𝛽𝑗

𝜕𝐴𝑗

𝜕𝑆𝑘)

𝐽

𝑗=1

The latter derivative is

𝛽𝑗

𝜕𝐴𝑗

𝜕𝑆𝑘= −

𝜕 ln 𝑍𝑗

𝜕𝑆𝑘

= −𝑒𝑆𝑘−𝛽𝑗𝑈𝑘

𝑍𝑗

Substituting back in:

0 = ∑(𝑐𝑗𝑘 − 𝑛𝑒𝑆𝑘−𝛽𝑗𝑈𝑘+𝛽𝑗𝐴𝑗)

𝐽

𝑗=1

for all 𝑘

We can solve for 𝑆𝑘:

𝑒𝑆𝑘 = [∑ 𝑐𝑗𝑘

𝐽

𝑗=1

] [𝑛 ∑ 𝑒−𝛽𝑗𝑈𝑘+𝛽𝑗𝐴𝑗

𝐽

𝑗=1

]

−1


Notice that this equation now provides us with a recipe for computing the density of states (the

LHS) based on histogram data. Let’s rewrite it without the index 𝑘 for clarity:

𝑒𝑆(𝑈) =𝑐𝑡𝑜𝑡(𝑈)

𝑛[∑ 𝑒−𝛽𝑗𝑈+𝛽𝑗𝐴𝑗

𝐽

𝑗=1

]

−1

Here, we have made the definition that 𝑐𝑡𝑜𝑡(𝑈) gives the total number of histogram counts of

energy 𝑈, at any temperature.

Even though this expression gives us a way to determine the entropy function from multiple sim-

ulation results, note that the RHS involves the free energy at every temperature. This free energy

depends on the entropy itself:

𝐴𝑗 = −𝛽𝑗−1 ln ∑ 𝑒𝑆(𝑈)−𝛽𝑗𝑈

𝑈

Iterative solution

How can we solve for 𝑆(𝑈)? Ferrenberg and Swendsen suggested an iterative solution:

1. An initial guess is made for the 𝐽 values of 𝐴𝑗. One can simply choose 𝐴𝑗 = 0 for all 𝑗.

2. The (discretized) entropy at every energy is computed using

𝑆(𝑈) = ln 𝑐𝑡𝑜𝑡(𝑈) − ln 𝑛 − ln ∑ 𝑒−𝛽𝑗𝑈+𝛽𝐴𝑗

𝐽

𝑗=1

for all 𝑈

3. The free energies are recalculated using

−𝛽𝑗𝐴𝑗 = ln ∑ 𝑒𝑆(𝑈)−𝛽𝑗𝑈

𝑈

for all 𝑗

4. Steps 2 and 3 are repeated until the free energies no longer change upon each iteration.

In practice, one typically checks to see if the free energies change less than some frac-

tional tolerance.

5. According to the above equations, the entropy function and the combinations 𝛽𝑗𝐴𝑗 can-

not be determined to within more than an additive constant. Thus, with each iteration,

one typically sets one of the free energies equal to zero at one temperature, 𝐴𝑗=1 = 0.


This iterative procedure can be fairly computationally expensive since it involves sums over all

temperatures and all discretized energies with each iteration. It is not uncommon to require on

the order of 100-1000 iterations to achieve good convergence.

However, once convergence has been achieved, energy averages at any temperature—even tem-

peratures not included in the original dataset—can be computed with the expression

℘(𝑈; 𝑇) ∝ 𝑒𝑆(𝑈)−𝛽𝑈

The constant of proportionality is determined by normalizing the probability distribution.

This general approach is termed multiple histogram reweighting, and it can be shown to be the

statistically optimal method for reweighting data from multiple simulations. That is, it makes the

maximum use of information and results in reweighted distributions for ℘(𝑈; 𝑇) that have the

least statistical error. Importantly, it provides statistically optimal estimates of free energy dif-

ferences 𝐴𝑗 − 𝐴𝑗+1.

Configurational weights

Using the multiple histogram technique, it is possible to define a weight associated with each

configuration 𝑖 at each temperature 𝑗 at the reweighting temperature 𝑇:

𝑤𝑖𝑗 =𝑒𝑆(𝑈𝑖𝑗)−𝛽𝑈𝑖𝑗

𝑐𝑡𝑜𝑡(𝑈𝑖𝑗)

In practice, the logarithm of the weights are maintained in the computer rather than the weights

themselves, so as to maintain precision.

Any property depending on the configurational coordinates can be determined by a sum over all

of the 𝑛𝐽 observations, using the weights:

⟨𝑋⟩ =∑ ∑ 𝑤𝑖𝑗𝑋𝑖𝑗

𝑛𝑖=1

𝐽𝑗=1

∑ ∑ 𝑤𝑖𝑗𝑛𝑖=1

𝐽𝑗=1

Here, 𝑋𝑖𝑗 is the value of the property 𝑋 for the 𝑖th configuration observed at temperature 𝑗. For

example, one can compute the mean potential energy,

⟨𝑈⟩ =∑ ∑ 𝑤𝑖𝑗𝑈𝑖𝑗

𝑛𝑖=1

𝐽𝑗=1

∑ ∑ 𝑤𝑖𝑗𝑛𝑖=1

𝐽𝑗=1


However, keep in mind that 𝑋 can be any property of interest. The multiple histogram technique

provides a way to determine its average at any arbitrary reweighting temperature 𝑇 and provides

a statistically optimal estimator of the average.

This approach also enables one to determine the distribution ℘(𝑋) using the weights:

℘(𝑋) ∝ ∑ ∑ 𝑤𝑖𝑗𝛿𝑋𝑖𝑗,𝑋

𝑛

𝑖=1

𝐽

𝑗=1

That is, for a given value of 𝑋, the probability ℘(𝑋) includes all of the weights from configurations

𝑖𝑗 that have 𝑋𝑖𝑗 = 𝑋. Again, the constant of proportionality stems from the probability normali-

zation condition.

Binless implementation

One possible drawback of this approach is that it requires us to specify a bin width 𝛿𝑈 for com-

puting histograms and tabulating energies. It is also possible to follow this derivation in the limit

that 𝛿𝑈 → 0. In this limit, the expression for the free energies are:

−𝛽𝑗𝐴𝑗 = − ln 𝑛 + ln [∑ ∑ 𝑒−𝛽𝑗𝑈𝑖𝑙 ( ∑ 𝑒−𝛽𝑚𝑈𝑖𝑙+𝛽𝑚𝐴𝑚

𝐽

𝑚=1

)

−1𝑛

𝑖=1

𝐽

𝑙=1

]

where 𝑙 and 𝑚 are also indices over temperature. As with the previous approach, this equation

must be iterated to solve for the free energies 𝐴𝑗. Notice that it involves a quadruple loop, once

over observations and three times over temperatures (the indices 𝑖, 𝑗, 𝑚). Therefore, conver-

gence can be much slower than the histogram version.

Once the free energies are determined using this expression, the expression for the configuration

weights is:

𝑤𝑖𝑗 ∝𝑒−𝛽𝑈𝑖𝑗

∑ 𝑒𝛽𝑙𝐴𝑙−𝛽𝑙𝑈𝑖𝑗𝐽𝑙=1

Precision issues

It is very important to keep in mind that the combined exponentiation and logarithm functions

in these expressions require careful treatment in numerical evaluation in the computer in order

to avoid precision inaccuracies. More on this issue is described in the document “Simulation best

practices.” As a quick example, consider the computation of


−𝐹 = ln ∑ 𝑒𝑤𝑖

𝑀

𝑖=1

To get around this numerical precision issues, we rearrange this expression so that the terms in

the exponential are better-behaved:

−𝐹 = ln [(𝑒𝑤max

𝑒𝑤max) ∑ 𝑒−𝑤𝑖

𝑀

𝑖=1

]

= 𝑤max + ln ∑𝑒𝑤𝑖

𝑒𝑤max

𝑀

𝑖=1

= 𝑤max + ln ∑ 𝑒𝑤𝑖−𝑤max

𝑀

𝑖=1

where

𝑤max = max 𝑤𝑖

Other thermodynamic potentials The multiple histogram technique is not limited to determining 𝑆(𝑈). It can also be used to de-

termine the joint functions 𝑆(𝑈, 𝑁) or 𝑆(𝑈, 𝑉) if histograms are taken not only in energy but in

particle number of volume fluctuations, respectively. Such might be the case in grand-canonical

or isothermal-isobaric MC simulations. The Ferrenberg-Swendsen equations change slightly in

each case, but the derivation proceeds conceptually as above. We will discuss one implementa-

tion in the context of phase equilibria, below.

Alternatively, one can find other underlying thermodynamic potentials, besides the entropy 𝑆,

using the reweighting approach. Imagine, for example, that we perform 𝐽 simulations in the

grand-canonical ensemble, each at the same temperature 𝑇 but with a different chemical poten-

tial 𝜇𝑗. For each, we collect a histogram in the particle number, 𝑐𝑗𝑘 where 𝑘 is now an index over

different values of 𝑁.

Statistical mechanics shows that the histogram of 𝑁 in the grand canonical ensemble relates to

the Helmholtz free energy,

℘(𝑁; 𝜇) =𝑄(𝑁)𝑒𝛽𝜇𝑁

Ξ(𝜇)

=𝑒−𝛽𝐴(𝑁)+𝛽𝜇𝑁

Ξ(𝜇)


Here, the dependence of 𝐴 and Ξ on 𝑇 and 𝑉 has been suppressed since these variables are

constant throughout.

The total likelihood is therefore

℘𝑡𝑜𝑡 = ∏ ∏ ℘(𝑁𝑖𝑗; 𝑇𝑗)

𝐽

𝑗=1

𝑛

𝑖=1

= ∏ ∏ (𝑒−𝛽𝐴(𝑁𝑖𝑗)+𝛽𝜇𝑗𝑁𝑖𝑗

Ξ𝑗

)

𝐽

𝑗=1

𝑛

𝑖=1

A derivation similar to the one above shows that the reweighting equations providing the solu-

tion to 𝐴(𝑁) and the 𝐽 values Ξ𝑗 are:

𝐴(𝑁) = ln 𝑐𝑡𝑜𝑡(𝑁) − ln 𝑛 − ln ∑ 𝑒𝛽𝜇𝑗𝑁+𝛽𝐹𝑗

𝐽

𝑗=1

for all 𝑁

−𝛽𝐹𝑗 = ln ∑ 𝑒𝐴(𝑁)+𝛽𝜇𝑗𝑁

𝑁

for all 𝑗

Notice here the change in roles relative to the previous example

• 𝑁 takes the place of 𝑈.

• 𝜇 takes the place of 𝑇.

• Instead of computing 𝑆(𝑈), we compute the underlying function 𝐴(𝑁). That is, we find

the partition function corresponding to variations in 𝑁 fluctuations at constant 𝑇, since

all of our simulations were at the same temperature.

• We compute 𝐹𝑗 = −𝑘𝐵𝑇 ln Ξ𝑗 at each chemical potential, rather than 𝐴𝑗 at each temper-

ature. By statistical mechanics, 𝐹𝑗 = −𝑃𝑗𝑉. Thus, the differences between these weights

provide information about the pressure differences.

It is possible to determine many different kinds of free-energetic quantities in this way. In gen-

eral,

• To compute a free energy or entropy 𝐹 as a function of some macroscopic parameter 𝑋,

the simulation must accomplish fluctuations in 𝑋. For example, 𝐴(𝑉) requires fluctua-

tions in 𝑉 and thus would necessitate use of the isothermal-isobaric ensemble.


• To have good statistical accuracy, a wide range of fluctuations in 𝑋 must be sampled. This

can be accomplished by performing multiple simulations at different values of the param-

eter conjugate to 𝑋. To compute 𝐴(𝑉), different pressures 𝑃 would be imposed for mul-

tiple simulations at the same temperature.

• The multiple-histogram reweighting procedure will compute 𝐹(𝑋) as well as the relative

partition functions at the different simulation conditions. In the example for 𝐴(𝑉), we

would find 𝐴(𝑉) as well as 𝐺𝑗, where 𝐺 is the Gibbs free energy of the isothermal-isobaric

ensemble.

Potentials of mean force (PMFs) The connection between a PMF and simulation observables stems from the probability distribu-

tion of the reaction coordinate of interest. Based upon the fact that the total partition function

can be expressed as a sum over the PMF partition function, we have that

℘(𝜉) =𝑒−𝛽𝐹(𝜉)

𝑍

That is, the probability that one will see configurations with any value of 𝜉 is proportional to a

Boltzmann factor involving the potential of mean force at 𝜉.

This approach enables us to measure PMFs in simulation by computing histograms along reaction

coordinates. We have,

𝐹(𝜉) = −𝑘𝐵𝑇 ln ℘(𝜉) − 𝑘𝐵𝑇 ln 𝑍

= −𝑘𝐵𝑇 ln ℘(𝜉) + const

Notice that 𝑍 provides an additive constant independent of the reaction coordinate. Typically,

we cannot determine the PMF in absolute units, which would require the determination of the

absolute free energy of the system. Instead, we can only compute 𝐹(𝜉) curves to within an ad-

ditive constant. This is not normally a problem, however, since we are usually only interested in

relative free energies along 𝜉.

One of the limitations of this basic approach is that we cannot compute large changes in free

energies due to the statistics of the histogram. Imagine that we examine a free energy difference

at two values of 𝜉, 𝐹(𝜉2) − 𝐹(𝜉1). Say that this value is equal to 4𝑘𝐵𝑇, which is a moderate

energy equal to about 2 kcal/mol at room temperature. The relative probabilities at these two

values would be

℘(𝜉2)

℘(𝜉1)= 𝑒−𝛽(𝐹(𝜉2)−𝐹(𝜉1))


= 𝑒−4

≈ 0.02

If we were to measure a histogram and the bin for 𝜉1 contained 1000 entries, the bin for 𝜉2 would

only contain ~20 entries. Thus, the statistical error for the latter would be quite high.

The main point here is that it is difficult to measure free energies for rarely sampled states. One

way around this, which we will discuss in a later lecture, is to bias our simulation to visit these

states. Though the introduction of the bias modifies the ensemble we are in, we can exactly

account for this fact in the computation of thermodynamic properties.

Weighted Histogram Analysis Method (WHAM)

If multiple simulations are performed at different state conditions (e.g., different 𝑇,𝑃, or 𝜇), the

Ferrenberg-Swendsen reweighting approach can also be used to compute potentials of mean

force at some set of target conditions. This is quite straightforward.

As before, imagine that 𝐽 simulations are performed at different conditions. In each simulation

𝑗, 𝑛 observations 𝑖 are made. In addition to computing the energy, one tallies the value of the

reaction coordinate for each observation:

𝜉𝑖𝑗

Then, the PMF at some final set of conditions (the reweighting temperature, chemical potential,

and/or pressure) is found by tallying the configurational weights to compute the distribution in

𝜉:

℘(𝜉) ∝ ∑ ∑ 𝑤𝑖𝑗𝛿𝜉𝑖𝑗,𝜉

𝑛

𝑖=1

𝐽

𝑗=1

Ultimately, this distribution will need to be performed in a discretized array if 𝜉 is a continuous

variable. Then, the PMF simply comes from

𝐹(𝜉) = −𝑘𝐵𝑇 ln ℘(𝜉)

Histogram methods for computing phase equilibria The computation of phase equilibria—saturation conditions, critical points, phase diagrams,

etc.—is a major enterprise in the molecular simulation community. There are many ways that

one might go about this, including the Gibbs ensemble, and they are discussed in detail in [Chipot

& Pohorille, Free Energy Calculations: Theory and Applications in Chemistry and Biology,

Springer, 2007]. Of all of these methods, however, the current gold-standard for phase


equilibrium computations is one that relies on multiple histogram reweighting. This approach is

often the quickest and has the maximum accuracy.

Here we will discuss these calculations in the context of liquid-vapor equilibria. Other phase be-

havior—such as liquid-solid, vapor-solid, or liquid-liquid (in mixtures)—can be treated using this

approach as well, although special techniques may be needed to treat structured phases.

Grand canonical perspective

The basic idea of our calculations is the following:

• For a given temperature, we want to find the conditions at which phase equilibrium oc-

curs. In a single component system, we only have one free parameter to specify this con-

dition in addition to 𝑇, per the Gibbs phase rule. This could be 𝑃 or 𝜇.

• We will focus on 𝜇 instead of 𝑃 and perform simulations in the grand canonical ensemble.

Thus, we want to find the value of 𝜇 at coexistence that generates an equilibrium between

liquid and vapor phases. We could equally choose 𝑃 and perform 𝑁𝑃𝑇 simulations; how-

ever, particle insertions and deletions are generally much more efficient than volume fluc-

tuations and thus we go with the former for computational reasons.

• If the state conditions place us at conditions of phase equilibrium (i.e., on the liquid-vapor

phase line), we expect the GCMC simulation to spontaneously fluctuate between a dense

liquid (low 𝑈, large 𝑁) and dilute gas (high 𝑈, small 𝑁). These fluctuations can be ob-

served by examining the joint distribution

℘(𝑈, 𝑁)

• At phase equilibrium, ℘(𝑈, 𝑁) will be bimodal, and the total probability weight under the

two peaks will be equal. One peak corresponds to the liquid phase, and the other to the

gas phase.

• We will find the conditions of phase equilibrium (𝑇, 𝜇) that result in a bimodal ℘(𝑈, 𝑁).

We will generate ℘(𝑈, 𝑁; 𝑇, 𝜇) by reweighting simulation data from multiple simulations,

each not necessarily on the phase coexistence line.

GCMC implementation

We need to gather enough histogram data so that we will be able to reweight ℘(𝑈, 𝑁; 𝑇, 𝜇) with

good statistical accuracy. We perform 𝐽 simulations in the grand-canonical ensemble at different

temperatures 𝑇𝑗 and chemical potentials 𝜇𝑗. For each we measure the joint distribution

℘(𝑈, 𝑁; 𝑇𝑗, 𝜇𝑗). The temperatures and chemical potentials should be chosen so that the energy


and particle number distributions all overlap with each other and span the complete range of

liquid-to-gas densities. Typically we perform the following kinds simulations:

• one near the (presumed) critical point that has large fluctuations in 𝑈 and 𝑁

• one in the gas phase

• several in the liquid phase, of varying temperature

The point is that we must have good histogram statistics for all the particle numbers and energies

that will ultimately be relevant at phase equilibrium.

Once these histograms are taken, Ferrenberg-Swendsen iterations are used to compute the free

energy at each state point. The relevant equations for GCMC simulations are:

𝑆(𝑈, 𝑁) = ln 𝑐𝑡𝑜𝑡(𝑈, 𝑁) − ln 𝑛 − ln ∑ 𝑒−𝛽𝑗𝑈+𝛽𝑗𝜇𝑗𝑁+𝛽𝑗𝐹𝑗

𝐽

𝑗=1

for all 𝑈, 𝑁

−𝛽𝐹𝑗 = ln ∑ ∑ 𝑒𝑆(𝑈,𝑁)−𝛽𝑗𝑈+𝛽𝑗𝜇𝑗𝑁

𝑁𝑈

for all 𝑗

This procedure is performed until values for the discretized entropy 𝑆(𝑈, 𝑁) and free energies

𝐹𝑗 = −𝑘𝐵𝑇𝑗 ln Ξ𝑗 converge. Keep in mind that one value of 𝐹 must be set equal to zero.

Once these values are determined, the joint distribution ℘(𝑈, 𝑁) can be computed for an arbi-

trary reweighting 𝑇, 𝜇 using

𝑈

𝑁

near-critical simulation

gas-phase simulation

liquid-phase simulation

lower-𝑇 liquid-phase

simulation


℘(𝑈, 𝑁; 𝑇, 𝜇) ∝ 𝑒𝑆(𝑈,𝑁)−𝛽𝑈+𝛽𝜇𝑁

The constant of proportionality is given by the normalization condition.

Using the reweighting equation, one finds phase equilibrium by adjusting 𝜇 given a value of 𝑇

until a bimodal distribution appears and the integral under each peak (the total probability) is

the same. Notice that this is a very fast operation since the Ferrenberg-Swendsen reweighting

only needs to be performed once, at the beginning, to determine 𝑆(𝑈, 𝑁).

The following figure shows what this distribution might look like, taken from a review article by

Panagiotopoulos [Panagiotopoulos, J Phys: Condens Matter 12, R25 (2000)].

By repeating this procedure at different reweighting temperatures, one can map out the phase

diagram. In the 𝑇-𝜌 plane, this might look like:

T

r

gas liquid

critical point

gas + liquid


The average density of each phase can be determined using:

⟨𝜌⟩𝐿 =1

𝑉

∑ ∑ 𝑁℘(𝑈, 𝑁)𝑁>𝑁∗𝑈

∑ ∑ ℘(𝑈, 𝑁)𝑁>𝑁∗𝑈 ⟨𝜌⟩𝐺 =

1

𝑉

∑ ∑ 𝑁℘(𝑈, 𝑁)𝑁<𝑁∗𝑈

∑ ∑ ℘(𝑈, 𝑁)𝑁<𝑁∗𝑈

Here, 𝑁∗ is the value of 𝑁 at the minimum between the two peaks in the distribution.

Pressures can be found using the fact that

𝛽𝑃𝑉 = ln Ξ(𝜇, 𝑇, 𝑉)

We cannot compute absolute pressures using this equation, because we cannot compute Ξ ab-

solutely (one must be set to zero). However, we can compare the pressures of two different

state points:

𝛽2𝑃2𝑉 − 𝛽1𝑃1𝑉 = lnΞ(𝜇2, 𝑇2, 𝑉)

Ξ(𝜇1, 𝑇1, 𝑉)

= ln∑ ∑ 𝑒𝑆(𝑈,𝑁)−𝛽2𝑈+𝛽2𝜇2𝑁

𝑁𝑈

∑ ∑ 𝑒𝑆(𝑈,𝑁)−𝛽1𝑈+𝛽1𝜇1𝑁𝑁𝑈

By letting one state point correspond to a very dilute, high temperature gas phase, we can com-

pute its absolute pressure using the ideal gas law. This equation then can be used to relate the

pressure at other state points back to the ideal gas reference.

Critical point reweighting and finite-size scaling

How can one predict the location of the critical point? It turns out that systems have a kind of

universal behavior at this point in the phase diagram. For many fluids, their behavior at the crit-

ical point follows the same trends as the three-dimensional Ising model (or, equivalently, the

lattice gas); that is, they fall in the Ising universality class or the Ising criticality class. In particu-

lar, the probability distribution ℘(𝑈, 𝑁) assumes a universal form.

We can locate the true critical point by finding reweighting values of 𝑇𝑐 , 𝜇𝑐 so that ℘(𝑈, 𝑁) maps

onto the universal Ising distribution, which has been computed to high accuracy by a number of

researchers.

Near the critical point, fluctuations become macroscopic in size. That means that computed

properties become sensitive to the size of the system being studied. In particular, the computed

𝑇𝑐 , 𝜇𝑐 can have a substantial dependence on the system size 𝑉. Fortunately, these parameters

have a scaling law behavior with 𝑉. To locate the infinite system size 𝑇𝑐 , 𝜇𝑐 one performs multiple

studies for different values of 𝑉, each time computing the critical point using the procedure

above. Then, one extrapolates to 𝑉 → ∞. An example of this taken from [Panagiotopoulos,

2000] is:


Documents

Histograms and free energies ChE210D - UCSB College of ...Shell/Che210d/Histograms.pdfthe free energy minimum • the free energy change upon binding, from the difference in free energy