Entropy Definition

8/9/2019 Entropy Definition

1/28

Lecture 5 Entropy

James Chou

BCMP201 Spring 2008


2/28

A measure of the amount of energy in a system that is available

for doing work; entropy increases as matter and energy in the

universe degrade to an ultimate state of inert uniformity.

A measure of the disorder of a system.

Differentiation of Heat (Q), dQ, is not an exact differential and

therefore cannot be integrated. Therefore we introduce an

integration factor (1/T) such that dQ/T can be integrated. And this

dQ/T is called entropy.

Some common definitions of entropy


3/28


4/28

An interesting observation

Random distribution of

kinetic energy through

random collisions

0-Vx Vx

Consider velocity in the y direction

0 Vx-Vx

f v x( ) =1

2!" 2exp

#v x2

2" 2

! 2= average of v

x " v

x ( )

2

! = v x

2=

k BT

m

0


5/28

Goal of this lecture:

Use the fundamental principle of maximum entropy to explain the physicalproperties of a complex system in equilibrium with the universe.


6/28

Maximum Entropy = Minimum Bias


7/28

Example: Rolling a die with 6 possible outcomes.

The only constraint we have is

P X =1( ) + P X = 2( ) + ...+ P X = 6( ) =1

Without additional information about the die, the most unbiased distribution is such that all

outcomes are equally probable.

P X =1( ) = P X = 2( ) = ... = P X = 6( ) = 1 6

Principle of Maximum-Entropy in Statistics

Given some information or constraints about a random variable, we should choose that

probability distribution for it, which is consistent with the given information, but has otherwisemaximum uncertainty associated with it.


8/28

Shannon’s Measure of Uncertainty

Shannon [1948] suggested the following measure of uncertainty, which is commonlyknown as the statistical entropy .

H = ! pi ln pii=1

N

"

1. H is a positive function of p1, p2 , …, pn.

2. H = 0 if one outcome has probability of 1.

3. H is maximum when the outcomes are equally likely.

In the case of the die, you will find the maximum entropy to be

H = ! pi ln pii=1

6

" = ln6 .


9/28

A quick review on logarithm

loge x = ln x, e = 2.73

ln AB( ) = ln A + ln B , ln A / B( ) = ln A ! ln B , d dx

ln x( ) = 1 x

Stirling approximation: ln N !( ) " N ln N ! N , for very large N


10/28

Shannon’s entropy in terms of the number of possible outcomes.

Example: the number of outcomes ! from rolling the die N times:

! = N !

Np1( )! Np2( )!... Np6( )!

ln! = ln N !" ln Npi( )!

i=1

6

#

Permutation of N numbers between 1 and 6

Factor out the redundant outcomes

Using Stirling’s approximation for very large N , ln N !! N ln N " N , ln# becomes

ln# = N ln N " Npiln Np

i

i=1

6

$ = N ln N " ln N Npi " N pi ln pii=1

6

$i=1

6

$ = " N pi ln pii=1

6

$


11/28

ln! = " N pi ln pi = NH i=1

6

#

H = Const ! ln"

Conclusion: ln! is linearly proportional to H . Therefore, maximizing thetotal number of possible outcomes is equivalent to maximizing Shannon’sstatistical entropy.

Statistical Entropy # of possible outcomes


12/28

Entropy in Statistical Physics

Definition of physical entropy:

S = const ! ln", " = # of possible microstates of a close system.

A microstate is the detailed state of a physical system.

Example: In an ideal gas, a microstate consists of the position and velocity of everymolecule in the system. So the number of microstates is just what Feynman said: the

number of different ways the inside of the system can be changed without changing theoutside.

Principle of maximum entropy (The second law of thermodynamics)

If a closed system is not in a state of statistical equilibrium, its macroscopic state will vary in time, until ultimately the system reaches a state of maximum entropy.Moreover, at equilibrium, all microstates are equally probable.


13/28

S = Const x ln (# of Velocity States X # of Position States)

# of velocity states does not change.# of position states does change

!S = S 2

" S 1

= Const # ln$2

v+ ln$

2

r" ln$

1

v" ln$

1

r

[ ]

An example of maximizing entropy:

!S = Const " ln 2V ( ) N

# lnV N ( ) = Const " N ln 2


14/28

What is temperature?

E1 E2

Not in Equilibrium Equilibrium

E = E 1+ E

2 = const. dE

1= !dE

2

S = Const " ln #1#

2( ) = S 1 E 1( ) + S 2 E 2( )

Maximize S,dS

dE 1

=dS

1

dE 1

+dS

2

dE 2

dE 2

dE 1

=dS

1

dE 1

! dS

2

dE 2

= 0

Picture from hyperphysics.phy-astr.gsu.edu

Temperature T is defined as 1 T = dS dE . The temperatures of bodies in equilibrium with

one another are equal.

At equilibrium,dS 1

dE 1

=

dS 2

dE 2


15/28

Since T is measured at a fixed number of particles N and volume V , a more stringent

definition is

T = dE dS ( ) N ,V .

Thus far, S is defined to be const.! ln "( ). If S is a dimension-less quantity, T has thedimensions of energy (e.g. in units of Joules (J)).

But J is too large a quantity. Example: Room temperature = 404.34 x 10- J !

What is the physical unit of T ?

It is more convenient to measure T in degrees Kelvin (K). The conversion factor betweenenergy and degree is the Boltzmann’s constant , k B = 1.38 X 10

-23 J / K. Hence we redefine

S and T by incorporating the conversion factor.

S = k Bln! and T "T /k

B .


16/28

What does T = dE dS ( ) N ,V mean?

Same change in entropy, but more energy is given away by the system initially with higher T . Hence temperature is a measure of the tendency of an object to spontaneously give upenergy to its surroundings.

e

2e

3eLower T

e

2e

3eHigher T

S 1= k

B ln

5!

2!2!

" #

% &

! E !S = e k B ln 3( )

S 2

= k B ln

5!

3!2!

" #

% &

S 1 = k B ln 5!

2!2!

" #

% & S 2 = k B ln

5!

3!2!

" #

% &

! E !S = 3e k B ln 3( )


17/28

One particle

!v " 4# v

2, $ is the speed in 3D.

N particles !v " 4# v3 N %1

& 4# v 3 N for large N .

Since v " E 1 2

,

S = k

Bln! =

3k B N

2ln E +Const.

Can we derive the equation of state of a gas (PV = nRT ) from the

concept of entropy?

! =!v "!

r !v = # of velocity states

!r = # of position states

Step 1: Evaluate S = k B ln! .


18/28

Step 2: Relate kinetic energy E to temperature.

S = k

B ln! =

3k B N

2ln E +Const.

1

T =dS

dE =

3k B N

2 E " E =

3

2k B NT =

3

2k B nN

0( )T

n = # of moles, N 0 = 6.02 x 1023

mol-1

R = k BN 0 = 8.314 J mol-1

K-1

(Rydberg constant)

Energy ofn mole of ideal gas:

E =3

2

nRT

.

Energy of one ideal gas molecule:

E =3

2k BT .


19/28

For a particle in a box, each collision with a wall

occurs in a time interval of 2 L v x , and change in

momentum (e.g. in x direction), ! p x is 2mv x .

F =! p

x

!t =

2mv x

2 L v x

= mv x2

L .

For N particles in a box,

F = mv x i2

L = Nm v x2

Li=1

N

" .

Step 3: Relate temperature to pressure.

SinceP = F L

2= Nm v

x

2V

and E = N

1

2m v

2= N

3

2m v

x

2

,

we obtain PV =2

3 E .

Finally, since E =3

2nRT , we obtain PV = nRT .

v x

2+ v

y

2+ v

z

2


20/28

How do we deal with the enormous complexity of a biological system?


21/28

Boltzmann and Gibbs Distribution

Goal: Describe the probability distribution of a molecule of interest, with energy E a , in

equilibrium with a macroscopic thermal reservoir with energy E B .

molecule of interest, a

Surrounding, B

In the joint system, the probability of the molecule of interest in a particular state,

E a , is

p E a( ) = ! B E B( ) " P0 = expS B E B( )

k B

$ %

' ( " P0 .

The second law says that at equilibrium, or

maximum entropy, all microstates are equally

probable, with a probability P0 .

S B E

B( ) = k B ln ! B E B( )( )


22/28

We can use the first-order Taylor’s expansion to approximate S B E B( ) because E B is very near E tot .

S B E

B( ) ! S B E tot ( )"dS

B E

tot ( )

dE B

E a

= S B E

tot ( ) "1

k BT E

a

E B

E tot

E a

S E B( )

p E a( ) = expS B E B( )

k B

" #

% & ' P0

?

E tot

= E B+ E

a ! E

B

Hence we obtain

p E a( ) = P0 ! exp S B E tot ( )

k B

# $

& ' exp ( E a

k BT

# $

& ' = A ! exp ( E a

k BT

# $

& '

IMPORTANT: The probability distribution of the molecule of interest in equilibrium with its surroundingdepends only on the temperature of the surrounding.

Boltzmann Distribution, also known

as the Gibbs Distribution


23/28

Random distribution of

kinetic energy through

random collisions

f v x( ) =1

2!" 2exp

#v x2

2" 2

! = v x

2=

k BT

m

P = Aexp ! E k BT

"

# $%

& ' = Aexp

!1

2mv

2

k BT

"

#

$$$

%

&

' ' '

= Aexp !v2

2 k

BT

m

" # $

% & '

"

#

$$$

%

&

' ' '

Now we can explain the velocity distribution at equilibrium using the

Boltzmann distribution


24/28

Example on rate constant and transition state

A k ! " ! B

! E

E a

Transition

state

A

B

The reaction rate constant, k [s-1], is

proportional to exp! E

a

RT

# $

& ' , where E a is the

activation energy, or energy barrier, in units of J mol-1.

k = Aexp ! E

a

RT

# $

& ' Arrhenius equation


25/28

Suppose Ea of a reaction is 100 kJ mol-1 and a catalyst lowers this to

80 kJ mol-1. Approximately how much faster will the reaction proceedwith the catalyst?

High energy barriers result in high specificity in a cellular signaling

pathway.

k catalyzed( )

k uncatalyzed( )=

exp !80 RT ( )

exp !100 RT ( )= e

8" 3000

RT ! 2.5 kJ mol-1 at room temperature


26/28

PrPC

PrPSc

! E ++

" 40 kcal mol-1=167.4 kJ mol

-1

Catalyzed by either mutation

or binding of PrPSc

k

The role of prion conformational switch in neurodegenerative diseases


27/28

What about low energy barrier?

The temperature-gated vanilloid receptor VR1, a pain receptor, is activated byheat T > 45 °C.

! E

E a

A

B

It was found experimentally that the probabilityof VR1 being in the open state is 0.04 at 40°C and 0.98 at 50 °C. What is the energy

barrier?


28/28

Take home messages

T = A measure of the tendency of an object to spontaneously give up energy to itssurroundings.

T = dE dS ( ) N ,V

The Boltzmann & Gibbs Distribution

p E a( ) = A ! exp" E ak BT

$ %

' (

Equilibrium = A system reaching a state of maximum entropy.Equilibrium = All microstates are equally probable.

S = k B ln!

Documents

Entropy Definition