Presentation is loading. Please wait.

Presentation is loading. Please wait.

Maximum Entropy, Maximum Entropy Production and their Application to Physics and Biology Roderick C. Dewar Research School of Biological Sciences The Australian.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Maximum Entropy, Maximum Entropy Production and their Application to Physics and Biology Roderick C. Dewar Research School of Biological Sciences The Australian."— Presentation transcript:

1 Maximum Entropy, Maximum Entropy Production and their Application to Physics and Biology Roderick C. Dewar Research School of Biological Sciences The Australian National University

2  Part 1: Maximum Entropy (MaxEnt) – an overview Part 2: Applying MaxEnt to ecology Part 3: Maximum Entropy Production (MEP) Part 4: Applying MEP to physics & biology

3 The problem: to predict “complex system” behaviour The solution: statistical mechanics - Boltzmann microstate counting (maximum probability) - Gibbs algorithm (MaxEnt) Applications of MaxEnt to equilibrium systems - micro-canonical, canonical, grand-canonical distributions Physical interpretation of MaxEnt - frequency interpretation - information theory interpretation (Jaynes) Extension to non-equilibrium systems (Jaynes) General properties of MaxEnt distributions Part 1: MaxEnt – an overview

4 The problem: to predict “complex system” behaviour The solution: statistical mechanics - Boltzmann microstate counting (maximum probability) - Gibbs algorithm (MaxEnt) Applications of MaxEnt to equilibrium systems - micro-canonical, canonical, grand-canonical distributions Physical interpretation of MaxEnt - frequency interpretation - information theory interpretation (Jaynes) Extension to non-equilibrium systems (Jaynes) General properties of MaxEnt distributions Part 1: MaxEnt – an overview

5 system energy in What is the problem? - to predict the macroscopic behaviour of systems having many interacting degrees of freedom cells, plants, ecosystems, economies, climates … environment matter in many interacting degrees of freedom energy out matter out open non-equilibrium

6 Poleward heat transport SW LW Latitudinal heat transport H = ? 170 W m -2 300 W m -2 TT

7 Cold plate, T c Hot plate, T h Ra < 1760 conduction TT Cold plate, T c Hot plate, T h Ra > 1760 convection H = ? Turbulent heat flow (Raleigh-Bénard convection)

8 F sw F lw + H + E C, H 2 0, O 2, N  T,  Ecosystem energy & mass fluxes

9 system energy in What is the problem? - to predict the macroscopic behaviour of systems having many interacting degrees of freedom cells, plants, ecosystems, economies, climates … environment matter in many interacting degrees of freedom energy out matter out open non-equilibrium statistical mechanics many degrees of freedom  statistical mechanics Global Circulation Models, Dynamic Ecosystem Models ….

10 The problem: to predict “complex system” behaviour The solution: statistical mechanics - Boltzmann microstate counting (maximum probability) - Gibbs algorithm (MaxEnt) Applications of MaxEnt to equilibrium systems - micro-canonical, canonical, grand-canonical distributions Physical interpretation of MaxEnt - frequency interpretation - information theory interpretation (Jaynes) Extension to non-equilibrium systems (Jaynes) General properties of MaxEnt distributions Part 1: MaxEnt – an overview

11 W(A) = number of microstates that give macrostate A Microstate i 1 Macrostate A = less detailed description Ludwig Boltzmann (1844 - 1906) The most probable macrostate A is the one with the largest W(A) (assume microstates are a priori equiprobable) S B (A) = k B log W(A) = Boltzmann entropy of macrostate A The most probable macrostate is the one of maximum entropy Boltzmann microstate counting Microstate i 2

12 Example: N independent distinguishable particles with fixed total energy E Macrostate A = {n j particles are in state j} Microstate i = {the m th particle is in state j m } ε3ε3 ε2ε2 ε1ε1 : maximise S = k B log W subject to (large N)

13 Boltzmann entropy  Clausius entropy β  1/k B T Given E, S max = k B logW max = k B ( β E + NlogZ)  under δ E = δ Q, S max changes by δ S max = k B β(δ Q) cf. Clausius thermodynamic entropy δ S TD = δ Q/T S max  S TD BUT: microstate counting only works for non-interacting particles

14 p i = probability that system is in microstate i Macroscopic predictions via J Willard Gibbs (1839 - 1903) Gibbs algorithm The Gibbs algorithm (MaxEnt) Maximise H = -  i p i log p i with respect to {p i } subject to the constraints (C) on the system But how do we construct p i ? ‘minimise the index of probability of phase’

15 The problem: to predict “complex system” behaviour The solution: statistical mechanics - Boltzmann microstate counting (maximum probability) - Gibbs algorithm (MaxEnt) Applications of MaxEnt to equilibrium systems - micro-canonical, canonical, grand-canonical distributions Physical interpretation of MaxEnt - frequency interpretation - information theory interpretation (Jaynes) Extension to non-equilibrium systems (Jaynes) General properties of MaxEnt distributions Part 1: MaxEnt – an overview

16 Closed, isolated Closed Open Three applications of MaxEnt (equilibrium systems) Microcanonical Canonical Grand-canonical System constraints (C) Distribution (p i )

17 Example 1: closed, isolated system in equilibrium C: N and E fixed Microstate i = any N-particle state with total energy E i restricted to E Precise description of i and E i depends on microscopic physics (CAN include particle interactions) Maximisesubject to basis for Boltzmann’s microstate counting

18 Example 2: closed system in equilibrium Microstate i = any N-particle state (no restriction on E i ) E Maximisesubject to C: N and fixed β  1/k B T H max  S TD

19 Example 3: open system in equilibrium Microstate i = any physically allowed microscopic state (no restriction on E i or N i ) E N Maximisesubject to C: and fixed β  1/k B T γ  -μ/k B T H max  S TD

20 The problem: to predict “complex system” behaviour The solution: statistical mechanics - Boltzmann microstate counting (maximum probability) - Gibbs algorithm (MaxEnt) Applications of MaxEnt to equilibrium systems - micro-canonical, canonical, grand-canonical distributions Physical interpretation of MaxEnt - frequency interpretation - information theory interpretation (Jaynes) Extension to non-equilibrium systems (Jaynes) General properties of MaxEnt distributions Part 1: MaxEnt – an overview

21 Frequency interpretation (Venn, Pearson, Fisher …) System has Ω a priori equiprobable microstates N independent identical systems, n i = no. of systems in state i p i describes a physical property of the real world (frequency) W = no. of microstates giving {n 1,n 2 … n Ω } p i = n i /N = frequency of microstate i MaxEnt coincides with large-N limit of Maximum Probability for multinomial W

22 p i represents our state of knowledge of the real world basic axioms for uncertainty H associated with p i  the unique uncertainty function is Applies to any discrete set of outcomes i p i = 1/6 (i = 1…6)  H = log 6 maximum uncertainty : p i = 0 (i = 1,2...5), p 6 = 1  H = 0 minimum uncertainty : Information theory interpretation (Shannon 1948, Jaynes 1957 …) Claude Shannon (1916-2001)

23 Jaynes (1957b, 1978) Q (= Σ i p i Q i ) reproducible under C it is sufficient to encode only the information C into p i … all information other than C is thrown away … but this is precisely what MaxEnt does! MaxEnt = max H subject to C H = -  i p i log p i = missing information about i Behaviour that is experimentally reproducible under conditions C must be theoretically predictable from C alone Edwin Jaynes (1922-1998)

24 Assumed constraints C experimental conditions conservation laws microstates (e.g. QM) Reproducible behaviour Q Max H subject to C  p i The prediction game Observed behaviour Q obs Q obs  Q  missing constraint MaxEnt test C  C'C  C' ESSENTIAL PHYSICS PREDICTION OBSERVATION

25 The problem: to predict “complex system” behaviour The solution: statistical mechanics - Boltzmann microstate counting (maximum probability) - Gibbs algorithm (MaxEnt) Applications of MaxEnt to equilibrium systems - micro-canonical, canonical, grand-canonical distributions Physical interpretation of MaxEnt - frequency interpretation - information theory interpretation (Jaynes) Extension to non-equilibrium systems (Jaynes) General properties of MaxEnt distributions Part 1: MaxEnt – an overview

26 Edwin Jaynes (aged 14 months) Information theory interpretation of MaxEnt general algorithm for predicting reproducible behaviour under given constraints can be extended to non-equilibrium systems (same principle, different constraints) ‘Maximum caliber principle’ (Jaynes 1980, 1996) cf. Feynman path integral formalism of QM!

27 A B The second law in a nutshell A  B reproducible  W B  W A ' = W A  S B  S A WBWB WAWA WA'WA'.. microscopic path in phase-space after Jaynes 1963, 1988 S = k B log W Liouville Theorem (Hamiltonian dynamics) reproducible macroscopic change

28 The problem: to predict “complex system” behaviour The solution: statistical mechanics - Boltzmann microstate counting (maximum probability) - Gibbs algorithm (MaxEnt) Applications of MaxEnt to equilibrium systems - micro-canonical, canonical, grand-canonical distributions Physical interpretation of MaxEnt - frequency interpretation - information theory interpretation (Jaynes) Extension to non-equilibrium systems (Jaynes) General properties of MaxEnt distributions Part 1: MaxEnt – an overview

29 Some general properties of MaxEnt distributions subject to m + 1 constraints C Response-fluctuation & reciprocity relations: Stability-convexity relation: Constitutive relation: Orthogonality: Partition function:

30 Summary of Lecture 1 … The problem to predict the behaviour of non-equilibrium systems with many degrees of freedom The proposed solution MaxEnt: a general information-theoretical algorithm for predicting reproducible behaviour under given constraints Boltzmann Gibbs Shannon Jaynes

Download ppt "Maximum Entropy, Maximum Entropy Production and their Application to Physics and Biology Roderick C. Dewar Research School of Biological Sciences The Australian."

Similar presentations

Statistical Mechanics and Evolutionary Theory

Statistical Mechanics and Evolutionary Theory
Dr Roger Bennett Rm. 23 Xtn. 8559 Lecture 19.

Dr Roger Bennett Rm. 23 Xtn Lecture 19.
The microcanonical ensemble Finding the probability distribution We consider an isolated system in the sense that the energy is a constant of motion. We.

The microcanonical ensemble Finding the probability distribution We consider an isolated system in the sense that the energy is a constant of motion. We.
Ch2. Elements of Ensemble Theory

Ch2. Elements of Ensemble Theory
–Introduction Second Law Weak nonlocality –Ginzburg-Landau equation –Schrödinger-Madelung equation –Digression: Stability and statistical physics –Discussion.

–Introduction Second Law Weak nonlocality –Ginzburg-Landau equation –Schrödinger-Madelung equation –Digression: Stability and statistical physics –Discussion.
Lecture 4. Entropy and Temperature (Ch. 3) In Lecture 3, we took a giant step towards the understanding why certain processes in macrosystems are irreversible.

Lecture 4. Entropy and Temperature (Ch. 3) In Lecture 3, we took a giant step towards the understanding why certain processes in macrosystems are irreversible.
We’ve spent quite a bit of time learning about how the individual fundamental particles that compose the universe behave. Can we start with that “microscopic”

We’ve spent quite a bit of time learning about how the individual fundamental particles that compose the universe behave. Can we start with that “microscopic”
The Maxwell-Boltzmann Distribution Valentim M. B. Nunes ESTT - IPT April 2015.

The Maxwell-Boltzmann Distribution Valentim M. B. Nunes ESTT - IPT April 2015.
1.The Statistical Basis of Thermodynamics 1.The Macroscopic & the Microscopic States 2.Contact between Statistics & Thermodynamics: Physical Significance.

1.The Statistical Basis of Thermodynamics 1.The Macroscopic & the Microscopic States 2.Contact between Statistics & Thermodynamics: Physical Significance.
Species-Abundance Distribution: Neutral regularity or idiosyncratic stochasticity? Fangliang He Department of Renewable Resources University of Alberta.

Species-Abundance Distribution: Neutral regularity or idiosyncratic stochasticity? Fangliang He Department of Renewable Resources University of Alberta.
Thermodynamics II I.Ensembles II.Distributions III. Partition Functions IV. Using partition functions V. A bit on gibbes.

Thermodynamics II I.Ensembles II.Distributions III. Partition Functions IV. Using partition functions V. A bit on gibbes.
Quantum statistics: a formal approach Reminder to axioms of quantum mechanics  state vector of the Hilbert space describes quantum mechanical system 

Quantum statistics: a formal approach Reminder to axioms of quantum mechanics  state vector of the Hilbert space describes quantum mechanical system 
MSEG 803 Equilibria in Material Systems 8: Statistical Ensembles Prof. Juejun (JJ) Hu

MSEG 803 Equilibria in Material Systems 8: Statistical Ensembles Prof. Juejun (JJ) Hu
AME 60614 Int. Heat Trans. D. B. GoSlide 1 Non-Continuum Energy Transfer: Gas Dynamics.

AME Int. Heat Trans. D. B. GoSlide 1 Non-Continuum Energy Transfer: Gas Dynamics.
Entropy. Optimal Value Example The decimal number 563 costs 10  3 = 30 units. The binary number 1000110011 costs 2  10 = 20 units.  Same value as decimal.

Entropy. Optimal Value Example The decimal number 563 costs 10  3 = 30 units. The binary number costs 2  10 = 20 units.  Same value as decimal.
Statistical Mechanics

Statistical Mechanics
Introduction to Thermostatics and Statistical Mechanics A typical physical system has N A = 6.023 X 10 23 particles. Each particle has 3 positions and.

Introduction to Thermostatics and Statistical Mechanics A typical physical system has N A = X particles. Each particle has 3 positions and.
Maximum Entropy Production and their

Maximum Entropy Production and their
Maximum Entropy, Maximum Entropy Production and their Application to Physics and Biology Roderick C. Dewar Research School of Biological Sciences The Australian.

Maximum Entropy, Maximum Entropy Production and their Application to Physics and Biology Roderick C. Dewar Research School of Biological Sciences The Australian.
1 Ch3. The Canonical Ensemble Microcanonical ensemble: describes isolated systems of known energy. The system does not exchange energy with any external.

1 Ch3. The Canonical Ensemble Microcanonical ensemble: describes isolated systems of known energy. The system does not exchange energy with any external.

Similar presentations

Ads by Google