1. Gibbs measures on trees
Elchanan Mossel,
U.C. Berkeley
9/22/2018

2. Gibbs Measures on Trees:
Lecture Plan
Gibbs Measures on Trees:
Uniqueness
Reconstruction
Mixing times on trees
Building Trees (Phylogeny)
Some analytical problems.
Gibbs Measures on Trees and Other Graphs
Mixing Times.
Belief Propagation.
The Replica Method.
9/22/2018

3. Gibbs Measures P[] = Z-1 £
A Gibbs Measure on a (finite) graph G=(V,E) is given by
Node potentials (v : v 2 V) and
Edge Potentials (e : e 2 E)
The probability of  = ((v) : v 2 V) 2 A|V| is given by
P[] = Z £
v 2 V v[(v)] £
e=(v,u) 2 Ee[(v),(u)] G
Gibbs measures introduced in Statistical Physics.
Essential in Machine Learning.
Also known as MRF’s, Graphical Models etc.
9/22/2018

4. Uniqueness and Reconstruction
Let (v,L) := ((w) : d(v,w) = L).
Let (v,L)(a) := P[ (v) = a | (v,L)] – P[(v) = a]
Let Gn be a family of Gibbs measures:
Uniqueness := limL ! 1 sup { |(v,L)|1 : v 2 Gn} = 0
Reconstruction := limL ! 1 sup { |(v,L)|2 : v 2 Gn}  0
Informally:
Uniqueness := 8 values of (v,L >> 1), (v) has same dist.
Reocn := (v) is typically independent of (v,L >> 1)
(v,L)
(v) L G
9/22/2018

5. Gibbs measures on trees
On a finite tree, a Gibbs
measure P can be written as:
Using recursions easy to calculate: P[(v) = . | (v,L)]
) Easy to determine uniqueness when extreme (v,L) are known (Ising, Potts, Independent sets …)
Open Problem 1: Given the d-ary tree and a general M, determine uniqueness.
Open Problem 2: Convex asymptotic geometry of
P[(v) = . | (v,L)] as L ! 1
P[] =
[(0)] £ {e = u ! v} Me(u),(v) + + + + - + + + - - + - + +
Assume Me are identical.
Tree is d-ary tree.
9/22/2018

6. Gibbs measures on trees – a story
Let Mi,j = P[hair(daughter) = j | hair(mother) = i]
Suppose we know the tree T of all mothers going back to Eve.
“Uniqueness”: Is there any assignment of hair color to current population that will yield information on Eve’s?
“Reconstruction”: Do we expect to have information on Eve’s hair color from current population?
9/22/2018

7. Reconstruction: Recursive Reconstruction
= Binary symmetric channel (BSC)
= Ising model (no external field)
T = 3-ary regular tree with Me = M for all edges.
Consider the recursive majority function.
Let pn := P[ n-fold rec-maj((0,n)) = (0) ] .
Let (p) = (1-) p +  (1-p) and g(p) = 3(p)+32(p)(1-(p))
p0 = 1 and pn+1 = g(pn) ) pn ! ½ if and only if (1-2) > 2/3.
) Reconstruction if  < 1/6.
Von-Neumann (56) for reliable noisy-computation.
Later: Evans-Schulmann93, Steel94, Mossel98.
9/22/2018

8. Spectral Reconstruction
Let M be the Ising (BSC) model on a b-ary tree T.
Let f(n) = Maj(n) = sign({(v) : v 2 Ln}).
Theorem (Higuchi 77):
limn P[0 = f(n)] > ½ if b(1-2)2 > 1.
) Reconstruction for ternary tree if  < ½ - (1/3)1/2.
Let M be any chain and T the b-ary tree
Let  be the 2nd eigenvalue of M in absolute value.
Claim[Kesten-Stigum66] b |  |2 > 1 ) Reconstruction.
b |  |2 =1 is also threshold for census [MosselPeres04] and robust [Janson-Mossel04] reconstruction.
9/22/2018

9. Non Reconstruction - Coupling
Copying rule. For i =+,-:
P[i ! i] =  = 1 – 2 
P[i ! Uniform] = 1– = 2 
Continuing down the tree, non-coupled elements form a branching process with parameter .
+ / -
+ / - = =
+ / - = = = = = = = = = =
If 2  · 1, branching process dies ) coupling.
) for  ¸ ¼ no reconstruction (this is not tight!)
The threshold for reconstruction is only for
Ising (BSC) model is given by 22 = 1.
9/22/2018

10. Ising Model on Binary Trees
low
interm.
high
bias
bias
no bias
“+”
boundary
“+”
boundary
bias
no bias
“typical”
boundary
2 2 > 1
“typical”
boundary
2  > 1
22 < 1
2  < 1
Unique Gibbs measure
The transition at 2 2 = 1 was proved by:
Bleher-Ruiz-Zagrebnov95,Evans-Kenyon-Peres-Schulman2000,Ioffe99,
Kenyon-Mossel-Peres-2001,Martinelli-Sinclair-Weitz2004.

11. Reconstruction for Markov models
So the threshold b  2 = 1 is important.
But [M-2000] it is not the threshold for extemality
Not even for 2 £ 2 markov chains.
Open: What is the threshold for q=3 Potts on binary tree?
Very Recent[Borgs-Chayes-M-Roch]: b  2 =1 for slightly asymmetric channels.
9/22/2018

12. Gibbs Measures on Trees:
Lecture Plan
Gibbs Measures on Trees:
Uniqueness
Reconstruction
Mixing times on trees
Building Trees (Phylogeny)
Some analytical problems.
Gibbs Measures on Trees and Other Graphs
Mixing Times.
Belief Propagation.
The Replica Method.
9/22/2018

13. Glauber dynamics: sampling Gibbs measures
Consider the following dynamics on configuration  of Gibbs measure G.
At rate 1:
Pick a vertex v uniformly at random, and update σ(v) according to the conditional probability given {σ(w): w ~ v}.
Easy: Converges to
Gibbs distribution.
Hard: How quickly?
Measure convergence in terms of Markov Operator. G
9/22/2018

14. Ising Model on Binary Trees
low
interm.
high
bias
bias
no bias
bias
no bias
“typical”
boundary
2 2 > 1
“typical”
boundary
2  > 1
22 < 1
2  < 1
Unique Gibbs measure
2 = (n1 + 2 log2  )
Reconstruction
No-Reconstruction, 2 =O(1)
9/22/2018
In Berger-Kenyon-M-Peres05

15. Relaxation time for the binary tree
On Trees: Fast mixing  No-Reconstruction.
Vs. Common wisdom: Fast mixing  Uniqueness.
Martinelli-Sinclair-Weitz05:
Log-Sob behaves in the same way as Spectral-Gap.
Study external-fields and boundary conditions …
9/22/2018

16. Gibbs Measures on Trees:
Lecture Plan
Gibbs Measures on Trees:
Uniqueness
Reconstruction
Mixing times on trees
Building Trees (Phylogeny)
Some analytical problems.
Gibbs Measures on Trees and Other Graphs
Mixing Times.
Belief Propagation.
The Replica Method.
9/22/2018

17. Phylogeny
“Phylogeny is the true evolutionary relationships between groups of living things”
Noah
Shem
Japheth
Ham
Cush
Kannan
Mizraim
9/22/2018

18. Pyhlogenetic Inference
In “phylogenetic inference”
The tree is unknown.
Given a sequence of collections of random variables at the leaves (“species”).
Collections are i.i.d.
Want to reconstruct the tree (un-rooted).
9/22/2018

19. Pyhlogenetic Reconstruction
9/22/2018

20. Markov Model of Evolution
… …
Simplest evolution model: binary symmetric channel
CFN Model:
Tree: T = (V,E)
Node states:
Mutation probabilities:
s(r)
pra
s(a)
prc
pab
pa3
s(b) 1
s(c)
pc4
pc5
pb1
pb2 1 1
0: Purines (A,G)
1: Pyrimidines (C,T)
s(1)
s(2)
s(3)
s(4)
s(5)
9/22/2018

21. Phylogenetic Inference Problem
Given: i.i.d. samples at the leaves
Task: Reconstruct the model, i.e. find tree and do so efficiently
Efficiency:
1) Computational: Running time of reconstruction algorithm
2) Information-theoretic: Sequence length required for successful reconstruction
Let n = # leaves (species)
k = length of sequences needed.
s(1)
s(2)
s(3)
s(4)
s(5) 1 1 1 1 1 1 1 1 1 1 1 1 1 1
pb2
prc +
pra
pa3
pc5
9/22/2018

22. Phylogeny: Conjectures and Results
Reconstruction
Phylogeny
Reconstruction
conj
k = O(log n)
No Reconstruction
conj
k = poly(n)
Percolation
critical  = 1/2
Random Cluster
MS03
Ising model
critical : 22 = 1
CFN
Mo04
DMR05
9/22/2018

23. Polynomial Lower Bound at High Mutations
Proof: ?
Known
* k
X=T L
q-L
9/22/2018

24. Logarithmic Reconstruction
Th2 [M 2004]: If T is an tree on n leaves s.t.
For all e, min < (e)< max and 22min > 1, max < 1.
Then k = O(log n – log ) characters suffice to infer the topology with probability 1- .
Caveat: Need a balanced tree – all leaves at the same distance from a root.
Th3 [Daskalakis-M-Roch 2005] Above result holds for general trees.
+ Cameron,Hill,Rao [2006]: Experimental performence.
9/22/2018

25. Two-Step Algorithm [M 2004]:
Balanced Trees
Two-Step Algorithm [M 2004]:
1) Reconstruct one (or a few) level(s) – using distance estimation.
2) Infer sequences at roots using recursive majority.
3) Start over
9/22/2018

26. General Trees [Daskalakis, M, Roch, 2005]
9/22/2018

27. Gibbs Measures on Trees:
Lecture Plan
Gibbs Measures on Trees:
Uniqueness
Reconstruction
Mixing times on trees
Building Trees (Phylogeny)
Some analytical problems.
Gibbs Measures on Trees and Other Graphs
Mixing Times.
Belief Propagation.
The Replica Method.
9/22/2018

28. Main analytical problems
How to analyze recursions of the random measures (,L)?
No general techniques are known (some easy methods follow).
Needed for
General boundary conditions:
Worst case (uniqueness)
Average case (Reconstruction)
Other.
non-regular trees (strong spatial mixing) and for
families of random trees (optimal error correcting codes).
9/22/2018

29. Gibbs Measures on Trees:
Lecture Plan
Gibbs Measures on Trees:
Uniqueness
Reconstrution
Mixing times on trees
Building Trees (Phylogeny)
Some analytical problems.
Gibbs Measures on Trees and Other Graphs
Mixing Times.
Belief Propagation.
The Replica Method.
9/22/2018

30. Conjecture: Uniqueness on tree / graphs
Consider Gibbs measures where
All edge potentials are identical: e =  for all e
All node potentials are trivial : v = 1 for all v.
Graph is regular of degree d.
Conjecture:
Gibbs measure unique on d-regular tree )
Gibbs measure unique on any family of d regular graphs.
Recently proved by Weitz for anti-ferromagnet Ising models.
Trivial for random graphs. G
9/22/2018 T

31. Conjecture: Uniqueness on tree / graphs
Very Recently [M-Weitz-Wormald-06]:
For the hard-core model:
Non-uniqueness of Gibbs measure on 3-regular tree )
Exp. Slow mixing on random 3-regular graphs.
Reconstruction on random 3-regular graphs.
Moral: Slow/Rapid mixing on “typical” graphs is determined by uniqueness on trees.
Still don’t really know how to prove for
4-regular graphs
Other models.
9/22/2018

32. Gibbs Measures on Trees:
Lecture Plan
Gibbs Measures on Trees:
Uniqueness
Reconstruction
Mixing times on trees
Building Trees (Phylogeny)
Some analytical problems.
Gibbs Measures on Trees and Other Graphs
Mixing Times.
Belief Propagation.
The Replica Method.
9/22/2018

33. Belief Propagation in AI
Belief Propagation (BP) is a popular method in AI/Coding for estimating marginal probabilities P[(0) = a] for a Gibbs measure G.
It is equivalent [TatikondaJordan02] to calculating marginal probabilities P[(0) = a] on the computation tree,T(G).
In particular, uniqueness on infinite computation tree ) convergence of BP.
Uniqueness + High girth )
Convergence to correct marginals
Open “problem”: Is uniqueness needed?
Why BP works also when girth is small? G T
9/22/2018

34. Belief Propagation in Coding
BP is used to decode Low Density Parity Check Codes [Gallager62]
Proved to be efficient without “uniqueness”[LMSS,RSU]
Recursive Analysis – up to girth of graph.
Open Problem: Is BP efficient beyond girth?
Open Problem: Can LDPC codes achieve Channel Capacity?
9/22/2018

35. Replica Symmetry Breaking in Physics
Replicas are recursive distributional equations used to calculate probabilities for spinglasses (random codes, random SAT problems).
Symmetric Replicas “” Belief Propogation.
Symmetry Breaking Replicas “” Survey Propogation.
[MezardMontanari06] Claim: Symmetry Breaks exactly when reconstruction emerges.
Open problem/Conjecture: Is the reconstruction threshold on d-ary tree the “right” threshold for spin-glasses on random d-regular graphs?
9/22/2018

36. Gibbs Measures on Trees:
Lecture Plan
Gibbs Measures on Trees:
Uniqueness
Reconstruction
Mixing times on trees
Building Trees (Phylogeny)
Some analytical problems.
Gibbs Measures on Trees and Other Graphs
Mixing Times.
Belief Propagation.
The Replica Method.
9/22/2018

37. 9/22/2018

38. A reminder: Markov Chains
A Markov Chain on a (finite) set S is given by an initial distribution  and transition probabilities  ti,j.
The probability of ((t))t=0T 2 AT+1 is given by
[(0)] £ t=0T-1  t(t),(t+1)  0 1 2
time 1 2 3
9/22/2018