1. Spatial decay of correlations and efficient methods for computing partition functions. David Gamarnik Joint work with Antar Bandyopadhyay (U of Chalmers), Dmitriy Rogozhnikov-Katz (MIT) June, 2006

2. Talk Outline Partition functions. Where do we see them ? Computing partition functions. Monte Carlo method. Correlation decay. Our results: computation tree, correlation decay and Deterministic algorithm for approximate computation of partition functions for matchings and colorings. Structural results and large deviations. Conclusions

3. Partition functions - feature in statistical mechanics Gibbs measure and Ising models computer science and combinatorics counting problems queueing theory product form loss networks electrical engineering coding theory statistics bayesian networks

4. Calls arrive as and request communication link Call is accepted only if no other link attached to is occupied Unaccepted call is lost Call duration is Queueing Example: loss system with shared resources

5. At any moment the set of occupied links is a matching The steady-state distribution is product form: - partition function.

6. Calls arrive as and occupy a node Call is accepted only if no neighbor is occupied Unaccepted call is lost Call duration is Example II: multicasting in a communication network

7. At any moment the set of occupied nodes is an independent set The steady-state distribution is product form: - partition function.

8. Calls arrive as and occupy a node and use frequency Call is accepted only if no neighbor is occupied and uses the same fr. Unaccepted call is lost Call duration is Example III: multicasting with many frequencies

9. At any moment the set of occupied nodes is a partial coloring The steady-state distribution is product form: - partition function.

10. Communication (matching) problem with From queueing to statistical physics - Gibbs distribution on Ising type models. Important object in stat mechanics. - inverse temperature - Monomer-dimer model.

11. Matching problem with From statistical physics to computer science total number of matchings in the graph (counting)

12. Can we compute partition function?... … easily when the underlying graph is a tree. Example (independent sets) This leads to

13. Theorem. Spitzer [75], Zachary [83,85], Kelly [85]. In -ary tree Is independent from the boundary condition (correlation decay) if and only if Ramanan, Sengupta, Zeidins, Mitra [2002] Related work on unicasting and multicasting on trees Implication: if the graph is locally-tree like, then computing marginals is possible in the regime

14. Computing partition function in general Valiant [1979] -- #P complexity class. Exact counting is hard for most of the counting problems (matchings, independent sets, colorings, etc. ) Focus – approximate counting. Our contribution: - use of correlation decay for - Deterministic (non-simulation based) algorithms for computing approximately partition functions for Matchings in low degree graphs Colorings in low degree graphs - Structural properties of partition functions in special classes of graphs

15. Existing approaches for computing partition function Main approximation method: Markov Chain Monte Carlo (MCMC) The MCMC is based on - computing the marginal distribution via simulation. - reducing partition function to marginals (cavity method). Jerrum, Valiant & Vazirani [86] Technical challenge: establishing rapid mixing

16. Computing partition functions using MCMC Jerrum [95].Coloring Vigoda [2000]. ColoringColoring Jerrum & Sinclair [89]Matchings Dyer, Frieze & Kannan [91]Volume of a convex body. Jerrum, Sinclair & Vigoda [2004].Permanents

17. (Temporal) Decay of correlations in Markov chains A Markov chain with transition matrix satisfies decay of correlation (mixes) if and only if it is aperiodic (Spatial) Decay of correlations Same thing, but time is replaced by a “spatial” distance

18. Correlation Decay A sequence of spatially (graph) related random variables exhibits a decay of correlation (long-range independence), if when is large Principle motivation - statistical phyisics. Uniqueness of Gibbs measures on infinite lattices, Dobrushin [60s].

19. What is known about correlation decay ? Spitzer [75], Zachary [83,85], Kelly [85]. Independent sets -ary tree J. van den Berg [98]Matchings Goldberg, Martin & Paterson [05]. Coloring. General graphs Jonasson [01]. Coloring. Regular trees Link between Correlation Decay and rapid mixing of MC: CD implies rapid mixing in subexp. growing graphs. Converse not true Kenyon, Mossel & Peres [01].

20. Our results: Theorem I. There exists a deterministic algorithm for computing approximately the partition function corresponding to matchings in graphs with constant degree (deterministic FPTAS), for arbitrary Related work: Weitz [2005]. Self-avoiding walk based algorithm for counting independent sets when

21. Algorithm and proof: Step I. Reduce computing partition function to computing marginals (cavity method) Thus computing marginals implies computing the partition function

22. Step II. Cavity recursion

25. Algorithm: repeat the recursion times. Initialize at the bottom arbitrarily. Compute recursively. - Computation tree

26. Proposition. The computation tree satisfies the decay of correlation property Proof: look at the recursion function: Introduce change of variables:

27. Mean Value Theorem: - contraction

30. Theorem II. There exists a deterministic algorithm for computing approximately the number of list colorings in triangle-free graphs when the size of each list is constant and for all nodes

31. Cavity recursion

32. x x x

33. We establish correlation decay for this recursion x x x

34. Why can’t we use conventional decay of correlation directly for counting by computing marginals locally for small (constant) ? Problem: We need accuracy in order to have accuracy

35. But:

36. Theorem III. The partition function of independent sets in every r-regular locally tree- like graphs satisfies when Structural results The decay of correlation property implies the following large deviations results:

37. Queueing/large deviations interpretation 1. In a multicasting model (independent sets) the probability that nobody is transmitting a signal is 2. The probability that the set of active nodes is is given as

38. These results are not “provable” using MCMC technique Structural results Theorem IV. The partition function of the number of q-colorings in every r-regular graph with large girth satisfies

39. Note: removing a node when computing marginals destroys regularity A fix comes from a rewiring trick Mezard-Parisi [05]. Lemma. The rewiring operation can be performed onpairs of nodes without creating small cycles.

40. Final thoughts and goals Queueing and stationarity. Consider a queueing version of the “matching” problem. Assume FIFO. Does the loss of stationarity occur before or after onset of long-range dependence?

41. Final thoughts and goals Create an implementable version of our algorithm (aka Belief Propagation). Our algorithm is only nominally efficient. Combining algorithm with importance sampling to handle large degree instances. Other counting problems: permanent, volume of a polyhedron. What other structures have the underlying computation tree satisfy the correlation decay property? Markov random fields?