Combinatorial optimization and the mean field model Johan Wästlund Chalmers University of Technology Sweden

Random instances of optimization problems

Typical distance between nearby points is of order n -1/2

Random instances of optimization problems A tour consists of n links, therefore we expect the total length of the minimum tour to scale like n 1/2 Beardwood-Halton-Hammersley (1959):

Mean field model of distance Distances X ij chosen as i.i.d. variables Given n and the distribution of distances, study the random variable L n If the distribution models distances in d dimensions, we expect L n to scale like n 1-1/d In particular, pseudo-dimension 1 means L n is asymptotically independent of n

Mean field model of distance The edges of a complete graph on n vertices are given i. i. d. nonnegative costs Exponential(1) distribution.

Mean field model of distance We are interested in the cost of the minimum matching, minimum traveling salesman tour etc, for large n.

Mean field model of distance Convergence in probability to a constant?

Matching Set of edges that gives a pairing of all points

Statistical Physics / C-S Spin configuration Hamiltonian Ground state energy Temperature Gibbs measure Thermodynamic limit Feasible solution Cost of solution Cost of minimal solution Artificial parameter T Gibbs measure n→∞

Statistical physics Replica-cavity method of statistical mechanics has given spectacular predictions for random optimization problems M. Mézard, G. Parisi 1980’s Limit of   /12 for minimum matching on the complete graph (Aldous 2000) Limit … for the TSP (Wästlund 2006)

A. Frieze (2004): “Up to now there has been almost no progress analysing this random model of the travelling salesman problem.” N. J. Cerf et al (1997): “Researchers outside physics remain largely unaware of the analytical progress made on the random link TSP.”

Non-rigorous derivation of the   /12 limit Matching problem on K n for large n. In principle, this requires even n, but we shall consider a relaxation Let the edges be exponential of mean n, so that the sequence of ordered edge costs from a given vertex is approximately a Poisson process of rate 1.

Non-rigorous derivation of the   /12 limit The total cost of the minimum matching is of order n. Introduce a punishment c>0 for not using a particular vertex. This makes the problem well-defined also for odd n. For fixed c, let n tend to infinity. As c tends to infinity, we expect to recover the behavior of the original problem.

Non-rigorous derivation of the   /12 limit For large n, suppose that the problem behaves in the same way for n-1 vertices. Choose an arbitrary vertex to be the root What does the graph look like locally around the root? When only edges of cost <2c are considered, the graph becomes locally tree-like

Non-rigorous derivation of the   /12 limit Non-rigorous replica-cavity method Aldous derived equivalent equations with the Poisson-Weighted Infinite Tree (PWIT)

Non-rigorous derivation of the   /12 limit Let X be the difference in cost between the original problem and that with the root removed. If the root is not matched, then X = c. Otherwise X =  i – X i, where X i is distributed like X, and  i is the cost of the i:th edge from the root. The X i ’s are assumed to be independent.

Non-rigorous derivation of the   /12 limit It remains to do some calculations. We have where X i is distributed like X

Non-rigorous derivation of the   /12 limit Let X  -u

Non-rigorous derivation of the   /12 limit Then if u>-c,

Non-rigorous derivation of the   /12 limit Henceis constant

Non-rigorous derivation of the   /12 limit The constant depends on c and holds when –c<u<c f(-u) f(u)

Non-rigorous derivation of the   /12 limit From definition, exp(-f(c)) = P(X=c) = proportion of vertices that are not matched, and exp(-f(-c)) = exp(0) = 1 e -f(u) + e -f(-u) = 2 – proportion of vertices that are matched = 1 when c = infinity.

Non-rigorous derivation of the   /12 limit

What about the cost of the minimum matching?

Non-rigorous derivation of the   /12 limit

Hence J = area under the curve when f(u) is plotted against f(-u)! Expected cost = n/2 times this area In the original setting = ½ times the area =   /12.

The equation has the explicit solution This gives the cost

The exponential bipartite assignment problem n

Exact formula conjectured by Parisi (1998) Suggests proof by induction Researchers in discrete math, combinatorics and graph theory became interested Generalizations…

Generalizations by Coppersmith & Sorkin to incomplete matchings Remarkable paper by M. Buck, C. Chan & D. Robbins (2000) Introduces weighted vertices Extremely close to proving Parisi’s conjecture!

Incomplete matchings n m

Weighted assignment problems Weights  1,…,  m,  1,…,  n on vertices Edge cost exponential of rate  i  j Conjectured formula for the expected cost of minimum assignment Formula for the probability that a vertex participates in solution (trivial for less general setting!)

The Buck-Chan-Robbins urn process Balls are drawn with probabilities proportional to weight 11 22 33

Proofs of the conjectures Two independent proofs of the Parisi and Coppersmith-Sorkin conjectures in 2003 (Nair, Prabhakar, Sharma and Linusson, Wästlund)

Rigorous method Relax by introducing an extra vertex Let the weight of the extra vertex go to zero Example: Assignment problem with  1 =…=  m =1,  1 =…=  n =1, and  m+1 =  p = P(extra vertex participates) p/n = P(edge (m+1,n) participates)

Rigorous method p/n = P (edge (m+1,n) participates) When  →0, this is Hence By Buck-Chan-Robbins urn theorem,

Rigorous method Hence Inductively this establishes the Coppersmith-Sorkin formula

Rigorous results Much simpler proofs of Parisi, Coppersmith- Sorkin, Buck-Chan-Robbins formulas Exact results for higher moments Exact results and limits for optimization problems on the complete graph

The 2-dimensional urn process 2-dimensional time until k balls have been drawn

Limit shape as n→∞ Matching: TSP/2-factor:

Mean field TSP If the edge costs are i.i.d and satisfy P(l<t)/t→1 as t→0 (pseudodimension 1), then as n →∞,

For the TSP, the replica-cavity approach gives

It follows that is constant, and = 1 by boundary conditions Replica-cavity prediction agrees with the rigorous result (Parisi 2006)

Further exact formulas

LP-relaxation of matching in the complete graph K n

Future work Explain why the cavity method gives the same equation as the limit shape in the urn process Reprove results of one method with the other Find the variance with the replica method Find rigorously the distribution of edge costs participating in the solution (there is an exact conjecture)