1. 1 Beyond NP Other complexity classes Phase transitions in P, PSPACE, … Structure Backbones, 2+p-SAT, small world topology, … Heuristics Constrainedness knife-edge, minimize constrainedness,...

2. Before we begin A little history...

3. 3 Where did this all start? zAt least as far back as 60s with Erdos & Renyi ythresholds in random graphs zLate 80s ypioneering work by Karp, Purdom, Kirkpatrick, Huberman, Hogg … zFlood gates burst yCheeseman, Kanefsky & Taylor’s IJCAI-91 paper

4. Other complexity classes Enough of the history, are phase transitions just in NP? Conjecture in Cheeseman et al paper that phase transitions distinguish P from NP.

5. 5 Random 2-SAT z2-SAT is P ylinear time algorithm zRandom 2-SAT displays “classic” phase transition yc/n < 1, almost surely SAT yc/n > 1, almost surely UNSAT ycomplexity peaks around c/n=1 x1 v x2, -x2 v x3, -x1 v x3, …

6. 6 Phase transitions in P z 2-SAT yc/n=1 z Horn SAT ytransition not “sharp” z Arc-consistency yrapid transition in whether problem can be made AC ypeak in (median) checks

7. 7 Phase transitions above NP zPSpace yQSAT (SAT of QBF)  x1  x2  x3. x1 v x2 & -x1 v x3

8. 8 Phase transitions above NP zPSpace-complete yQSAT (SAT of QBF) ystochastic SAT ymodal SAT zPP-complete ypolynomial-time probabilistic Turing machines ycounting problems y#SAT(>= 2^n/2) [Bailey, Dalmau, Kolaitis IJCAI-2001]

9. 9 Exact phase boundaries in NP zRandom 3-SAT is only known within bounds y3.26 < c/n < 4.596 zRecent result gives an exact NP phase boundary y1-in-k SAT at c/n = 2/k(k-1) y2nd order transition (like 2- SAT and unlike 3-SAT) Are there any NP phase boundaries known exactly? 1st order transitions not a characteristic of NP as has been conjectured

10. Structure Can we identify structure in (random) problems that makes problems hard? How do we model structural features found in real problems? How does such structure affect phase transition behaviour?

11. 11 Backbone zVariables which take fixed values in all solutions yalias unit prime implicates zLet f k be fraction of variables in backbone yin random 3-SAT c/n < 4.3, f k vanishing (otherwise adding clause could make problem unsat) c/n > 4.3, f k > 0 discontinuity at phase boundary!

12. 12 Backbone zSearch cost correlated with backbone size yif f k non-zero, then can easily assign variable “wrong” value ysuch mistakes costly if at top of search tree zOne source of “thrashing” behaviour ycan tackle with randomization and rapid restarts see Carla’s section Can we adapt algorithms to offer more robust performance guarantees?

13. 13 Backbone zBackbones observed in structured problems yquasigroup completion problems (QCP) colouring partial Latin squares zBackbones also observed in optimization and approximation problems ycoloring, TSP, blocks world planning … see [Slaney, Walsh IJCAI-2001] Can we adapt algorithms to identify and exploit the backbone structure of a problem?

14. 14 2+p-SAT z Morph between 2-SAT and 3-SAT yfraction p of 3-clauses yfraction (1-p) of 2-clauses z 2-SAT is polynomial (linear) yphase boundary at c/n =1 ybut no backbone discontinuity here! z 2+p-SAT maps from P to NP yp>0, 2+p-SAT is NP-complete

15. 15 2+p-SAT phase transition

16. 16 2+p-SAT phase transition c/n p

17. 17 2+p-SAT phase transition z Lower bound yare the 2-clauses (on their own) UNSAT? yn.b. 2-clauses are much more constraining than 3-clauses z p <= 0.4 ytransition occurs at lower bound y3-clauses are not contributing!

18. 18 2+p-SAT backbone zf k becomes discontinuous for p>0.4 ybut NP-complete for p>0 ! zsearch cost shifts from linear to exponential at p=0.4 zsimilar behavior seen with local search algorithms Search cost against n

19. Structure How do we model structural features found in real problems? How does such structure affect phase transition behaviour?

20. 20 The real world isn’t random? zVery true! Can we identify structural features common in real world problems? zConsider graphs met in real world situations ysocial networks yelectricity grids yneural networks y...

21. 21 Real versus Random z Real graphs tend to be sparse ydense random graphs contains lots of (rare?) structure z Real graphs tend to have short path lengths yas do random graphs z Real graphs tend to be clustered yunlike sparse random graphs L, average path length C, clustering coefficient (fraction of neighbours connected to each other, cliqueness measure) mu, proximity ratio is C/L normalized by that of random graph of same size and density

22. 22 Small world graphs z Sparse, clustered, short path lengths z Six degrees of separation yStanley Milgram’s famous 1967 postal experiment yrecently revived by Watts & Strogatz yshown applies to: xactors database xUS electricity grid xneural net of a worm x...

23. 23 An example z1994 exam timetable at Edinburgh University y59 nodes, 594 edges so relatively sparse ybut contains 10-clique zless than 10^-10 chance in a random graph yassuming same size and density zclique totally dominated cost to solve problem

24. 24 Small world graphs zTo construct an ensemble of small world graphs ymorph between regular graph (like ring lattice) and random graph yprob p include edge from ring lattice, 1-p from random graph real problems often contain similar structure and stochastic components?

25. 25 Small world graphs z ring lattice is clustered but has long paths z random edges provide shortcuts without destroying clustering

26. 26 Small world graphs

27. 27 Small world graphs

28. 28 Colouring small world graphs

29. 29 Small world graphs z Other bad news ydisease spreads more rapidly in a small world z Good news ycooperation breaks out quicker in iterated Prisoner’s dilemma

30. 30 Other structural features It’s not just small world graphs that have been studied zLarge degree graphs yBarbasi et al’s power-law model [Walsh, IJCAI 2001] zUltrametric graphs yHogg’s tree based model zNumbers following Benford’s Law y1 is much more common than 9 as a leading digit! prob(leading digit=i) = log(1+1/i) ysuch clustering, makes number partitioning much easier

31. Heuristics What do we understand about problem hardness at the phase boundary? How can this help build better heuristics?

32. 32 Looking inside search z Constrainedness “knife- edge” yproblems are critically constrained between SAT and UNSAT z Suggests branching heuristics yalso insight into branching mistakes

33. 33 Inside SAT phase transition zRandom 3-SAT, c/n =4.3 zDavis Putnam algorithm ytree search through space of partial assignments yunit propagation zClause to variable ratio c/n drops as we search => problems become less constrained Aside: can anyone explain simple scaling? c/n against depth/n

34. 34 Inside SAT phase transition z But (average) clause length, k also drops => problems become more constrained z Which factor, c/n or k wins? yLook at kappa which includes both! Aside: why is there again such simple scaling? Clause length, k against depth/n

35. 35 Constrainedness knife-edge kappa against depth/n

36. 36 Constrainedness knife-edge zSeen in other problem domains ynumber partitioning, … zSeen on “real” problems yexam timetabling (alias graph colouring) zSuggests branching heuristic y“get off the knife-edge as quickly as possible” yminimize or maximize-kappa heuristics must take into account branching rate, max-kappa often therefore not a good move!

37. 37 Minimize constrainedness zMany existing heuristics minimize-kappa yor proxies for it zFor instance yKarmarkar-Karp heuristic for number partitioning yBrelaz heuristic for graph colouring yFail-first heuristic for constraint satisfaction y… zCan be used to design new heuristics yremoving some of the “black art”

38. 38 Beyond NP Other complexity classes Phase transitions in P, PSPACE, … Structure Backbones, 2+p-SAT, small world topology, … Heuristics Constrainedness knife-edge, minimize constrainedness,...