1. UBC March 20071 The Evergreen Project: How To Learn From Mistakes Caused by Blurry Vision in MAX-CSP Solving Karl J. Lieberherr Northeastern University Boston joint work with Ahmed Abdelmeged, Christine Hang and Daniel Rinehart

2. UBC March 20072 Introduction Boolean MAX-CSP(G) for rank d, G = set of relations of rank d –Input Input = Bag of Constraint Constraint = Relation + Set of Variable Relation = int. // Relation number < 2 ^ (2 ^ d) in G Variable = int –Output (0,1) assignment to variables which maximizes the number of satisfied constraints. Example Input: G = {22} of rank 3 –22:1 2 3 0 –22:1 2 4 0 –22:1 3 4 0 1in3 has number 22 M = {1 !2 !3 !4} satisfies all

3. UBC March 20073 Decision: MAX-CSP(G,f) MAX-CSP({22},f): Given a MAX-CSP({22}) instance (a bag of constraints using relation 22 = 1in3) expressed in n variables which may assume only the values 0 or 1, find an assignment to the n variables which satisfies at least the fraction f of the constraints. Example: Constraints use 1in3 = 22. 22:1 2 3 0 22:1 2 4 0 22:1 3 4 0 22: 2 3 4 0

4. UBC March 20074 MAX-CSP Search approach: Look for forced variables before making a decision (as in Sudoku) Look-forward: make informed decisions –Abstract representation based on look-ahead polynomials Look-backward: avoid past mistakes –Transition system based on superresolution

5. UBC March 20075 Organization of Solver look backlook forward

6. UBC March 20076 Look-ahead polynomial The look-ahead polynomial computes the expected fraction of satisfied constraints among all random assignments that are produced with bias p.

7. UBC March 20077 Consider an instance: 40 variables, 1000 constraints (1in3) 1, …,40 22: 6 7 9 0 22: 12 27 38 0 Abstract representation: reduce the instance to look-ahead poly. 3p(1-p) 2

8. UBC March 20078 3p(1-p) 2 for MAX-CSP({22})

9. UBC March 20079 SAT Rank 2 example 9 constraints 14 : 1 2 0 14 : 3 4 0 14 : 5 6 0 7 : 1 3 0 7 : 1 5 0 7 : 3 5 0 7 : 2 4 0 7 : 2 6 0 7 : 4 6 0 14: 1 2 = or(1 2) 7: 1 3 = or(!1 !3) What is the look-ahead polynomial?

10. UBC March 200710 appmean = lookahead is an approximation of the true mean Blurry vision What do we learn from the abstract representation? set 1/3 of the variables to true (maximize). the best assignment will satisfy at least 7/9 constraints. very useful but the vision is blurred in the “middle”. excellent peripheral vision

11. UBC March 200711 Forget about computation... Focus on purely mathematical question first Algorithmic solution will follow Mathematical question: Given a MAX- CSP(G,f) instance. For which fractions f is there always an assignment satisfying fraction f of the constraints? In which constraint systems is it impossible to satisfy many constraints?

12. UBC March 200712 Simple example MAX-CSP({22},f): For f <= u: problem has always a solution For f = u +  : problem has not always a solution,    u  critical transition point always (fluid) not always (solid)

13. UBC March 200713 3p(1-p) 2 for MAX-CSP({22})

14. UBC March 200714 The Magic Number u = 4/9

15. UBC March 200715 Look-ahead Polynomial F is a MAX-CSP(G) instance. N is an arbitrary assignment. The look-ahead polynomial la F,N (p) computes the expected fraction of satisfied constraints of F when each variable in N is flipped with probability p.

16. UBC March 200716 The general case MAX-CSP(G) G = {R 1, … }, t R (F) = fraction of constraints in F that use R. x = p

17. UBC March 200717

18. UBC March 200718 General Dichotomy Theorem MAX-CSP(G,f): For each finite set G of relations there exists an algebraic number t G For f <= t G : MAX-CSP(G,f) has polynomial solution For f = t G +  : MAX-CSP(G,f) is NP-complete,   t G  critical transition point easy (fluid) hard (solid) due to Lieberherr/Specker polynomial solution: Use maximally biased coin. Derandomize.

19. UBC March 200719 Observations The look-ahead polynomial look-forward approach has not been used in state-of- the-art MAX-SAT and Boolean MAX-CSP solvers. Often a fair coin is used. The optimally biased coin is often significantly better.

20. UBC March 200720

21. UBC March 200721 Where we are Introduction Look-forward Look-back Packed truth tables

22. UBC March 200722 Observation Optimally biased coin technique based on look-ahead polynomials is “best-possible”. If we could improve it by a trillionth in polynomial time, then P=NP. We improve it now by learning new constraints that will influence the polynomial.

23. UBC March 200723 Algorithm plan start with assignment N = all zero. while (proof incomplete) { –try to improve N by creating new assignment from scratch using optimally biased coin to flip the assignments; success: Update N; failure: learn a new constraint that will prevent same mistake and will “improve” the polynomial. }

24. UBC March 200724 N 0 ={!v 1,!v 2,!v 3,!v 4 }

25. UBC March 200725 N 0 ‘ ={v 1,!v 2,!v 3,!v 4 }

26. UBC March 200726 Transition Rules Unit-Propagation (UP): M || F || SR || N → Mk || F || SR || N if k is undefined in M, and unsat (M¬k,SR) > 0 or unsat(M¬k,F) ≥ unsat(N,F).

27. UBC March 200727 Transition Rules Decide (D): M || F || SR || N → Mk d || F || SR || N if k is undefined in M, and v(k) occurs in some constraint of F.

28. UBC March 200728 Transition Rules Update: M || F || SR || N → M || F || SR || M if M is complete, and unsat(M,F) < unsat(N,F).

29. UBC March 200729 Transition Rules Restart: M || F || SR || N → { } || F || SR || N

30. UBC March 200730 Transition Rules Finale: M || F || SR || N → M || F || SR || N if Φ SR or unsat(N,F) = 0.

31. UBC March 200731 Transition Rules Semi-Superresolution (SSR): NewSR = V (¬k), where k M d M || F || SR || N → M || F || SR, NewSR || N if unsat(M,SR) > 0 or unsat(M,F) ≥ unsat(N,F).

32. UBC March 200732 Transition Rules

33. UBC March 200733 Transition Rules (cont.)

34. UBC March 200734 Transition Manager

35. UBC March 200735 Where we are Introduction Look-forward Look-back Packed truth tables

36. UBC March 200736 Requirements The look-ahead polynomial can be computed efficiently. Requires efficient truth table analysis. Reduction of an instance must be efficient. Efficiently compute the forced variables. Each relation has a unique representation.

37. UBC March 200737 Packed Truth tables

38. UBC March 200738 end for now

39. UBC March 200739 Rank 2 example 14 : 1 2 0 14 : 3 4 0 14 : 5 6 0 7 : 1 3 0 7 : 1 5 0 7 : 3 5 0 7 : 2 4 0 7 : 2 6 0 7 : 4 6 0

40. UBC March 200740 appmean is an approximation of the true mean

41. UBC March 200741

42. UBC March 200742 Transition Manager

43. UBC March 200743

44. UBC March 200744 MAX-CSP: Superresolution and P-Optimality Karl J. Lieberherr Northeastern University Boston joint work with Ahmed Abdelmeged, Christine Hang and Daniel Rinehart

45. UBC March 200745

46. UBC March 200746 Binomial Distribution

47. UBC March 200747

48. UBC March 200748 Example x1 + x2 + x3 = 1 x1 + x2 + + x4 = 1 can satisfy 6/7 x1 + x3 + x4 = 1 x1 + x2 + + x5 = 1 x1 + x3 + x5 = 1 x2 + x3 + x5 =1

49. UBC March 200749 maximize 3x(1-x) 2

50. UBC March 200750 Organization of Solver look backlook forward