1. UBC March 20071 The Evergreen Project: How To Learn From Mistakes Caused by Blurry Vision in MAX-CSP Solving Karl J. Lieberherr Northeastern University Boston joint work with Ahmed Abdelmeged, Christine Hang and Daniel Rinehart

2. Boolean MAX-CSP solving for CSU 670 UBC March 20072 1.Do nothing: The algorithm for the classic version finds a reasonable assignment. Both the randomized and derandomized algorithm work for non-symmetric formulas. But you can do much better. 2.Chronological Backtracking 3.Non-Chronological Backtracking: around page 26.

3. UBC March 20073

4. 4

5. 5 Introduction Boolean MAX-CSP(G) for rank d, G = set of relations of rank d –Input Input = Bag of Constraint Constraint = Relation + Set of Variable Relation = int. // Relation number < 2 ^ (2 ^ d) in G Variable = int –Output (0,1) assignment to variables which maximizes the number of satisfied constraints. Example Input: G = {22} of rank 3 –22:1 2 3 0 –22:1 2 4 0 –22:1 3 4 0 1in3 has number 22 M = {1 !2 !3 !4} satisfies all

6. UBC March 20076 Decision: MAX-CSP(G,f) MAX-CSP({22},f): Given a MAX-CSP({22}) instance (a bag of constraints using relation 22 = 1in3) expressed in n variables which may assume only the values 0 or 1, find an assignment to the n variables which satisfies at least the fraction f of the constraints. Example: Constraints use 1in3 = 22. 22:1 2 3 0 22:1 2 4 0 22:1 3 4 0 22: 2 3 4 0

7. UBC March 20077 MAX-CSP Search approach: Look for forced variables before making a decision (as in Sudoku) Look-forward: make informed decisions –Abstract representation based on look-ahead polynomials Look-backward: avoid past mistakes –Transition system based on superresolution

8. UBC March 20078 Organization of Solver look backlook forward

9. UBC March 20079 Look-ahead polynomial The look-ahead polynomial computes the expected fraction of satisfied constraints among all random assignments that are produced with bias p.

10. UBC March 200710 Consider an instance: 40 variables, 1000 constraints (1in3) 1, …,40 22: 6 7 9 0 22: 12 27 38 0 Abstract representation: reduce the instance to look-ahead poly. 3p(1-p) 2

11. UBC March 200711 3p(1-p) 2 for MAX-CSP({22})

12. UBC March 200712 SAT Rank 2 example 9 constraints 14 : 1 2 0 14 : 3 4 0 14 : 5 6 0 7 : 1 3 0 7 : 1 5 0 7 : 3 5 0 7 : 2 4 0 7 : 2 6 0 7 : 4 6 0 14: 1 2 = or(1 2) 7: 1 3 = or(!1 !3) What is the look-ahead polynomial?

13. UBC March 200713 appmean = lookahead is an approximation of the true mean Blurry vision What do we learn from the abstract representation? set 1/3 of the variables to true (maximize). the best assignment will satisfy at least 7/9 constraints. very useful but the vision is blurred in the “middle”. excellent peripheral vision

14. UBC March 200714 Forget about computation... Focus on purely mathematical question first Algorithmic solution will follow Mathematical question: Given a MAX- CSP(G,f) instance. For which fractions f is there always an assignment satisfying fraction f of the constraints? In which constraint systems is it impossible to satisfy many constraints?

15. UBC March 200715 Simple example MAX-CSP({22},f): For f <= u: problem has always a solution For f = u +  : problem has not always a solution,    u  critical transition point always (fluid) not always (solid)

16. UBC March 200716 3p(1-p) 2 for MAX-CSP({22})

17. UBC March 200717 The Magic Number u = 4/9

18. UBC March 200718 Look-ahead Polynomial F is a MAX-CSP(G) instance. N is an arbitrary assignment. The look-ahead polynomial la F,N (p) computes the expected fraction of satisfied constraints of F when each variable in N is flipped with probability p.

19. UBC March 200719 The general case MAX-CSP(G) G = {R 1, … }, t R (F) = fraction of constraints in F that use R. x = p

20. UBC March 200720

21. UBC March 200721 General Dichotomy Theorem MAX-CSP(G,f): For each finite set G of relations there exists an algebraic number t G For f <= t G : MAX-CSP(G,f) has polynomial solution For f = t G +  : MAX-CSP(G,f) is NP-complete,   t G  critical transition point easy (fluid) hard (solid) due to Lieberherr/Specker polynomial solution: Use maximally biased coin. Derandomize.

22. UBC March 200722 Observations The look-ahead polynomial look-forward approach has not been used in state-of- the-art MAX-SAT and Boolean MAX-CSP solvers. Often a fair coin is used. The optimally biased coin is often significantly better.

23. UBC March 200723

24. UBC March 200724 Where we are Introduction Look-forward Look-back Packed truth tables

25. UBC March 200725 Observation Optimally biased coin technique based on look-ahead polynomials is “best-possible”. If we could improve it by a trillionth in polynomial time, then P=NP. We improve it now by learning new constraints that will influence the polynomial.

26. UBC March 200726 Algorithm plan start with assignment N = all zero. while (proof incomplete) { –try to improve N by creating new assignment from scratch using optimally biased coin to flip the assignments; success: Update N; failure: learn a new constraint that will prevent same mistake and will “improve” the polynomial. }

27. UBC March 200727 N 0 ={!v 1,!v 2,!v 3,!v 4 }

28. UBC March 200728 N 0 ‘ ={v 1,!v 2,!v 3,!v 4 }

29. UBC March 200729 Transition Rules Unit-Propagation (UP): M || F || SR || N → Mk || F || SR || N if k is undefined in M, and unsat (M¬k,SR) > 0 or unsat(M¬k,F) ≥ unsat(N,F).

30. UBC March 200730 Transition Rules Decide (D): M || F || SR || N → Mk d || F || SR || N if k is undefined in M, and v(k) occurs in some constraint of F.

31. UBC March 200731 Transition Rules Update: M || F || SR || N → M || F || SR || M if M is complete, and unsat(M,F) < unsat(N,F).

32. UBC March 200732 Transition Rules Restart: M || F || SR || N → { } || F || SR || N

33. UBC March 200733 Transition Rules Finale: M || F || SR || N → M || F || SR || N if Φ SR or unsat(N,F) = 0.

34. UBC March 200734 Transition Rules Semi-Superresolution (SSR): NewSR = V (¬k), where k M d M || F || SR || N → M || F || SR, NewSR || N if unsat(M,SR) > 0 or unsat(M,F) ≥ unsat(N,F).

35. UBC March 200735 Transition Rules

36. UBC March 200736 Transition Rules (cont.)

37. UBC March 200737 Transition Manager

38. UBC March 200738 Where we are Introduction Look-forward Look-back Packed truth tables

39. UBC March 200739 Requirements The look-ahead polynomial can be computed efficiently. Requires efficient truth table analysis. Reduction of an instance must be efficient. Efficiently compute the forced variables. Each relation has a unique representation.

40. UBC March 200740 Packed Truth tables

41. UBC March 200741 end for now

42. UBC March 200742 Rank 2 example 14 : 1 2 0 14 : 3 4 0 14 : 5 6 0 7 : 1 3 0 7 : 1 5 0 7 : 3 5 0 7 : 2 4 0 7 : 2 6 0 7 : 4 6 0

43. UBC March 200743 appmean is an approximation of the true mean

44. UBC March 200744

45. UBC March 200745 Transition Manager

46. UBC March 200746

47. UBC March 200747 MAX-CSP: Superresolution and P-Optimality Karl J. Lieberherr Northeastern University Boston joint work with Ahmed Abdelmeged, Christine Hang and Daniel Rinehart

48. UBC March 200748

49. UBC March 200749 Binomial Distribution

50. UBC March 200750

51. UBC March 200751 Example x1 + x2 + x3 = 1 x1 + x2 + + x4 = 1 can satisfy 6/7 x1 + x3 + x4 = 1 x1 + x2 + + x5 = 1 x1 + x3 + x5 = 1 x2 + x3 + x5 =1

52. UBC March 200752 maximize 3x(1-x) 2

53. UBC March 200753 Organization of Solver look backlook forward