UBC March The Evergreen Project: How To Learn From Mistakes Caused by Blurry Vision in MAX-CSP Solving Karl J. Lieberherr Northeastern University Boston joint work with Ahmed Abdelmeged, Christine Hang and Daniel Rinehart

Boolean MAX-CSP solving for CSU 670 UBC March Do nothing: The algorithm for the classic version finds a reasonable assignment. Both the randomized and derandomized algorithm work for non-symmetric formulas. But you can do much better. 2.Chronological Backtracking 3.Non-Chronological Backtracking: around page 26.

UBC March 20073

4

5 Introduction Boolean MAX-CSP(G) for rank d, G = set of relations of rank d –Input Input = Bag of Constraint Constraint = Relation + Set of Variable Relation = int. // Relation number < 2 ^ (2 ^ d) in G Variable = int –Output (0,1) assignment to variables which maximizes the number of satisfied constraints. Example Input: G = {22} of rank 3 –22: –22: –22: in3 has number 22 M = {1 !2 !3 !4} satisfies all

UBC March Decision: MAX-CSP(G,f) MAX-CSP({22},f): Given a MAX-CSP({22}) instance (a bag of constraints using relation 22 = 1in3) expressed in n variables which may assume only the values 0 or 1, find an assignment to the n variables which satisfies at least the fraction f of the constraints. Example: Constraints use 1in3 = : : : :

UBC March MAX-CSP Search approach: Look for forced variables before making a decision (as in Sudoku) Look-forward: make informed decisions –Abstract representation based on look-ahead polynomials Look-backward: avoid past mistakes –Transition system based on superresolution

UBC March Organization of Solver look backlook forward

UBC March Look-ahead polynomial The look-ahead polynomial computes the expected fraction of satisfied constraints among all random assignments that are produced with bias p.

UBC March Consider an instance: 40 variables, 1000 constraints (1in3) 1, …,40 22: : Abstract representation: reduce the instance to look-ahead poly. 3p(1-p) 2

UBC March p(1-p) 2 for MAX-CSP({22})

UBC March SAT Rank 2 example 9 constraints 14 : : : : : : : : : : 1 2 = or(1 2) 7: 1 3 = or(!1 !3) What is the look-ahead polynomial?

UBC March appmean = lookahead is an approximation of the true mean Blurry vision What do we learn from the abstract representation? set 1/3 of the variables to true (maximize). the best assignment will satisfy at least 7/9 constraints. very useful but the vision is blurred in the “middle”. excellent peripheral vision

UBC March Forget about computation... Focus on purely mathematical question first Algorithmic solution will follow Mathematical question: Given a MAX- CSP(G,f) instance. For which fractions f is there always an assignment satisfying fraction f of the constraints? In which constraint systems is it impossible to satisfy many constraints?

UBC March Simple example MAX-CSP({22},f): For f <= u: problem has always a solution For f = u +  : problem has not always a solution,    u  critical transition point always (fluid) not always (solid)

UBC March p(1-p) 2 for MAX-CSP({22})

UBC March The Magic Number u = 4/9

UBC March Look-ahead Polynomial F is a MAX-CSP(G) instance. N is an arbitrary assignment. The look-ahead polynomial la F,N (p) computes the expected fraction of satisfied constraints of F when each variable in N is flipped with probability p.

UBC March The general case MAX-CSP(G) G = {R 1, … }, t R (F) = fraction of constraints in F that use R. x = p

UBC March

UBC March General Dichotomy Theorem MAX-CSP(G,f): For each finite set G of relations there exists an algebraic number t G For f <= t G : MAX-CSP(G,f) has polynomial solution For f = t G +  : MAX-CSP(G,f) is NP-complete,   t G  critical transition point easy (fluid) hard (solid) due to Lieberherr/Specker polynomial solution: Use maximally biased coin. Derandomize.

UBC March Observations The look-ahead polynomial look-forward approach has not been used in state-of- the-art MAX-SAT and Boolean MAX-CSP solvers. Often a fair coin is used. The optimally biased coin is often significantly better.

UBC March

UBC March Where we are Introduction Look-forward Look-back Packed truth tables

UBC March Observation Optimally biased coin technique based on look-ahead polynomials is “best-possible”. If we could improve it by a trillionth in polynomial time, then P=NP. We improve it now by learning new constraints that will influence the polynomial.

UBC March Algorithm plan start with assignment N = all zero. while (proof incomplete) { –try to improve N by creating new assignment from scratch using optimally biased coin to flip the assignments; success: Update N; failure: learn a new constraint that will prevent same mistake and will “improve” the polynomial. }

UBC March N 0 ={!v 1,!v 2,!v 3,!v 4 }

UBC March N 0 ‘ ={v 1,!v 2,!v 3,!v 4 }

UBC March Transition Rules Unit-Propagation (UP): M || F || SR || N → Mk || F || SR || N if k is undefined in M, and unsat (M¬k,SR) > 0 or unsat(M¬k,F) ≥ unsat(N,F).

UBC March Transition Rules Decide (D): M || F || SR || N → Mk d || F || SR || N if k is undefined in M, and v(k) occurs in some constraint of F.

UBC March Transition Rules Update: M || F || SR || N → M || F || SR || M if M is complete, and unsat(M,F) < unsat(N,F).

UBC March Transition Rules Restart: M || F || SR || N → { } || F || SR || N

UBC March Transition Rules Finale: M || F || SR || N → M || F || SR || N if Φ SR or unsat(N,F) = 0.

UBC March Transition Rules Semi-Superresolution (SSR): NewSR = V (¬k), where k M d M || F || SR || N → M || F || SR, NewSR || N if unsat(M,SR) > 0 or unsat(M,F) ≥ unsat(N,F).

UBC March Transition Rules

UBC March Transition Rules (cont.)

UBC March Transition Manager

UBC March Where we are Introduction Look-forward Look-back Packed truth tables

UBC March Requirements The look-ahead polynomial can be computed efficiently. Requires efficient truth table analysis. Reduction of an instance must be efficient. Efficiently compute the forced variables. Each relation has a unique representation.

UBC March Packed Truth tables

UBC March end for now

UBC March Rank 2 example 14 : : : : : : : : : 4 6 0

UBC March appmean is an approximation of the true mean

UBC March

UBC March Transition Manager

UBC March

UBC March MAX-CSP: Superresolution and P-Optimality Karl J. Lieberherr Northeastern University Boston joint work with Ahmed Abdelmeged, Christine Hang and Daniel Rinehart

UBC March

UBC March Binomial Distribution

UBC March

UBC March Example x1 + x2 + x3 = 1 x1 + x2 + + x4 = 1 can satisfy 6/7 x1 + x3 + x4 = 1 x1 + x2 + + x5 = 1 x1 + x3 + x5 = 1 x2 + x3 + x5 =1

UBC March maximize 3x(1-x) 2

UBC March Organization of Solver look backlook forward