Instructor: Aaron Roth aaroth@cis.upenn.edu CIS 262 Automata, Computability, and Complexity Spring 2019 http://www.seas.upenn.edu/~cse262/ Instructor: Aaron Roth aaroth@cis.upenn.edu Lecture: April 17, 2019

NP-Complete Problems A problem A is called NP-complete if 1. A is in NP 2. A is NP-hard: for every language L in NP, L <P A Cook’s Theorem (1972): SAT is NP-complete Proof: Need to show that every problem in NP poly-time-reduces to SAT We showed that COLOR <P SAT COLOR : Given graph G and k, can G be colored using k colors ? Need to generalize the proof using poly-time nondeterministic TMs Will get back to this later…

Invalidity INVALID = {<j> | there exists a truth assignment r s.t. r(j) = 0 } We know that INVALID is in NP Show that SAT <P INVALID Reduction: Algorithm for SAT using algorithm R for INVALID Given a Boolean formula j, Check if R accepts ~ j : if so, accept else reject Since SAT is NP-complete, it follows that INVALID is NP-complete Claim: If a problem A is in NP and B <P A for a known NP-complete problem B, then A is also NP-complete Proof : ??

NP-Complete Problems NP Invalid Coloring Indep Set SAT CNF-SAT Clique Hamiltonian Path 3SAT A large class of problems are all computationally “equivalent’ An efficient algorithm for one can be used to solve others We have seen some reductions, and will see a few more

Formulas in Conjunctive Normal Form (CNF) Boolean formulas: Boolean variables x, y, z … Logical operators: And (&), Not (~), Or (|) Formula in CNF: Nesting of logical operators is restricted ~ at lowest level, | at mid level, & at top level Example: ( x | ~y | u ) & ( ~x | y | ~z ) & ( ~u | y | ~z ) Literal : a variable or a negated variable Clause : OR (also called disjunction) of literals CNF Formula : AND (also called conjunction) of clauses

3CNF A CNF Boolean formula is a conjunction (&) of clauses, where each clause is a disjunction (|) of literals, where each literal is a variable or a negated variable A CNF formula is in 3CNF if each clause has only three literals ( x | y | ~z ) & (~x | u | w ) & (~u | w | z ) & ( z | ~w | ~y ) : 3CNF (x | z | ~y | u ) & ( ~x | y | z ) : CNF but not 3CNF ( x & y ) | (~x & ~ y) : not CNF

SAT, CNF-SAT, and 3SAT SAT: Given a Boolean formula j, check if it is satisfiable CNF-SAT: Given a Boolean formula j in CNF, check if it is satisfiable 3SAT: Given a Boolean formula j in 3CNF, check if it is satisfiable Claim: SAT, CNF-SAT, 3SAT are in NP Claim: SAT <P CNF-SAT Given a boolean formula j, we can construct a formula y in CNF such that j is satisfiable iff y is satisfiable and |y| is polynomial in |j| Details of proof: see textbook Claim CNF-SAT <P 3SAT (later today) Next: 3SAT <P IndepSet It follows that SAT <P IndepSet, and IndepSet is NP-complete

Satisfying Assignment for 3CNF Formulas ( x | y | ~z ) & (~x | y | w ) & (~y | w | z ) & ( z | ~x | ~y ) To make this formula True, each clause must be made True, so at least one literal from each clause must be made True Search for satisfying assignment for a formula in 3CNF Pick one literal from each clause (e.g. either x or y or ~z from clause 1) Cannot choose conflicting literals, that is, a variable and its negation (e.g. cannot pick literal x from clause 1 and literal ~x from clause 2) For above formula, we can pick x, w, w, and ~y Satisfying assignment: x=1, w=1, y=0 (value of z doesn’t matter)

Reducing 3SAT to IndepSet ( x | y | ~z ) & (~x | y | w ) & (~y | w | z ) & ( z | ~x | ~y ) Satisfying assignment: One literal per clause, but no conflicting literals Goal: Construct a graph G such that independent sets of certain size in that graph correspond to satisfying assignment of formula x ~z y ~x w y ~y z w z ~y ~x Graph: Node for each literal and edges between literals of same clause + Edges connecting conflicting literals in different clauses Indep set of size 4 in graph  Satisfying assignment to formula

Reducing 3SAT to IndepSet Given a Boolean formula j in 3CNF with k clauses, construct a graph G with 3k nodes, one per each literal add edges between nodes corresponding to literals in same clause add edges between nodes corresponding to conflicting literals G has an independent set with k nodes iff j is satisfiable G has k triangles, one per clause, so indep set of size k must include one node from each triangle. Chosen nodes cannot correspond to conflicting literals Thus to satisfy j, just set literal for each chosen node to True To check if j is satisfiable, check if <G,k> is in IndepSet Size of G is polynomial in | j |, so 3SAT <P IndepSet

Reducing CNF-SAT to 3SAT CNF-SAT: Given a Boolean formula j in CNF, check if it is satisfiable Given formula is a conjunction of clauses, but some clauses may not have exactly 3 literals ( x ) & ( x | ~y | z ) & ( x | y | z | w) & ( ~y | z | ~w | u | ~v ) … Goal: How to replace such clauses with 3-literal clauses without affecting satisfiability ? Handling clauses with < 3 literals: trivial, just duplicate literals x is same as x | x | x x | ~y is same as x | ~y | ~y So let’s assume each clause has 3 or more literals

Reducing CNF-SAT to 3SAT Consider clause ( x | y | z | w) Introduce a fresh (i.e. new) variable x’ Intended meaning: x’ should be equivalent to z | w Replace original clause by (x | y | x’) Add clauses ( x’ | ~z) , ( x’ | ~w), and ( z | w | ~x’) (these are constraints for x’  (z | w) and (z | w)  x’ in CNF form) Claim: In a satisfying assignment for ( x’ | ~z ) & ( x’ | ~w) & ( z | w | ~x’) either x’ = 1 and either z =1 or w=1 or x’ = 0 and both z= 0 and w =0 Claim: ( x | y | z | w) & j is satisfiable if and only if (x | y | x’) & ( x’ | ~z ) & ( x’ | ~w) & ( z | w | ~x’) & j is satisfiable

Reducing CNF-SAT to 3SAT Consider clause ( x | … | u | v ) with four or more literals Introduce a fresh variable x’ to stand for u | v Replace original clause by (x | … | x’) Add clauses ( x’ | ~u) , ( x’ | ~v), and ( u | v | ~x’) New clauses have <= 3 literals Replacement clause has one fewer literal than the original one Apply this process repeatedly until no clauses with >3 literals Each step preserves satisfiability Number of steps is linear in size of original formula Final formula is in 3CNF and has size polynomial in original formula Thus CNF-SAT <P 3SAT

3SAT vs 2SAT 2CNF: Conjunction of clauses, where each clause has 2 literals 2SAT : Given a formula j in 2CNF, is it satisfiable ? Why can’t we reduce 3SAT to 2SAT using same idea of previous proof? Introducing a fresh variable x’ to stand for u | v to reduce number of literals in a clause needs a clause with 3 literals ( u | v | ~x’) 2SAT is in P there is a nice poly-time algorithm to solve 2SAT

SAT is NP-complete Cook’s Theorem (1972): SAT is NP-complete Proof: 1. SAT is in NP (easy to show) 2. SAT is NP-hard Consider a language L in NP Need to show that L <P SAT Consider a poly-time verifier V for L A given string w is in L iff there is some certificate c s.t. V accepts <w,c> Suppose |w| = n Suppose time complexity of V is f(n), which must be O(nk) Let N = f(n) Goal: construct a formula j such that | j | is polynomial in n and is satisfiable if and only if there exists c such that V accepts <w,c>

Computation of Verifier TM a b a a b # 0 1 0 1 1 _ _ _ _ q0 V executes for only N steps V uses only N tape cells Input to V looks like: Input w # Certificate c , followed by blanks Question : Given w, is there c such that V accepts c Desired formula depends on w and V Suppose TM V = (Q, q0, qa, qr, G, S, _, d )

Encoding Computation of V a b b a b # a # b a a _ _ _ _ q3 TM V = (Q, q0, qa, qr, G, S, _, d ) Uses only N tape cells and executes for N steps A configuration can be encoded as (u , q, v) with u, v in G* with |u|+|v|=N So a configuration can be encoded as a vector of length N+1 e.g.: above configuration is [ a b b a q3 a b # a # b a a _ _ _ _] Hence, the entire execution can be encoded as a (N+1) x (N+1) matrix C[ i , j ] : element of G U Q First row of C = Initial configuration of V i-th row of C = configuration after (i-1) steps (N+1)-th row of C = Configuration upon termination

Checking is w in L ? w is in L iff V accepts w # c for some c Is there an accepting execution of V on some input of the form w # c ? Is there a (N+1) x (N+1) matrix C with elements in G U Q such that (1) First row of C starts with q0 w # Note: value of certificate c is left unspecified (2) For i = 2, … (N+1), i-th row encodes successor of configuration encoded by (i-1)-th row according transition function of V (3) Last row encodes accepting configuration, i.e. state equals qa

Encoding using Boolean Variables Are there variables C[i,j], for i,j=1,…(N+1), ranging over G U Q such that constraints (1), (2), and (3) from previous slide are satisfied But we have only Boolean variables ?? Suppose elements in G U Q are numbered 1, 2, … H Boolean variables B[i, j, k], for i,j = 1, …, (N+1), and k = 1, … H C[i,j] = a is encoded by setting B[i,j,a] = 1 (and B[i,j,a’] = 0 for a !=a’ ) Desired Boolean formula is AND of many formulas including following: for every i,j=1,…(N+1), there is exactly one a such that B[i,j,a] is True

Initialization Constraint (1) First row of C starts with q0 w # Note: value of certificate c is left unspecified B[ 1, 1, q0 ] & B[ 1, 2, w1] & B[ 1, 3, w2] & … & B[1, n+1, wn] & B[1, n+2, #] Note: remaining values in first row (i=1) are unconstrained Satisfying assignment will tell us what the certificate is

Acceptance Constraint Is there a (N+1) x (N+1) matrix C with elements in G U Q such that (3) Last row encodes accepting configuration, i.e. state equals qa B[ N+1, 1, qa ] | B[ N+1, 2, qa ] | … | B[ N+1, N+1, qa ]

Adjacent Rows are Successor Configurations Is there a (N+1) x (N+1) matrix C with elements in G U Q such that (2) For i = 2, … (N+1), i-th row encodes successor of configuration encoded by (i-1)-th row according transition function of V a a b a a b q2 a b … _ a a b a a b a’ q3 b … _ (assuming d(q2, a) = (q3, a’, R) a a b a a b q4 a’ b’ … _ (assuming d(q3, b) = (q4, b’, L) Observation: C[i, j] depends only on C[ i-1,j-1], C[i-1, j], C[i-1,j+1], C[i-1,j+2]

Adjacent Rows are Successor Configurations a a b a a b q2 a b … _ a a b a a b a’ q3 b … _ (assuming d(q2, a) = (q3, a’, R) a a b a a b q4 a’ b’ … _ (assuming d(q3, b) = (q4, b’, L) Observation: C[i, j] depends only on C[ i-1,j-1], C[i-1, j], C[i-1,j+1], C[i-1,j+2] Desired formula says that for all i, j, value of C[i,j] is consistent with values C[ i-1,j-1], C[i-1, j], C[i-1,j+1], C[i-1,j+2] and transition function of V For example: B[i-1,j,q2] & B[i-1,j+1,a]  B[i,j,a’] & B[i,j+1,q3] Details a bit tedious

Completing the proof Given a verifier V of time complexity f(n) for an NP language L and input w, construct a formula j with (N+1)*(N+1)*|G U Q| variables Satisfying assignment for the formula j corresponds to filling in a (N+1) x (N+1) matrix C with elements in G U Q such that (1) First row of C starts with q0 w # (2) For i = 2, … (N+1), i-th row encodes successor of configuration encoded by (i-1)-th row according transition function of V (3) Last row encodes accepting configuration, i.e. state equals qa Note: satisfying assignment for variables for the rest of the first row gives the value of the certificate c that makes V accept Thus to check if w is in L, we can check if j is satisfiable And this proves Cook’s theorem that SAT is NP-complete