Theory of Computing Lecture 19 MAS 714 Hartmut Klauck

Subset Sum Input: set S of natural numbers, target t Is there a subset S’ µ S such that the elements in S’ sum to t? Theorem: Subset Sum is NP-complete Reduction from 3-SAT

Subset Sum Given formula of m clauses and n variables – no variable is unused, x i and : x i not in same clause Create 2 numbers for each variable: v i, w i 2 numbers for each clause: r c, s c Numbers are base 10 and have n+m digits t: 1 in each variable digit and 4 in each clause digit v i : 1 at digit i and 1 for each clause digit with literal x i w i : 1 at digit i and 1 for each clause digit with literal : x i r c : 1 at clause digit c s c : 2 at clause digit c

Subset Sum Suppose sum of a subset of numbers equals t Clearly: cannot contain both v i and w i Set x i =1 iff v i is in the subset Claim: Satisfying for F r c and s c can contribute at most 3 to a clause digit, but the sum is 4, hence the v i or w i in the subset must have digit 1 for the clause-> clause in F is satisfied

Subset Sum Suppose there is a satisfying assignment to F If x i =1 take v i into the subset, else w i Sums all numbers, consider n most significant digits: all are 1 Consider the last m digits of the sum: – For each clause digit value is at least 1, and up to 3 Add appropriate numbers r c, s c to make the sum equal to t

3-Dimensional Matching X,Y,Z are sets of n elements each E is a subset of X £ Y £ Z A 3D-matching is a set of triples that do not share vertices Question: Is there a perfect matching, i.e., one that contains every vertex in exactly one edge? Recall: 2D-matching is in P

NP-hardness We reduce from 3-SAT Formula F with n variables and m clauses Variable gadget for x i : Vertices: {a i,1,…,a i,2m } [ {b i,1,...,b i,2m } Edges: (a i,j, a i,j+1, b i,j ) In a perfect matching only b i,j for odd j or for even j can be present (else some a i,j covered twice) Covering even b i,j means x i =1

3DM Clause gadget: – Vertices {c j, d j } – Edges: connect clause gadget nodes with variable gadgets: literal x i : connect with b i,2j, else with b i,2j-1 Observation: Must choose exactly one edge per clause gadget. This covers an odd or even b i,j vertex. Can cover a-vertices only if either odd or even b i,j are used, not both. More details: Need some extra gadgets to cover unused b-vertices

3DM Suppose F is satisfiable. Choose the edges for the variable gadgets as in x Then for a clause (l 1 Ç l 2 Ç l 3 ) all true literals have un-covered b-vertices that can be covered using an edge that covers the clause gadget vertices Similarly, given a perfect matching we can find a satisfying assignment

co-NP co-NP is the class of complements of languages that are in NP It is widely believed that NP  co-NP This would imply that NP Å co-NP does not contain NP-complete problems – Example: IntegerFactorization (decision version) – We don’t know a polynomial time algorithm, but the problem is likely not NP-complete

Intermediate Problems Ladner’s Theorem – Unless P=NP, there exist problems in NP that are neither in P nor NP-complete Candidate problems (their decision versions): – Integer Factorization – Graph Isomorphism – Discrete Logarithm – Group Isomorphism – Finding approximate short vectors in lattices Some of these problems can be used to define public key cryptosystems – Such systems are hard to break if the underlying problems are hard (e.g. for factorization/discrete log)

Other completeness notions P-completeness – Reductions must be computed in logarithmic space – implies (probably) that a problem cannot be solved in space polylog n or parallel time polylog n EXP-completeness – Implies that the problem cannot be solved in polynomial time NEXP-completeness – Problem is not in NP PSPACE-completeness – Problem is likely not in NP – Many two-player games like generalized chess with time bound

New topic: Computability Question 1: Are all Boolean functions computable? Answer 1: yes, by Boolean circuit families – But those are not uniform, seems like cheating, circuit family is an infinite object Question 2: Are all languages L µ {0,1}* computable by Turing machines ? Answer 2: NO.

Argument number 1 A Turing machine can be described by a finite string – list the elements of the 8-tuple – Alternative: C++ program We can identify finite strings with natural numbers – treat each character as a digit Conclusion: there are infinitely many Turing machines, but the set of all Turing machines is countable – A set S is countable if there is an injective mapping from S to the natural numbers – An infinite, countable set has the same cardinality as the natural numbers – Example: the rational numbers are countable, the reals are not

Argument number 1 Theorem: the set of languages L µ {0,1}* is uncountable – Indeed has the same cardinality as the real numbers Conclusion: – Almost all languages are not computable by a TM No matter how much time the TM can spend computing

Proof of the theorem First we use Cantor’s method of diagonalization to show that there is a language that cannot be computed Let M 1, M 2,… denote an enumeration of all Turing machines Let x 1, x 2,… denotes an enumeration of all inputs in {0,1}* Consider the matrix A that has rows labeled by x i and columns labeled by M j and that contains 1 if M j accepts x i and 0 otherwise

Proof of the claim Each column j corresponds to the language of strings accepted by M j Consider the complement of the diagonal of A – 0 with 1 and vice versa Consider the language L that contains string x i iff the i- th diagonal element is 0 L is not equal to any of the languages computed by the M j ! Hence L is an un-computable language [For languages we use both terms computable/decidable to mean that the language can be decided by a TM that halts on all inputs]

Proof of the claim The same can be done if we replace the columns with any enumeration of languages Hence the set of all languages is not countable, i.e., there is no injective mapping from the set of all languages to the natural numbers Note: the set of all languages has the same cardinality as the reals – Find an injective mapping from the reals to the set of all languages This means that `almost all’ languages are not computable But what about an explicit example?

Universality To answer this question we need another property of Turing machine Universality means that there is a Turing machine that can simulate all other Turing machines Compare: actual computers are universal, we generally don’t buy/build a new computer for a new application

Universality Fix some way to encode Turing machines – Write for the encoding of TM M (as a string) Definition: A universal Turing machine is a TM that on input,x accepts if and only if M accepts on x Theorem: There is a universal TM U

Proof Note: Since U must be finite, we cannot simply read into the internal state of U – in fact we cannot even read a state of M into U’s internal state Copy input x to a second tape (we assume that M has 1 tape) A third tape contains the current state of M – w.l.o.g. this is 0 2 Q We need to show that one step of M can be simulated Idea: search the transition function table in to determine the new state given the tape symbol on tape 2 and the current state on tape 3, update both, move the head on tape 2 Each step tests if the current state is accepting/rejecting We have a machine U that can simulate any 1-tape TM using 3 tapes Running time: we are interested in the running time in terms of |x| Fact: a universal TM U with time O(C n log n) exist, where C depends on M, and U has the same number of tapes as M

Halting Problem On an input x, a machine M might not halt – Infinite loop It would be nice to be able to decide if this happens! HALT={,x: M halts on input x} Theorem: – HALT is not computable

Proof We try a similar argument to Cantor’s Consider a TM H that decides HALT – halts on all inputs and gives the correct output Consider inputs, – These belong to HALT, if M halts on its encoding We define a new machine M that does the following: – On input x, run H on x,x [use U, note that H halts] – If H reports that x halts on x, go into an infinite loop – Else halt

Proof What does M do on ? If M halts on, H will say so, and M will go into an infinite loop on If M does not halt on, then H will say it does not halt, and M will halt on Contradiction! Hence H does not exist!

Reductions Definition: L many-one reduces to S is there is a computable function f such that x 2 L, f(x) 2 S Notation: L · S Conclusion: if L is not decidable then S is not decidable More general notion of reductions: Turing reductions (using oracle Turing machine)

More uncomputable problems We are in a similar situation as in the theory of NP-completeness We have an uncomputable/undecidable problem, find more by reductions

Example H={ : M halts on some input x 2 {0,1}*} Theorem: H is not computable because HALT · H Reduction: given,x we construct a new TM M x – M x ignores its own input but writes x on the tape and then simulates M on x Clearly:,x 2 HALT, 2 H Note: The description of has to be computable from and x

Example L M denotes the language accepted by M – the set of strings that M accepts NEMPTY={ : L M  ; } Theorem: E is not decidable because HALT · NEMPTY Reduction: – Given,x we create a new machine M x – M x reads the input and compares it to x – If input is not equal to x, then M x rejects – If input is equal to x, then M x simulates M – If M halts, accept We have:,x 2 HALT, 2 NEMPTY