02/01/11CMPUT 671 Lecture 11 CMPUT 671 Hard Problems Winter 2002 Joseph Culberson Home Page

02/01/11CMPUT 671 Lecture 12 What are you doing?

02/01/11CMPUT 671 Lecture 13 What am I doing? What are you doing?

02/01/11CMPUT 671 Lecture 14 What is Hard? Problems and Complexity –P, NP, PSPACE; between and beyond –Worst case, Randomized, Average case What am I doing? What are you doing?

02/01/11CMPUT 671 Lecture 15 What is Hard? Problems and Complexity –P, NP, PSPACE; between and beyond –Worst case, Randomized, Average case Instances and Algorithms –Some instances may be easy for some algorithms –Hard instances for classes of algorithms What am I doing? What are you doing?

02/01/11CMPUT 671 Lecture 16 What is Hard? Problems and Complexity –P, NP, PSPACE; between and beyond –Worst case, Randomized, Average case Instances and Algorithms –Some instances may be easy for some algorithms –Hard instances for classes of algorithms Ensembles and Phase Transitions - Most instances may be easy - Hard instances may be “concentrated” - Hard random instances for classes of algorithms - Generation What am I doing? What are you doing?

02/01/11CMPUT 671 Lecture 17 Background Asymptotic notation and algorithm analysis Problems, Complexity Classes, Reductions and Completeness (P to NPc) Backtrack Search, Stochastic Search Graphs and some graph theory Probability (as background to random graphs and structures)

02/01/11CMPUT 671 Lecture 18 Problems A Problem is a set of instances and queries. An instance is a structure (versus language) Queries are universal e.g. Graph k-coloring Instance: A graph G and integer k Query: Is G k-colorable? e.g. MST union TSP Instance: A weighted graph G, integers B 1,B 2 Query: Does G have a spanning tree of weight B 1 or a tour of weight B 2 ? (Set one to 0)

02/01/11CMPUT 671 Lecture 19 Measuring Resources Time Instance I n = Size(I); Formally, the number of bits in an encoding (logarithmic model) but mostly we will use the uniform model, e.g. the number of vertices and edges of a graph. T(I) = number of steps in computation to answer the query. Formally, on a model of computation, typically a RAM.

02/01/11CMPUT 671 Lecture 110 Measuring Resources Time (continued) Worst Case T(n) = max{ T(I) : size(I) = n } Average Case A(n) = sum T(I)* Prob(I) : size(I) = n Note: distribution dependent! Typically all instances of size n equally likely, but this may not be representative of the “real world”

02/01/11CMPUT 671 Lecture 111 Measuring Resources SPACE Similar, but measures number of memory cells required. Formally, each cell must be of fixed size, e.g. number of bits fixed. For certain purposes, e.g. O(log n) memory requirements, we exclude size of input memory. (Likely not relevant to this course)

02/01/11CMPUT 671 Lecture 112 Asymptotic Notation

02/01/11CMPUT 671 Lecture 113 Types of Problems Optimization Given G find the minimum number of colors required to color it. Functional Given G find an optimal coloring Given G and k find a k-coloring if one exists Enumeration and Generation (listing) Decision (Yes/No) Given G and k is there a k-coloring of G?

02/01/11CMPUT 671 Lecture 114 The Class P P is the set of decision problems for which there exists a polynomial time algorithm. i.e. O(n k ) for some fixed k. n is the size of the input. See Garey and Johnson, Brassard and Bratley, Sipser, Cormen et al.

02/01/11CMPUT 671 Lecture 115 The Class NP NP is the set of decision problems for which there exists a non-deterministic algorithm that solves the yes instances of the problem in polynomial time. (G&J, B&B) The algorithm non-deterministically writes out a solution (certificate) and in time polynomial in the input size verifies that the answer is yes. E.g. Coloring: Certificate= c:V -> {1..k}; verify for each edge e = (u,v), c(u) not = c(v)

02/01/11CMPUT 671 Lecture 116 Class CO-NP The set of problems whose NO instances are solved by a poly-time non-deterministic algorithm; the complement of NP wrt Yes/No In Language Terms Note that P is in the intersection of NP and CO-NP … Why?

02/01/11CMPUT 671 Lecture 117 We no of know certificate we can use to verify that G is not k-colorable in poly-time. The Questions: P ?= NP ?= CO-NP ? The first asks if there is a polynomial time deterministic algorithm to solve all NP problems The second asks is there a polynomial time non- deterministic algorithm to solve all of CO-NP

02/01/11CMPUT 671 Lecture 118 Reductions Polynomial time many one reduction (transformation) from problem P to problem Q f:P->Q Maps instances x in Y(P) iff f(x) in Y(Q) f is computable by a polynomial time deterministic algorithm.

02/01/11CMPUT 671 Lecture 119 Reductions Example: Graph 3-coloring to 3-SAT Can also add 2-clauses restricting each vertex to at most one color

02/01/11CMPUT 671 Lecture 120 Class NPc A subclass of NP, Q in NPc iff for every problem P in NP there is a polynomial time many reduction from P to Q. Cook’s theorem showed that SAT is in NPc. Create a set of clauses which model the possible execution of the NDTM that solves the problem.

02/01/11CMPUT 671 Lecture 121 Showing a Problem is in NPc To show a problem Q is NP-complete Show Q is in NP, that is, specify a certificate that can be verified in polynomial time Choose a problem P in NPc and provide a polynomial time reduction from P to Q. The second alone shows it is NP-hard

02/01/11CMPUT 671 Lecture 122 Only applies to worst case