week 10Complexity of Algorithms1 Hard Computational Problems Some computational problems are hard Despite a numerous attempts we do not know any efficient algorithms for these problems We are also far away from the proof that these problems are indeed hard to solve, in other words NP=P or NP  P, this is a question …

week 10Complexity of Algorithms2 Decision/Optimisation Problems A decision problem (DP) is a computational problem for which the intended output is either yes or no In an optimisation problem (OP) we rather try to maximise or minimise some value An OP can be turned into a DP if we add a parameter k, and then ask whether the optimal value in OP is at most or at least k Note that if a DP is hard, then its related its optimisation version must be hard too

week 10Complexity of Algorithms3 Decision/Optimisation Problems Example: Optimisation problem - Given graph G with integer weights on its edges. What is the weight of a minimum spanning tree (MST) in G? Decision problem – Given graph G with integer weights on its edges, and an integer k. Does G have a minimum spanning tree of weight at most k?

week 10Complexity of Algorithms4 Problems and Languages We say that the algorithm A accepts an input string x if A outputs yes on input x A decision problem can be viewed as a set L of (binary) strings – the strings that should be accepted by an algorithm that correctly solves the problem We often refer to L as a language We say that an algorithm A accepts a language L if A outputs yes for each x in L and outputs no otherwise

week 10Complexity of Algorithms5 The Complexity Class P The complexity class P is the set of all decision problems (or languages) L that can be solved in worst-case polynomial time That is, there is an algorithm A that if x  L, then on input x, A outputs yes in time p(n), where n is the size (length) of x and p(n) is a polynomial

week 10Complexity of Algorithms6 The Complement of a Language The complement of a language L consists of all strings that are not in L If we have a p(n) time algorithm A that accepts L (i.e., L is in P) we can construct a p(n) time algorithm B (based on A) that accepts the complement of L, i.e., Run algorithm A on input string x for p(n) steps If A outputs yes, then B outputs no If A outputs no or give no output, then B outputs yes If L is in P, the complement of L is in P too!

week 10Complexity of Algorithms7 The Complexity Class NP An algorithm that chooses (by a really good guess!) some number of non-deterministic bits during its execution is called a non-deterministic algorithm We say that an algorithm A non-deterministically accepts a string x if there exists a choice of non- deterministic bits that leads to the ultimate answer yes The complexity class NP is the set of decision problems (or languages) L that can be non- deterministically accepted in polynomial time Obviously P  NP

week 10Complexity of Algorithms8 The Complement of L in NP Note that the definition of class NP does not address the running time of rejection (which might be very long) And indeed even knowing that we can choose an appropriate number of non-deterministic bits for all strings in L in NP we cannot assure that such a choice is feasible for the complement of L In fact there is a class co-NP that consists of all languages whose complements are in NP Many researchers believe that co-NP  NP

week 10Complexity of Algorithms9 The P = NP Question Computer scientists do not know for certain whether P = NP or not We also do not know whether P = NP  co-NP However there is a common believe that P is different then both NP and co-NP, as well as they intersection

week 10Complexity of Algorithms10 Hamiltonian Cycle is NP Hamiltonian-Cycle is the problem that takes a graph G as an input and asks whether there is a simple (Hamiltonian) cycle in G that visits every vertex of G exactly once The non-deterministic algorithm chooses a cycle (represented by a sequence of non- deterministic bits) and then it checks deterministically whether this cycle is indeed a Hamiltonian cycle in G

week 10Complexity of Algorithms11 Boolean Circuit A Boolean circuit is a directed graph where each node, called a logic gate corresponds to a simple Boolean function AND, OR, or NOT The incoming edges for a logic gate correspond to inputs for its Boolean function and the outgoing edges correspond to the outputs

week 10Complexity of Algorithms12 Boolean Circuit (example)

week 10Complexity of Algorithms13 Circuit-SAT is in NP Circuit-Sat is the problem that takes an input a Boolean circuit with a single output node, and asks whether there is an assignment of values to the circuit’s inputs so that its output value is 1 The non-deterministic algorithm chooses an assignment of input bits (represented by a sequence of non-deterministic bits) and then it checks deterministically whether this input generates output 1

week 10Complexity of Algorithms14 Vertex Cover Given a graph G=(V,E), a vertex cover for G is a subset C  V, s.t., for every edge (v,w) in E, v  C or w  C The optimisation problem is to find as small a vertex cover as possible Vertex-Cover is the decision problem that takes a graph G and an integer k as input, and asks whether there is a vertex cover for G containing at most k vertices

week 10Complexity of Algorithms15 Vertex-Cover is in NP Suppose we are given an integer k and a graph G The non-deterministic algorithm chooses a subset of vertices C  V, s.t., ¦C¦  k, (represented by a sequence of non- deterministic bits) and then it checks deterministically whether this subset C is an appropriate vertex cover

week 10Complexity of Algorithms16 Polynomial-Time Reducibility We say that a language L, defining some decision problem, is polynomial-time reducible to a language M, if there is a function f computable in polynomial time, that takes an input x to L, and transforms it to an input f(x) of M, s.t., x  L if and only if f(x)  M We use notation L poly  M to signify that language L is polynomial-time reducible to language M

week 10Complexity of Algorithms17 NP-hardness We say that a language M, defining some decision problem, is NP-hard if every other language L in NP is polynomial-time reducible to M, i.e., M is NP-hard, if for every L  NP, L poly  M If a language M is NP-hard and it belongs to NP itself, then M is NP-complete NP-complete problem is, in a very formal sense, one of the hardest problems in NP, as far as polynomial-time reducibility is concerned

week 10Complexity of Algorithms18 The Cook-Levin Theorem Circuit-Sat is NP-complete Proof [sketch]: A computation steps of any (reasonable) algorithm can be simulated by layers in appropriately constructed (in polynomial time and size) Boolean circuit

week 10Complexity of Algorithms19 The Cook-Levin Theorem

week 10Complexity of Algorithms20 Other NP-complete Problems We have just noted that there is at least one NP-complete problem Using polynomial-time reducibility we can show existence of other NP- complete problems according to Lemma: If L 1 poly  L 2 and L 2 poly  L 3 then L 1 poly  L 3

week 10Complexity of Algorithms21 Types of reduction Let M be known NP-complete problem. The types of reductions are: By restriction: noting that known NP-complete problem is a special case of our problem L Local replacement: dividing instances of M and L into basic units, and then showing how each basic unit of M can be locally converted into a basic unit of L Component design: building components for an instance of L that will enforce important structural functions for instances of M

week 10Complexity of Algorithms22 Important NP-complete Problems

week 10Complexity of Algorithms23 Conjunctive Normal Form A Boolean formula is in conjunctive normal form (CNF) if it is formed as a collection of clauses combined using operator AND (·), where each clause is formed by literals (variables or their negations) combined using operator OR (+), e.g.,

week 10Complexity of Algorithms24 CNF-SAT & 3SAT Problem CNF-SAT takes a Boolean formula in CNF form as input and asks if there is an assignment of Boolean values to its variables so that the formula evaluates to 1 (i.e., formula is satisfiable) 3SAT is CNF-SAT in which each clause has exactly 3 literals Fact: CNF-SAT and 3-SAT are NP-complete

week 10Complexity of Algorithms25 Vertex-Cover is NP-complete We can show that Vertex-Cover is NP-hard by reducing 3SAT problem to it in polynomial time This reduction is an example of a reduction from a logic problem to a graph problem It also illustrates an application of the component design proof technique

week 10Complexity of Algorithms26 Vertex-Cover is NP-complete

week 10Complexity of Algorithms27 Vertex-Cover is NP-complete

week 10Complexity of Algorithms28 Vertex-Cover is NP-complete

week 10Complexity of Algorithms29 Approximation Schemes One way of dealing with NP-completeness for optimisation problems is to use an approximation algorithm The goal of an approximation algorithm is to come as close to the optimum value as possible Such an algorithm typically runs much faster than an algorithm that strives for exact solution

week 10Complexity of Algorithms30 Approximation Schemes Let c(S) be the value of a solution S delivered by an algorithm A to an optimisation problem P and OPT will be the optimal solution for P We say that A is a  -approximation algorithm for a minimisation problem P if c(S)   ·OPT And A is is a  -approximation algorithm for a maximisation problem P if c(S)   ·OPT

week 10Complexity of Algorithms31 Polynomial-Time Approximation Scheme (PTAS) There are some problems for which we can construct  -approximation algorithms that run in polynomial time with  =1+ , for any fixed value  > 0 The running time of such collection of algorithms depends both on n, the size of an input and also on a fixed value  We refer to such collection of algorithms as a polynomial-time approximation scheme, or PTAS If the running time is polynomial in both n and 1/  we have a fully polynomial-time approximation scheme