How Hard Can It Be?

Tractable versus Intractability Given an algorithm A that solves a problem. If the worst case time efficiency of A is O(p(n)), for some polynomial p, then A is said to solve the problem in polynomial time. If a problem can be solved in polynomial time it is said to be tractable. 11/29/2018 DRAFT

P A decision problem is a problem with one of two answers, e.g., yes/no, 1/0, or true/false. P = {decision problems solvable by a deterministic algorithm in O(p(n)) time} = {tractable decision problems} What problem would you be solving if it had a best algorithm with the largest degree polynomial you have experienced? How “tractable” is a problem if the degree of p(n) is large? 11/29/2018 DRAFT

Decidability Are all decision problems solvable (decidable)? A possibly undecidable problem: Does the Collatz sequence, a.k.a., the 3 n + 1 sequence always have finite length? Does the sequence, C(t) = 3*C(t-1)+1, if C(t-1) is odd, and C(t) = C(t-1)/2, if C(t-1) is even, C(t0)an integer such that C(t0)>1, always reach C(t) = 1 in a finite number of steps? 11/29/2018 DRAFT

Existence of Functions that are Not Turing Computable Recall from the theory of computation that: Computability of a function means one can find a Turing machine that computes the function The set Turing machines is countable; thus, they can be put in 1-1 correspondence with the counting numbers, N¥ Consider the collection of functions f:N→N How many such functions are there? 2|N|>>|N| Thus, there are functions not Turing computable ¥ See, e.g., Dennis, Denning and Qualitz, Machines, Languages, and Computation, Prentice-Hall, 1978, pp. 497. 11/29/2018 DRAFT

An Example: The Halting Problem The halting problem: Find an algorithm A such that: Given any program P and any input I for P, The algorithm decides whether P will finish processing with I in finite time 11/29/2018 DRAFT

A Proof By Contradiction that the Halting Problem is Undecidable Let P be any program, I any input for P, and suppose A is an algorithm that can decide if P halts on I: 𝐴(𝑃, 𝐼)= 1, 𝑖𝑓 𝑃 ℎ𝑎𝑙𝑡𝑠 𝑜𝑛 𝐼 0, 𝑖𝑓 𝑃 𝑑𝑜𝑒𝑠 𝑛𝑜𝑡 ℎ𝑎𝑙𝑡 𝑜𝑛 𝐼 Let A operate on P with P as input in order to define a program Q: Q P = ℎ𝑎𝑙𝑡𝑠, 𝑖𝑓 𝐴 𝑃,𝑃 =0, 𝑖.𝑒., 𝑃 𝑑𝑜𝑒𝑠 𝑛𝑜𝑡 ℎ𝑎𝑙𝑡 𝑑𝑜𝑒𝑠 𝑛𝑜𝑡 ℎ𝑎𝑙𝑡, 𝑖𝑓 𝐴 𝑃,𝑃 =1, 𝑖.𝑒., 𝑃 ℎ𝑎𝑙𝑡𝑠 𝑜𝑛 𝑃 11/29/2018 DRAFT

Example Program Properties of Program Q Halts if A returns False; Does not halt if A returns = True #Program Q import random TIMELIMIT = 5 def A(): #Mimics an algorithm that decides whether a program terminates given some input if random.random()<0.5: return True else: return False result = A() if result: print('Algorithm A returned "HALTS"') print('Algorithm A returned "DOES NOT HALT" and program Q HALTS') elapsedtime=0 while result and (elapsedtime< TIMELIMIT): elapsedtime+=1 print(elapsedtime, 'Q does not HALT') if elapsedtime >= TIMELIMIT: print('Timed out') 11/29/2018 DRAFT

Proof (Continued) On the previous slide is a program Q: Q P = ℎ𝑎𝑙𝑡𝑠, 𝑖𝑓 𝐴 𝑃,𝑃 =0, 𝑖.𝑒., 𝑃 𝑑𝑜𝑒𝑠 𝑛𝑜𝑡 ℎ𝑎𝑙𝑡 𝑑𝑜𝑒𝑠 𝑛𝑜𝑡 ℎ𝑎𝑙𝑡, 𝑖𝑓 𝐴 𝑃,𝑃 =1, 𝑖.𝑒., 𝑃 ℎ𝑎𝑙𝑡𝑠 𝑜𝑛 𝑃 What happens with Q(Q)? Q Q = ℎ𝑎𝑙𝑡𝑠, 𝑖𝑓 𝐴 𝑄,𝑄 =0, 𝑖.𝑒., 𝑄 𝑑𝑜𝑒𝑠 𝑛𝑜𝑡 ℎ𝑎𝑙𝑡 𝑑𝑜𝑒𝑠 𝑛𝑜𝑡 ℎ𝑎𝑙𝑡, 𝑖𝑓 𝐴 𝑄,𝑄 =1, 𝑖.𝑒., 𝑄 ℎ𝑎𝑙𝑡𝑠 𝑜𝑛 𝑄 Thus, Q must halt when Q does not halt, and Q does not halt when Q halts—a contradiction. 11/29/2018 DRAFT

An Analogy Consider the following statements: p: The statement q is true. q: The statement p is false. What can be said about the truth of the statements p and q? 11/29/2018 DRAFT

Nondeterministic Algorithms Nondeterministic algorithms possess two properties: Candidate solutions may be generated by random guessing Verification of validity and evaluation of quality are of O(p(n)) for some polynomial p 11/29/2018 DRAFT

Example: A TSP Algorithm Repeat: Generate a random permutation of the city labels (choose a trip by guessing) Evaluate the length of the trip (validate and evaluate) How hard is this? This is not a decision problem but could be used to derive a decision problem. 11/29/2018 DRAFT

NP Consider the nondeterministic algorithms with worst case run time O(p(n)) for some polynomial p. NP = {decision problems solvable by a nondeterministic polynomial algorithm} 11/29/2018 DRAFT

Polynomially Reducible Given two decision problems D1 and D2. Suppose there exists a transformation t of instances of D1 into instances of D2 such that: All “yes” instances of D1 are mapped to instances of D2 that produce “yes” responses All “no” instances of D1 are mapped to instances of D2 that produce “no” responses t(.) is O(p(n)) for some polynomial Then D1 is said to be polynomially reducible to D2 11/29/2018 DRAFT

NP-Complete A decision problem D is said to be NP-Complete if: D is in NP Every other problem in NP is polynomially reducible to D 11/29/2018 DRAFT

Satisfiability Consider a logic expression in conjunctive normal form (CNF), e.g., (x1 ∨ ¬x5 ∨ x4) ∧ (¬x1 ∨ x5 ∨ x3 ∨ x4)… Does there exist an assignment of truth values for the propositions x1, x2, x3…, that makes the expression true? 11/29/2018 DRAFT

NP-Hard A problem is NP-Hard, if satisfiability reduces to it Any problem in NP is thus at least as hard as an NP-complete problem. Many optimization problems 11/29/2018 DRAFT

The Halting Problem, Again The Halting Problem is Not in NP: It is undecidable, so there is no algorithm, no matter what the complexity, to solve it. NP-hard: Because satisfiability reduces to it: Construct an algorithm A with a propositional formula X as input, X having n logic variables A searches all 2n combinations of logic values to determine if X is satisfiable If so, A stops; otherwise A enters an infinite loop A halts if and only if X is satisfiable Note: If there were a polynomial algorithm for the halting problem, the mapping would allow satisfiability to be solved using A and X as its inputs. 11/29/2018 DRAFT