Theory of Computation 1 Theory of Computation Peer Instruction Lecture Slides by Dr. Cynthia Lee, UCSD are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Based on a work at Cynthia Lee, UCSDCreative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported Licensewww.peerinstruction4cs.org

WHY IS IT CALLED DIAGONALIZATION? Looking back on diagonalization proof that A TM is undecidable 2

Diagonalization to prove that |R| > |N| z cannot be in f(n) for any n in N, because z differs from f(n) in f(n)’s nth decimal place Easy to see why we call this “diagonalization” 3 nf(n) 1.d 1 1 d 1 2 d 1 3 d 1 4 d 1 5 d 1 6 d 1 7 d 1 8 d 1 9 … 2.d 2 1 d 2 2 d 2 3 d 2 4 d 2 5 d 2 6 d 2 7 d 2 8 d 2 9 … 3.d 3 1 d 3 2 d 3 3 d 3 4 d 3 5 d 3 6 d 3 7 d 3 8 d 3 9 … 4.d 4 1 d 4 2 d 4 3 d 4 4 d 4 5 d 4 6 d 4 7 d 4 8 d 4 9 … 5.d 5 1 d 5 2 d 5 3 d 5 4 d 5 5 d 5 6 d 5 7 d 5 8 d 5 9 … 6.d 6 1 d 6 2 d 6 3 d 6 4 d 6 5 d 6 6 d 6 7 d 6 8 d 6 9 … 7.d 7 1 d 7 2 d 7 3 d 7 4 d 7 5 d 7 6 d 7 7 d 7 8 d 7 9 … 8.d 8 1 d 8 2 d 8 3 d 8 4 d 8 5 d 8 6 d 8 7 d 8 8 d 8 9 … 9.d 9 1 d 9 2 d 9 3 d 9 4 d 9 5 d 9 6 d 9 7 d 9 8 d 9 9 … ……

D( ): Run M ATM (M( )), if acc->rej, if rej->acc … … M1AccRejAccRej M2Acc M3Rej M4AccRejAccRejAccRej M5Acc Rej... DRej … 4 What goes in the green region? (a)Acc, Rej, Rej, Rej (b)Rej, Acc, Acc, Acc (c)Not enough information Why do we call the A TM undecidability proof “diagonalization”?

D( ): Run M ATM (M( )), if acc->rej, if rej->acc 5 Why do we call the A TM undecidability proof “diagonalization”? What goes in the green region? (a)Rej (b)Acc (c)Not enough information (d) Other! Paradox! … … M1AccRejAccRej M2Acc M3Rej M4AccRejAccRejAccRej M5Acc Rej... DRej …

REDUCTIONS USING A TM EXAMPLES Changing one problem into another 6

Thm.: TWO = { | for all w in L(M), w has exactly two 1s} is undecidable Proof by contradiction: Assume TWO is decidable by TM D TWO. Use D TWO to construct TM D ATM that decides A TM. D ATM ( ): – String Z = “Z(x): \n” + “Count the number of 1s in x. If it is 2, accept.\n” + “Simulate” + + “ on input “ + w + “. If it accepts, accept. If it rejects, reject. ”; – Call D TWO (Z) //D TWO is a decider for TWO – If D TWO (Z) accepts, reject, else accept. But A TM is undecidable, a contradiction. So the assumption is false and T is undecidable. QED. 7 What is L(M’)? (a) Σ* (b) {x | x has exactly two 1s} (c) Σ * if M(w) accepts, and {x|x has exactly two 1s} if M(w) does not accept (d) Σ * if M(w) accepts, and the empty set if M(w) rejects.

We just did a reduction We showed that if we have a solution to TWO, then we have a solution to A TM. What did we show exactly? a)A TM reduces to TWO. b)TWO reduces to A TM. c)TWO and A TM reduce to each other. d)None of the above or more than one of the above. 8

Thm. T = { | M is a TM and both “101” and “111” are in L(M)} is undecidable Proof by contradiction: (Reduce from A TM.) Assume T is decidable by TM M T. Use M T to construct TM D ATM that decides A TM. D ATM ( ): – String Z = “Z(x): \n” + “If x != “111” and x != “101” then reject. \n” + “Simulate” + + “ on input “ + w + “. If it accepts, accept. If it rejects, reject. ”; – Run M T (Z). If it accepts then accept, otherwise reject. But A TM is undecidable, a contradiction. So the assumption is false and T is undecidable. QED. 9 What is L(Z)? (a) Σ * (b) {“101”, “111”} (c) {“101”,”111”} if M(w) accepts, otherwise empty set (d) Σ * if M(w) accepts, and {“101”,”111”} if M(w) does not accept

We just did a reduction We showed that if we have a solution to T, then we have a solution to A TM. What did we show exactly? a)A TM reduces to T. b)T reduces to A TM. c)T and A TM reduce to each other. d)None of the above or more than one of the above. 10

Thm. T = { | M is a TM that accepts w R whenever it accepts w} is undecidable Proof by contradiction: (Reduce from A TM.) Assume T is decidable by TM M T. Use M T to construct TM D ATM that decides A TM. D ATM ( ): – String Z = “Z(x): \n” + “If x != “01” and x != “10” then reject \n” + “If x == “01” accept \n” + “ ??? ”; – Run M T (Z). If it accepts then accept, otherwise reject. But A TM is undecidable, a contradiction. So the assumption is false and T is undecidable. QED. 11 How do we finish Z? (a)Run M(w), if it accepts then accept. If it rejects then reject (might loop in which case obviously Z loops). (b)Run M(w), if it accepts then reject. If it rejects then accept (might loop in which case obviously Z loops).

We just did a reduction We showed that if we have a solution to T, then we have a solution to A TM. What did we show exactly? a)A TM reduces to T. b)T reduces to A TM. c)T and A TM reduce to each other. d)None of the above or more than one of the above. 12

Rice’s Theorem Let P be any non-trivial property of the language of a Turing machine The problem of determining whether a given Turing machine’s language has property P is undecidable It doesn’t matter what the property is! – Non-trivial here means that P is a property of the language of at least one, but not all, Turing machines – Also, P must be a property of the language of the Turing machine, i.e., if L(TM 1 ) = L(TM 2 ), then P is true of L(TM 1 ) iff P is true of L(TM 2 ) 13

PROPERTIES OF REDUCTIONS What can we conclude from a reduction?

MYSTERY_LANG reduces to A CFG Which of the following is true (given the above statement is true): a)You can reduce from MYSTERY_LANG to A CFG. b)MYSTERY_LANG is decidable. c)A decider for A CFG (if it exists) could be used to decide MYSTERY_LANG. d)All of the above 15

A TM reduces to MYSTERY_LANG Which of the following is true (given the above statement is true): a)You can reduce to MYSTERY_LANG from A TM. b)MYSTERY_LANG is undecidable. c)A decider for MYSTERY_LANG (if it exists) could be used to decide A TM. d)All of the above 16

MYSTERY_LANG reduces to A TM Which of the following is true (given the statement above is true): a)You can reduce to A TM from MYSTERY_LANG. b)MYSTERY_LANG is undecidable. c)A decider for A TM (if it exists) could be used to decide MYSTERY_LANG. d)All of the above e)Other 17

MEASURING COMPLEXITY OF DECIDABLE PROBLEMS BIG-O Review 18

Review: How Big-O Simplifies Analysis The number of transitions Turing Machine M takes when run on input w can be written as a function of n=|w| like this: f(n) = 10n n Which best represents a tight bound on the big-O running time of M(w)? (where n = |w|) a)O(n 3 + n 2 ) b)O(n 5 ) c)O(n 3 ) d)O(200n 2 ) 19

Counting Big-O Let L = { | Grammar G generates at least one string of length | | } (Σ = {0,1}) TM M decides L: M( ): – For each string x in {0,1}* that is of length | | Run TM M ACFG ( ), if accept, then accept // TM that decides A CFG – (if didn’t accept yet) Reject. Which best represents a tight bound on the big-O running time of M( )? (where n = | |) a)O(n 2 ) b)O(n 4 ) c)O(2 n ) d)O(2 n 2 ) e)O(4 n ) 20

MEASURING COMPLEXITY OF DECIDABLE PROBLEMS P AND NP Complexity Classes 21

Polynomial Time A language L can be decided in polynomial time if there exists a TM recognizing L that, on input n characters long, will accept or reject in O(n k ) steps in the worst case, for some constant k – (i.e., n 2n doesn’t count) 22

P and NP P is the class of languages that can be decided in polynomial time on a deterministic, single- tape TM NP is the class of languages that can be decided in polynomial time on a non- deterministic TM – Alternate definition of NP: problems where, if somebody gives you a solution, you can check it for correctness in polynomial time on a deterministic, single-tape TM – NP stands for non-determinisitic polynomial 23

Complexity Classes The red part is not actually known!! P = NP?? P NP?? Is it possible that NP P? Can you prove or disprove it? 24

P = NP ???????? Perhaps the greatest theoretical computation question of our time Most people suspect no, but it has never been proven… If you can prove it one way or the other, you win $1,000,000 from the Clay Institute 25

A CLUE IN THE P ≠/= NP PUZZLE? THE CLASS NP-COMPLETE Complexity Classes 26

Polynomial-time Mapping Reducibility If A B (we can transform an instance of problem A into something that can be solved with one call to a decider for problem B), and M B uses a polynomial number of steps in the worst case, then as long as the process of reducing A to B takes a polynomial number of steps… A B A is in P! 27

Mapping Reducibility and NP-Completeness There are a class of problems that are known to be in NP (can be solved in polynomial time using nondeterminism), but no deterministic polynomial time algorithm has ever been discovered to solve any of them – Boolean equation satisfiability – Traveling salesman – Various graph problems (Clique, Independent Set, …) – Various scheduling optimization problems – FreeCell (Windows game), Sudoku – Map coloring Amazingly, all these problems can be to each other This means that a polynomial-time solution to ONE is a polynomial time solution to ALL !! 28

3-SAT and CLIQUE (p.275) 29

If P ≠ NP 30

If P = NP 31

CONSEQUENCES OF P ≠/= NP WHAT’S AT STAKE? Wonderful!!! Doom!!! 32

If P = NP, what would the consequences be? a)Many cryptographic systems are no longer secure, so all your Facebook, Gmail and online credit card transactions become insecure in a way they weren’t before b)Many important optimizations may now be computationally practical, improving our lives and efficiency in many things c)It takes as long to verify the correctness of an answer to problems in NP as it does to find the correct answer in the first place d)Other/None/More than one 33