2. Hard problems November 29. 2001

3. Administrivia Ps 8 –Handed out Tuesday (12/4) –Due Tuesday (12/11) or Tuesday (12/18) your choice Ps 9 –Handed out Thursday (12/6) –In 3 parts Part a due 12/11 Part b due 12/13 Part c due 12/14

4. The big picture We built a computer We built an operating system We attached the computer to a network We wrote a compiler We designed algorithms We attached the computer to a network Next, we want to see if there are limitations –Mathematical proofs that things cannot be done???

5. Simple unsolvable problem This sentence is false.

6. Limitations of Computers? Possible impossible tasks for computers? Emotion Beating the world chess champion Creative thought Others?

7. Limitations of Computers? How to show that such tasks are impossible? Find a cleanly defined problem Prove a theorem that says it can’t be solved Theorem is proved using axioms (like logic gates) that are assembled (like state machines and computers) to justify the result The underlying field is called mathematical logic

8. Simple task --The “Hello” Assignment You write a computer program which outputs “Hello!” and stops. Assignment promises full credit for ANY program that outputs “Hello!” and then halts, and no credit otherwise. Assignment is to be done on a computer that is as fast as you like and has as much memory as you like.

9. The “Hello” Assignment Can I write a program that automatically grades your homework?

10. The “Hello” Assignment Can I write a program that automatically grades your homework? Sample homework Function GradeHELLO (Program P) Execute P, and store output in output If (output = “Hello!”) Then Return “Pass” Else Return “Fail” End If End Function

11. The “Hello” Assignment Possible glitch –What if the program never stops? Program P Do While (1 > 0) Print “Hello!” End While End Program

12. The “Hello” Assignment Student’s program runs forever! This means the grading program would run forever. It would never tell me that the student should fail… Program P Do While (1 > 0) Print “Hello!” End While End Program

13. The “Hello” Assignment Can just stop the Program after 1 hour and fail the student if the program hasn’t stopped yet. Program P Do While (1 > 0) Print “Hello!” End While End Program Program P Print “Hello!” Do While (1 hour hasn’t passed) Do Nothing End While End Program

14. The “Hello” Assignment Other kind of program you could write Program P Initialize x = 0 Do While (x is not a proof of Fermat’s Last Theorem) x = x + 1 End While Print “Hello!” End Program

15. The “Hello” Assignment Other kind of program you could write Program P Initialize x = 0 Do While (x is not a proof of Fermat’s Last Theorem) x = x + 1 End While Print “Hello!” End Program Interpret x as a sequence of mathematical statements…

16. The “Hello” Assignment Other kind of program you could write Grading program might say to fail you. But technically you deserve to Pass. Unless the proof is wrong Program P Initialize x = 0 Do While (x is not a proof of Fermat’s Last Theorem) x = x + 1 End While Print “Hello!” End Program

17. The “Hello” Assignment We keep running in to problems… Could it be that writing a computer program to grade such a simple assignment is impossible? YES! Moreover we can prove that it is impossible.

18. The Halting Problem Can we design a program that tells whether other programs ever halt on their own input. So, for program P Output “halts” if P(P) eventually halts Output “never halts” if P(P) never halts Theorem (Alan Turing 1937): No computer program can solve the halting problem.

19. Proof by Contradiction Suppose there was a computer program called HALT, which solved the halting problem. Consider the following program: Program NonConformist (Program P) If ( HALT(P) = “never halts” ) Then Halt Else Do While (1 > 0) Print “Hello!” End While End If End Program

20. Proof by Contradiction Does NonConformist(NonConformist) halt? Yes? That means HALT(NonConformist) = “never halts” No? That means HALT(NonConformist) = “halts” Program NonConformist (Program P) If ( HALT(P) = “never halts” ) Then Halt Else Do While (1 > 0) Print “Hello!” End While End If End Program Contradiction!

21. Undecidability Halting Problem: Given a computer program P: Output “halts” if P(P) eventually halts Output “never halts” if P(P) never halts We’ve shown that the Halting Problem is Undecidable – no computer program can ever solve it, no matter how powerful the computer is.

22. Undecidability Many other problems are undecidable, too. Moreover, the same ideas that we used to prove undecidability can be used to prove a very disturbing statement.

23. GÖDEL’S INCOMPLETENESS THEOREM Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework.In 1931, Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework. “Not every true statement has a proof”“Not every true statement has a proof”

24. GÖDEL’S INCOMPLETENESS THEOREM Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework.In 1931, Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework. But wait, if it can’t be proved, how do we know it is true?But wait, if it can’t be proved, how do we know it is true?

25. GÖDEL’S INCOMPLETENESS THEOREM We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…

26. GÖDEL’S INCOMPLETENESS THEOREM We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework… But that larger logical framework also has a true statement that cannot be proved!But that larger logical framework also has a true statement that cannot be proved!

27. GÖDEL’S INCOMPLETENESS THEOREM We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework… But that larger logical framework also has a true statement that cannot be proved!But that larger logical framework also has a true statement that cannot be proved! …and so on, and so on……and so on, and so on… No matter when we stop, there will always be some true statement that doesn’t have a proof!No matter when we stop, there will always be some true statement that doesn’t have a proof!

28. GÖDEL’S INCOMPLETENESS THEOREM Until Gödel came along, Mathematicians were searching for the “one true logical framework” in which everything could be proved.Until Gödel came along, Mathematicians were searching for the “one true logical framework” in which everything could be proved. Their faith was shattered by Gödel.Their faith was shattered by Gödel. For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.

29. What Computers Can’t Do In Your Lifetime We’ve now seen an example of a problem that computers can’t solve, even if computers have unlimited speed and unlimited time. Are there more real world problems (eg that arise in business, science, … ) that can be solved but take billions of years?

30. “Search” Problems We’ll focus on what we’ll call “search problems”. (In CS terms, we’ll be talking about what are termed NP problems) Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it.

31. Example: Coloring

32. Example: Coloring with 3 colors Suppose we are given a collections of circles. Some circles are connected to others. Rule: No connected circles can have the same color. You only have three colors (Green, Red, Yellow) Is there a valid coloring? Note: easy to check validity. Hard to find?

33. Search Problems: More Precise Definition Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it. A bit more precisely, we require that there is a small circuit that can quickly check the validity of a solution. For coloring, it would just be a circuit that checks that for every pair of connected circles, the colors are different.

34. Example: Traveling Salesman Problem A salesman wants to visit n different cities. He knows the costs associated with flights between each city. Can he visit all the cities spending less than $B total? Note: Easy to check that a given flight plan visits all the cities and costs less than $B. Seems hard to find…

35. Search Problems… Unfortunately, we don’t know how to solve these and many other search problems with a computer in our lifetimes for large inputs. For many years, computer scientists wondered which search problems could be solved, and which couldn’t. Just because computer scientists couldn’t solve the Coloring problem for 40 years doesn’t mean it is impossible, right? Proof of Fermat’s theorem took over 300 years

36. Search Problems… We still don’t know how to prove that it is impossible to solve them quickly… But now we are much more confident that they really are impossible to solve quickly.

37. Breakthrough Another surprise: In the early 70’s, Cook, Levin, and Karp showed us that if we can solve the Coloring problem quickly, then we can solve ALL search problems quickly! But general search problems are defined in terms of circuits, not colors…

38. Coloring Circuits? Suppose we think of : Green as meaning True. Red as meaning False Yellow as meaning nothing

39. Coloring Circuits? TF X making a NOT gate by coloring NOT X

40. T F X Y Output XY FF FT TF TT

41. T F X Y XY FF FT TF TT T F

42. T F X Y XY FF FT TF TT

43. T F X Y XY FF FT TF TT

44. T F X Y XY FF FT TF TT

45. T F X Y XY FF FT TFT TT

46. XY FFF FTT TFT TTT T F X Y

47. XYOR FFF FTT TFT TTT = T F X Y Output

48. Coloring ! And you thought coloring was for kids.. In fact, we can encode any circuit into a collection of connected circles waiting to be colored. Thus, we can efficiently reduce any search problem to Coloring. If we can solve Coloring quickly, we can solve any search problem quickly!

49. Many more Traveling Salesman Problem is also as “hard” as any search problem. Search Problems with this property are called complete problems Although we don’t know how to prove that they are hard, we know that if we solve one of them, we solve all search problems! Gives us more confidence that they really are hard.

50. Wrap-up Today we saw problems that computers can’t solve at all, and also problems that computers probably can’t solve in our lifetimes. Too much bad news? Could it ever be useful to have hard problems? Yes! Next time: Cryptography