The Halting Problem Can we design a program that, given any other program and its input, tells whether that program will halt when run on that input? Output of this program, called HALT, when applied to program P and input I, is “halts” if P(I) eventually halts Output of this program is “never halts” if P(I) never halts Theorem (Alan Turing 1937): No computer program can solve the halting problem.

Proof by Contradiction Assume the hypothesis to be proven is false, i.e. there is a computer program that can solve the halting problem Show that this assumption leads to a contradiction Do it by creating a program and an input to it that generate this contradiction

Proof by Contradiction Suppose there was a computer program, called HALT, which solved the halting problem. We can write a new program that uses HALT and leads to a contradiction: Program NonConformist (Program P) If ( HALT(P) = “never halts” ) Then Halt Else Do While (1 > 0) Print “Hello!” End While End If End Program

Proof by Contradiction Does NonConformist(NonConformist) halt? Note: It calls HALT(NonConformist) Yes? Means HALT(NonConformist) = never halts No? That means HALT(NonConformist) = halts Contradiction: There exists a program (NonConformist) for which the HALT program gives the wrong answer Program NonConformist (Program P) If ( HALT(P) = “never halts” ) Then Halt Else Do While (1 > 0) Print “Hello!” End While End If End Program

Undecidability Halting Problem: Given a computer program P and input “input”: Output “halts” if P(input) eventually halts Output “never halts” if P(input) never halts We’ve shown that the Halting Problem is Undecidable – no computer program can ever solve it, no matter how powerful the computer is.

Undecidability Many other problems are undecidable, too. Moreover, the same ideas that we used to prove undecidability can be used to prove a very disturbing statement.

GÖDEL’S INCOMPLETENESS THEOREM Until Gödel came along, Mathematicians were searching for the “one true logical framework,” i.e. a framework in which every true statement can be proved.Until Gödel came along, Mathematicians were searching for the “one true logical framework,” i.e. a framework in which every true statement can be proved. Their faith was shattered by Gödel.Their faith was shattered by Gödel. For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.

GÖDEL’S INCOMPLETENESS THEOREM Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework.In 1931, Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework. “Not every true statement has a proof”“Not every true statement has a proof”

GÖDEL’S INCOMPLETENESS THEOREM Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework.In 1931, Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework. But wait, if it can’t be proved, how do we know it is true?But wait, if it can’t be proved, how do we know it is true?

GÖDEL’S INCOMPLETENESS THEOREM We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…

GÖDEL’S INCOMPLETENESS THEOREM We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework… But that larger logical framework also has a true statement that cannot be proved!But that larger logical framework also has a true statement that cannot be proved!

GÖDEL’S INCOMPLETENESS THEOREM We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework… But that larger logical framework also has a true statement that cannot be proved!But that larger logical framework also has a true statement that cannot be proved! …and so on, and so on……and so on, and so on… No matter when we stop, there will always be some true statement that doesn’t have a proof!No matter when we stop, there will always be some true statement that doesn’t have a proof!

GÖDEL’S INCOMPLETENESS THEOREM Until Gödel came along, Mathematicians were searching for the “one true logical framework,” i.e. a framework in which every true statement can be proved.Until Gödel came along, Mathematicians were searching for the “one true logical framework,” i.e. a framework in which every true statement can be proved. Their faith was shattered by Gödel.Their faith was shattered by Gödel. For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.

More Undecidable Problems? Post's Correspondence Problem (PCP) An instance of PCP of size s is a finite set of pairs of strings (g i, h i ) [for i = 1...s; s>=1] over some alphabet . A solution is a sequence i 1 i 2... i n of selections from each set of strings (g i, h i ) such that the strings g i1 g i2... g in and h i1 h i2... h in formed by concatenation are identical.

Sample PCP g 1 = abah 1 = abaa g 2 = bbabh 2 = abab g 3 = baaah 3 = a g 4 = ah 4 = bb So, 1,3,1,2 would correspond to aba baaa aba bbab from g’s abaa a abaa abab from h’s Not a solution!

Another Sample PCP g 1 = aba h 1 = abaa g 2 = bbab h 2 = abab g 3 = baaa h 3 = a g 4 = a h 4 = bb 1,4,2,1,3 corresponds to aba a bbab aba baaa from g abaa bb abab abaa a from h Solution!

PCP is undecidable? PCP shown to be undecidable by Post in What about PCP with limited-size inputs PCP with size 2 has been proved decidable. PCP with size 7 has been proved undecidable The decidablility of problems with size between 3 and 6 is still pending.

What Computers Can’t Do In Your Lifetime We’ve now seen examples of problems that computers can’t solve, even if computers have unlimited speed and unlimited time. Are there more real world problems (eg that arise in business, science, … ) that can be solved but take far too much time to be solved in practice?

“Search” Problems We’ll focus on what we’ll call “search problems”. (In Computer Science terms, we’ll be talking about what are termed NP problems) Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it. Recognizing a good solution is easy Problem is finding it

Example: Coloring

Example: Coloring with 3 colors Suppose we are given a collection of circles (nodes). Some circles connect to others by edges, forming a graph Rule: No two connected circles can have the same color. You only have three colors (Green, Red, Yellow) Is there a valid coloring? Note: easy to check validity. Hard to find?

Search Problems: More Precise Definition Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it. A bit more precisely, we require that there is a small circuit that can quickly check the validity of a solution. For coloring, it would just be a circuit that checks that for every pair of connected circles, the colors are different.

Another Example: Traveling Salesperson Problem A saleswoman wants to visit n different cities. She knows the costs associated with flights between each city. Can she visit all the cities spending less than $B in total? Note: Easy to check that a given flight plan visits all the cities and costs less than $B. Seems hard to find the flight plan …

Search Problems… Unfortunately, we don’t know how to solve these and many other search problems with a computer in our lifetimes for large inputs. Large graphs, large number of cities For many years, computer scientists wondered which search problems could be solved, and which couldn’t. But just because computer scientists couldn’t solve the Coloring problem for 40 years doesn’t mean it is impossible, right? Proof of Fermat’s Last Theorem took over 300 years

Search Problems… Can we prove that it is impossible to solve them quickly for large inputs? Unfortunately, not yet But now we are much more confident that they really are impossible to solve quickly How do we make progress toward determining an answer?

Breakthrough: Reduction A surprise: In the early 70’s, Cook, Levin, and Karp showed us that if we can solve the Coloring problem quickly, then we can solve ALL search problems quickly! But general search problems are defined in terms of circuits (that can validate their solutions), not colors… Need to map circuits to graph coloring problems Then we can map the circuit for any search problem to the corr. graph coloring problem

Coloring and Circuits Suppose we think of : Green as meaning True. Red as meaning False Yellow as meaning nothing

What is the circuit for this graph? TF Valid coloring for this graph leads to a NOT gate X Input NOT X Output So this is a graph for a NOT circuit

T F X Y Output XY FF FT TF TT

T F X Y XY FF FT TF TT T F

T F X Y XY FF FT TF TT

T F X Y XY FF FT TF TT

T F X Y XY FF FT TF TT

T F X Y XY FF FT TFT TT

XY FFF FTT TFT TTT T F X Y

XYOR FFF FTT TFT TTT = T F X Y Output

Coloring ! And you thought coloring was for kids.. In fact, we can encode any circuit into a collection of connected circles (graph) waiting to be colored. Any valid coloring for the graph conforms to the circuit (which verifies a search problem) Thus, we can efficiently reduce any search problem to Coloring. If we can solve Coloring quickly, we can solve any search problem quickly!

Many more Traveling Salesperson Problem is also as “hard” as any search problem. Search Problems with this property are called complete problems Although we don’t know how to prove that they are hard, we know that if find a way to solve one of them quickly, we can solve all search problems quickly! Gives us more confidence that they really are hard.

Recap: Hard search problems A hard search problem is a problem where It is hard to find a solution It is easy to check possible solutions for validity A complete search problem is a problem that is as hard as any search problem Search problem is believed to be hard because No one found a way to solve any of the complete search problems quickly

What does “Quickly” mean? In time polynomial in the size of the input E.g. if one could solve an n-city TSP in time –n 2 +2n+5 –or even n n …+8n 2 +5n+6 –Then these are a Polynomial-time algorithms (quick) But if one can only solve it in time: –2 n –or 2 n +6 n-1 +…+3 –Then these are Exponential-time algorithms (slow)

n 2 versus 2 n n = 10 –n 2 = 100 –2 n = 1024 n = 100 –n 2 = 10,000 –2 n = (1024) 10 > 1 trillion * 1 trillion * 1 million Hence we call exponential-time algorithms slow

Classes of search problems In computer-science terminology: NP = All Search Problems P = Problems we can solve quickly We believe that P  NP, i.e. not every search problem can be solved quickly on a computer.

NP-Complete Problems Coloring is complete In particular, we can reduce solving any search problem to finding a valid coloring for some collection of circles! So, if we could solve Coloring quickly, then P = NP That’s why we believe Coloring can’t be solved quickly by any computer. We call such problems NP-Complete.