1. 1 Computability Tractable, Intractable and Non-computable functions

2. 2 Efficiency A computer program should be totally correct, but it should also execute as quickly as possible (time-efficiency) use memory wisely (storage-efficiency) How do we compare programs (or algorithms in general) with respect to execution time? various computers run at different speeds due to different processors compilers optimize code before execution the same algorithm can be written differently depending on the programming language used

3. 3 Order of Complexity For very large n, we express the number of operations as the order of complexity. Order of complexity for worst-case behavior is often expressed using Big-O notation: Number of operationsOrder of Complexity nO(n) n/2 + 6O(n) 2n + 9O(n) Usually doesn't matter what the constants are... we are only concerned about the highest power of n.

4. 4 O(n) ("Linear") n (amount of data) Number of Operations n n/2 + 6 2n + 9

5. 5 Order of Complexity Number of operationsOrder of Complexity n 2 O(n 2 ) 2n 2 + 7O(n 2 ) n 2 /2 + 5n + 2O(n 2 ) Usually doesn't matter what the constants are... we are only concerned about the highest power of n.

6. 6 O(n 2 ) ("Quadratic") n (amount of data) Number of Operations n 2 /2 + 5n + 2 2n 2 + 7 n2n2

7. 7 Order of Complexity Number of operationsOrder of Complexity log 2 nO(log n) log 10 nO(log n) 2(log 2 n) + 5O(log n) The logarithm base is not written in big O notation since all that matters is that the function is logarithmic.

8. 8 O(log n) ("Logarithmic") n (amount of data) Number of Operations log 2 n log 10 n 2(log 2 n) + 5

9. 9 Comparing Big O Functions n (amount of data) Number of Operations O(2 n ) O(1) O(n log n) O(log n) O(n 2 ) O(n)

10. 10 Searching & Sorting WORST CASE Order Of Complexity on N data elements Linear SearchO(N) Binary SearchO(log N) Selection SortO(N 2 ) Bubble SortO(N 2 ) Merge SortO(N log N) Quick SortO(N 2 ) Sort + Binary SearchO(N log N) + O(log N) = O(N log N)

11. 11 Comparing Algorithms Assume an algorithm processes n data values. If each operation takes 1  s to execute, how many  s will it take to run the algorithm on 100 data values if the algorithm has the following number of computations? Number of ComputationsExecution Time n100  s n log 2 n665  s n 2 10,000  s n 3 1,000,000  s = 1 sec 2 n > 10 30  s n!> 10 160  s

12. 12 Decision Problems A specific set of computations are classified as decision problems. An algorithm describes a decision problem if its output is simply YES or NO, depending on whether a certain property holds for its input. Example: Given a set of n shapes, can these shapes be arranged into a rectangle?

13. 13 Monkey Puzzle Problem Given: A set of n square cards whose sides are imprinted with the upper and lower halves of colored monkeys. n is a square number, such that n = m 2. Cards cannot be rotated. Problem: Determine if an arrangement of the n cards in an m X m grid exists such that each adjacent pair of cards display the upper and lower half of a monkey of the same color. Source: www.dwheeler.com (2002)

14. 14 Example Images from: Simonas Šaltenis, Aalborg University, simas@cs.auc.dk 1 23 456 7 89

15. 15 Analysis Simple algorithm: Pick one card for each cell of m X m grid. Verify if each pair of touching edges make a full monkey of the same color. If not, try another arrangement until a solution is found or all possible arrangements are checked. Answer "YES" if a solution is found. Otherwise, answer "NO" if all arrangements are analyzed and no solution is found.

16. 16 Analysis If there are n = 9 cards (m = 3): To fill the first cell, we have 9 card choices. To fill the second cell, we have 8 card choices left. To fill the third cell, we have 7 card choices remaining. etc. The total number of unique arrangements for n = 9 cards is: 9 * 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 = 362,880

17. 17 Analysis For n cards, the number of arrangements to examine is n! (n factorial) If we can analyze one arrangement in a microsecond: nTime to analyze all arrangements 9362,880  s = 0.36288 s 1620,922,789,888,000  s ≈ 242 days 2515,511,210,043,330,985,984,000,000  s ≈ 491,520,585,955 years

18. 18 Map Coloring Given a map of n territories, can the map be colored using k colors such that no two adjacent territories are colored with the same color? k=4: Answer is always yes. k=2: Only if the map contains no point that is the junction of an odd number of territories.

19. 19 Map Coloring

20. 20 Map Coloring Given a map of 48 territories, can the map be colored using 3 colors such that no two adjacent territories are colored with the same color? Pick a color for California (3 choices) Pick a color for Nevada (3 choices)... There are 3 48 = 79,766,443,076,872,509,863,361 possible colorings. No one has come up with a better algorithmic solution that works in general for any map, so far.

21. 21 Classifications Algorithms that are O(n k ) for some fixed k are polynomial-time algorithms. O(1), O(log n), O(n), O(n log n), O(n 2 ) reasonable, tractable All other algorithms are super-polynomial-time algorithms. O(2 n ), O(n!), O(n n ) unreasonable, intractable

22. 22 Traveling Salesperson Given: a weighted graph of nodes representing cities and edges representing flight paths (weights represent cost) Is there a route that takes the salesperson through every city and back to the starting city with cost no more than k? The salesperson can visit a city only once (except for the start and end of the trip).

23. 23 Traveling Salesperson A B D C G E F 12 6 4 5 9 8 10 7 11 3 7 7 Is there a route with cost at most 52?YES (Route above costs 50.) Is there a route with cost at most 48?YES? NO?

24. 24 Traveling Salesperson If there are n cities, what is the maximum number of routes that we might need to compute? Worst-case: There is a flight available between every pair of cities. Compute cost of every possible route. Pick a starting city Pick the next city (n-1 choices remaining) Pick the next city (n-2 choices remaining)... Maximum number of routes: (n-1)! = O(n!) how to build a route

25. 25 P and NP The class P consists of all those decision problems that can be solved on a deterministic sequential machine (e.g. a computer) in an amount of time that is polynomial with respect to the size of the input The class NP consists of all those decision problems whose positive solutions can be verified in polynomial time given the right information. from Wikipedia

26. 26 NP Complete The class NPC consists of all those problems in NP that are least likely to be in P. Each of these problems is called NP Complete. Monkey puzzle, Traveling salesperson, Hamiltonian path, map coloring, satisfiability are all in NPC. Every problem in NPC can be transformed to another problem in NPC. If there were some way to solve one of these problems in polynomial time, we should be able to solve all of these problems in polynomial time.

27. 27 Complexity Classes NP Problems P Problems NP Complete Problems But does P = NP? If P ≠ NP, then all decision problems can be broken down into this classification scheme. If P = NP, then all three classes are one and the same. The Clay Mathematics Institute is offering a $1M prize for the first person to prove P = NP or P ≠ NP. ( http://www.claymath.org/millennium/P_vs_NP/) We know that P < NP, since any problem that can be solved in polynomial time can certainly have a solution verified in polynomial time.

28. 28

29. 29 It gets worse... Tractable Problems Problems that have reasonable, polynomial- time solutions Intractable Problems Problems that have no reasonable, polynomial- time solutions Noncomputable Problems Problems that have no algorithms at all to solve them

30. 30 Noncomputability and Undecidability An algorithmic problem that has no algorithm is called noncomputable. If the noncomputable algorithm requires a yes/no answer, the problem is called undecidable. Example: Given any set of any number of different tile designs (examples shown above), with an infinite number of each type of tile, can we tile any area with these tiles so that like colored edges touch? This problem is undecidable!

31. 31 Tiling Problem YES Note the periodicity in the tiling.

32. 32 Tiling Problem NO For this 3 X 3 room, if we try all 3 9 tiling configurations, no tiling works.

33. 33 Tiling Problem Possible algorithm: If we find a repeating pattern, report YES. If we find a floor we cannot tile, report NO. BUT: there are some tilings which have no repeating pattern!

34. 34 Tiling Problem The only way to know if this set of tiles can tile every finite-sized floor is to evaluate every possible floor. BUT: there are an infinite number of finite-sized floors! So we could never answer YES in this case. This problem is undecidable.

35. 35 Another Undecidable Problem: The Barber Paradox Suppose there is a town with one male barber; and that every man in the town keeps himself clean-shaven: some shave themselves and some are shaved by the barber. Only the barber can shave another man. The barber shaves all and only those men who do not shave themselves. Does the barber shave himself?

36. 36 Program Termination Can we determine if a program will terminate given a valid input? Example: 1. Input x 2. While x is not equal to 1, do the following: (a) Subtract 2 from x. Does this algorithm terminate when x = 15105? Does this algorithm terminate when x = 2008?

37. 37 Program Termination Another Example: 1. Input x 2. While x is not equal to 1, do the following: (a) If x is even, divide x by 2. (b) Otherwise, Set x to 3x + 1. Does this algorithm terminate for x = 15? Does this algorithm terminate for x = 105? Does this algorithm terminate for any positive x?

38. 38 The Halting Problem Can we write a general program Q that takes as its input any program P and an input I and determines if program P will terminate (halt) when run with input I? It will answer YES if P terminates successfully on input I. It will answer NO if P never terminates on input I. This computational problem is undecidable! No such general program Q can exist! It doesn’t matter how powerful the computer is. It doesn’t matter how much time we devote to the computation. The proof of this involves contradiction.

39. 39 start end Does this algorithm end? yes no Contradiction

40. 40 Contradiction isn't just for computer scientists...