CS4HS at Marquette University

a) Find the sum of 4 and 7 b) Sort a list of words in alphabetical order c) State the most beautiful phrase in the English language d) Name the smallest even number greater than 2 that cannot be written as the sum of two primes

 A problem is a relationship between input data and output data  For every set of input values, there is only one correct set of output  A specific set of input and output is called an instance  Adding 4 and 7 is an instance of the general problem of adding integers

 An algorithm is a solution to a problem  It gives a finite series of steps that you can follow from any legal input data to reach the correct output data  Can you give an example?

 How do you measure how long an algorithm takes?  We can describe the number of steps the algorithm takes as a function of the size of input data (usually abbreviated as n)  How does the number of steps needed to solve an instance grow as the input grows?

 Let's say an algorithm takes 34n 3 – 5n n steps for an input of size n  Ugh.  Since we care about growth when n gets big, the biggest term is what matters  Since different computers are better at different operations, we ignore constant coefficients  We say 34n 3 – 5n n is O(n 3 )  Cubic growth is what matters

 How long does it take to do addition by hand?  Let’s assume the numbers have n digits  n additions + n carries  Running time: O(n)

 How long does it take to do multiplication by hand? 123 x _492__  Let’s assume the numbers have n digits  (n multiplications + n carries) x n digits + (n + 1 digits) x n additions  Running time: O(n 2 )

 I've got a list of n numbers  You want to look at my list to see if your favorite number (42) is on it  How many numbers do you have to look at to see if my list has your number?  How long does that take in Big-Oh notation?

 Is there any way to go smaller than O(n)?  What if the numbers were processed in some way beforehand?  What if you knew that the numbers were sorted in ascending order?

 Repeatedly divide the search space in half  We play a high-low game  Again, let's say we're looking for Check the middle (Too high) Check the middle (Too low) Check the middle (Too low) Check the middle (Found it!) 42

 This is a classic interview question asked by Microsoft, Amazon, and similar companies  Imagine that you have 9 red balls  One of them is just slightly heavier than the others, but so slightly that you can’t feel it  You have a very accurate two pan balance you can use to compare balls  Find the heaviest ball in the smallest number of weighings

 It’s got to be 8 or fewer  We could easily test one ball against every other ball  There must be some cleverer way to divide them up  Something that is related somehow to binary search

 We can divide the balls in half each time  If those all balance, it must be the one we left out to begin with

 How?  They key is that you can actually cut the number of balls into three parts each time  We weigh 3 against 3, if they balance, then we know the 3 left out have the heavy ball  When it’s down to 3, weigh 1 against 1, again knowing that it’s the one left out that’s heavy if they balance

 The cool thing is…  Yes, this is “cool” in the CS sense, not in the real sense  Anyway, the cool thing is that we are trisecting the search space each time  This means that it takes log 3 n weighings to find the heaviest ball  We could do 27 balls in 3 weighings, 81 balls in 4 weighings, etc.

 CS people have written hundreds of papers just about how to sort things better  Bubble sort is a very simple sorting algorithm  It is not the fastest algorithm  Make a number of passes  In each pass, swap each contiguous pair of items if they're out of order  Keep making passes until no swaps are necessary

 Run through the whole list, swapping any entries that are out of order No swap Swap 45 0 No swap Swap No swap Swap

 Everyone stand up  We're going to use the bubble sort algorithm to sort us all by birth month and day (you can keep the year to yourself)  In each pass, we sequentially swap pairs that are out of order  We keep making passes until no one needs swapping  We are guaranteed that the last person is put in the right place after each pass

 How bad could it be?  What if the array was in reverse- sorted order?  One pass would only move the largest number to the bottom  We would need n – 1 passes to sort the whole array  What's the total time for bubble sort?

 How does Google Maps find the shortest route from Philadelphia to Milwaukee?  It simplifies all the streets into a graph  A graph has nodes and edges  Nodes represent locations  Edges are (parts of) streets

 We can write a number next to each edge which gives its weight  Weight can represent time, distance, cost: anything, really  The shortest path (lowest total weight) is not always obvious A A B B D D C C E E 2 8 6

A B D C E D E C B A  Take a moment and try to find the shortest path from A to E.  The shortest path has length

 On a graph of that size, it isn’t hard to find the shortest path  A Google Maps graph has millions and millions of nodes  How can we come up with an algorithm that will always find the shortest path from one node to another?  Dijkstra's algorithm adds the closest node we can find, expanding our set of stuff we know the best way to get to

NotationMeaning sStarting node d(v)d(v) The best distance from s to v found so far d(u, v) The direct distance between nodes u and v S A set which contains the nodes for which we know the shortest path from s V A set which contains the nodes for which we do not yet know the shortest path from s

1. Start with two sets, S and V:  S has the starting node in it  V has everything else 2. Set the distance to all nodes in V to ∞ 3. Find the node u in V that is closest to a node in S 4. For every neighbor v of u in V  If d(v) > d(u) + d(u,v)  Set d(v) = d(u) + d(u,v) 5. Move u from V to S 6. If V is not empty, go back to Step 2

Sets SV AB, C, D, E Sets SV A, BC, D, E Sets SV A, B, CD, E Sets SV A, B, C, DE Sets SV A, B, C, D, E Noded(u)d(u) A0 B5 C8 D∞ E A A B B D D C C E E Noded(u)d(u) A0 B5 C7 D11 E16 Noded(u)d(u) A0 B5 C7 D10 E16 Noded(u)d(u) A0 B5 C7 D10 E16 Noded(u)d(u) A0 B5 C7 D10 E14 Finding the shortest distance from A to all other nodes

A A B B D D C C E E A B C D E  The traveling salesman problem (TSP) also wants to find the shortest path, but it adds two requirements  You have to return to the point where you start  You have to visit all the cities exactly once  Like a UPS guy who wants to make all his dropoffs and return to the shipping center

 TSP seems easy!  There must be some way we can adapt Dijkstra's algorithm to work  Suggestions?  The greedy approach always adds the nearest neighbor  A brute force approach tries all possible tours and sees which is shortest

 We are tempted to always take the closest neighbor, but there are pitfalls GreedyOptimal

 In a completely connected graph, we can try any sequence of nodes  If there are n nodes, there are (n – 1)! tours  For 30 locations, 29! =  If we can check 1,000,000,000 tours per second, it will only take about 20,000 times the age of the universe to check them all  We will (eventually) get the best answer!

 TSP is just one NP-complete problem  These problems are pretty useful (scheduling, optimizing resource usage, packing boxes)  No running-times for NP-complete problems are known that are better than exponential O(2 n )  All these problems can easily be converted into each other  If you could solve one efficiently, you could solve them all efficiently  The Clay Mathematics Institute has a $1 million prize for such an algorithm  You get the same prize if you could prove that no such algorithm is possible

 A Turing machine is a model of computation  It consists of a head, an infinitely long tape, a set of possible states, and an alphabet of characters that can be written on the tape  A finite list of rules says what it should write and whether it should move left or right given the current symbol and state A A

 Computational power means the ability to solve a problem  The ability to run an algorithm  Speed is not related to computational power  Turing machines have as much computational power as any known computer  Any algorithm that can be run on any computer can be run on a Turing machine

 There are some computational problems that you cannot design an algorithm for  No computer can ever solve the general problem (though sometimes you can solve instances of the problem)  Examples  It's impossible to write a program that can tell if any program will run to completion

 Douglas Hofstadter uses the metaphor of turntables  Imagine that evil people design records that will shake turntables apart when they're played  Maybe turntable A can play record A and turntable B can play record B  However, if turntable A plays record B, it will shatter

 Turing machines can perform all possible computations  It's possible to encode the way a Turing machine works such that another Turing machine can read it  It's easy to make a slight change to a Turing machine so that it gives back the opposite answer (or goes into an infinite loop)

 You've got a Turing machine M with encoding m  You want to see if M will halt on input x  Assume there is a machine H that can take encoding m and input x  H(m,x) is YES if it halts  H(m,x) is NO if it loops forever  We create (evil) machine E that takes description m and runs H(m,m)  If H(m,m) is YES, E loops forever  If H(m,m) is NO, E returns YES H H mxmx YES NO H H YES NO E m Loop forever YES

 Let's say that e is the description of E  What happens if you feed description e into E?  E(e) says what E will do with itself as input  If it returns YES, that means that E on input e loops forever  But it can't, because it just returned YES  If it loops forever, then E on input e would return YES  But it can't, because it's looping forever!  Our assumption that machine H exists must be wrong