1 Algorithms CS/APMA 202 Rosen section 2.1 Aaron Bloomfield

2 What is an algorithm? An algorithm is “a finite set of precise instructions for performing a computation or for solving a problem” A program is one type of algorithm A program is one type of algorithm All programs are algorithms Not all algorithms are programs! Directions to somebody’s house is an algorithm Directions to somebody’s house is an algorithm A recipe for cooking a cake is an algorithm A recipe for cooking a cake is an algorithm The steps to compute the cosine of 90 ° is an algorithm The steps to compute the cosine of 90 ° is an algorithm

3 Some algorithms are harder than others Some algorithms are easy Finding the largest (or smallest) value in a list Finding the largest (or smallest) value in a list Finding a specific value in a list Finding a specific value in a list Some algorithms are a bit harder Sorting a list Sorting a list Some algorithms are very hard Finding the shortest path between Miami and Seattle Finding the shortest path between Miami and Seattle Some algorithms are essentially impossible Factoring large composite numbers Factoring large composite numbers In section 2.2, we’ll see how to rate how “hard” algorithms are

4 Algorithm 1: Maximum element Given a list, how do we find the maximum element in the list? To express the algorithm, we’ll use pseudocode Pseudocode is kinda like a programming language, but not really Pseudocode is kinda like a programming language, but not really

5 Algorithm 1: Maximum element Algorithm for finding the maximum element in a list: procedure max (a 1, a 2, …, a n : integers) max := a 1 for i := 2 to n if max < a i then max := a i {max is the largest element}

6 procedure max (a 1, a 2, …, a n : integers) max := a 1 for i := 2 to n if max < a i then max := a i max := a 1 for i := 2 to n if max < a i then max := a i Algorithm 1: Maximum element a1a1a1a1 a2a2a2a2 a3a3a3a3 a4a4a4a4 a5a5a5a5 a6a6a6a6 a7a7a7a7 a8a8a8a8 a9a9a9a9 a 10 max i

7 Maximum element running time How long does this take? If the list has n elements, worst case scenario is that it takes n “steps” Here, a step is considered a single step through the list Here, a step is considered a single step through the list

8 Properties of algorithms Algorithms generally share a set of properties: Input: what the algorithm takes in as input Input: what the algorithm takes in as input Output: what the algorithm produces as output Output: what the algorithm produces as output Definiteness: the steps are defined precisely Definiteness: the steps are defined precisely Correctness: should produce the correct output Correctness: should produce the correct output Finiteness: the steps required should be finite Finiteness: the steps required should be finite Effectiveness: each step must be able to be performed in a finite amount of time Effectiveness: each step must be able to be performed in a finite amount of time Generality: the algorithm should be applicable to all problems of a similar form Generality: the algorithm should be applicable to all problems of a similar form

9 Searching algorithms Given a list, find a specific element in the list We will see two types Linear search Linear search a.k.a. sequential search Binary search Binary search

10 Algorithm 2: Linear search Given a list, find a specific element in the list List does NOT have to be sorted! List does NOT have to be sorted! procedure linear_search (x: integer; a 1, a 2, …, a n : integers) i := 1 while ( i ≤ n and x ≠ a i ) i := i + 1 if i ≤ n then location := i else location := 0 {location is the subscript of the term that equals x, or it is 0 if x is not found}

11 procedure linear_search (x: integer; a 1, a 2, …, a n : integers) i := 1 while ( i ≤ n and x ≠ a i ) i := i + 1 if i ≤ n then location := i else location := 0 i := 1 while ( i ≤ n and x ≠ a i ) i := i + 1 if i ≤ n then location := i else location := 0 Algorithm 2: Linear search, take a1a1a1a1 a2a2a2a2 a3a3a3a3 a4a4a4a4 a5a5a5a5 a6a6a6a6 a7a7a7a7 a8a8a8a8 a9a9a9a9 a 10 i x3 location8

12 procedure linear_search (x: integer; a 1, a 2, …, a n : integers) i := 1 while ( i ≤ n and x ≠ a i ) i := i + 1 if i ≤ n then location := i else location := 0 i := 1 while ( i ≤ n and x ≠ a i ) i := i + 1 if i ≤ n then location := i else location := 0 Algorithm 2: Linear search, take a1a1a1a1 a2a2a2a2 a3a3a3a3 a4a4a4a4 a5a5a5a5 a6a6a6a6 a7a7a7a7 a8a8a8a8 a9a9a9a9 a 10 i x11 location0

13 Linear search running time How long does this take? If the list has n elements, worst case scenario is that it takes n “steps” Here, a step is considered a single step through the list Here, a step is considered a single step through the list

14 Algorithm 3: Binary search Given a list, find a specific element in the list List MUST be sorted! List MUST be sorted! Each time it iterates through, it cuts the list in half procedure binary_search (x: integer; a 1, a 2, …, a n : increasing integers) i := 1{ i is left endpoint of search interval } j := n{ j is right endpoint of search interval } while i < j begin m :=  (i+j)/2  { m is the point in the middle } if x > a m then i := m+1 else j := m end if x = a i then location := i else location := 0 {location is the subscript of the term that equals x, or it is 0 if x is not found}

15 Algorithm 3: Binary search, take a1a1a1a1 a2a2a2a2 a3a3a3a3 a4a4a4a4 a5a5a5a5 a6a6a6a6 a7a7a7a7 a8a8a8a8 a9a9a9a9 a 10 ijm i := 1 j := n procedure binary_search (x: integer; a 1, a 2, …, a n : increasing integers) while i < j begin m :=  (i+j)/2  if x > a m then i := m+1 else j := m end if x = a i then location := i else location := 0 i := 1 j := n while i < j begin m :=  (i+j)/2  if x > a m then i := m+1 else j := m end if x = a i then location := i 1x location7

16 Algorithm 3: Binary search, take a1a1a1a1 a2a2a2a2 a3a3a3a3 a4a4a4a4 a5a5a5a5 a6a6a6a6 a7a7a7a7 a8a8a8a8 a9a9a9a9 a 10 ijm i := 1 j := n procedure binary_search (x: integer; a 1, a 2, …, a n : increasing integers) while i < j begin m :=  (i+j)/2  if x > a m then i := m+1 else j := m end if x = a i then location := i else location := 0 i := 1 j := n while i < j begin m :=  (i+j)/2  if x > a m then i := m+1 else j := m end if x = a i then location := I else location := 0 1x location0 8

17 Algorithm 3: Binary search A somewhat alternative view of what a binary search does…

18 Quick survey I felt I understood the search algorithms… I felt I understood the search algorithms… a) Very well b) With some review, I’ll be good c) Not really d) Not at all

19 Today’s demotivators

20 Binary search running time How long does this take (worst case)? If the list has 8 elements It takes 3 steps It takes 3 steps If the list has 16 elements It takes 4 steps It takes 4 steps If the list has 64 elements It takes 6 steps It takes 6 steps If the list has n elements It takes log 2 n steps It takes log 2 n steps

21 Sorting algorithms Given a list, put it into some order Numerical, lexicographic, etc. Numerical, lexicographic, etc. We will see two types Bubble sort Bubble sort Insertion sort Insertion sort

22 Algorithm 4: Bubble sort One of the most simple sorting algorithms Also one of the least efficient Also one of the least efficient It takes successive elements and “bubbles” them up the list procedure bubble_sort (a 1, a 2, …, a n ) for i := 1 to n-1 for j := 1 to n-i if a j > a j +1 then interchange a j and a j +1 { a 1, …, a n are in increasing order }

23 Algorithm 4: Bubble sort procedure bubble_sort (a 1, a 2, …, a n ) for i := 1 to n-1 for j := 1 to n-i if a j > a j +1 then interchange a j and a j +1 a1a1a1a1 a2a2a2a2 a3a3a3a3 a4a4a4a4 a5a5a5a5 a6a6a6a6 a7a7a7a7 a8a8a8a8 a9a9a9a9 a 10 ij for i := 1 to n-1 for j := 1 to n-i if a j > a j +1 then interchange a j and a j

24 Algorithm 4: Bubble sort An example using physical objects…

25 Bubble sort running time Bubble sort algorithm: for i := 1 to n-1 for j := 1 to n-i if a j > a j +1 then interchange a j and a j +1 Outer for loop does n-1 iterations Inner for loop does n-1 iterations the first time n-1 iterations the first time n-2 iterations the second time n-2 iterations the second time … 1 iteration the last time 1 iteration the last time Total: (n-1) + (n-2) + (n-3) + … = (n 2 -n)/2 We can say that’s “about” n 2 time We can say that’s “about” n 2 time

26 Algorithm 5: Insertion sort Another simple (and inefficient) algorithm It starts with a list with one element, and inserts new elements into their proper place in the sorted part of the list procedure insertion_sort (a 1, a 2, …, a n ) for j := 2 to n begin i := 1 while a j > a i i := i +1 m := a j for k := 0 to j-i-1 a j-k := a j-k-1 a i := m end { a 1, a 2, …, a n are sorted } take successive elements in the list find where that element should be in the sorted portion of the list move all elements in the sorted portion of the list that are greater than the current element up by one put the current element into it’s proper place in the sorted portion of the list

27 Insertion sort running time for j := 2 to n begin i := 1 while a j > a i i := i +1 m := a j for k := 0 to j-i-1 a j-k := a j-k-1 a i := m end { a 1, a 2, …, a n are sorted } Outer for loop runs n-1 times In the inner for loop: Worst case is when the while keeps i at 1, and the for loop runs lots of times Worst case is when the while keeps i at 1, and the for loop runs lots of times If i is 1, the inner for loop runs 1 time (k goes from 0 to 0) on the first iteration, 1 time on the second, up to n-2 times on the last iteration If i is 1, the inner for loop runs 1 time (k goes from 0 to 0) on the first iteration, 1 time on the second, up to n-2 times on the last iteration Total is … + n-2 = (n-1)(n-2)/2 We can say that’s “about” n 2 time We can say that’s “about” n 2 time

28 Quick survey I felt I understood the sort algorithms… I felt I understood the sort algorithms… a) Very well b) With some review, I’ll be good c) Not really d) Not at all

29 Comparison of running times Searches Linear: n steps Linear: n steps Binary: log 2 n steps Binary: log 2 n steps Binary search is about as fast as you can get Binary search is about as fast as you can getSorts Bubble: n 2 steps Bubble: n 2 steps Insertion: n 2 steps Insertion: n 2 steps There are other, more efficient, sorting techniques There are other, more efficient, sorting techniques In principle, the fastest are heap sort, quick sort, and merge sort These each take take n * log 2 n steps In practice, quick sort is the fastest, followed by merge sort

30 Greedy algorithms Ignore this part…

31 Quick survey I felt I understood the material in this slide set… I felt I understood the material in this slide set… a) Very well b) With some review, I’ll be good c) Not really d) Not at all

32 Quick survey The pace of the lecture for this slide set was… The pace of the lecture for this slide set was… a) Fast b) About right c) A little slow d) Too slow

33 Quick survey How interesting was the material in this slide set? Be honest! How interesting was the material in this slide set? Be honest! a) Wow! That was SOOOOOO cool! b) Somewhat interesting c) Rather borting d) Zzzzzzzzzzz