Invitation to Computer Science 6th Edition Chapter 3 The Efficiency of Algorithms

Invitation to Computer Science, 6th Edition Objectives In this chapter, you will learn about: Attributes of algorithms Measuring efficiency Analysis of algorithms When things get out of hand 2

Invitation to Computer Science, 6th Edition Introduction An important part of computer science –Finding algorithms to solve problems of interest Are some algorithms better than others? 3

Invitation to Computer Science, 6th Edition Attributes of Algorithms We expect correctness from our algorithms Program maintenance –Fixes errors uncovered through repeated use –Extends program to meet new requirements Ease of understanding –Desirable characteristic of an algorithm Elegance –Algorithmic equivalent of style 4

Invitation to Computer Science, 6th Edition Attributes of Algorithms (continued) Efficiency –Term used to describe an algorithm’s careful use of resources Benchmarks –Useful for rating one machine against another and for rating how sensitive a particular algorithm is with respect to variations in input on one particular machine 5

Invitation to Computer Science, 6th Edition Measuring Efficiency Analysis of algorithms –The study of the efficiency of algorithms –An important part of computer science Efficiency measured as a function relating size of input to time or space used For one input size, best case, worst case, and average case behavior must be considered The  notation captures the order of magnitude of the efficiency function 6

Invitation to Computer Science, 6th Edition7

8

Sequential Search Search for NAME among a list of n names Start at the beginning and compare NAME to each entry until a match is found 9

Invitation to Computer Science, 6th Edition Figure 3.1 Sequential Search Algorithm 10

Invitation to Computer Science, 6th Edition11

Invitation to Computer Science, 6th Edition12

Invitation to Computer Science, 6th Edition Figure 3.2 Number of Comparisons to Find NAME in a List of n Names Using Sequential Search 13

Invitation to Computer Science, 6th Edition14

Invitation to Computer Science, 6th Edition Order of Magnitude - Order n Order of magnitude n –Anything that varies as a constant times n (and whose graph follows the basic shape of n) Sequential search –An Θ(n) algorithm in both the worst case and the average case 15

Invitation to Computer Science, 6th Edition16

Invitation to Computer Science, 6th Edition17

Invitation to Computer Science, 6th Edition Figure 3.3 Work = 2n 18

Invitation to Computer Science, 6th Edition Figure 3.4 Work = cn for Various Values of c 19

Invitation to Computer Science, 6th Edition Figure 3.5 Growth of Work = cn for Various Values of c 20

Invitation to Computer Science, 6th Edition21

Invitation to Computer Science, 6th Edition Selection Sort Selection sort algorithm –Sorts in ascending order Subtask within selection sort –Task of finding the largest number in a list When selection sort algorithm begins –The largest-so-far value must be compared to all the other numbers in the list –If there are n numbers in the list, n – 1 comparisons must be done 22

Invitation to Computer Science, 6th Edition Figure 3.6 Selection Sort Algorithm 23

Invitation to Computer Science, 6th Edition Figure 3.7 Comparisons Required by Selection Sort 24

Invitation to Computer Science, 6th Edition Selection Sort (continued) Selection sort algorithm –Does n exchanges, one for each position in the list to put the correct value in that position Space efficiency of the selection sort –Original list occupies n memory locations –Storage is needed for: The marker between the unsorted and sorted sections Keeping track of the largest-so-far value and its location in the list 25

Invitation to Computer Science, 6th Edition Figure 3.8 An Attempt to Exchange the Values at X and Y 26

Invitation to Computer Science, 6th Edition Figure 3.9 Exchanging the Values at X and Y 27

Invitation to Computer Science, 6th Edition28

Invitation to Computer Science, 6th Edition29 Selection Sort (continued) Count comparisons of largest so far against other values Find Largest, given m values, does m-1 comparisons Selection sort calls Find Largest n times,  Each time with a smaller list of values  Cost = n-1 + (n-2) + … = n(n-1)/2

Invitation to Computer Science, 6th Edition30 Selection Sort (continued) Time efficiency  Comparisons: n(n-1)/2  Exchanges: n (swapping largest into place)  Overall:  (n 2 ), best and worst cases Space efficiency  Space for the input sequence, plus a constant number of local variables

Invitation to Computer Science, 6th Edition31 Order of Magnitude – Order n 2 All functions with highest-order term cn 2 have similar shape An algorithm that does cn 2 work for any constant c is order of magnitude n 2, or  (n 2 )

Invitation to Computer Science, 6th Edition32 Order of Magnitude – Order n 2 Anything that is  (n 2 ) will eventually have larger values than anything that is  (n), no matter what the constants are An algorithm that runs in time  (n) will outperform one that runs in  (n 2 )

Invitation to Computer Science, 6th Edition Order of Magnitude - Order n 2 (continued) Order of magnitude n 2, or Θ(n 2 ) –An algorithm that does cn 2 work for any constant c Selection sort –An Θ(n 2 ) algorithm (in all cases) Sequential search –An Θ(n) algorithm (in the worst case) 33

Invitation to Computer Science, 6th Edition Figure 3.10 Work 5 cn 2 for Various Values of c 34

Invitation to Computer Science, 6th Edition Figure 3.11 A Comparison of n and n 2 35

Invitation to Computer Science, 6th Edition Figure 3.12 For Large Enough n, 0.25n 2 Has Larger Values Than 10n 36

Invitation to Computer Science, 6th Edition Order of Magnitude - Order n 2 (continued) Part of the job of program documentation –To make clear any assumptions or restrictions about the input size the program was designed to handle Comparing algorithm efficiency –Only makes sense if there is a choice of algorithms for the task at hand 37

Invitation to Computer Science, 6th Edition Figure 3.13 A Comparison of Two Extreme Q(n 2 ) and Q(n) Algorithms 38

Invitation to Computer Science, 6th Edition39 Analysis of Algorithms Multiple algorithms for one task may be compared for efficiency and other desirable attributes Data cleanup problem Search problem Pattern matching

Invitation to Computer Science, 6th Edition Analysis of Algorithms Given a collection of numbers, find and remove all zeros Possible algorithms  Shuffle-left  Copy-over  Converging-pointers 40

Invitation to Computer Science, 6th Edition The Shuffle-Left Algorithm Scans list from left to right When a zero is found, copy each remaining data item in the list one cell to the left Value of legit –Originally set to the length of the list –Is reduced by 1 every time a 0 is encountered 41

Invitation to Computer Science, 6th Edition Figure 3.14 The Shuffle-Left Algorithm for Data Cleanup 42

Invitation to Computer Science, 6th Edition The Shuffle-Left Algorithm (continued) To analyze time efficiency: –Identify the fundamental units of work the algorithm performs Examinations Copying numbers –Best case occurs when the list has no 0 values because no copying is required –Worst case occurs when the list has all 0 values 43

Invitation to Computer Science, 6th Edition44 The Shuffle-Left Algorithm (continued) Time efficiency  Count examinations of list values and copies  Best case No copies, n examinations  (n)  Worst case Shift at each pass, n passes n 2 copies plus n examinations  (n 2 )

Invitation to Computer Science, 6th Edition The Shuffle-Left Algorithm (continued) Space-efficient –n slots for n values, plus a few local variables –Only requires four memory locations to store the quantities n, legit, left, and right 45

Invitation to Computer Science, 6th Edition The Copy-Over Algorithm Scans the list from left to right, copying every legitimate (non-zero) value into a new list that it creates Every list entry is examined to see whether it is 0 Every non-zero list entry is copied once 46

Invitation to Computer Science, 6th Edition Figure 3.15 The Copy-Over Algorithm for Data Cleanup 47

Invitation to Computer Science, 6th Edition48 The Copy-Over Algorithm Use a second list  Copy over each nonzero element in turn Time efficiency  Count examinations and copies  Best case All zeros n examinations and 0 copies  (n)

Invitation to Computer Science, 6th Edition49 The Copy-Over Algorithm (continued) Time efficiency (continued)  Worst case No zeros n examinations and n copies  (n) Space efficiency  2n slots for n values, plus a few extraneous variables

Invitation to Computer Science, 6th Edition50 The Copy-Over Algorithm (continued) Time/space tradeoff  Algorithms that solve the same problem offer a tradeoff One algorithm uses more time and less memory Its alternative uses less time and more memory

Invitation to Computer Science, 6th Edition The Converging-Pointers Algorithm Swap zero values from left with values from right until pointers converge in the middle 51

Invitation to Computer Science, 6th Edition Figure 3.16 The Converging-Pointers Algorithm for Data Cleanup 52

Invitation to Computer Science, 6th Edition53 The Converging-Pointers Algorithm Time efficiency  Count examinations and swaps  Best case n examinations, no swaps  (n)

Invitation to Computer Science, 6th Edition54 The Converging-Pointers Algorithm (continued) Time efficiency (continued)  Worst case n examinations, n swaps  (n) Space efficiency  n slots for the values, plus a few extra variables

Invitation to Computer Science, 6th Edition Figure 3.17 Analysis of Three Data Cleanup Algorithms 55

Invitation to Computer Science, 6th Edition The Converging-Pointers Algorithm (continued) In an Θ(n) algorithm: –The work is proportional to n In an Θ(n 2 ) algorithm: –The work is proportional to the square of n 56

Invitation to Computer Science, 6th Edition57

Invitation to Computer Science, 6th Edition58 Binary Search Algorithm Given ordered data  Search for NAME by comparing to middle element  If not a match, restrict search to either lower or upper half only  Each pass eliminates half the data

Invitation to Computer Science, 6th Edition Binary Search Procedure –First looks for NAME at roughly the halfway point in list –If name equals NAME, search is over –If NAME comes alphabetically before name at halfway point, search is narrowed to the front half of list –If NAME comes alphabetically after name at halfway point, search is narrowed to the back half of the list –Algorithm halts when NAME is found 59

Invitation to Computer Science, 6th Edition Figure 3.18 Binary Search Algorithm (list must be sorted) 60

Invitation to Computer Science, 6th Edition Figure 3.19 Binary Search Tree for a 7-Element List 61

Invitation to Computer Science, 6th Edition62 Binary Search Algorithm (continued) Time Efficiency  Best case 1 comparison  (1)  Worst case lg n comparisons  lg n: The number of times n can be divided by two before reaching 1  (lg n)

Invitation to Computer Science, 6th Edition Binary Search (continued) Logarithm of n to the base 2 –Number of times a number n can be cut in half and not go below 1 As n doubles: –lg n increases by only 1, so lg n grows much more slowly than n Worst case and average case –Θ(lg n) comparisons Works only on a list that has already been sorted 63

Invitation to Computer Science, 6th Edition Figure 3.20 Values for n and lg n 64

Invitation to Computer Science, 6th Edition Figure 3.21 A Comparison of n and lg n 65

Invitation to Computer Science, 6th Edition66 Binary Search Algorithm (continued) Tradeoff  Sequential search Slower, but works on unordered data  Binary search Faster (much faster), but data must be sorted first

Invitation to Computer Science, 6th Edition67

Invitation to Computer Science, 6th Edition Figure 2.16 Final Draft of the Pattern-Matching Algorithm 68

Invitation to Computer Science, 6th Edition69 Pattern-Matching Algorithm Analysis involves two measures of input size  m: length of pattern string  n: length of text string Unit of work  Comparison of a pattern character with a text character

Invitation to Computer Science, 6th Edition Pattern-Matching Usually involves a pattern length that is short compared to the text length –That is, when m is much less than n Time Efficiency  Best case Pattern does not match at all n - m + 1 comparisons  (n) 70

Invitation to Computer Science, 6th Edition Pattern-Matching Time Efficiency  Worst case Pattern almost matches at each point (m)(n - m + 1) comparisons  (m x n) 71

Text: AAAAAAAAA Pattern: AAAB Invitation to Computer Science, 6th Edition72

Invitation to Computer Science, 6th Edition Figure 3.22 Order-of-Magnitude Time Efficiency Summary 73

Invitation to Computer Science, 6th Edition74 When Things Get Out of Hand Polynomially bound algorithms  Work done is no worse than a constant multiple of n 2 Intractable algorithms  Run in worse than polynomial time  Examples Hamiltonian circuit Bin-packing

Invitation to Computer Science, 6th Edition When Things Get Out of Hand Graph –A collection of nodes and connecting edges Hamiltonian circuit –A path through a graph that begins and ends at the same node and goes through all other nodes exactly once 75

Invitation to Computer Science, 6th Edition Figure 3.23 Four Connected Cities 76

Invitation to Computer Science, 6th Edition Figure 3.24 Hamiltonian Circuits among All Paths from A in Figure 3.23 with Four Links 77

Invitation to Computer Science, 6th Edition When Things Get Out of Hand (continued) Exponential algorithm –An Θ(2 n ) algorithm Brute force algorithm –One that beats the problem into submission by trying all possibilities Intractable problem –No polynomially bounded algorithm exists Approximation algorithms –Do not solve the problem, but provide a close approximation to a solution 78

Invitation to Computer Science, 6th Edition Figure 3.27 A Comparison of Four Orders of Magnitude 79

Invitation to Computer Science, 6th Edition Figure 3.25 Comparisons of lg n, n, n 2, and 2 n 80

Invitation to Computer Science, 6th Edition Figure 3.26 Comparisons of lg n, n, n 2, and 2 n for Larger Values of n 81

Invitation to Computer Science, 6th Edition82

Invitation to Computer Science, 6th Edition83

Invitation to Computer Science, 6th Edition Summary of Level 1 Level 1 (Chapters 2 and 3) explored algorithms –Chapter 2 Pseudocode Sequential, conditional, and iterative operations Algorithmic solutions to various practical problems –Chapter 3 Desirable properties for algorithms Time and space efficiencies of a number of algorithms 84

Invitation to Computer Science, 6th Edition Summary Desirable attributes in algorithms –Correctness –Ease of understanding (clarity) –Elegance –Efficiency Efficiency –An algorithm’s careful use of resources is extremely important 85