Lecture 6 Efficiency of Algorithms (2) (S&G, ch.3) 12/4/2018 8:29 AM Lecture 6 Efficiency of Algorithms (2) (S&G, ch.3) 12/4/2018 CS 100 - Lecture 6 CS 100

Read S&G chapter 4 Warning: There are printing errors in ch. 4 in formulas! We will send out corrections by email 12/4/2018 CS 100 - Lecture 6

slide courtesy of S. Levy Lecture 6 12/4/2018 8:29 AM Sorting One of the most common activities of a computer is sorting data. Arranging data into numerical or alphabetical order for purposes of Reports by category Summarizing data Searching data 12/4/2018 CS 100 - Lecture 6 slide courtesy of S. Levy CS 100

slide courtesy of S. Levy Lecture 6 12/4/2018 8:29 AM Sorting (2) Given: n, N1, N2, …, Nn Want: Rearrangement M1, M2, …, Mn where M1 …  Mn according to the appropriate understanding of . There are many well-known algorithms for doing this. Preference may be due to list size degree of order already present ease of programming program available 12/4/2018 CS 100 - Lecture 6 slide courtesy of S. Levy CS 100

Sorting - Selection Sort Lecture 6 12/4/2018 8:29 AM Sorting - Selection Sort Strategy: find the largest element in list swap it with last element in list find next largest swap it with next to last element in list etc Variables: U - Marker for unsorted section L - Position of largest so far C - Current position 12/4/2018 CS 100 - Lecture 6 slide courtesy of S. Levy CS 100

Selection Sort - The Algorithm: High Level Lecture 6 12/4/2018 8:29 AM Selection Sort - The Algorithm: High Level Set U to n Repeat until U = 1 Search For Largest of N [1] to N [U] and put its location in L Exchange N [L] and N [U] Set U to U – 1 Stop The inner loop finds the largest in N[1] … N[U] 12/4/2018 CS 100 - Lecture 6 CS 100

Selection Sort - The Algorithm Lecture 6 12/4/2018 8:29 AM Selection Sort - The Algorithm Set U to n Repeat until U = 1 Set L to 1 Set C to 2 Repeat until C > U If N [C] > N [L] then Set L to C Set C to C + 1 Exchange N [L] and N [U] Set U to U – 1 Stop Search For Largest of N [1], …, N [U] and put location in L The inner loop finds the largest in N[1] … N[U] 12/4/2018 CS 100 - Lecture 6 slide courtesy of S. Levy CS 100

Selection Sort - Example (Pass 1) Lecture 6 Selection Sort - Example (Pass 1) 12/4/2018 8:29 AM 6 5 19 9 12 U L C 6 5 19 9 12 U L C 6 5 19 9 12 U L C 6 5 19 9 12 U L C 6 5 19 9 12 U L C 6 5 12 9 19 U L C 6 5 19 9 U L C 12 12/4/2018 CS 100 - Lecture 6 slide courtesy of S. Levy CS 100

Selection Sort - Example (Pass 2) Lecture 6 Selection Sort - Example (Pass 2) 12/4/2018 8:29 AM 6 5 12 9 U L C 19 6 12 9 19 U L C 5 6 5 9 19 U L C 12 6 5 12 19 U L C 9 6 5 12 9 19 U L C 6 12 9 U L C 19 5 12/4/2018 CS 100 - Lecture 6 slide courtesy of S. Levy CS 100

Selection Sort - Example (Pass 3) Lecture 6 Selection Sort - Example (Pass 3) 12/4/2018 8:29 AM 6 12 U L C 19 9 5 6 5 9 12 U L C 19 6 5 12 U L C 19 9 6 5 19 LU C 12 9 6 5 9 12 U L C 19 12/4/2018 CS 100 - Lecture 6 slide courtesy of S. Levy CS 100

Selection Sort - Example (Pass 4) Lecture 6 Selection Sort - Example (Pass 4) 12/4/2018 8:29 AM 6 5 12 U L C 19 9 5 6 12 U 19 9 6 5 12 U L C 19 9 5 9 12 19 LU C 6 12/4/2018 CS 100 - Lecture 6 slide courtesy of S. Levy CS 100

Efficiency Analysis of Selection Sort Lecture 6 12/4/2018 8:29 AM Efficiency Analysis of Selection Sort Makes n – 1 passes through the list Each pass uses Search For Largest, on the unsorted part of the list, to find the largest element If the unsorted part has length U, Search For Largest takes U – 1 comparisons 12/4/2018 CS 100 - Lecture 6 CS 100

Efficiency of Selection Sort (2) Lecture 6 12/4/2018 8:29 AM Efficiency of Selection Sort (2) To sort n element list: n – 1 comparisons to find largest n – 2 comparisons to find second largest … 2 comparisons to find largest of last three 1 comparisons to find largest of last two Total number of comparisons: 12/4/2018 CS 100 - Lecture 6 CS 100

A Useful Formula 12/4/2018 CS 100 - Lecture 6 Lecture 6 12/4/2018 8:29 AM A Useful Formula 12/4/2018 CS 100 - Lecture 6 CS 100

A Useful Formula (2) 12/4/2018 CS 100 - Lecture 6

Best, Worst, & Average 12/4/2018 CS 100 - Lecture 6 Lecture 6 12/4/2018 8:29 AM Best, Worst, & Average Note that for some common (and useful) algorithms, the worst case is quite bad, but the (observed) average case is quite good. 12/4/2018 CS 100 - Lecture 6 CS 100

Motivation for Asymptotic Complexity Lecture 6 12/4/2018 8:29 AM Motivation for Asymptotic Complexity We are interested in comparing the efficiency of various algorithms, independent of computer running them Therefore, we choose to ignore: the absolute time taken by the instructions constant initialization overhead time taken by the less frequently executed instructions 12/4/2018 CS 100 - Lecture 6 CS 100

Motivation for Asymptotic Complexity (2) Lecture 6 12/4/2018 8:29 AM Motivation for Asymptotic Complexity (2) Not concerned about efficiency for small problems (they’re all quick enough) Usually interested in using on large problems We investigate how resource usage varies on progressively larger problems Which is asymptotically better, i.e., for sufficiently large problems 12/4/2018 CS 100 - Lecture 6 CS 100

Quadratic vs. Linear Growth Lecture 6 12/4/2018 8:29 AM Quadratic vs. Linear Growth Variable curve is y = n x2 (n in [0, 10]) On big enough problem, quadratic algorithm takes more time than linear algorithm 12/4/2018 CS 100 - Lecture 6 CS 100

Quadratic vs. Linear Growth (2) Lecture 6 12/4/2018 8:29 AM Quadratic vs. Linear Growth (2) Note the linear algorithm has higher fixed overhead and a quicker rate of growth Quadratic behavior eventually dominates quadratic algorithm 12/4/2018 CS 100 - Lecture 6 CS 100

Equivalent Complexity For our purposes: An algorithm taking time 10n sec. is as good as one taking 2n sec. because absolute time for individual instructions doesn’t matter An algorithm taking 2n + 10 sec. is as good as one taking 2n sec. because fixed overhead doesn’t matter An algorithm taking n2 + 2n sec. is as good as one taking n2 sec. less frequently executed instructions don’t matter 12/4/2018 CS 100 - Lecture 6

Complexity Classes No growth: (1) Linear growth: (n) Lecture 6 12/4/2018 8:29 AM Complexity Classes No growth: (1) Linear growth: (n) Quadratic growth: (n2) Cubic growth: (n3) Polynomial growth: (nk) for any k Exponential growth: (kn) for any k  (theta) is called the “exact order” of a function Call Q the “exact order” of a function 12/4/2018 CS 100 - Lecture 6 CS 100

Comparison of Growth Orders Lecture 6 12/4/2018 8:29 AM Comparison of Growth Orders grey: constant yellow: logarithmic blue: linear light blue: quadratic green: cubic purple: exponential (2n) red: exponential (3n) 12/4/2018 CS 100 - Lecture 6 CS 100

Combinatorial Explosion Consider a game with k possible moves on each play Therefore: k possible first moves k2 possible replies k3 possible replies to the replies, etc. In general, must consider kn possibilities to look n moves ahead (exponential algorithm) “Brute force” solutions to many problems lead to “combinatorial explosion” 12/4/2018 CS 100 - Lecture 6

Intractable Problems For some problems, all known algorithms are exponential-time No known polynomial-time algorithms — not even (n100)! Sometimes there are efficient approximation algorithms give good, but not perfect answer may be good enough for practical purposes 12/4/2018 CS 100 - Lecture 6

Limitations of Asymptotic Complexity Note that 1,000,000 n is (n), but 0.000001 n2 is (n2) Paradox: 1,000,000 n hrs >> 0.000001 n2 hrs (for n << 1012), but (n) is “more efficient” than (n2) Not useful if concerned with absolute real-time constraints (e.g., must respond in 2 sec.) Not useful if concerned with fixed size or bounded problems (e.g., n < 1,000,000) Sometimes, “the constants matter” 12/4/2018 CS 100 - Lecture 6