Copyright © 2014 Curt Hill Algorithm Analysis How Do We Determine the Complexity of Algorithms

Two Flavors of Complexity Time and Space Time complexity deals with how many instructions need to be executed based on the number of inputs Space complexity deals with how much memory is required, also based on number of inputs –This is dependent on the data structures used –Book will not discuss this one Copyright © 2014 Curt Hill

Time Complexity We typically focus on one or several types of instructions These may be any of the following types: –Comparisons –Assignment statements Copyright © 2014 Curt Hill

Why These? There is considerable difference in the speeds of these instructions over expenses of various computers Moreover, there may be other support instructions that may be present or absent depending on the machine language used Yet these factors generally get rolled into the constants in Big O and similar notations Copyright © 2014 Curt Hill

Max of Sequence procedure max(a 1, a 2, …a n : integer) max := a 1 {Assignment on name} for i=2 to n if max < a i then max := a i return max {All done} There is an obvious and a subtle comparison Similarly there are obvious assignments and subtle Copyright © 2014 Curt Hill

Analysis For comparisons –N-1 comparisons in the if –The for uses N compares on whether i has reached its limit –Thus 2N – 1 comparisons Assignments is harder for it varies –Best case, first is max, just 1 –Worst case, list is in ascending order N –Average case, N/2 We would call this O(N) Copyright © 2014 Curt Hill

Bubble Copyright © 2014 Curt Hill procedure bubble(a 1,a 2 …a n :integer) swapped: Boolean n: integer; do swapped := false for j := 1 to n-1 if a j < a j+1 temp := a j a j := a j+1 a j+1 := temp swapped := true n := n – 1 until not swapped

Commentary This bubble sort is slightly better than that given in Rosen The largest element always sinks to the bottom, so why check it again? –This is the n:=n-1 It also stops sorting when there are no interchanges However, these do complicate the analysis Copyright © 2014 Curt Hill

Inner loop contents Inside the inner loop there will be two comparisons –One for loop, one for if Zero or four assignments 1 increment of control variable Copyright © 2014 Curt Hill

How many loops? Copyright © 2014 Curt Hill

Cases Copyright © 2014 Curt Hill

Commentary Typically sorts fall into three big O categories O(n 2 ) – bad sorts –Bubble, selection, insertion and others O(n log n) – good sorts –Quick, Heap, Merge Weird cases –Shell O(n ~1.25 ) Copyright © 2014 Curt Hill

Tractable and not Algorithms with explosively growing Big Os are called intractable Polynomial and slower growing are tractable –Algorithms with exponents larger than four are unusual Exponential, factorial and faster growing algorithms are generally intractable Copyright © 2014 Curt Hill

Tractable and Intractable Copyright © 2014 Curt Hill O(n) O(n log n) ,4699,966 n2n2 1002,50010,00090,0001 x 10 7 n3n ,0001 x x x n2n 1024~10 16 ~ ~ Big n!3.6 x 10 7 ~10 65 ~ ~ Big

Worst and Average There are instances of problems where worst case estimates are intractable, but average cases are not We typically attempt to solve these, but if the program runs longer than desired we quit In problems that we know are intractable we may have to settle for an approximate solution Copyright © 2014 Curt Hill

P and NP Those problems with known solutions belong to a class labeled P There is also a class of problems labeled NP which are believed to have intractable solutions, but if a solution is known it can be checked in polynomial time There is also the NP complete set of problems –If any of them can be solved in polynomial time then all of them can be solved in polynomial time Copyright © 2014 Curt Hill

Traveling Salesman Problem Given a set of cities with distances between them Find the shortest route that visits each city exactly once and returns to the city of origin Variations are NP-Complete with others being even worse NP-Hard Copyright © 2014 Curt Hill

Knapsack Problem Given various articles of different weights and volumes Find the set which has a weight less than some maximum with the greatest volume Copyright © 2014 Curt Hill

Bin Packing Problem Given a set bins of a given volume Pack a set of items which are of differing volumes into a minimum number of bins If the number of bins is restricted to one this becomes the knapsack problem Copyright © 2014 Curt Hill

Worse than intractable? The halting problem show us that there are also unsolvable problems Clearly worse than intractable Copyright © 2014 Curt Hill

Exercises 3.3 –3, 7, 11, 17, 23 Copyright © 2014 Curt Hill