Algorithmic complexity EECE.3220 Data Structures Instructor: Dr. Michael Geiger Spring 2019 Lecture 7: Algorithmic complexity

Announcements/reminders Program 1 due Friday, 2/8 All programs to be submitted via Blackboard Submit a single .zip file containing all files for this assignment Additional instructor support for course Grader: Shubham Tikare Office hours: Tues 9-11 AM, Ball 301E (ECE Conf Rm) Tutor: Felipe Loera Hours available on CLASS site (link also on home page) 5/4/2019 Data Structures: Lecture 7

Data Structures: Lecture 7 Lecture outline Review C++ strings Algorithmic complexity 5/4/2019 Data Structures: Lecture 7

Data Structures: Lecture 7 Review: Strings String data type found in <string> library Concepts to work with strings Relational operators: ==, !=, <, >, <=, >= Character-by-character comparison using ASCII Concatenation: +, += Choosing single character: [], at() at() provides boundary checking Substrings: substr() function If s1 = “ECE 264”  s1.substr(0,3) = “ECE”  3 chars starting at position 0 s1.substr(4) = “264”  all chars from position 4 to end of string Checking string length: length(), empty() functions 5/4/2019 Data Structures: Lecture 7

Data Structures: Lecture 7 Algorithm Efficiency How do we measure efficiency? Space utilization – amount of memory required Time required to accomplish the task Time efficiency depends on: size of input speed of machine quality of source code quality of compiler 5/4/2019 Data Structures: Lecture 7

Data Structures: Lecture 7 Algorithm Efficiency We can count the number of times operations are executed Gives measure of efficiency of an algorithm So we measure computing time as: T(n) = computing time of an algorithm for input of size n = number of times the operations are executed Analysis typically focuses on worst-case execution time Occasionally analyze average execution time 5/4/2019 Data Structures: Lecture 7

Example: Calculating the Mean Task # times executed Initialize the sum to 0 1 Initialize index i to 0 1 While i < n do following n+1 a) Add x[i] to sum n b) Increment i by 1 n Return mean = sum/n 1 Total 3n + 4 5/4/2019 Data Structures: Lecture 7

Computing Time Order of Magnitude As number of inputs increases T(n) = 3n + 4 grows at a rate proportional to n Thus T(n) has "order of magnitude" n The computing time of an algorithm on input of size n, T(n) said to have order of magnitude f(n), written T(n) is O(f(n)), if there is a constant C such that: T(n) < Cf(n) for sufficiently large values of n 5/4/2019 Data Structures: Lecture 7

Data Structures: Lecture 7 Big O Notation More commonly, we say the complexity of the algorithm is O(f(n)). Example: for the Mean-Calculation Algorithm: T(n) is O(n) Constants and multiplicative factors are ignored Use the slowest-growing function possible Most informative about execution time Technically, an algorithm with complexity O(n) has complexity O(n2), O(n3), etc … 5/4/2019 Data Structures: Lecture 7

Data Structures: Lecture 7 Examples For each function, find worst case execution time & order of magnitude (a) int F(int n) { int i, res; 1 if (n < 2) 2 return 1; 3 else { 4 res = 1; 5 for (i=2; i<=n; i++) 6 res *= i; 7 return res; } (b) unsigned F(unsigned n){ 1 unsigned res = 0;   2 for (i=0; i<n+1; i++) 3 for (j=0; j<n+1; j++) 4 res = res + j; 5 return res; } 5/4/2019 Data Structures: Lecture 7

Example solution—part (a) int F(int n) { int i, res; 1 if (n < 2) 1 2 return 1; 1 3 else { 1 4 res = 1; 1 5 for (i=2; i<=n; i++) 2n (see 5a-5c) 5a Set i = 2 1 5b Test i <= n n (# iterations + 1) 5c i++ n-1 (body runs n-1 times) 6 res *= i; n-1 7 return res; 1 } If condition evaluated in both cases If case—1 other statement (return) T(n) = 2 = O(1) Else case—execute statements in red T(n) = 3 + 3n = O(n)  Worst case 5/4/2019 Data Structures: Lecture 7

Example solution—part (b) unsigned F(unsigned n){ 1 unsigned res = 0; 1 2 for (i=0; i<n+1; i++) 2n+4 (see 2a-2c) 2a Set i = 0 1 2b Test i < n+1 n+2 (# iterations + 1) 2c i++ n+1 (body runs n+1 times) 3 for (j=0; j<n+1; j++) 2n2+6n+4 3a Set j = 0 (n+1)*1 = n+1 3b Test j < n+1 (n+1)*(n+2) = n2+3n+2 3c j++ (n+1)*(n+1) = n2+2n+1 4 res = res + j; (n+1)*(n+1) = n2+2n+1 5 return res; 1 } In nested loop, inner loop goes through all iterations for every outer loop iteration T(n) = 3n2 + 10n + 11 = O(n2) 5/4/2019 Data Structures: Lecture 7

Worst case analysis examples Following slides present pseudocode and analysis for three common array operations Linear search Binary search Selection sort Pseudocode: describes steps of algorithm in code-like fashion, but doesn’t correspond to any actual language 5/4/2019 Data Structures: Lecture 7

Worst case analysis: linear search Search entire array of n values for item; found = true and loc = item position if successful; otherwise, found = false Set found = false Set loc = 0 While loc < n and not found, do following: If item == a[loc] then Set found = true Else Increment loc by 1 5/4/2019 Data Structures: Lecture 7

Worst case analysis: linear search (2) Worst case: item is not in list Algorithm will go through all elements in array In all cases Lines 1 & 2 execute once In worst case Line 3 executes n+1 times Lines 4 & 6 execute n times Therefore, T(n) = 3n + 3 = O(n) 5/4/2019 Data Structures: Lecture 7

Worst case analysis: binary search Searching ordered array much more efficient Search array of n ascending values for item; found = true and loc = item position if successful; otherwise, found = false Set found = false Set first = 0 Set last = n - 1 While first ≤ last and not found, do following: Calculate loc = (first + last) / 2 If item < a[loc] then Set last = loc – 1 // Search first half Else if item > a[loc] then Set first = loc + 1 // Search second half Else Set found = true // Item found 5/4/2019 Data Structures: Lecture 7

Worst case analysis: binary search (2) Algorithm splits list into smaller sublist to be searched Loop control statement again limiting factor Each time, sublist size ≤ ½ previous sublist size Total number of loop iterations: 1 + (# iterations to produce sublist of size 1) If k = # iterations to produce sublist of size 1, n / 2k < 2  n < 2k * 2  n < 2k+1  log2n < k + 1 Therefore, in worst case (item is larger than everything in list) line 4 executed 2 + log2n times T(n) = O(log2n) 5/4/2019 Data Structures: Lecture 7

Worst case analysis: selection sort Algorithm to sort array of n elements into ascending order On ith pass, first find smallest element in sublist x[i] … x[n-1], then place that value in position i For i = 0 to n – 2 do the following Set smallPos = i Set smallest = x[smallPos] For j = i+1 to n-1 do the following If x[j] < smallest then Set smallPos = j Set x[smallPos] = x[i] Set x[i] = smallest 5/4/2019 Data Structures: Lecture 7

Worst case analysis: selection sort (2) Outer loop condition (1) executed n times Statements inside outer loop but not in inner loop (2, 3, 8, 9) executed n – 1 times Inner loop If i = 0, (4) executed n times, (5,6,7) n – 1 times If i = 1, (4) executed n-1 times, (5,6,7) n – 2 times … i = n-2, (4) executed 2 times, (5,6,7) 1 time In total (4) executed n(n+1)/2 – 1 times (5,6,7) executed n(n+1)/2 – 2 times Therefore T(n) = n + 4(n-1) + n(n+1)/2 – 1 + 3(n(n-1)/2) = 2n2 + 4n – 5 = O(n2) 5/4/2019 Data Structures: Lecture 7

Data Structures: Lecture 7 Final notes Next time Abstract data types Classes Reminders: Program 1 due Friday, 2/8 All programs to be submitted via Blackboard Submit a single .zip file containing all files for this assignment Additional instructor support for course Grader: Shubham Tikare Office hours: Tues 9-11 AM, Ball 301E (ECE Conf Rm) Tutor: Felipe Loera Hours available on CLASS site (link also on home page) 5/4/2019 Data Structures: Lecture 7