1. Algorithmic complexity
EECE.3220 Data Structures
Instructor:
Dr. Michael Geiger
Spring 2019
Lecture 7:
Algorithmic complexity

2. Announcements/reminders
Program 1 due Friday, 2/8
All programs to be submitted via Blackboard
Submit a single .zip file containing all files for this assignment
Additional instructor support for course
Grader: Shubham Tikare
Office hours: Tues 9-11 AM, Ball 301E (ECE Conf Rm)
Tutor: Felipe Loera
Hours available on CLASS site (link also on home page)
5/4/2019
Data Structures: Lecture 7

3. Data Structures: Lecture 7
Lecture outline
Review
C++ strings
Algorithmic complexity
5/4/2019
Data Structures: Lecture 7

4. Data Structures: Lecture 7
Review: Strings
String data type found in <string> library
Concepts to work with strings
Relational operators: ==, !=, <, >, <=, >=
Character-by-character comparison using ASCII
Concatenation: +, +=
Choosing single character: [], at()
at() provides boundary checking
Substrings: substr() function
If s1 = “ECE 264”
 s1.substr(0,3) = “ECE”  3 chars starting at position 0
s1.substr(4) = “264”  all chars from position 4 to end of string
Checking string length: length(), empty() functions
5/4/2019
Data Structures: Lecture 7

5. Data Structures: Lecture 7
Algorithm Efficiency
How do we measure efficiency?
Space utilization – amount of memory required
Time required to accomplish the task
Time efficiency depends on:
size of input
speed of machine
quality of source code
quality of compiler
5/4/2019
Data Structures: Lecture 7

6. Data Structures: Lecture 7
Algorithm Efficiency
We can count the number of times operations are executed
Gives measure of efficiency of an algorithm
So we measure computing time as:
T(n) = computing time of an algorithm for input of size n = number of times the operations are executed
Analysis typically focuses on worst-case execution time
Occasionally analyze average execution time
5/4/2019
Data Structures: Lecture 7

7. Example: Calculating the Mean
Task # times executed
Initialize the sum to 0 1
Initialize index i to 0 1
While i < n do following n+1
a) Add x[i] to sum n
b) Increment i by 1 n
Return mean = sum/n 1
Total n + 4
5/4/2019
Data Structures: Lecture 7

8. Computing Time Order of Magnitude
As number of inputs increases
T(n) = 3n + 4 grows at a rate proportional to n
Thus T(n) has "order of magnitude" n
The computing time of an algorithm on input of size n, T(n) said to have order of magnitude f(n), written T(n) is O(f(n)), if there is a constant C such that:
T(n) < Cf(n) for sufficiently large values of n
5/4/2019
Data Structures: Lecture 7

9. Data Structures: Lecture 7
Big O Notation
More commonly, we say the complexity of the algorithm is O(f(n)).
Example: for the Mean-Calculation Algorithm: T(n) is O(n)
Constants and multiplicative factors are ignored
Use the slowest-growing function possible
Most informative about execution time
Technically, an algorithm with complexity O(n) has complexity O(n2), O(n3), etc …
5/4/2019
Data Structures: Lecture 7

10. Data Structures: Lecture 7
Examples
For each function, find worst case execution time & order of magnitude
(a) int F(int n) { int i, res; 1 if (n < 2) 2 return 1; 3 else { 4 res = 1; 5 for (i=2; i<=n; i++) 6 res *= i; 7 return res; }
(b)
unsigned F(unsigned n){
1 unsigned res = 0;
2 for (i=0; i<n+1; i++)
3 for (j=0; j<n+1; j++)
res = res + j;
5 return res; }
5/4/2019
Data Structures: Lecture 7

11. Example solution—part (a)
int F(int n) { int i, res; 1 if (n < 2) 1 2 return 1; 1 3 else { 1 4 res = 1; 1 5 for (i=2; i<=n; i++) 2n (see 5a-5c) 5a Set i = 2 1 5b Test i <= n n (# iterations + 1) 5c i++ n-1 (body runs n-1 times) 6 res *= i; n-1 7 return res; 1 }
If condition evaluated in both cases
If case—1 other statement (return)
T(n) = 2 = O(1)
Else case—execute statements in red
T(n) = 3 + 3n = O(n)  Worst case
5/4/2019
Data Structures: Lecture 7

12. Example solution—part (b)
unsigned F(unsigned n){ 1 unsigned res = 0; 1 2 for (i=0; i<n+1; i++) 2n+4 (see 2a-2c) 2a Set i = 0 1 2b Test i < n+1 n+2 (# iterations + 1) 2c i++ n+1 (body runs n+1 times) 3 for (j=0; j<n+1; j++) 2n2+6n+4 3a Set j = 0 (n+1)*1 = n+1 3b Test j < n+1 (n+1)*(n+2) = n2+3n+2 3c j++ (n+1)*(n+1) = n2+2n+1 4 res = res + j; (n+1)*(n+1) = n2+2n+1 5 return res; 1 }
In nested loop, inner loop goes through all iterations for every outer loop iteration
T(n) = 3n2 + 10n + 11 = O(n2)
5/4/2019
Data Structures: Lecture 7

13. Worst case analysis examples
Following slides present pseudocode and analysis for three common array operations
Linear search
Binary search
Selection sort
Pseudocode: describes steps of algorithm in code-like fashion, but doesn’t correspond to any actual language
5/4/2019
Data Structures: Lecture 7

14. Worst case analysis: linear search
Search entire array of n values for item; found = true and loc = item position if successful; otherwise, found = false
Set found = false
Set loc = 0
While loc < n and not found, do following:
If item == a[loc] then
Set found = true
Else
Increment loc by 1
5/4/2019
Data Structures: Lecture 7

15. Worst case analysis: linear search (2)
Worst case: item is not in list
Algorithm will go through all elements in array
In all cases
Lines 1 & 2 execute once
In worst case
Line 3 executes n+1 times
Lines 4 & 6 execute n times
Therefore, T(n) = 3n + 3 = O(n)
5/4/2019
Data Structures: Lecture 7

16. Worst case analysis: binary search
Searching ordered array much more efficient
Search array of n ascending values for item; found = true and loc = item position if successful; otherwise, found = false
Set found = false
Set first = 0
Set last = n - 1
While first ≤ last and not found, do following:
Calculate loc = (first + last) / 2
If item < a[loc] then
Set last = loc – 1 // Search first half
Else if item > a[loc] then
Set first = loc + 1 // Search second half
Else
Set found = true // Item found
5/4/2019
Data Structures: Lecture 7

17. Worst case analysis: binary search (2)
Algorithm splits list into smaller sublist to be searched
Loop control statement again limiting factor
Each time, sublist size ≤ ½ previous sublist size
Total number of loop iterations:
1 + (# iterations to produce sublist of size 1)
If k = # iterations to produce sublist of size 1,
n / 2k < 2
 n < 2k * 2
 n < 2k+1
 log2n < k + 1
Therefore, in worst case (item is larger than everything in list) line 4 executed 2 + log2n times
T(n) = O(log2n)
5/4/2019
Data Structures: Lecture 7

18. Worst case analysis: selection sort
Algorithm to sort array of n elements into ascending order
On ith pass, first find smallest element in sublist x[i] … x[n-1], then place that value in position i
For i = 0 to n – 2 do the following
Set smallPos = i
Set smallest = x[smallPos]
For j = i+1 to n-1 do the following
If x[j] < smallest then
Set smallPos = j
Set x[smallPos] = x[i]
Set x[i] = smallest
5/4/2019
Data Structures: Lecture 7

19. Worst case analysis: selection sort (2)
Outer loop condition (1) executed n times
Statements inside outer loop but not in inner loop (2, 3, 8, 9) executed n – 1 times
Inner loop
If i = 0, (4) executed n times, (5,6,7) n – 1 times
If i = 1, (4) executed n-1 times, (5,6,7) n – 2 times
… i = n-2, (4) executed 2 times, (5,6,7) 1 time
In total
(4) executed n(n+1)/2 – 1 times
(5,6,7) executed n(n+1)/2 – 2 times
Therefore
T(n) = n + 4(n-1) + n(n+1)/2 – 1 + 3(n(n-1)/2)
= 2n2 + 4n – 5 = O(n2)
5/4/2019
Data Structures: Lecture 7

20. Data Structures: Lecture 7
Final notes
Next time
Abstract data types
Classes
Reminders:
Program 1 due Friday, 2/8
All programs to be submitted via Blackboard
Submit a single .zip file containing all files for this assignment
Additional instructor support for course
Grader: Shubham Tikare
Office hours: Tues 9-11 AM, Ball 301E (ECE Conf Rm)
Tutor: Felipe Loera
Hours available on CLASS site (link also on home page)
5/4/2019
Data Structures: Lecture 7