Introduction to Algorithms: Verification, Complexity, and Searching Andy Wang Data Structures, Algorithms, and Generic Programming.

Introduction to Algorithms: Verification, Complexity, and Searching Andy Wang Data Structures, Algorithms, and Generic Programming

Lecture Overview Components of an algorithm Sequential search algorithms Binary search algorithms Proving algorithm correctness Computational complexity TVector Retrospective

Algorithm Components Required components Assumptions Asserted outcomes Body Proof Optional components Time complexity Space complexity

Sequential Search Goal Find a specified value in a collection of values Idea Walk through the collection and test each value A simple, commonly used, algorithm

Sequential Search Requirements A way to differentiate things in the collection A starting position A way to move on to the next thing in the collection A way to stop

Sequential Search Algorithm Assumptions Collection L of data of type T Can iterate through L with begin(), next(), end(); Outcomes Decide whether t is in L Return boolean (true/false)

Sequential Search Algorithm (2) Body (in pseudocode) for (T item = begin(L); item != end(L); item = next(L)) { if (t == item) return true; } if (t == item) return true; return false;

Binary Search Goal Find a value in a collection of values Idea Divide and conquer

Binary Search (2) Requirements Collection must be “array”-like Can use an index to jump to any array element Collection must be sorted Efficiency Very fast No extra space required

Binary Search—the idea You are heading down to an exotic restaurant… with something exotic on your mind… Chocolate……….Garlic……….Pasta … do not try this at home …

There are 8 items on the menu… sorted alphabetically… 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream

Binary Search—the idea The list is quite long…… it’s time to do a binary search… 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream

Binary Search—the idea 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream Search range: 0 - 7 Compare the middle element  Chocolate garlic pasta < Chocolate martini

Binary Search—the idea 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream Search range: 0 - 3 Compare the middle element  Chocolate garlic pasta > Blood pudding

Binary Search—the idea 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream Search range: 2 - 3 Compare the middle element  Chocolate garlic pasta == Chocolate garlic pasta

Binary Search—the idea Yum……………………………………

Binary Search Algorithm Three versions Binary_search Lower_bound Upper_bound

Binary Search Algorithm (2) Assumptions Collection L of data type of T with size sz L is sorted Element t of type T Outcomes Binary_search: true if t in L; false, otherwise Lower_bound: smallest j, where t <= L[j] Upper_bound: smallest j, where t < L[j]

Lower_bound Code unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } return low; }

Lower_bound Code 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream t = “Chocolate Garlic Pasta” low = 0; high = 7; while (0 < 7) { mid = (0 + 7) / 2 = 3; if (L[3] < t) { low = mid + 1; } else { high = mid; } return low;

Lower_bound Code 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream t = “Chocolate Garlic Pasta” low = 2; high = 2; while (2 < 2) { … } return low = 2;

Upper_bound Code unsigned int upper_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { mid = (low + high) / 2; if (t < L[mid]) { high = mid; } else { low = mid + 1; } return low; }

Upper_bound Code 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream t = “Chocolate Garlic Pasta” low = 0; high = 7; while (0 < 7) { mid = (0 + 7) / 2 = 3; if (t < L[3]) { high = mid; } else { low = mid + 1; } return low;

Upper_bound Code 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream t = “Chocolate Garlic Pasta” low = 3; high = 2; while (3 < 2) { … } return low = 3;

Binary_search Code unsigned int binary_search(T* L, unsigned sz, T t) { unsigned int lb = lower_bound(L, sz – 1, t); if (lb < sz) { if (t == L[lb]) { return true; } return false; }

If there are duplicate entries… 0. Baby beer 1. Blood pudding 2. Chocolate garlic pasta 3. Chocolate garlic pasta 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream Lower bound  Upper bound 

If t is not in L… 0. Baby beer 1. Blood pudding 2. Chocolate banana crepe 3. Chocolate martini 4. Death by Chocolate 5. Garlic ice cream 6. Popcorn-flavored jelly beans 7. Saffron ice cream Lower bound and upper bound 

Issues of Proof Correctness Termination Correct outcome Performance Time complexity Space complexity

Correctness and Loop Invariants Correctness Loop termination State when entering the loop State when exiting the loop Loop invariants Conditions that remain true for each iteration Mathematical induction

What can go wrong? for (j = 0; j < n; ++j) { compute(j); } void compute(unsigned int &j) { --j; } n < 0 at the beginning

Invariants—Sequential Search boolean sequential_search { for (T item = first(L); item != end(L); item = next(L)) { // item is not the final item in the collection // current item has not been examined  progress // L is finite if (t == item) { return true; } // t does not match the current item } if (t == item) { return true; } return false; }

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; }

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } t does not have to be L

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low –1 ] < t <= L[high] (if index is valid) } return low; } low = mid + 1 = (old_low + high)/2 + 1 <= (old_low + high)/2 + 1 < (high + high)/2 + 1 = high + 1 low < high + 1 low <= high

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low – 1] < t <= L[high] mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } high = mid = (low + old_high)/2 > (low + old_high)/2 - 1 > (low + low)/2 - 1 high > low – 1 high >= low

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } high – low = high – (mid + 1) = high – mid – 1 < high - mid = high – (old_low + high)/2 <= high – (old_low + high)/2 < high – (old_low + old_low)/2 = old_high – old_low high – low < old_high – old_low

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } high – low = high – (mid + 1) = high – mid – 1 < high - mid = high – (old_low + high)/2 <= high – (old_low + high)/2 = (high – old_low)/2 = (old_high – old_low)/2 high – low < (old_high – old_low)/2

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } high – low = mid – low = (low + old_high)/2 – low <= (low + old_high)/2 – low = (old_low + old_high)/2 – old_low < (old_high + old_high)/2 – old_low = old_high – old_low high – low < old_high – old_low

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } high – low = mid – low = (low + old_high)/2 – low <= (low + old_high)/2 – low = (old_high - low)/2 = (old_high - old_low)/2 high – low <= (old_high – old_low)/2

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } L[low - 1] = L[mid] < t Since high is not changed, t <= L[high]

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } Since low is unchanged, L[low] <= t L[high] = L[mid] >= t

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } Termination: (3) shows that the loop can terminate (4) shows progress

Invariants—Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { // (1) low < high // (2) L[low - 1] < t <= L[high] (if index is valid) mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } // (3) low <= high // (4) high – low has decreased // (5) L[low - 1] < t <= L[high] (if index is valid) } return low; } Return value: (5) smallest t <= L[j], since L[j = t

Computational Complexity Compares growth of two functions Independent of constant multipliers and lower-order effects Metrics “Big O” Notation “Big Omega” Notation “Big Theta” Notation

Big “O” Notation f(n) < O(g(n)) iff  c, n 0 > 0 | 0 = n 0 f(n) cg(n) n0n0

Big “Omega” Notation  (g(n)) < f(n) iff  c, n 0 > 0 | 0 = n 0 f(n) cg(n) n0n0

Big “Theta” Notation f(n) =  (g(n)) iff  c 1, c 2, n 0 > 0 | 0 = n 0 f(n) c 1 g(n) n0n0 c 2 g(n)

Examples 3n 2 + 17  (1),  (n) O(n 2 ), O(n 3 )  (n 2 )

Analogous to Real Numbers f(n) = O(g(n)) (a < b) f(n) =  (g(n)) (a > b) f(n) =  (g(n))(a = b)

Transitivity f(n) = O(g(n)) (a < b) f(n) =  (g(n)) (a > b) f(n) =  (g(n))(a = b) f(n) = O(g(n)) and g(n) = O(h(n)) f(n) = O(h(n))

Anti-symmetry f(n) = O(g(n)) (a < b) f(n) =  (g(n)) (a > b) f(n) =  (g(n))(a = b) f(n) = O(g(n)) iff g(n) =  (f(n))

Symmetry f(n) = O(g(n)) (a < b) f(n) =  (g(n)) (a > b) f(n) =  (g(n))(a = b) f(n) =  (g(n)) iff g(n) =  (f(n))

Reflexivity f(n) = O(g(n)) (a < b) f(n) =  (g(n)) (a > b) f(n) =  (g(n))(a = b) f(n) = O(f(n)), f(n) =  (f(n)), f(n) =  (f(n))

Dichotomy f(n) = O(g(n)) (a < b) f(n) =  (g(n)) (a > b) f(n) =  (g(n))(a = b) If f(n) = O(g(n)), g(n) =  (f(n)), f(n) =  (g(n))

Complexity Analysis Steps Find n = size of input Find an atomic activity to count Find f(n) = the number of atomic activities done by an input size of n Complexity of an algorithm = complexity of f(n)

Algorithm Complexity—Loops for (j = 0; j < n; ++j) { // 3 atomics } Complexity =  (3n) =  (n)

Loops with Break for (j = 0; j < n; ++j) { // 3 atomics if (condition) break; } Upper bound =  (3n) =  (n) Lower bound =  (3) =  (1) Complexity = O(n)

Loops in Sequence for (j = 0; j < n; ++j) { // 3 atomics } for (j = 0; j < n; ++j) { // 5 atomics } Complexity =  (3n + 5n) =  (n)

Nested Loops for (j = 0; j < n; ++j) { // 2 atomics for (k = 0; k < n; ++k) { // 3 atomics } Complexity =  ((2 + 3n)n) =  (n 2 )

Sequential Search for (T item = begin(L); item != end(L); item = next(L)) { if (t == item) return true; } if (t == item) return true; return false; Input size: n Atomic computation: comparison Complexity = O(n)

Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } return low; } Input size: n Atomic computation: comparison Complexity = k iterations x 1 comparison/loop  (log(n))

Iteration Count for Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } return low; } Iter # search space 1n 2n/2 3n/4 kn/2 (k-1) n/2 (k-1) = 1 n = 2 (k-1) log 2 (n) = k - 1

Iteration Count for Binary Search unsigned int lower_bound(T* L, unsigned max, T t) { unsigned int low = 0, mid, high = max; while (low < high) { mid = (low + high) / 2; if (L[mid] < t) { low = mid + 1; } else { high = mid; } return low; } n/2 (k-1) = 1 n = 2 (k-1) log 2 (n) = k - 1 log 2 (n) + 1 = k Complexity function f(n) = log(n) iterations x 1 comparison/loop =  (log(n))

TVector Retrospective Runtime complexity =  (1) PopBack(), Clear(), Front(), Back(), Empty() Size(), Capacity, [] Amortized runtime complexity =  (1) PushBack() Runtime complexity =  (n) Constructors, Destructor, Display(), Dump(), =

Introduction to Algorithms: Verification, Complexity, and Searching Andy Wang Data Structures, Algorithms, and Generic Programming.

Similar presentations

Presentation on theme: "Introduction to Algorithms: Verification, Complexity, and Searching Andy Wang Data Structures, Algorithms, and Generic Programming."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Introduction to Algorithms: Verification, Complexity, and Searching Andy Wang Data Structures, Algorithms, and Generic Programming.

Similar presentations

Presentation on theme: "Introduction to Algorithms: Verification, Complexity, and Searching Andy Wang Data Structures, Algorithms, and Generic Programming."— Presentation transcript:

Similar presentations

About project

Feedback