Getting Started Introduction to Algorithms Jeff Chastine

Hard to find Symbols in PPT ƒßΘΟΣΦΩαβθωο‹›←→↔∑∞∫≠≤ ≥≈≡ ☺☻ Jeff Chastine

A Little Boy and His Teacher A troublesome student (named Bob) was asked to sum the numbers between 1 and 100 in order to keep him busy. He came up with this formula: ∑ i=1 n i = n(n+1) 2 Why does this work? Jeff Chastine

When n = =11, 2+9=11, 3+8=11, etc… You do this n/2 times Jeff Chastine

Sorting Problem Input: A sequence of n numbers Output: A permutation of the original such that a' 1 ≤ a' 2 ≤ … ≤ a' n Sorting is fundamental to computer science, so we’ll be studying several different solutions to it Jeff Chastine

Insertion Sort Uses two “hands” – Left – initially empty – Right – initially the original array Move a card from the right hand to the left Find the correct position by going from right to left (in the already sorted left hand) We say that insertion sort is sorted in place (no additional memory needed) Jeff Chastine

Insertion Sort 1 for j ← 2 to length[A] 2do key ← A[ j ] 3// Insert A[ j ] into the sorted sequence A[ j – 1] 4i ← j - 1 5while i > 0 and A[i] > key 6do A[i+1] ← A[i] 7i ← i - 1 8A[i+1] ← key A = ‹ 5, 2, 4, 6, 1, 3 › Jeff Chastine

Correctness of Insertion Sort We can use loop invariants: – Initialization – true prior to first iteration – Maintenance – remains true before the next iteration – Termination – remains true after the loop terminates At the start of each iteration, the subarray A [1.. j -1] is in sorted order Jeff Chastine

Correctness of Insertion Sort Initialization: when j = 2, A [1.. j – 1] holds a single element Maintenance: inner loop moves elements to the right until the proper position is found. A[ j ] is inserted into the correct position Termination: j = n + 1, which is beyond n Jeff Chastine

Analyzing Algorithms Analyzing an algorithm usually means determining how much computational time is taken to solve a given problem Input size usually means the number of items in the input (elements to be sorted, number of bits, number of nodes in a graph) Running time is the number of primitive operations executed (and is device independent) Jeff Chastine

Analysis of Insertion Sort Worst case: sorted in descending order (runs as a quadratic an 2 + bn + c, you’ll see) Best case scenario: numbers sorted in ascending order (linear function n) Why? This loop won't have to run! 5 while i > 0 and A[i] > keyc 5 6do A[i+1] ← A[i]c 6 7i ← i - 1 c 7 Jeff Chastine

Insertion Sort 1 for j ← 2 to length[A]c 1 n 2do key ← A[ j ]c 2 n-1 3// Insert …c 3 n-1 4i ← j - 1c 4 n-1 5while i > 0 and A[i] > keyc 5 ∑ 6do A[i+1] ← A[i]c 6 ∑ 7i ← i - 1c 7 ∑ 8A[i+1] ← keyc 8 n-1 j=2 n n n tjtj (t j - 1) Jeff Chastine

T(n)= c 1 n + c 2 (n-1) + c 4 (n-1) + c 5 ((n(n+1))/2-1) + c 6 ((n(n-1))/2) + c 7 ((n(n-1))/2) + c 8 = (c 5 /2 + c 6 /2 + c 7 /2) n 2 + (c 1 +c 2 +c 4 +c 5 /2-c 6 /2-c 7 /2+c 8 ) n - (c 2 +c 4 +c 5 +c 8 ) j=2 j = n n(n+1) 2 ∑ Thanks Bob! → Jeff Chastine

Rate of Growth The rate of growth is what we're interested in Only consider leading term (other terms are insignificant, as you will see) Also ignore leading term's coefficient a – Constants are less significant than rate of growth Therefore, we say worst-case for insertion sort is Θ(n 2 ) What is the best case for this algorithm? What about the average/expected case? Jeff Chastine

The Divide-and-Conquer Approach These algorithms are recursive in structure Call themselves with a subset of the given problem Then combine solutions back together Question: how to recursively fill in the screen? Jeff Chastine

M ERGE S ORT Divide n-element array into two subsection of n/2 size Conquer: sort the two subsections recursively using Merge Sort Merge the sorted subarrays to produce sorted answer Note: a unit of 1 is, by definition, sorted. Jeff Chastine

The Code M ERGE -S ORT (A, p, r) 1if p < r 2then q ←(p+r)/2 3 M ERGE -S ORT (A, p, q) 4 M ERGE -S ORT (A, q+1, r) 5 M ERGE (A, p, q, r) Jeff Chastine

M ERGE S ORT (Divide) Jeff Chastine

M ERGE S ORT (Merge – where the work’s done) Jeff Chastine

Analysis of M ERGE S ORT Analyzed with a recurrence equation, where – T(n) is the running time of the problem – We divide the problem into a problems of size 1/b – It takes D (n) time to divide each problem – It takes C (n) time to combine each problem T(n) actually comes out to be Θ (n lg n) T(n) = Θ (1) if n < c aT(n/b) + D(n) + C(n) otherwise { Jeff Chastine

Analysis of M ERGE -S ORT Divide: only takes constant time O(1) to compute the middle of the array Conquer: solve by creating two sub-problems of size n/2 Combine: combine the two n/2 arrays, taking n time T(n) = 2T(n/2) +  (n) Jeff Chastine

T(n)T(n)

cn T(n/2) Jeff Chastine

cn cn/2 T(n/4) Jeff Chastine

cn cn/2 cn/4 cccccccc Jeff Chastine

cn cn/2 cn/4 cccccccc cn Jeff Chastine

cn cn/2 cn/4 cccccccc log 2 n + 1 cn Total: cn lg n + cn Jeff Chastine

Why log 2 n levels? Let i be the height of the tree (top i==0) The level below the top has 2 i nodes, each contributing c(n/2 i ) amount of work = cn Assume that number of levels for 2 i nodes has a height of lg2 i + 1 Next level adds 2 i+1 nodes Therefore, lg 2 i+1 = (i + 1) + 1 cn(lg n + 1) = cn lg n + cn Jeff Chastine