1. Getting Started Introduction to Algorithms Jeff Chastine

2. Hard to find Symbols in PPT ƒßΘΟΣΦΩαβθωο‹›←→↔∑∞∫≠≤ ≥≈≡ ☺☻ Jeff Chastine

3. A Little Boy and His Teacher A troublesome student (named Bob) was asked to sum the numbers between 1 and 100 in order to keep him busy. He came up with this formula: ∑ i=1 n i = n(n+1) 2 Why does this work? Jeff Chastine

4. When n = 10 12345678910 1 2 3 4 5 1+10=11, 2+9=11, 3+8=11, etc… You do this n/2 times Jeff Chastine

5. Sorting Problem Input: A sequence of n numbers Output: A permutation of the original such that a' 1 ≤ a' 2 ≤ … ≤ a' n Sorting is fundamental to computer science, so we’ll be studying several different solutions to it Jeff Chastine

6. Insertion Sort Uses two “hands” – Left – initially empty – Right – initially the original array Move a card from the right hand to the left Find the correct position by going from right to left (in the already sorted left hand) We say that insertion sort is sorted in place (no additional memory needed) Jeff Chastine

7. Insertion Sort 1 for j ← 2 to length[A] 2do key ← A[ j ] 3// Insert A[ j ] into the sorted sequence A[ j – 1] 4i ← j - 1 5while i > 0 and A[i] > key 6do A[i+1] ← A[i] 7i ← i - 1 8A[i+1] ← key A = ‹ 5, 2, 4, 6, 1, 3 › Jeff Chastine

8. Correctness of Insertion Sort We can use loop invariants: – Initialization – true prior to first iteration – Maintenance – remains true before the next iteration – Termination – remains true after the loop terminates At the start of each iteration, the subarray A [1.. j -1] is in sorted order Jeff Chastine

9. Correctness of Insertion Sort Initialization: when j = 2, A [1.. j – 1] holds a single element Maintenance: inner loop moves elements to the right until the proper position is found. A[ j ] is inserted into the correct position Termination: j = n + 1, which is beyond n Jeff Chastine

10. Analyzing Algorithms Analyzing an algorithm usually means determining how much computational time is taken to solve a given problem Input size usually means the number of items in the input (elements to be sorted, number of bits, number of nodes in a graph) Running time is the number of primitive operations executed (and is device independent) Jeff Chastine

11. Analysis of Insertion Sort Worst case: sorted in descending order (runs as a quadratic an 2 + bn + c, you’ll see) Best case scenario: numbers sorted in ascending order (linear function n) Why? This loop won't have to run! 5 while i > 0 and A[i] > keyc 5 6do A[i+1] ← A[i]c 6 7i ← i - 1 c 7 Jeff Chastine

12. Insertion Sort 1 for j ← 2 to length[A]c 1 n 2do key ← A[ j ]c 2 n-1 3// Insert …c 3 n-1 4i ← j - 1c 4 n-1 5while i > 0 and A[i] > keyc 5 ∑ 6do A[i+1] ← A[i]c 6 ∑ 7i ← i - 1c 7 ∑ 8A[i+1] ← keyc 8 n-1 j=2 n n n tjtj (t j - 1) Jeff Chastine

13. T(n)= c 1 n + c 2 (n-1) + c 4 (n-1) + c 5 ((n(n+1))/2-1) + c 6 ((n(n-1))/2) + c 7 ((n(n-1))/2) + c 8 = (c 5 /2 + c 6 /2 + c 7 /2) n 2 + (c 1 +c 2 +c 4 +c 5 /2-c 6 /2-c 7 /2+c 8 ) n - (c 2 +c 4 +c 5 +c 8 ) j=2 j = n n(n+1) 2 ∑ Thanks Bob! → Jeff Chastine

14. Rate of Growth The rate of growth is what we're interested in Only consider leading term (other terms are insignificant, as you will see) Also ignore leading term's coefficient a – Constants are less significant than rate of growth Therefore, we say worst-case for insertion sort is Θ(n 2 ) What is the best case for this algorithm? What about the average/expected case? Jeff Chastine

15. The Divide-and-Conquer Approach These algorithms are recursive in structure Call themselves with a subset of the given problem Then combine solutions back together Question: how to recursively fill in the screen? Jeff Chastine

16. M ERGE S ORT Divide n-element array into two subsection of n/2 size Conquer: sort the two subsections recursively using Merge Sort Merge the sorted subarrays to produce sorted answer Note: a unit of 1 is, by definition, sorted. Jeff Chastine

17. The Code M ERGE -S ORT (A, p, r) 1if p < r 2then q ←(p+r)/2 3 M ERGE -S ORT (A, p, q) 4 M ERGE -S ORT (A, q+1, r) 5 M ERGE (A, p, q, r) Jeff Chastine

18. M ERGE S ORT (Divide) 52461326 5 24 61 32 6 5 2 4 61 3 2 6 5 2 4 6 1 3 2 6 Jeff Chastine

19. M ERGE S ORT (Merge – where the work’s done) 52461326 2 54 61 32 6 2 4 5 61 2 3 6 1 2 2 3 4 5 6 6 Jeff Chastine

20. Analysis of M ERGE S ORT Analyzed with a recurrence equation, where – T(n) is the running time of the problem – We divide the problem into a problems of size 1/b – It takes D (n) time to divide each problem – It takes C (n) time to combine each problem T(n) actually comes out to be Θ (n lg n) T(n) = Θ (1) if n < c aT(n/b) + D(n) + C(n) otherwise { Jeff Chastine

21. Analysis of M ERGE -S ORT Divide: only takes constant time O(1) to compute the middle of the array Conquer: solve by creating two sub-problems of size n/2 Combine: combine the two n/2 arrays, taking n time T(n) = 2T(n/2) +  (n) Jeff Chastine

22. T(n)T(n)

23. cn T(n/2) Jeff Chastine

24. cn cn/2 T(n/4) Jeff Chastine

25. cn cn/2 cn/4 cccccccc Jeff Chastine

26. cn cn/2 cn/4 cccccccc cn Jeff Chastine

27. cn cn/2 cn/4 cccccccc log 2 n + 1 cn Total: cn lg n + cn Jeff Chastine

28. Why log 2 n levels? Let i be the height of the tree (top i==0) The level below the top has 2 i nodes, each contributing c(n/2 i ) amount of work = cn Assume that number of levels for 2 i nodes has a height of lg2 i + 1 Next level adds 2 i+1 nodes Therefore, lg 2 i+1 = (i + 1) + 1 cn(lg n + 1) = cn lg n + cn Jeff Chastine