Lecture 3 Induction & Sort(1) Algorithm design techniques: induction Selection sort, Insertion sort, Shell sort...
ROADMAP Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort... ‣ Proofs and comparisons 5/27/2016Xiaojuan Cai2
Techniques based on Recursions Induction or Tail recursion Iteration, Proof: invariants by induction Non-overlapping subproblems Divide-and-conquer Overlapping subproblems Dynamic programming 5/27/2016Xiaojuan Cai3
Techniques based on Recursions Induction or Tail recursion ‣ Iteration, ‣ Proof: invariants by induction Non-overlapping subproblems ‣ Divide-and-conquer Overlapping subproblems ‣ Dynamic programming 5/27/2016Xiaojuan Cai4
Induction example 9, 8, 9, 6, 2, 56 Selection sort Problem size: 6 2, 8, 9, 6, 9, 56 Problem size: 5 Algorithm Design: problem size n --> n-1 Correctness Proof: by mathematical induction. 5/27/2016Xiaojuan Cai5
Permutation /27/2016Xiaojuan Cai6 Problem: Permuation Input: an array of n elements Output: the permutations of n elements
Permutation1 5/27/2016Xiaojuan Cai7
Permutation2 5/27/2016Xiaojuan Cai8
Permutation a1a1 a2a2 a3a3 a4a4 Permutation 1: Permutation 2: a1a1 a2a2 a3a3 a4a4 5/27/2016Xiaojuan Cai9
Permutation 1 5/27/2016Xiaojuan Cai10 Ω(n ⋅ n!)
Permutation 2 5/27/2016Xiaojuan Cai11 Ω(n ⋅ n!)
Quiz Which output is in alphabetical order? A. permutation1 B. permutation2 C. both D. none of above
Polynomial Problem: PolynomialEval Input: a[0...n], and x Output: the value of p(x) William George Horner (source from Wikipedia) Horner’s rule 5/27/2016Xiaojuan Cai13
Horner’s rule 5/27/2016Xiaojuan Cai14
Finding majority element Problem: MajorityElement Input: a sequence of integers A[1...n] Output: An integer a in A that appears more than n/2 times, otherwise, output none none 1 Observation If two different elements in the original sequence are removed, then the majority in the original sequence remains the majority in the new sequence. 5/27/2016Xiaojuan Cai15
Finding majority element Θ(n) 5/27/2016Xiaojuan Cai16
Proof of correctness Lemma (Invariants) Given a[1..N], Candidates(i) returns the majority element of a[i...N] if there exists a majority element. Proof. (Inductive proof) Base step. Lemma holds for i = N Inductive step. Induction Hypothesis. Assume for j > i, the lemma holds. Prove it holds for i. 5/27/2016Xiaojuan Cai17
Radix sort All numbers in the array consists of EXACTLY k digits
Proof of correctness Lemma (Invariants) In Radix sort, if the i-th digits are sorted, then the numbers consisting of i, i-1,..,1 digits are sorted.. Proof. (Inductive proof) Base step. Lemma holds for i = 1 Inductive step. Induction Hypothesis. Assume for j < i, the lemma holds. Prove it holds for i. 5/27/2016Xiaojuan Cai19
Where are we? Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort... ‣ Proofs and comparisons 5/27/2016Xiaojuan Cai20
? 5/27/2016Xiaojuan Cai21
? 5/27/2016Xiaojuan Cai22
? 5/27/2016Xiaojuan Cai23
Selection sort i jmin Best caseWorst case #Comparison: #Interchange: 5/27/2016Xiaojuan Cai24
Selection sort in final order ↑ Lemma (Invariants) Entries the left of ↑ (including ↑) fixed and in ascending order. No entry to right of ↑ is smaller than any entry to the left of ↑. 5/27/2016Xiaojuan Cai25
Insertion sort Best caseWorst case #Comparison: #Move: 5/27/2016Xiaojuan Cai26
Insertion sort in order ↑ not yet seen Lemma (Invariants) Entries to the left of ↑ (including ↑) are in ascending order. Entries to the right of ↑ have not yet been seen. 5/27/2016Xiaojuan Cai27
Shell sort (3-1) Correctness: The last round of shell sort is insertion sort. 5/27/2016Xiaojuan Cai28
Comparison worstaveragebestremarks selection N 2 / 2 N exchanges insertion N 2 / 2N 2 / 4Nfor small N or partially ordered shell ??Ntight code, subquadratic 5/27/2016Xiaojuan Cai29
Stability and in-place Definition: Stability A stable sort preserves the relative order of items with equal keys. Definition: In-place A sorting algorithm is in-place if the extra memory it uses ≤ c log N. in-place?stable? selection ?? insertion ?? shell ?? 5/27/2016Xiaojuan Cai30
Comparison in- place? stable?worst averag e bestremarks selection xN 2 / 2 N exchanges insertion xxN 2 / 2N 2 / 4N for small N or partially ordered shell x??Ntight code, subquadratic 5/27/2016Xiaojuan Cai31
Conclusion 5/27/2016Xiaojuan Cai32 Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort... ‣ Proofs and comparisons
Next permutation 5/27/2016Xiaojuan Cai Next_perm Next_perm
Exercises Download hw3.pdf from our course homepage Due on next Tuesday 5/27/2016Xiaojuan Cai34