Lecture 3 Induction Algorithm design techniques: induction Selection sort, Insertion sort, Shell sort ...

ROADMAP Algorithm design techniques: induction Majority element Permutation Radix sort Selection sort, Insertion sort, Shell sort ... Proofs and comparisons 11/19/2018 Xiaojuan Cai

Techniques based on Recursions Induction or Tail recursion Iteration, Proof: invariants by induction Non-overlapping subproblems Divide-and-conquer Overlapping subproblems Dynamic programming 11/19/2018 Xiaojuan Cai

Techniques based on Recursions Induction or Tail recursion Iteration, Proof: invariants by induction Non-overlapping subproblems Divide-and-conquer Overlapping subproblems Dynamic programming 11/19/2018 Xiaojuan Cai

Induction example Selection sort Problem size: 6 9, 8, 9, 6, 2, 56 2, 8, 9, 6, 9, 56 Algorithm Design: problem size n --> n-1 Correctness Proof: by mathematical induction. 11/19/2018 Xiaojuan Cai

Finding majority element Problem: MajorityElement Input: a sequence of integers A[1...n] Output: An integer a in A that appears more than n/2 times, otherwise, output none. 1 2 3 3 4 2 2 2 3 2 none 1 5 1 1 4 2 1 3 1 1 1 Observation If two different elements in the original sequence are removed, then the majority in the original sequence remains the majority in the new sequence. 11/19/2018 Xiaojuan Cai

Finding majority element 11/19/2018 Xiaojuan Cai

Proof of correctness Lemma (Invariants) Given a[1..N], Candidates(i) returns the majority element of a[i...N] if there exists one. Proof. (Inductive proof) Base step. Lemma holds for i = N Inductive step. Induction Hypothesis. Assume for j > i, the lemma holds. Prove it holds for i. 11/19/2018 Xiaojuan Cai

Permutation Problem: Permuation Input: an array of n elements Output: the permutations of n elements 1 2 3 1 3 2 2 1 3 2 3 1 3 1 2 3 2 1 11/19/2018 Xiaojuan Cai

Permutation1 11/19/2018 Xiaojuan Cai

Permutation2 11/19/2018 Xiaojuan Cai

Which output is in alphabetical order? Quiz Which output is in alphabetical order? A. permutation1 B. permutation2 C. both D. none of above Lemma1. Assume P[1,..,m,m+1,…,n] = a_1,…,a_m,a_m+1,…,a_n, then Perm1(m) will output a_1,…,a_m-1 + all the permutations of a_m,a_m+1,…,a_n. And after Perm1(m), P does not change the value Lemma2. Before perm2(m), there are n-m + 1 slots in P, and Perm2(m) will fill these slots with all the permutations of m,m+1,…,n. And after Perm2(m), P does not change.

All numbers in the array consists of EXACTLY k digits. Radix sort All numbers in the array consists of EXACTLY k digits. 7467 1247 3275 6792 9187 9134 4675 1239 6792 9134 3275 4675 7467 1247 9187 1239 9134 1239 1247 7467 3275 4675 9187 6792 9134 9187 1239 1247 3275 7467 4675 6792 1239 1247 3275 4675 6792 7467 9134 9187

Proof of correctness Lemma (Invariants) In Radix sort, if the i-th digits are sorted, then the numbers consisting of i, i-1, ..,1 digits are sorted.. Proof. (Inductive proof) Base step. Lemma holds for i = 1 Inductive step. Induction Hypothesis. Assume for j < i, the lemma holds. Prove it holds for i. 11/19/2018 Xiaojuan Cai

Where are we? Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort ... Proofs and comparisons 11/19/2018 Xiaojuan Cai

? Gypsy folk dance Selection sort 11/19/2018 Xiaojuan Cai

? Romanian folk dance Insertion sort 11/19/2018 Xiaojuan Cai

? Shell sort Hungarian folk dance 11/19/2018 Xiaojuan Cai

Selection sort Lemma (Invariants) Entries the left of ↑ (including ↑) fixed and in ascending order. No entry to right of ↑ is smaller than any entry to the left of ↑. ↑ in final order 11/19/2018 Xiaojuan Cai

Insertion sort Lemma (Invariants) Entries to the left of ↑ (including ↑) are in ascending order. Entries to the right of ↑ have not yet been seen. in order ↑ not yet seen 11/19/2018 Xiaojuan Cai

Shell sort (3-1) 2 13 6 24 43 1 51 9 10 Correctness: The last round of shell sort is insertion sort. 11/19/2018 Xiaojuan Cai

Comparison http://www.sorting-algorithms.com/ worst average best remarks selection N 2 / 2 N exchanges insertion N 2 / 4 N for small N or partially ordered shell ? tight code, subquadratic 11/19/2018 Xiaojuan Cai

Stability and in-place Definition: Stability A stable sort preserves the relative order of items with equal keys. Definition: In-place A sorting algorithm is in-place if the extra memory it uses ≤ c log N. in-place? stable? selection ？ insertion shell 11/19/2018 Xiaojuan Cai

tight code, subquadratic Comparison in-place? stable? worst average best remarks selection x N 2 / 2 N exchanges insertion N 2 / 4 N for small N or partially ordered shell ? tight code, subquadratic 11/19/2018 Xiaojuan Cai

Conclusion Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort ... Proofs and comparisons 11/19/2018 Xiaojuan Cai

Next permutation 1 3 2 Next_perm 2 1 3 1 4 3 2 5 Next_perm 1 4 3 5 2 11/19/2018 Xiaojuan Cai