Lecture 3 Induction & Sort(1) Algorithm design techniques: induction Selection sort, Insertion sort, Shell sort...

ROADMAP Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort... ‣ Proofs and comparisons 5/27/2016Xiaojuan Cai2

Techniques based on Recursions Induction or Tail recursion Iteration, Proof: invariants by induction Non-overlapping subproblems Divide-and-conquer Overlapping subproblems Dynamic programming 5/27/2016Xiaojuan Cai3

Techniques based on Recursions Induction or Tail recursion ‣ Iteration, ‣ Proof: invariants by induction Non-overlapping subproblems ‣ Divide-and-conquer Overlapping subproblems ‣ Dynamic programming 5/27/2016Xiaojuan Cai4

Induction example 9, 8, 9, 6, 2, 56 Selection sort Problem size: 6 2, 8, 9, 6, 9, 56 Problem size: 5 Algorithm Design: problem size n --> n-1 Correctness Proof: by mathematical induction. 5/27/2016Xiaojuan Cai5

Permutation /27/2016Xiaojuan Cai6 Problem: Permuation Input: an array of n elements Output: the permutations of n elements

Permutation1 5/27/2016Xiaojuan Cai7

Permutation2 5/27/2016Xiaojuan Cai8

Permutation a1a1 a2a2 a3a3 a4a4 Permutation 1: Permutation 2: a1a1 a2a2 a3a3 a4a4 5/27/2016Xiaojuan Cai9

Permutation 1 5/27/2016Xiaojuan Cai10 Ω(n ⋅ n!)

Permutation 2 5/27/2016Xiaojuan Cai11 Ω(n ⋅ n!)

Quiz Which output is in alphabetical order? A. permutation1 B. permutation2 C. both D. none of above

Polynomial Problem: PolynomialEval Input: a[0...n], and x Output: the value of p(x) William George Horner (source from Wikipedia) Horner’s rule 5/27/2016Xiaojuan Cai13

Horner’s rule 5/27/2016Xiaojuan Cai14

Finding majority element Problem: MajorityElement Input: a sequence of integers A[1...n] Output: An integer a in A that appears more than n/2 times, otherwise, output none none 1 Observation If two different elements in the original sequence are removed, then the majority in the original sequence remains the majority in the new sequence. 5/27/2016Xiaojuan Cai15

Finding majority element Θ(n) 5/27/2016Xiaojuan Cai16

Proof of correctness Lemma (Invariants) Given a[1..N], Candidates(i) returns the majority element of a[i...N] if there exists a majority element. Proof. (Inductive proof) Base step. Lemma holds for i = N Inductive step. Induction Hypothesis. Assume for j > i, the lemma holds. Prove it holds for i. 5/27/2016Xiaojuan Cai17

Radix sort All numbers in the array consists of EXACTLY k digits

Proof of correctness Lemma (Invariants) In Radix sort, if the i-th digits are sorted, then the numbers consisting of i, i-1,..,1 digits are sorted.. Proof. (Inductive proof) Base step. Lemma holds for i = 1 Inductive step. Induction Hypothesis. Assume for j < i, the lemma holds. Prove it holds for i. 5/27/2016Xiaojuan Cai19

Where are we? Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort... ‣ Proofs and comparisons 5/27/2016Xiaojuan Cai20

? 5/27/2016Xiaojuan Cai21

? 5/27/2016Xiaojuan Cai22

? 5/27/2016Xiaojuan Cai23

Selection sort i jmin Best caseWorst case #Comparison: #Interchange: 5/27/2016Xiaojuan Cai24

Selection sort in final order ↑ Lemma (Invariants) Entries the left of ↑ (including ↑) fixed and in ascending order. No entry to right of ↑ is smaller than any entry to the left of ↑. 5/27/2016Xiaojuan Cai25

Insertion sort Best caseWorst case #Comparison: #Move: 5/27/2016Xiaojuan Cai26

Insertion sort in order ↑ not yet seen Lemma (Invariants) Entries to the left of ↑ (including ↑) are in ascending order. Entries to the right of ↑ have not yet been seen. 5/27/2016Xiaojuan Cai27

Shell sort (3-1) Correctness: The last round of shell sort is insertion sort. 5/27/2016Xiaojuan Cai28

Comparison worstaveragebestremarks selection N 2 / 2 N exchanges insertion N 2 / 2N 2 / 4Nfor small N or partially ordered shell ??Ntight code, subquadratic 5/27/2016Xiaojuan Cai29

Stability and in-place Definition: Stability A stable sort preserves the relative order of items with equal keys. Definition: In-place A sorting algorithm is in-place if the extra memory it uses ≤ c log N. in-place?stable? selection ？？ insertion ？？ shell ？？ 5/27/2016Xiaojuan Cai30

Comparison in- place? stable?worst averag e bestremarks selection xN 2 / 2 N exchanges insertion xxN 2 / 2N 2 / 4N for small N or partially ordered shell x??Ntight code, subquadratic 5/27/2016Xiaojuan Cai31

Conclusion 5/27/2016Xiaojuan Cai32 Algorithm design techniques: induction Permutation Majority element Radix sort Selection sort, Insertion sort, Shell sort... ‣ Proofs and comparisons

Next permutation 5/27/2016Xiaojuan Cai Next_perm Next_perm

Exercises Download hw3.pdf from our course homepage Due on next Tuesday 5/27/2016Xiaojuan Cai34