1. Lecture 3 Induction Algorithm design techniques: induction
Selection sort, Insertion sort, Shell sort ...

2. ROADMAP Algorithm design techniques: induction
Majority element
Permutation
Radix sort
Selection sort, Insertion sort, Shell sort ...
Proofs and comparisons
11/19/2018
Xiaojuan Cai

3. Techniques based on Recursions
Induction or Tail recursion
Iteration,
Proof: invariants by induction
Non-overlapping subproblems
Divide-and-conquer
Overlapping subproblems
Dynamic programming
11/19/2018
Xiaojuan Cai

4. Techniques based on Recursions
Induction or Tail recursion
Iteration,
Proof: invariants by induction
Non-overlapping subproblems
Divide-and-conquer
Overlapping subproblems
Dynamic programming
11/19/2018
Xiaojuan Cai

5. Induction example Selection sort Problem size: 6 9, 8, 9, 6, 2, 56
2, 8, 9, 6, 9, 56
Algorithm Design: problem size n --> n-1
Correctness Proof: by mathematical induction.
11/19/2018
Xiaojuan Cai

6. Finding majority element
Problem: MajorityElement
Input: a sequence of integers A[1...n]
Output: An integer a in A that appears more than n/2 times, otherwise, output none.
none 1
Observation
If two different elements in the original sequence are removed, then the majority in the original sequence remains the majority in the new sequence.
11/19/2018
Xiaojuan Cai

7. Finding majority element
11/19/2018
Xiaojuan Cai

8. Proof of correctness Lemma (Invariants)
Given a[1..N], Candidates(i) returns the majority element of a[i...N] if there exists one.
Proof. (Inductive proof)
Base step. Lemma holds for i = N
Inductive step.
Induction Hypothesis. Assume for j > i, the lemma holds.
Prove it holds for i.
11/19/2018
Xiaojuan Cai

9. Permutation Problem: Permuation Input: an array of n elements
Output: the permutations of n elements
1 2 3
1 3 2
2 1 3
2 3 1
3 1 2
3 2 1
11/19/2018
Xiaojuan Cai

10. Permutation1
11/19/2018
Xiaojuan Cai

11. Permutation2
11/19/2018
Xiaojuan Cai

12. Which output is in alphabetical order?
Quiz
Which output is in alphabetical order?
A. permutation1
B. permutation2
C. both
D. none of above
Lemma1. Assume P[1,..,m,m+1,…,n] = a_1,…,a_m,a_m+1,…,a_n, then Perm1(m) will output a_1,…,a_m-1 + all the permutations of a_m,a_m+1,…,a_n. And after Perm1(m), P does not change the value
Lemma2. Before perm2(m), there are n-m + 1 slots in P, and Perm2(m) will fill these slots with all the permutations of m,m+1,…,n. And after Perm2(m), P does not change.

13. All numbers in the array consists of EXACTLY k digits.
Radix sort
All numbers in the array consists of EXACTLY k digits.
7467
1247
3275
6792
9187
9134
4675
1239
6792
9134
3275
4675
7467
1247
9187
1239
9134
1239
1247
7467
3275
4675
9187
6792
9134
9187
1239
1247
3275
7467
4675
6792
1239
1247
3275
4675
6792
7467
9134
9187

14. Proof of correctness Lemma (Invariants)
In Radix sort, if the i-th digits are sorted, then the numbers consisting of i, i-1, ..,1 digits are sorted..
Proof. (Inductive proof)
Base step. Lemma holds for i = 1
Inductive step.
Induction Hypothesis. Assume for j < i, the lemma holds.
Prove it holds for i.
11/19/2018
Xiaojuan Cai

15. Where are we? Algorithm design techniques: induction
Permutation
Majority element
Radix sort
Selection sort, Insertion sort, Shell sort ...
Proofs and comparisons
11/19/2018
Xiaojuan Cai

16. ?
Gypsy folk dance
Selection sort
11/19/2018
Xiaojuan Cai

17. ?
Romanian folk dance
Insertion sort
11/19/2018
Xiaojuan Cai

18. ?
Shell sort
Hungarian folk dance
11/19/2018
Xiaojuan Cai

19. Selection sort Lemma (Invariants)
Entries the left of ↑ (including ↑) fixed and in ascending order.
No entry to right of ↑ is smaller than any entry to the left of ↑. ↑
in final order
11/19/2018
Xiaojuan Cai

20. Insertion sort Lemma (Invariants)
Entries to the left of ↑ (including ↑) are in ascending order.
Entries to the right of ↑ have not yet been seen.
in order ↑
not yet seen
11/19/2018
Xiaojuan Cai

21. Shell sort (3-1) 2 13 6 24 43 1 51 9 10
Correctness: The last round of shell sort is insertion sort.
11/19/2018
Xiaojuan Cai

22. Comparison http://www.sorting-algorithms.com/ worst average best
remarks
selection
N 2 / 2
N exchanges
insertion
N 2 / 4 N
for small N or partially ordered
shell ?
tight code, subquadratic
11/19/2018
Xiaojuan Cai

23. Stability and in-place
Definition: Stability
A stable sort preserves the relative order of items with equal keys.
Definition: In-place
A sorting algorithm is in-place if the extra memory it uses ≤ c log N.
in-place?
stable?
selection ？
insertion
shell
11/19/2018
Xiaojuan Cai

24. tight code, subquadratic
Comparison
in-place?
stable?
worst
average
best
remarks
selection x
N 2 / 2
N exchanges
insertion
N 2 / 4 N
for small N or
partially ordered
shell ?
tight code, subquadratic
11/19/2018
Xiaojuan Cai

25. Conclusion Algorithm design techniques: induction
Permutation
Majority element
Radix sort
Selection sort, Insertion sort, Shell sort ...
Proofs and comparisons
11/19/2018
Xiaojuan Cai

26. Next permutation 1 3 2 Next_perm 2 1 3 1 4 3 2 5 Next_perm 1 4 3 5 2
11/19/2018
Xiaojuan Cai