1. Algorithms: the big picture
Algorithm is sequence of operations that transforms input to output
Example: Sort N items
input a sequence (a1, a2, … aN)
output a permutation of input sequence that is sorted
Most often items to be sorted do not have a numerical value
example: medical records
define keys that point to satellite data
sort keys

2. Algorithms begin with an idea
Example: Insertion sort
In partially sorted array, find where next item belongs
Create storage allocation and insert the item
Repeat until all items are sorted
Important questions about this idea:
will it work?
how much storage is required?
how fast does it run?
To get answers to these questions, we need a pseudocode.

3. To write a pseudocode, a picture may be helpful: Insertion sort on 6 items.
Sorts in place beginning at left end. 1st item is sorted by default. Next item to right of partially sorted array is placed in temporary storage. Leaves room to shift sorted items to right to open up the correct place to insert the item being sorted

4. Note: indents show the structure of pseudocode
Worst case tj = j

5. Line-by-line accounting of operations
Note: loop failure is
counted as operation
i.e. # of times an item in the
sorted part has to be moved
to open the correct place
for the item being sorted

6. Worst case analysis: tj = j
In worst-case, every element in A[1…j-1] sorted part of the array must be moved to the right
With tj = j sums can be evaluated because they are related to the “arithmetic” sum

7. Order of growth Our primary interest is
“How does runtime change with increasing input size?”
Linear: T(N) = a + bN
Quadratic : T(N) = a + bN + cN2
Logarithmic: T(N) = aN(log(bN))
The coefficients in these expressions are hard to calculate and
probably depend on properties of the input other than size.
To avoid this difficulty, we use “order of growth”
Linear: T(N) = order(N)
Quadratic : T(N) = order(N2)
Logarithmic: T(N) = order(N(log(N)))
Order of growth concept is expanded in Chapter 3

8. Prove by induction on integers that
Structured proof:
Base case: for some value of n (n=1 in case above)
what is the RHS equal to?
what is the LHS equal to?
are they the same?
Setup equation that reflects the strategy for proof:
In this class we will use “if S(n-1) then S(n)”
Inductive hypothesis (IH): an assumption that reflects the strategy and can be used on RHS of your setup equation.
Application of IH: equations with S(n) on the LHS and all algebraic manipulations on the RHS.

9. I.H. Proof that Setup to use “if S(n-1) then S(n)”
example of induction on integers
using if S(n-1) then S(n)
I.H.

10. Assignment 1: due 1/9/19
Geometric sum: Prove by induction on integers that
Give a structured proof using the technique if S(n-1) then S(n).
Include the following:
base case
setup
inductive hypothesis
application of inductive hypothesis with all algebra on RHS of equals

11. Review: Worst case analysis of insertion sort: tj = j
including the test that fails
With tj = j sums can be evaluated by relating them to the arithmetic sum

12. Use the arithmetic sum to evaluate the sums in the analysis of insertion sort runtime
(n-1)n/2

13. We find the polynomial dependence of runtime on input size in worst case

14. Line-by-line analysis of insertion sort: worst case
Collect terms: T(n) = a + bn + cn2
This is an upper bound that has the possibility of being equal
to the runtime
In notation of Chapter 3, T(n) = O(n2)

15. Worst case: T(n) = a + bn + cn2 = order of growth = n2
In asymptotic notation T(n) = O(n2)
All quadratic terms come from analysis of “while” statement
Was algebra really necessary?
Worst case tj = j

16. Loop invariant and correctness
Loop invariant = statement about iterative pseudo-codes
Prove that the statement is true by induction on integers
Initialization: true on 1st iteration (base case)
Maintenance: if true on ith iteration, also true on i+1 (if S(n) then S(n+1))
At termination: truth of loop invariant shows that the algorithm is correct

17. Application of loop invariant to insertion sort
Loop invariant: At the start of the jth iteration of for-loop 1-8, A(1…j-1) is sorted
Initialization: On the 1st iteration j=2. A(1…j-1) = A(1) is sorted by default.
Maintenance: By I.H. at start of jth iteration A(1…j-1) is sorted. On the jth iteration A(j) is
inserted into A(1…j-1) at its proper order; hence at the start of j+1 iteration A(1…j) is sorted
At termination: j = n+1; therefore A(1…j-1) = A(1…n). Whole array is sorted.

18. Merge sort: A new idea about sorting

19. Recursive sorting algorithm
Divide-and-conquer phase: Recursively divide the problem into smaller pieces until you can get a solution by default. Push merge instructions onto stack
Execution phase: Given default solution, assemble the full solution from successively larger pieces using merge instructions on the stack.
On 1st call to Merge-Sort p = 1, r = n
On recursive calls, the these dummy
variables have different values
Eventually, the test if p < r will fail and
stacked instructions to merge sorted
subarrays will execute.
Example of execution phase
for input (5, 2, 4, 7, 1, 3, 2, 6)

20. For recursive algorithms, analysis of runtime begins with
a recurrence relation that describes divide and conquer
T(n) = mT(n/k) + “overhead”
Overhead includes everything not involved in solving m
problems of size n/k
For Merge-Sort, m = k = 2 and “overhead” is cost of merging

21. With an even number of items, Merge-Sort is described by the recurrence relation
T(2k) = 2T(2k-1) + cost of merging
To evaluate “cost of merging” we need a pseudocode for
Merge(A,p,q,r), where sorted subarrays A(p…q) and
A(q+1…r) are merged in sorted order

22. Cost of merging 2 sorted arrays containing a total of n elements
Heuristic analysis:
Consider 2 card decks sorted in increasing order and turned face-up
decks not necessarily of equal size
Compare the value of cards showing
Place card of smaller value face down on the 3rd stack
If one deck is empty, place all of the other deck on the 3rd stack
How many comparison must be made?
How many comparison must be made, worst case?
Why is it important that the decks to be merged are sorted?

23. Use of “sentinel” cards
Merge(A,p,q,r) uses the fact that subarrays A(p…q) and A(q+1…r) are sorted.
All of the elements of A(p…q) and A(q+1…r) that have not been merged are larger than those in the merged array.
The possibility exists that all of the elements of A(p…q) have been merged while some elements remain in A(q+1…r), or vice versa.
Sentinel cards, with infinite value, are a way to ensure that this does not happen.

24. Pseudocode No nested loops.
Cost is linear in total number of items to be merged = r – p +1

25. Sentinel cards make algorithm simpler but maybe slower

26. A linear Merge algoarithm is sufficient to make Merge-Sort asymptotically optimal.
Recurrence for runtime becomes T(2k) = 2T(2k-1) + c2k for an even number of items.
In general, T(n)=T(floor(n/2)) + T(ceiling(n/2)) + cn
In chapter 4 we show that this recurrence has order of growth nlog(n)
In chapter 8 we show that order(nlog(n)) is an asymptotic lower bound on runtime of
sorting algorithm based on comparison of value.

27.  Assignment 2: due 1/11/19
Prove by induction on integers that recurrence
T(2k) = 2 if k=1
T(2k) = 2T(2k-1) + 2k if k>1
has solution T(2k) = k2k
Is this solution T(n)=nlg(n)?
Why does the recursion have 2 parts?
What are the base cases?
What is the “setup” for if S(n-1) then S(n)?
What is the inductive hypothesis?
What is the application of the inductive hypothesis?

28. Prove by iteration that recurrence
T(2k) = 2 if k=1
T(2k) = 2T(2k-1) + 2k if k>1
has solution T(2k) = k2k