Welcome to Cpt S 450 Design and Analysis of Algorithms Syllabus and Introduction

CptS 450 Design and Analysis of Algorithms Spring 2015 Instructor: John Miller, West 134E Class web page can be found at

Objectives: To present the mathematical basis for quantitative analysis of algorithms. To introduce students to various types of algorithms and analysis techniques Textbook: “Introduction to Algorithms” 3th Edition by Corman, et al Course content: Chapters 1-2: Introduction Chapter 3: Growth of functions Chapter 4: Solving recurrences Chapter 5: Probabilistic analysis Selected topics from the following: Chapters 6-9: Sorting algorithms Chapters 15-17: Advanced design and analysis techniques Chapters 22-26: Graph algorithms Final exam

Required assignments: Prior approval is required for late submission homework assignments. No partial credit on homework. Full credit for corrected homework. Criteria for student evaluation: homework 34%, quizzes 33%, final exam 33%.

Academic integrity: Academic integrity will be strongly enforced in this course. Any student caught cheating on any assignment will be given an F grade for the course and will be reported to the Office Student Standards and Accountability. Cheating is defined in the Standards for Student Conduct WAC (3). It is strongly suggested that you read and understand these definitions: I encourage you to work with classmates on assignments. However, each student must turn in original work. No copying will be accepted. Students who violate WSU’s Standards of Conduct for Students will receive an F as a final grade in this course, will not have the option to withdraw from the course and will be reported to the Office Student Standards and Accountability. Cheating is defined in the Standards for Student Conduct WAC (3). It is strongly suggested that you read and understand these definitions: Academic integrity is the cornerstone of the university. Any student who attempts to gain an unfair advantage over other students by cheating, will fail the assignment and be reported to the Office Student Standards and Accountability. Cheating is defined in the Standards for Student Conduct WAC (3).

Campus Safety Statement WSU Tri-Cities is committed to maintaining a safe environment for its faculty, staff and students. The Campus Safety Plan can be found at See also the WSU Office of Emergency Management site at Up-to-date WSU emergency alerts are available at Disability Services Reasonable Accommodations Statement: Reasonable accommodations are available for students who have a documented disability. Classroom accommodation forms are available through the Disability Services Office. If you have a documented disability, even if it’s temporary, make an appointment as soon as possible with the Disability Services Coordinator, Cherish Tijerina Pearson, West Building, Room 269J, at or You will need to provide your instructor with the appropriate classroom accommodation form. The form should be completed and submitted during the first week of class. Late notification can delay your accommodations or cause them to be unavailable. All accommodations for disabilities must be approved through the Disability Services Coordinator.

Algorithm is sequence of operations that transforms input to output Example: Sort N items input sequence (a 1, a 2, … a N ) output some permutations that is sorted Most often items to be sorted do not have a numerical value example: medical records define keys that point to satellite data Algorithms: the big picture

Example: Insertion sort In partially sorted intermediates, find where next item belongs Create storage allocation and insert the item Repeat until all items are sorted Important questions about this idea: will it work? how much storage is required? are there problems with numerical precision? how fast does it run? To get answers to these questions, we need a pseudocode. Algorithm begins with an idea

Operation of insertion sort on 6 items To write a pseudocode, a picture may be helpful Starting with the second item, we copy it to a new variable. This creates a storage allocation that we fill with items in the partially sorted intermediate if their value is larger than the value of the item being sorted. Note: this algorithm “sorts in place”

Note: indents show the structure of pseudocode Worst case t j = j

Our primary interest is “How does runtime change with increasing input size?” Linear: T(N) = a + bN Quadratic : T(N) = a + bN + cN 2 Logarithmic: T(N) = aN(log(bN)) The coefficients in these expressions are hard to calculate and probably depend on properties of the input other than size. To avoid this difficulty, we use “order of growth” Linear: T(N) = order(N) Quadratic : T(N) = order(N 2 ) Logarithmic: T(N) = order(N(log(N))) Order of growth concept is expanded in Chapter 3 In this class we focus on runtime

Line-by-line accounting of operations

In worst-case must compare a j to every element in A[1…j-1] With t j = j sums can be evaluated Worst case analysis: t j = j

Proof that example of induction on integers using if S(n-1) then S(n)

With t j = j sums can be evaluated Non of the sums needed are exactly the arithmetic sum Worst case analysis: t j = j

Use the arithmetic sum to evaluate the sums in the analysis of insertion sort runtime

We find the polynomial dependence of runtime on input size in worst case

Line-by-line analysis of insertion sort: worst case Collect terms: T(n) = a + bn + cn 2 This is an upper bound that has the possibility of being equal to the runtime In notation of Chapter 3, T(n) = O(n 2 )

Worst case t j = j T(n) = a + bn + cn 2 All quadratic terms come from analysis of “while” statement Was algebra really necessary?

Loop invariant and correctness Loop invariant = statement about iterative pseudo-code To prove that the statement is true by induction we need Initialization: true before 1 st iteration Maintenance: if true on i th iteration, also true on i+1 Termination: truth shows that the algorithm is correct

Application of loop invariant to insertion sort

Merge sort: A new idea about sorting

Divide-and-conquer phase: Recursively divide the problem into smaller pieces until get a solution by default Execution phase: Given default solution, assemble the full solution from successively larger pieces For efficiency analysis, consider execution phase only Example of execution phase for input (5, 2, 4, 7, 1, 3, 2, 6) Recursive sorting algorithm

For recursive algorithm, analysis of runtime begins with a recurrence relation that describes divide and conquer T(N) = mT(N/k) + “overhead” Overhead includes everything not involved in solving m problems of size N/k

Example: merge sort with even number of items T(2 k ) = 2T(2 k-1 ) + cost of merging To evaluate “cost of merging” need a pseudocode for Merge(A,p,q,r), where subarrays A(p) to A(q) and A(q+1) to A(r) are merged in sorted order

Use of “sentinel” cards

No nested loops. Cost is linear in total number of items to be merged = r - p

Merge does not use the fact that files to be merged are sorted Sentinel cards make algorithm simpler but maybe slower

Although a faster Merge algorithm may exist, a linear Merge is sufficient to make Merge-Sort asymptotically optimal. Recurrence for runtime becomes T(2 k ) = 2T(2 k-1 ) + c2 k for an even number of items. In general, T(N)=2T(| N/2 |) + cN In chapter 4 show that this recurrence has order of growth Nlog(N) In chapter 8 show that order(Nlog(N)) is an asymptotic lower bound runtime of sorting algorithm based on comparison of values.

CptS 450 Spring 2015 Homework Assignment 1: due 1/21/15 Prove by induction on integers that recurrence T(2 k ) = 2 if k=1 T(2 k ) = 2T(2 k-1 ) + 2 k if k>1Note: assuming c = 1 has solution T(2 k ) = k2 k Hint: use if S(n-1) then S(n)

Analysis of T(n)=2T(n/2)+cn by trees (n is even) n/2 istop = 1 All 3 trees are equally valid but not equally useful Smallest tree just graphical representation of recurrence As tree expands, overhead appears at each branch point At each level get the total overhead Calculate number of levels of D&C Sum cost of levels Express as order of growth

ex p39 Sort A[1…n] by recursively sorting A[1…n-1] then insert A[n] (as in Insertion-Sort). Write recurrence for worst-case run time. What is the solution of this recurrence?