Lecture 5 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure.

Slides:



Advertisements
Similar presentations
CS3024-FAZ1 Mathematical Analysis of Recursive Algorithms Design and Analysis of Algorithms (CS3024) 28/02/2006.
Advertisements

CSE Lecture 3 – Algorithms I
Analysis of Algorithms
1 Recursive Algorithm Analysis Dr. Ying Lu RAIK 283: Data Structures & Algorithms September 13, 2012.
Divide and Conquer Strategy
CS4413 Divide-and-Conquer
Lecture 10 Jianjun Hu Department of Computer Science and Engineering University of South Carolina CSCE350 Algorithms and Data Structure.
DIVIDE AND CONQUER APPROACH. General Method Works on the approach of dividing a given problem into smaller sub problems (ideally of same size).  Divide.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 5.
Lecture 8 Jianjun Hu Department of Computer Science and Engineering University of South Carolina CSCE350 Algorithms and Data Structure.
CS Section 600 CS Section 002 Dr. Angela Guercio Spring 2010.
Liang, Introduction to Java Programming, Sixth Edition, (c) 2007 Pearson Education, Inc. All rights reserved L17 (Chapter 23) Algorithm.
Data Structures, Spring 2004 © L. Joskowicz 1 Data Structures – LECTURE 3 Recurrence equations Formulating recurrence equations Solving recurrence equations.
Design and Analysis of Algorithms - Chapter 21 Analysis of Algorithms b Issues: CorrectnessCorrectness Time efficiencyTime efficiency Space efficiencySpace.
Data Structures, Spring 2006 © L. Joskowicz 1 Data Structures – LECTURE 3 Recurrence equations Formulating recurrence equations Solving recurrence equations.
CS 104 Introduction to Computer Science and Graphics Problems Data Structure & Algorithms (3) Recurrence Relation 11/11 ~ 11/14/2008 Yang Song.
CSC 2300 Data Structures & Algorithms January 30, 2007 Chapter 2. Algorithm Analysis.
Design and Analysis of Algorithms - Chapter 41 Divide and Conquer The most well known algorithm design strategy: 1. Divide instance of problem into two.
Chapter 2: Fundamentals of the Analysis of Algorithm Efficiency
Liang, Introduction to Java Programming, Eighth Edition, (c) 2011 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
Analysis of Algorithm.
CS Discrete Mathematical Structures Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, 9:30-11:30a.
CHAPTER 7: SORTING & SEARCHING Introduction to Computer Science Using Ruby (c) Ophir Frieder at al 2012.
Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina CSCE350 Algorithms and Data Structure.
Liang, Introduction to Java Programming, Seventh Edition, (c) 2009 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
Analysis of Algorithm Lecture 3 Recurrence, control structure and few examples (Part 1) Huma Ayub (Assistant Professor) Department of Software Engineering.
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved ADT Implementation:
1 Chapter 24 Developing Efficient Algorithms. 2 Executing Time Suppose two algorithms perform the same task such as search (linear search vs. binary search)
1 Recursion Algorithm Analysis Standard Algorithms Chapter 7.
{ CS203 Lecture 7 John Hurley Cal State LA. 2 Execution Time Suppose two algorithms perform the same task such as search (linear search vs. binary search)
Analyzing Recursive Algorithms A recursive algorithm can often be described by a recurrence equation that describes the overall runtime on a problem of.
CS 3343: Analysis of Algorithms
Design and Analysis of Algorithms - Chapter 21 Analysis of Algorithms b Issues: CorrectnessCorrectness Time efficiencyTime efficiency Space efficiencySpace.
Analysis of Algorithms
Complexity of algorithms Algorithms can be classified by the amount of time they need to complete compared to their input size. There is a wide variety:
Project 2 due … Project 2 due … Project 2 Project 2.
Analysis of Algorithm Efficiency Dr. Yingwu Zhu p5-11, p16-29, p43-53, p93-96.
1 7.Algorithm Efficiency What to measure? Space utilization: amount of memory required  Time efficiency: amount of time required to process the data Depends.
1 7.Algorithm Efficiency What to measure? Space utilization: amount of memory required  Time efficiency: amount of time required to process the data.
Getting Started Introduction to Algorithms Jeff Chastine.
CS 361 – Chapters 8-9 Sorting algorithms –Selection, insertion, bubble, “swap” –Merge, quick, stooge –Counting, bucket, radix How to select the n-th largest/smallest.
1/6/20161 CS 3343: Analysis of Algorithms Lecture 2: Asymptotic Notations.
2-0 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Theoretical.
UNIT-I FUNDAMENTALS OF THE ANALYSIS OF ALGORITHM EFFICIENCY ANALYSIS AND DESIGN OF ALGORITHMS CHAPTER 2:
Searching Topics Sequential Search Binary Search.
Lecture 4 Jianjun Hu Department of Computer Science and Engineerintg University of South Carolina CSCE350 Algorithms and Data Structure.
BINARY SEARCH CS16: Introduction to Data Structures & Algorithms Thursday February 12,
Lecture #3 Analysis of Recursive Algorithms
Chapter 15 Running Time Analysis. Topics Orders of Magnitude and Big-Oh Notation Running Time Analysis of Algorithms –Counting Statements –Evaluating.
chap10 Chapter 10 Recursion chap10 2 Recursive Function recursive function The recursive function is a kind of function that calls.
Mathematical Analysis of Recursive Algorithm CSG3F3 Lecture 7.
Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
1 7.Algorithm Efficiency These factors vary from one machine/compiler (platform) to another  Count the number of times instructions are executed So, measure.
Algorithmic Foundations COMP108 COMP108 Algorithmic Foundations Algorithm efficiency Prudence Wong
1 ADT Implementation: Recursion, Algorithm Analysis Chapter 10.
LECTURE 9 CS203. Execution Time Suppose two algorithms perform the same task such as search (linear search vs. binary search) and sorting (selection sort.
CSG523/ Desain dan Analisis Algoritma
CSG523/ Desain dan Analisis Algoritma
Algorithm Analysis 1.
Theoretical analysis of time efficiency
Analysis of Algorithms
Chapter 10 Recursion Instructor: Yuksel / Demirer.
Introduction to the Design and Analysis of Algorithms
Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency
Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency
CS 3343: Analysis of Algorithms
Data Structures Review Session
Fundamentals of the Analysis of Algorithm Efficiency
At the end of this session, learner will be able to:
Fundamentals of the Analysis of Algorithm Efficiency
Presentation transcript:

Lecture 5 Jianjun Hu Department of Computer Science and Engineering University of South Carolina CSCE350 Algorithms and Data Structure

Outline  How to analyze the Time Efficiency of non-recursive algorithms  How to analyze time efficiency of recursive algorithms

Analyze the Time Efficiency of An Algorithm Nonrecursive Algorithm Recursive Algorithm

Element Uniqueness Check whether all the elements in a given array are distinct Input: An array A[0…n-1] Output: Return “true” if all the elements in A are distinct and “false” otherwise

Selection sort

Insertion Sort

Insert Sorting—running time cases  The best case input is an array that is already sorted. In this case insertion sort has a linear running time (i.e., O(n)). During each iteration, the first remaining element of the input is only compared with the right-most element of the sorted subsection of the array.O  The worst case input is an array sorted in reverse order. In this case every iteration of the inner loop will scan and shift the entire sorted subsection of the array before inserting the next element. For this case insertion sort has a quadratic running time (i.e., O(n 2 )).  The average case is also quadratic, which makes insertion sort impractical for sorting large arrays. However, insertion sort is one of the fastest algorithms for sorting arrays containing fewer than ten elements.

Improvement of Insert sorting  Shell sort: two simple variants requiring O(n 3/2 ) and O(n 4/3 ) running time.  binary insertion sort  In 2004 Bender, Farach-Colton, and Mosteiro published a new variant of insertion sort called library sort or gapped insertion sort high probability in O(n log n) time

Example: Find the Number of Binary Digits Find the Number of Binary Digits in the Binary Representation of a Positive Decimal Integer

How to identify inefficiency and speed it up? What is it doing? Time complexity? How to speed it up?

How to identify inefficiency and speed it up? Matrix multiplication For i  0 to n-1 do for j  0 to n-1 do C[i,j]  0 for k  0 to n-1 do C[i,j]  C[i,j]+A[i,k]*B[k,j]

Example Recursive evaluation of n ! Recursive algorithm for n! Input size: n Basic operation: multiplication “*” Let M(n) be the number of multiplications needed to compute n!, then To compute Factorial(n-1) To multiply Factorial(n-1) by n

Solve the Recurrence Therefore, the number of multiplications needed to compute n! in this algorithm is n.  The complexity of this algorithm is

Time Efficiency of Recursive Algorithms Steps in mathematical analysis of recursive algorithms: Decide on parameter n indicating input size Identify algorithm’s basic operation Determine worst, average, and best case for input of size n Set up a recurrence relation and initial condition(s) for C(n)- the number of times the basic operation will be executed for an input of size n (alternatively count recursive calls). Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution (see Appendix B)

Another Example: Tower of Hanoi n different-size disks, 3 pegs, move disks from the left peg to the right one using the middle one as an auxiliary Rules: you can move one disk each time and it is forbidden to place a larger disk on top of a smaller one Design and algorithm and analyze its complexity

Recursive Algorithm Input size: n (disks) Basic operation: one move of a disk Initial condition: n=1  only one direct move To build the recurrence: suppose you have a way to move n-1 disks. Then you can move the top n-1 disks from the left peg to the middle peg using the right peg as an auxiliary. Move the bottom disk from the left peg to the right peg. Move n-1 disks from the middle peg to the right peg using the left one as an auxiliary.

Illustration

Algorithm Complexity Let M(n) be the number of needed moves Initialization M(1)=1 Recurrence

Important Recurrence Types: One (constant) operation reduces problem size by one. T(n) = T(n-1) + c T(1) = d Solution: T(n) = (n-1)c + d linear A pass through input reduces problem size by one. T(n) = T(n-1) + cn T(1) = d Solution: T(n) = [n(n+1)/2 – 1] c + d quadratic One (constant) operation reduces problem size by half. T(n) = T(n/2) + c T(1) = d Solution: T(n) = c lg n + d logarithmic A pass through input reduces problem size by half. T(n) = 2T(n/2) + cn T(1) = d Solution: T(n) = cn lg n + d n n log n

A General Divide-and-Conquer Recurrence: Master Theorem T(n) = aT(n/b) + f (n) where f (n) ∈ Θ (n k ) a < b k T(n) ∈ Θ (n k ) a = b k T(n) ∈ Θ (n k lg n ) a > b k T(n) ∈ Θ ( n log b a ) Note: the same results hold with O instead of Θ.

Example: Find the Number of Binary Digits (Recursive Algorithm) Find the Number of Binary Digits in the Binary Representation of a Positive Decimal Integer using a recursive algorithm Recurrence