Design and Analysis of Algorithms Chapter 2.1 1 Analysis of Algorithms Dr. Ying Lu August 28, 2012

Slides:



Advertisements
Similar presentations
Introduction to Algorithms Rabie A. Ramadan rabieramadan.org 2 Some of the sides are exported from different sources.
Advertisements

Computational Complexity 1. Time Complexity 2. Space Complexity.
Fall 2006CENG 7071 Algorithm Analysis. Fall 2006CENG 7072 Algorithmic Performance There are two aspects of algorithmic performance: Time Instructions.
Chapter 3 Growth of Functions
Chapter 2: Fundamentals of the Analysis of Algorithm Efficiency
Asymptotic Growth Rate
CS 206 Introduction to Computer Science II 09 / 10 / 2008 Instructor: Michael Eckmann.
Introduction to Analysis of Algorithms
Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Design and Analysis of Algorithms - Chapter 21 Analysis of Algorithms b Issues: CorrectnessCorrectness Time efficiencyTime efficiency Space efficiencySpace.
Concept of Basic Time Complexity Problem size (Input size) Time complexity analysis.
CS 206 Introduction to Computer Science II 01 / 28 / 2009 Instructor: Michael Eckmann.
Liang, Introduction to Java Programming, Eighth Edition, (c) 2011 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 2 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.
Analysis of Algorithms COMP171 Fall Analysis of Algorithms / Slide 2 Introduction * What is Algorithm? n a clearly specified set of simple instructions.
Algorithm Analysis (Big O)
Asymptotic Growth Rates Themes –Analyzing the cost of programs –Ignoring constants and Big-Oh –Recurrence Relations & Sums –Divide and Conquer Examples.
Lecture 2 We have given O(n 3 ), O(n 2 ), O(nlogn) algorithms for the max sub-range problem. This time, a linear time algorithm! The idea is as follows:
Liang, Introduction to Java Programming, Seventh Edition, (c) 2009 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
Analysis and Design of Algorithms. According to math historians the true origin of the word algorism: comes from a famous Persian author named ál-Khâwrázmî.
Last Class: Graph Undirected and directed (digraph): G=, vertex and edge Adjacency matrix and adjacency linked list
Program Performance & Asymptotic Notations CSE, POSTECH.
Chapter 2.6 Comparison of Algorithms modified from Clifford A. Shaffer and George Bebis.
1 Chapter 24 Developing Efficient Algorithms. 2 Executing Time Suppose two algorithms perform the same task such as search (linear search vs. binary search)
{ CS203 Lecture 7 John Hurley Cal State LA. 2 Execution Time Suppose two algorithms perform the same task such as search (linear search vs. binary search)
Lecture 2 Computational Complexity
Algorithm Efficiency CS 110: Data Structures and Algorithms First Semester,
1 Computer Algorithms Lecture 3 Asymptotic Notation Some of these slides are courtesy of D. Plaisted, UNC and M. Nicolescu, UNR.
Design and Analysis of Algorithms - Chapter 21 Analysis of Algorithms b Issues: CorrectnessCorrectness Time efficiencyTime efficiency Space efficiencySpace.
J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 2. Relevant Mathematics: Classifying Functions Input Size Time f(i) = n  (n)
Analysis of Algorithm Efficiency Dr. Yingwu Zhu p5-11, p16-29, p43-53, p93-96.
CMPT 438 Algorithms. Why Study Algorithms? Necessary in any computer programming problem ▫Improve algorithm efficiency: run faster, process more data,
DR. Gatot F. Hertono, MSc. Design and Analysis of ALGORITHM (Session 2)
Fundamentals of Algorithms MCS - 2 Lecture # 8. Growth of Functions.
Design and Analysis of Algorithms Chapter Asymptotic Notations* Dr. Ying Lu August 30, RAIK.
The Time Complexity of an Algorithm Specifies how the running time depends on the size of the input. CSE 3101Z Design and Analysis of Algorithms.
CMSC 341 Asymptotic Analysis. 2 Complexity How many resources will it take to solve a problem of a given size? –time –space Expressed as a function of.
Algorithmic Analysis Charl du Plessis and Robert Ketteringham.
Chapter 9 Efficiency of Algorithms. 9.3 Efficiency of Algorithms.
1 Algorithms  Algorithms are simply a list of steps required to solve some particular problem  They are designed as abstractions of processes carried.
2-0 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Theoretical.
Algorithm Analysis (Big O)
UNIT-I FUNDAMENTALS OF THE ANALYSIS OF ALGORITHM EFFICIENCY ANALYSIS AND DESIGN OF ALGORITHMS CHAPTER 2:
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Chapter.
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 2 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.
E.G.M. PetrakisAlgorithm Analysis1  Algorithms that are equally correct can vary in their utilization of computational resources  time and memory  a.
Lecture 4 Jianjun Hu Department of Computer Science and Engineerintg University of South Carolina CSCE350 Algorithms and Data Structure.
BITS Pilani Pilani Campus Data Structure and Algorithms Design Dr. Maheswari Karthikeyan Lecture1.
GC 211:Data Structures Week 2: Algorithm Analysis Tools Slides are borrowed from Mr. Mohammad Alqahtani.
LECTURE 2 : fundamentals of analysis of algorithm efficiency Introduction to design and analysis algorithm 1.
Algorithmic Foundations COMP108 COMP108 Algorithmic Foundations Algorithm efficiency Prudence Wong.
Algorithmic Foundations COMP108 COMP108 Algorithmic Foundations Algorithm efficiency Prudence Wong
Lecture 3COMPSCI.220.S1.T Running Time: Estimation Rules Running time is proportional to the most significant term in T(n) Once a problem size.
Data Structures I (CPCS-204) Week # 2: Algorithm Analysis tools Dr. Omar Batarfi Dr. Yahya Dahab Dr. Imtiaz Khan.
Algorithm Analysis 1.
Theoretical analysis of time efficiency
Analysis of Algorithms
Analysis of algorithms
Analysis of algorithms
Thinking about Algorithms Abstractly
Introduction to Algorithms
Analysis of algorithms
Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency
Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency
Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency
Analysis of algorithms
Chapter 2.
Fundamentals of the Analysis of Algorithm Efficiency
Analysis of algorithms
Fundamentals of the Analysis of Algorithm Efficiency
Presentation transcript:

Design and Analysis of Algorithms Chapter Analysis of Algorithms Dr. Ying Lu August 28, RAIK 283: Data Structures & Algorithms

Design and Analysis of Algorithms Chapter b Giving credit where credit is due: Most of the lecture notes are based on the slides from the Textbook’s companion websiteMost of the lecture notes are based on the slides from the Textbook’s companion website Several slides are from Jeff Edmonds of the York UniversitySeveral slides are from Jeff Edmonds of the York University I have modified them and added new slidesI have modified them and added new slides RAIK 283: Data Structures & Algorithms

The Time Complexity of an Algorithm

Specifies how the running time depends on the size of the input

Design and Analysis of Algorithms Chapter Purpose

6 Purpose b To estimate how long a program will run. b To estimate the largest input that can reasonably be given to the program. b To compare the efficiency of different algorithms. b To help focus on the parts of code that are executed the largest number of times. b To choose an algorithm for an application.

Design and Analysis of Algorithms Chapter Purpose (Example) b Suppose a machine that performs a million floating- point operations per second (10 6 FLOPS), then how long an algorithm will run for an input of size n=50? 1) If the algorithm requires n 2 such operations:1) If the algorithm requires n 2 such operations:

Design and Analysis of Algorithms Chapter Purpose (Example) b Suppose a machine that performs a million floating- point operations per second (10 6 FLOPS), then how long an algorithm will run for an input of size n=50? 1) If the algorithm requires n 2 such operations:1) If the algorithm requires n 2 such operations: – second

Design and Analysis of Algorithms Chapter Purpose (Example) b Suppose a machine that performs a million floating- point operations per second (10 6 FLOPS), then how long an algorithm will run for an input of size n=50? 1) If the algorithm requires n 2 such operations:1) If the algorithm requires n 2 such operations: – second 2) If the algorithm requires 2 n such operations:2) If the algorithm requires 2 n such operations: –A) Takes a similar amount of time (t < 1 sec) –B) Takes a little bit longer time (1 sec < t < 1 year) –C) Takes a much longer time (1 year < t)

Design and Analysis of Algorithms Chapter Purpose (Example) b Suppose a machine that performs a million floating- point operations per second (10 6 FLOPS), then how long an algorithm will run for an input of size n=50? 1) If the algorithm requires n 2 such operations:1) If the algorithm requires n 2 such operations: – second 2) If the algorithm requires 2 n such operations:2) If the algorithm requires 2 n such operations: –over 35 years!

Design and Analysis of Algorithms Chapter Time Complexity Is a Function Specifies how the running time depends on the size of the input. A function mapping “size” of input “time” T(n) executed.

Design and Analysis of Algorithms Chapter Definition of Time?

Design and Analysis of Algorithms Chapter Definition of Time b # of seconds (machine, implementation dependent). b # lines of code executed. b # of times a specific operation is performed (e.g., addition).

Design and Analysis of Algorithms Chapter Theoretical analysis of time efficiency Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size b Basic operation: the operation that contributes most towards the running time of the algorithm. T(n) ≈ c op C(n) T(n) ≈ c op C(n) running time execution time for basic operation Number of times basic operation is executed input size

Design and Analysis of Algorithms Chapter Input size and basic operation examples Problem Input size measure Basic operation Search for key in a list of n items Multiply two matrices of floating point numbers Compute a n Graph problem

Design and Analysis of Algorithms Chapter Input size and basic operation examples Problem Input size measure Basic operation Search for key in a list of n items Number of items in the list: n Key comparison Multiply two matrices of floating point numbers Compute a n Graph problem

Design and Analysis of Algorithms Chapter Input size and basic operation examples Problem Input size measure Basic operation Search for key in a list of n items Number of items in the list: n Key comparison Multiply two matrices of floating point numbers Dimensions of matrices Floating point multiplication Compute a n Graph problem

Design and Analysis of Algorithms Chapter Input size and basic operation examples Problem Input size measure Basic operation Search for key in list of n items Number of items in list n Key comparison Multiply two matrices of floating point numbers Dimensions of matrices Floating point multiplication Compute a n n Floating point multiplication Graph problem

Design and Analysis of Algorithms Chapter Input size and basic operation examples Problem Input size measure Basic operation Search for key in list of n items Number of items in list n Key comparison Multiply two matrices of floating point numbers Dimensions of matrices Floating point multiplication Compute a n n Floating point multiplication Graph problem #vertices and/or edges Visiting a vertex or traversing an edge

Design and Analysis of Algorithms Chapter Theoretical analysis of time efficiency Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size

Design and Analysis of Algorithms Chapter Which Input of size n? Efficiency also depends on the particular input b For instance: search a key in a list of n letters Problem input: a list of n lettersProblem input: a list of n letters How many different inputs?How many different inputs?

Design and Analysis of Algorithms Chapter Which Input of size n? Efficiency also depends on the particular input For instance: search a key in a list of n letters There are 26 n inputs of size n. Which do we consider for the time efficiency C(n)?

Design and Analysis of Algorithms Chapter Best-case, average-case, worst-case b Worst case: W(n) – maximum over inputs of size n b Best case: B(n) – minimum over inputs of size n b Average case: A(n) – “average” over inputs of size n NOT the average of worst and best caseNOT the average of worst and best case Under some assumption about the probability distribution of all possible inputs of size n, calculate the weighted sum of expected C(n) (numbers of basic operation repetitions) over all possible inputs of size n.Under some assumption about the probability distribution of all possible inputs of size n, calculate the weighted sum of expected C(n) (numbers of basic operation repetitions) over all possible inputs of size n.

Design and Analysis of Algorithms Chapter Example: Sequential search b Problem: Given a list of n elements and a search key K, find an element equal to K, if any. b Algorithm: Scan the list and compare its successive elements with K until either a matching element is found (successful search) or the list is exhausted (unsuccessful search) b Worst case b Best case b Average case

Design and Analysis of Algorithms Chapter An example b Compute gcd(m, n) by applying the algorithm based on checking consecutive integers from min(m, n) down to gcd(m, n) b Input size? b Best case? b Worst case? b Average case?

Design and Analysis of Algorithms Chapter Types of formulas for basic operation count b Exact formula e.g., C(n) = n(n-1)/2 e.g., C(n) = n(n-1)/2 b Formula indicating order of growth with specific multiplicative constant e.g., C(n) ≈ 0.5 n 2 e.g., C(n) ≈ 0.5 n 2 b Formula indicating order of growth with unknown multiplicative constant e.g., C(n) ≈ cn 2 e.g., C(n) ≈ cn 2

Design and Analysis of Algorithms Chapter Types of formulas for basic operation count b Exact formula e.g., C(n) = n(n-1)/2 e.g., C(n) = n(n-1)/2 b Formula indicating order of growth with specific multiplicative constant e.g., C(n) ≈ 0.5 n 2 e.g., C(n) ≈ 0.5 n 2 b Formula indicating order of growth with unknown multiplicative constant e.g., C(n) ≈ cn 2 e.g., C(n) ≈ cn 2 b Most important: Order of growth within a constant multiple as n→∞

Design and Analysis of Algorithms Chapter Asymptotic growth rate b A way of comparing functions that ignores constant factors and small input sizes b O(g(n)): b Θ (g(n)): b Ω(g(n)):

Design and Analysis of Algorithms Chapter Asymptotic growth rate b A way of comparing functions that ignores constant factors and small input sizes b O(g(n)): class of functions f(n) that grow no faster than g(n) b Θ (g(n)): class of functions f(n) that grow at same rate as g(n) b Ω(g(n)): class of functions f(n) that grow at least as fast as g(n)

Design and Analysis of Algorithms Chapter Table 2.1

Classifying Functions Giving an idea of how fast a function grows without going into too much detail.

Design and Analysis of Algorithms Chapter Which are more alike?

Design and Analysis of Algorithms Chapter Which are more alike? Mammals

Design and Analysis of Algorithms Chapter Which are more alike?

Design and Analysis of Algorithms Chapter Which are more alike? Dogs

Design and Analysis of Algorithms Chapter Classifying Animals Vertebrates BirdsMammals Reptiles Fish Dogs Giraffe

Design and Analysis of Algorithms Chapter Which are more alike? n 1000 n2n2 2n2n

Design and Analysis of Algorithms Chapter Which are more alike? Polynomials n 1000 n2n2 2n2n

Design and Analysis of Algorithms Chapter Which are more alike? 1000n 2 3n 2 2n 3

Design and Analysis of Algorithms Chapter Which are more alike? Quadratic 1000n 2 3n 2 2n 3

Design and Analysis of Algorithms Chapter Classifying Functions? Functions

Design and Analysis of Algorithms Chapter Classifying Functions Functions Constant Logarithmic Poly Logarithmic Polynomial Exponential Factorial (log n) 5 n5n5 2 5n 5 log n5 n!

Design and Analysis of Algorithms Chapter Classifying Functions? Polynomial

Design and Analysis of Algorithms Chapter Classifying Functions Polynomial LinearQuadratic Cubic ? 5n 2 5n 5n 3 5n 4

Design and Analysis of Algorithms Chapter Logarithmic b log 10 n = # digits to write n b log 2 n = # bits to write n = 3.32 log 10 n b log(n 1000 ) = 1000 log(n) Differ only by a multiplicative constant

Design and Analysis of Algorithms Chapter Poly Logarithmic (log n) 5 = log 5 n

Design and Analysis of Algorithms Chapter Which grows faster? log 1000 n n 0.001

Design and Analysis of Algorithms Chapter Poly Logarithmic << Polynomial For sufficiently large n log 1000 n << n 0.001

Design and Analysis of Algorithms Chapter Which grows faster? n n 2

Design and Analysis of Algorithms Chapter Linear << Quadratic For sufficiently large n n << n 2

Design and Analysis of Algorithms Chapter Which grows faster? n n

Design and Analysis of Algorithms Chapter Polynomial << Exponential For sufficiently large n n 1000 << n

Design and Analysis of Algorithms Chapter Ordering Functions Functions Constant Logarithmic Poly Logarithmic Polynomial Exponential (log n) 5 n5n5 2 5n 5 log n5 << Factorial n!

Design and Analysis of Algorithms Chapter Which Functions are Constant? 5 1,000,000,000, sin(n)

Design and Analysis of Algorithms Chapter Which Functions are Constant? 5 1,000,000,000, sin(n) Yes No Yes

Design and Analysis of Algorithms Chapter Which Functions are “Constant”? 5 1,000,000,000, sin(n) The running time of the algorithm is a “Constant” if it does not depend significantly on the size of the input.

Design and Analysis of Algorithms Chapter Which Functions are “Constant”? 5 1,000,000,000, sin(n) Yes No Yes Lie in between 7 9 The running time of the algorithm is a “Constant” It does not depend significantly on the size of the input.

Design and Analysis of Algorithms Chapter Which Functions are Quadratic? n n n 2 5n 2 + 3n + 2log n

Design and Analysis of Algorithms Chapter Which Functions are Quadratic? n n n 2 5n 2 + 3n + 2log n Lie in between

Design and Analysis of Algorithms Chapter Which Functions are Quadratic? n n n 2 5n 2 + 3n + 2log n Ignore low-order terms Ignore multiplicative constants. Ignore "small" values of n. Write θ(n 2 ).

Design and Analysis of Algorithms Chapter Examples f(n)g(n)O(g(n))?Ω(g(n))?Θ(g(n))? 1)ln 2 nn  2)nknk cncn 3)n sinn 4)2n2n 2 n/2 5)n lgc c lgn 6)lg(n!)lg(n n )