DATA STRUCTURES AND ALGORITHMS Lecture Notes 1 Prepared by İnanç TAHRALI.

Slides:



Advertisements
Similar presentations
Discrete Structures CISC 2315
Advertisements

Lecture: Algorithmic complexity
 The running time of an algorithm as input size approaches infinity is called the asymptotic running time  We study different notations for asymptotic.
CSE 373: Data Structures and Algorithms Lecture 5: Math Review/Asymptotic Analysis III 1.
CSE332: Data Abstractions Lecture 2: Math Review; Algorithm Analysis Tyler Robison Summer
Asymptotic Growth Rate
Cutler/HeadGrowth of Functions 1 Asymptotic Growth Rate.
Introduction to Analysis of Algorithms
Analysis of Algorithms Review COMP171 Fall 2005 Adapted from Notes of S. Sarkar of UPenn, Skiena of Stony Brook, etc.
Asymptotic Analysis Motivation Definitions Common complexity functions
Data Structures, Spring 2004 © L. Joskowicz 1 Data Structures – LECTURE 2 Elements of complexity analysis Performance and efficiency Motivation: analysis.
Chapter 2: Fundamentals of the Analysis of Algorithm Efficiency
Tirgul 2 Asymptotic Analysis. Motivation: Suppose you want to evaluate two programs according to their run-time for inputs of size n. The first has run-time.
CHAPTER 2 ANALYSIS OF ALGORITHMS Part 1. 2 Big Oh and other notations Introduction Classifying functions by their asymptotic growth Theta, Little oh,
Analysis of Algorithms COMP171 Fall Analysis of Algorithms / Slide 2 Introduction * What is Algorithm? n a clearly specified set of simple instructions.
CSE 373 Data Structures and Algorithms Lecture 4: Asymptotic Analysis II / Math Review.
Instructor Neelima Gupta
Asymptotic Notations Iterative Algorithms and their analysis
Algorithm Efficiency CS 110: Data Structures and Algorithms First Semester,
Iterative Algorithm Analysis & Asymptotic Notations
2IL50 Data Structures Fall 2015 Lecture 2: Analysis of Algorithms.
CS 221 Analysis of Algorithms Instructor: Don McLaughlin.
1 COMP3040 Tutorial 1 Analysis of algorithms. 2 Outline Motivation Analysis of algorithms Examples Practice questions.
Analyzing algorithms & Asymptotic Notation BIO/CS 471 – Algorithms for Bioinformatics.
CSCI 3160 Design and Analysis of Algorithms Tutorial 1
Asymptotic Analysis-Ch. 3
Algorithms Growth of Functions. Some Notation NNatural numbers RReal numbers N + Positive natural numbers R + Positive real numbers R * Non-negative real.
Program Efficiency & Complexity Analysis. Algorithm Review An algorithm is a definite procedure for solving a problem in finite number of steps Algorithm.
Algorithm Analysis CS 400/600 – Data Structures. Algorithm Analysis2 Abstract Data Types Abstract Data Type (ADT): a definition for a data type solely.
Asymptotic Growth Rates  Themes  Analyzing the cost of programs  Ignoring constants and Big-Oh  Recurrence Relations & Sums  Divide and Conquer 
CE 221 Data Structures and Algorithms Chapter 2: Algorithm Analysis - I Text: Read Weiss, §2.1 – Izmir University of Economics.
DATA STRUCTURES AND ALGORITHMS Lecture Notes 2 Prepared by İnanç TAHRALI.
Algorithm Analysis Part of slides are borrowed from UST.
Chapter 2 Computational Complexity. Computational Complexity Compares growth of two functions Independent of constant multipliers and lower-order effects.
Data Structures and Algorithms IT12112
Algorithm Analysis Algorithm Analysis Lectures 3 & 4 Resources Data Structures & Algorithms Analysis in C++ (MAW): Chap. 2 Introduction to Algorithms (Cormen,
1/6/20161 CS 3343: Analysis of Algorithms Lecture 2: Asymptotic Notations.
Spring 2015 Lecture 2: Analysis of Algorithms
E.G.M. PetrakisAlgorithm Analysis1  Algorithms that are equally correct can vary in their utilization of computational resources  time and memory  a.
Vishnu Kotrajaras, PhD.1 Data Structures
CSE 421 Algorithms Richard Anderson Winter 2009 Lecture 4.
1 Chapter 2 Algorithm Analysis All sections. 2 Complexity Analysis Measures efficiency (time and memory) of algorithms and programs –Can be used for the.
DR. Gatot F. Hertono, MSc. Design and Analysis of ALGORITHM (Session 2)
1 Chapter 2 Algorithm Analysis Reading: Chapter 2.
Analysis of Algorithms Spring 2016CS202 - Fundamentals of Computer Science II1.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 2.
Chapter 2 Algorithm Analysis
Analysis of Algorithms
Analysis of Algorithms
CS 3343: Analysis of Algorithms
Time Complexity Analysis Neil Tang 01/19/2010
Algorithm Analysis Neil Tang 01/22/2008
Algorithm Analysis Lectures 3 & 4
Asymptotic Growth Rate
Analysis of Algorithms
BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS
Chapter 2: Fundamentals of the Analysis of Algorithm Efficiency
Analysis of Algorithms
Analysis of Algorithms
Advanced Analysis of Algorithms
Chapter 2.
CE 221 Data Structures and Algorithms
At the end of this session, learner will be able to:
Advanced Analysis of Algorithms
Estimating Algorithm Performance
Big Omega, Theta Defn: T(N) = (g(N)) if there are positive constants c and n0 such that T(N)  c g(N) for all N  n0 . Lingo: “T(N) grows no slower than.
Analysis of Algorithms
Richard Anderson Autumn 2015 Lecture 4
Analysis of Algorithms
Presentation transcript:

DATA STRUCTURES AND ALGORITHMS Lecture Notes 1 Prepared by İnanç TAHRALI

2 ROAD MAP What is an algorithm ? What is a data structure ? Analysis of An Algorithm Asymptotic Notations –Big Oh Notation –Omega Notation –Theta Notation –Little o Notation Rules about Asymptotic Notations

3 What is An Algorithm ? Definition : A finite, clearly specified sequence of instructions to be followed to solve a problem.

4 What is An Algorithm ? int Sum (int N) PartialSum  0 i  1 foreach (i > 0) and (i<=N) PartialSum  PartialSum + (i*i*i) increase i with 1 return value of PartialSum int Sum (int N) { int PartialSum = 0 ; for (int i=1; i<=N; i++) PartialSum += i * i * i; return PartialSum; } Problem : Write a program to calculate

5 Properties of an Algorithm Effectiveness –simple –can be carried out by pen and paper Definiteness –clear –meaning is unique Correctness –give the right answer for all possible cases Finiteness –stop in reasonable time

6 Is function F an algorithm? int F (int m) { n = m; while (n>1) { if (n/2 == 0) n=n/2; else n=3*n+1; } return m; }

7 What is a Data Structure ? Definition : An organization and representation of data –representation data can be stored variously according to their type –signed, unsigned, etc. example : integer representation in memory –organization the way of storing data changes according to the organization –ordered, inordered, tree example : if you have more then one integer ?

8 Properties of a Data Structure ? Efficient utilization of medium Efficient algorithms for –creation –manipulation (insertion/deletion) –data retrieval (Find) A well-designed data structure allows using little –resources –execution time –memory space

9 The Process of Algorithm Development Design –divide&conquer, greedy, dynamic programming Validation –check whether it is correct Analysis –determine the properties of algorithm Implementation Testing –check whether it works for all possible cases

10 Analysis of Algorithm Analysis investigates –What are the properties of the algorithm? in terms of time and space –How good is the algorithm ? according to the properties –How it compares with others? not always exact –Is it the best that can be done? difficult !

11 Mathematical Background Assume the functions for running times of two algorthms are found ! For input size N Running time of Algorithm A = T A (N) = 1000 N Running time of Algorithm B = T B (N) = N 2 Which one is faster ?

12 NTATA TBTB sec10 -4 sec sec10 -2 sec sec sec100 sec sec10000 sec If the unit of running time of algorithms A and B is µsec Mathematical Background So which algorithm is faster ?

13 Mathematical Background If N T B (N) o/wT B (N) > T A (N) Compare their relative growth ?

14 Mathematical Background Is it always possible to have definite results? NO ! The running times of algorithms can change because of the platform, the properties of the computer, etc. We use asymptotic notations (O, Ω, θ, o) compare relative growth compare only algorithms

15 Big Oh Notation (O) Provides an “upper bound” for the function f Definition : T(N) = O (f(N)) if there are positive constants c and n 0 such that T(N) ≤ cf(N) when N ≥ n 0 –T(N) grows no faster than f(N) –growth rate of T(N) is less than or equal to growth rate of f(N) for large N –f(N) is an upper bound on T(N) not fully correct !

16 Big Oh Notation (O) Analysis of Algorithm A 1000 N ≤ cN if c= 2000 and n 0 = 1 for all N is right

17 Examples 7n+5 = O(n) for c=8 and n 0 =5 7n+5 ≤ 8nn>5 = n 0 7n+5 = O(n 2 ) for c=7 and n 0 =2 7n+5 ≤ 7n 2 n≥n 0 7n 2 +3n = O(n) ?

18 Advantages of O Notation It is possible to compare of two algorithms with running times Constants can be ignored. –Units are not important O(7n 2 ) = O(n 2 ) Lower order terms are ignored –O(n 3 +7n 2 +3) = O(n 3 )

19 Running Times of Algorithm A and B T A (N) = 1000 N = O(N) T B (N) = N 2 = O(N 2 ) A is asymptotically faster than B !

20 Omega Notation (Ω) Definition : T(N) = Ω (f(N)) if there are positive constants c and n 0 such that T(N) ≥ c f(N) when N≥ n 0 –T(N) grows no slower than f(N) –growth rate of T(N) is greater than or equal to growth rate of f(N) for large N –f(N) is a lower bound on T(N) not fully correct !

21 Omega Notation Example: n 1/2 =  (lg n). for c = 1 and n 0 = 16 Let n > 16 c*(lg n) ≤ n 1/2

22 Omega Notation Theorem: f(N) = O(g(n)) g(n) = Ω(f(N)) Proof: f(N) ≤ c 1 g(n) g(n) ≥ c 2 f(N) divide the left side with c 1 1/c 1 f(N) ≤ g(n) g(n) ≥ c 2 f(N) if we choose c 2 as 1/c 1 then theorem is right.

23 Omega Notation 7n 2 + 3n + 5 = O(n 4 ) 7n 2 + 3n + 5 = O(n 3 ) 7n 2 + 3n + 5 = O(n 2 ) 7n 2 + 3n + 5 = Ω(n 2 ) 7n 2 + 3n + 5 = Ω(n) 7n 2 + 3n + 5 = Ω(1) n 2 and 7n 2 + 3n + 5 grows at the same rate 7n 2 + 3n + 5 = O(n 2 ) = Ω(n 2 ) = θ (n 2 )

24 Theta Notation (θ) Definition : T(N) = θ (h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N)) –T(N) grows as fast as h(N) –growth rate of T(N) and h(N) are equal for large N –h(N) is a tight bound on T(N) not fully correct !

25 Theta Notation (θ) Example : T(N) = 3N 2 T(N) = O(N 4 ) T(N) = O(N 3 ) T(N) = θ(N 2 )  best

26 Little O Notation (o) Definition : T(N) = o(p(N)) if T(N) = O(p(N)) and T(N)≠θ(p(N)) –f(N) grows strictly faster than T(N) –growth rate of T(N) is less than the growth rate of f(N) for large N –f(N) is an upperbound on T(N) (but not tight) not fully correct !

27 Little O Notation (o) Example : T(N) = 3N 2 T(N) = o(N 4 ) T(N) = o(N 3 ) T(N) = θ(N 2 )

28 RULES RULE 1: if T 1 (N) = O(f(N)) and T 2 (N) = O(g(N)) then a) T 1 (N) + T 2 (N) = max (O(f(N)), O(g(N))) b) T 1 (N) * T 2 (N) = O(f(N) * g(N)) You can prove these ? Is it true for θ notation ? What about Ω notation?

29 RULES RULE 2: if T(N) is a polynomial of degree k T(N) = a k N k + a k-1 N k-1 + … + a 1 N + a 0 then T(N) = θ(N k )

30 RULES RULE 3: log k N = o(N) for any constant k logarithm grows very slowly !

31 Some Common Functions c = o (log N) => c=O(log N) but c≠Ω(logN) log N = o(log 2 N) log 2 N = o(N) N = o(N log N) N = o (N 2 ) N 2 = o (N 3 ) N 3 = o (2 N )

32 Example : T(N) = 4N 2 T(N) = O(2N 2 ) correct but bad style T(N) = O(N 2 ) drop the constants T(N) = O(N 2 +N) correct but bad style T(N) = O(N 2 ) ignore low order terms

33 Another Way to Compute Growth Rates

34 Example : f(N) = 7N 2 g(N) = N 2 + N

35 Example : f(N) = N logNg(N) = N 1.5 compare logN with N 0.5 compare log 2 N with N compare log 2 N with o(N) N logN = o(N 1.5 )