3.3 Complexity of Algorithms

Slides:



Advertisements
Similar presentations
Growth-rate Functions
Advertisements

College of Information Technology & Design
MATH 224 – Discrete Mathematics
Discrete Structures CISC 2315
CompSci 102 Discrete Math for Computer Science
Computational Complexity 1. Time Complexity 2. Space Complexity.
Wednesday, 11/25/02, Slide #1 CS 106 Intro to CS 1 Wednesday, 11/25/02  QUESTIONS??  Today:  More on sorting. Advanced sorting algorithms.  Complexity:
Analysis of Algorithms. Time and space To analyze an algorithm means: –developing a formula for predicting how fast an algorithm is, based on the size.
© 2006 Pearson Addison-Wesley. All rights reserved6-1 More on Recursion.
CSE115/ENGR160 Discrete Mathematics 03/10/11
Algorithm Efficiency and Sorting Bina Ramamurthy CSE116A,B.
Data Structure Algorithm Analysis TA: Abbas Sarraf
Analysis of Algorithm.
1 Section 2.3 Complexity of Algorithms. 2 Computational Complexity Measure of algorithm efficiency in terms of: –Time: how long it takes computer to solve.
Algorithm Analysis (Big O)
February 17, 2015Applied Discrete Mathematics Week 3: Algorithms 1 Double Summations Table 2 in 4 th Edition: Section th Edition: Section th.
COMP s1 Computing 2 Complexity
1 Complexity Lecture Ref. Handout p
Time Complexity Dr. Jicheng Fu Department of Computer Science University of Central Oklahoma.
CSCI 2670 Introduction to Theory of Computing November 10, 2005.
Chapter Complexity of Algorithms –Time Complexity –Understanding the complexity of Algorithms 1.
Week 2 CS 361: Advanced Data Structures and Algorithms
Discrete Mathematics Algorithms. Introduction  An algorithm is a finite set of instructions with the following characteristics:  Precision: steps are.
1 Chapter 24 Developing Efficient Algorithms. 2 Executing Time Suppose two algorithms perform the same task such as search (linear search vs. binary search)
Algorithm Analysis. Algorithm An algorithm is a clearly specified set of instructions which, when followed, solves a problem. recipes directions for putting.
1.2. Comparing Algorithms. Learning outcomes Understand that algorithms can be compared by expressing their complexity as a function relative to the size.
(C) 2010 Pearson Education, Inc. All rights reserved. Java How to Program, 8/e.
Analysis of Algorithms
Chapter 3 Sec 3.3 With Question/Answer Animations 1.
Algorithm Evaluation. What’s an algorithm? a clearly specified set of simple instructions to be followed to solve a problem a way of doing something What.
Searching. RHS – SOC 2 Searching A magic trick: –Let a person secretly choose a random number between 1 and 1000 –Announce that you can guess the number.
Ch 18 – Big-O Notation: Sorting & Searching Efficiencies Our interest in the efficiency of an algorithm is based on solving problems of large size. If.
Big Oh Algorithms are compared to each other by expressing their efficiency in big-oh notation Big O notation is used in Computer Science to describe the.
Fall 2002CMSC Discrete Structures1 Enough Mathematical Appetizers! Let us look at something more interesting: Algorithms.
MCA-2012Data Structure1 Algorithms Rizwan Rehman CCS, DU.
Asymptotic Notation (O, Ω, )
Starting Out with C++ Early Objects Seventh Edition by Tony Gaddis, Judy Walters, and Godfrey Muganda Modified for use by MSU Dept. of Computer Science.
Data Structure Introduction.
New Mexico Computer Science For All Algorithm Analysis Maureen Psaila-Dombrowski.
Counting Discrete Mathematics. Basic Counting Principles Counting problems are of the following kind: “How many different 8-letter passwords are there?”
1 Algorithms  Algorithms are simply a list of steps required to solve some particular problem  They are designed as abstractions of processes carried.
Sorting and Searching by Dr P.Padmanabham Professor (CSE)&Director
Module #7: Algorithmic Complexity Rosen 5 th ed., §2.3.
Liang, Introduction to Java Programming, Sixth Edition, (c) 2007 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
27-Jan-16 Analysis of Algorithms. 2 Time and space To analyze an algorithm means: developing a formula for predicting how fast an algorithm is, based.
تصميم وتحليل الخوارزميات عال311 Chapter 3 Growth of Functions
Searching Topics Sequential Search Binary Search.
Copyright © 2014 Curt Hill Algorithm Analysis How Do We Determine the Complexity of Algorithms.
Concepts of Algorithms CSC-244 Unit 3 and 4 Algorithm Growth Rates Shahid Iqbal Lone Computer College Qassim University K.S.A.
Complexity of Algorithms Fundamental Data Structures and Algorithms Ananda Guna January 13, 2005.
Ch03-Algorithms 1. Algorithms What is an algorithm? An algorithm is a finite set of precise instructions for performing a computation or for solving a.
Chapter 3 Chapter Summary  Algorithms o Example Algorithms searching for an element in a list sorting a list so its elements are in some prescribed.
Section 1.7 Comparing Algorithms: Big-O Analysis.
GC 211:Data Structures Week 2: Algorithm Analysis Tools Slides are borrowed from Mr. Mohammad Alqahtani.
1 Algorithms Searching and Sorting Algorithm Efficiency.
Discrete Mathematics Chapter 2 The Fundamentals : Algorithms, the Integers, and Matrices. 大葉大學 資訊工程系 黃鈴玲.
Data Structures I (CPCS-204) Week # 2: Algorithm Analysis tools Dr. Omar Batarfi Dr. Yahya Dahab Dr. Imtiaz Khan.
Applied Discrete Mathematics Week 2: Functions and Sequences
Analysis of Algorithms
CSE15 Discrete Mathematics 03/13/17
Lesson Objectives Aims Understand the following: The big O notation.
Teach A level Computing: Algorithms and Data Structures
Enough Mathematical Appetizers!
Computation.
CS 2210 Discrete Structures Algorithms and Complexity
Applied Discrete Mathematics Week 6: Computation
Discrete Mathematics CS 2610
Algorithmic Complexity
At the end of this session, learner will be able to:
CS 2210 Discrete Structures Algorithms and Complexity
Presentation transcript:

3.3 Complexity of Algorithms Linear Search Algorithm Determine if 3 is in the following lists A= { 1 4 8 -1 2 } B= { 3 4 8 6 5 } C= { 1 2 4 9 10 11 12 14 19 10 -11 0 } D= { 1 3 2 4 } For each list ( problem ), how much time will the algorithm take to finish the problem ?

3.3 Complexity of Algorithms Recall that the linear search algorithm go through the list from the beginning of the list until reaching the number ( 3 ) or the end of list. Procedure linear search (x: integer a1, a2, . . . , an ) i=1; while ( i <= n and x =/= ai ) i := i+1 if i <= n, then location := i else location := 0

3.3 Complexity of Algorithms A= { 1 4 8 -1 2 } ( exist at the end of the list) B= { 3 4 8 6 5 } (exist at the beginning of the list) C= { 1 2 4 9 10 11 12 14 19 10 -11 0 } D= { 1 3 2 4 } (exit after comparing two elements) List B will finish before lists A, B and C, even though list B is longer than D. List C will take the longest to finish.

3.3 Complexity of Algorithms B= { 3 4 8 6 5 } C= { 1 2 4 9 10 11 12 14 19 10 -11 0 } D= { 1 3 2 4 } Clearly, the time the algorithm takes to finish Depends on the size of the problem (list). Even for problems of the same size, there are worst-case, best case, and average case.

3.3 Complexity of Algorithms Time Complexity: Analysis of the time required to solve a problem of a particular size. Space Complexity: Analysis of the computer memory required to solve a problem of a particular size.

3.3 Complexity of Algorithms Describe the (worst-case) time complexity of the linear search algorithm. Procedure linear search (x: integer a1, a2, . . . , an ) i=1; while ( i <= n and x =/= ai ) i := i+1 if i <= n, then location := i else location := 0 The time complexity depends on the number of comparisons because they are the basic operations used. So how many comparisons are there if (in the worst-case) you have to go to the end of the list?

3.3 Complexity of Algorithms At each step of the loop, there are two comparisons. Outside the loop, there is one more comparison. Therefore, if the list has n elements, we will have 2 n + 1 comparisons. In our machine, if each comparison takes k seconds (say, k = 0.00001), then for a list of size n, we’d expect that the algorithm to finish in k* (2 n + 1) seconds. This result is linear (degree one) in n. If list A is twice as long as list B, it will take twice as long to finish A than B. The complexity (in this case) is denote as O(n) (big-O of n, we will discuss this next week 3.2).

3.3 Complexity of Algorithms Note that we will never know the exact time the algorithm will take to finish a given problem (that depends also on the machine you use). The main issue addressed by time complexity analysis is how does the algorithm’s performance scale with the size of the problem. The previous example O(n) shows a linear time complexity. There are other examples, such as O(n2), O( log n)….. If the time complexity is O(log n), you have to square n to double the running time (if you just double n, the difference in running time is almost negligible).

3.3 Complexity of Algorithms How about average complexity? Definition: the average number of operations used to solve the problem over all inputs of a given size. The time depends on where the element is located in the list. So given a list of size n. If x is in the first position in the list, it will take 3 comparisons. If it is in the second position, it will take 5 comparisons ….. So average is ( 3 + 5 + … + (2n+1) )/n = (2 ( 1+2+ … +n) +n)/n = n+2. Again, it depends linearly on n.

3.3 Complexity of Algorithms What is the worst-case complexity of bubble sort ?

3.3 Complexity of Algorithms Example { 1 –11 50 6 8 –1} Use Bubble Sort (sort in increasing order} After first pass {-11 1 6 8 –1 50} (5 = 6 -1 comparisons) After Second Pass {-11 1 6 -1 8 50} (4 = 5 -1 comparisons) After Third Pass {-11 1 -1 6 8 50} (3 = 4 - 1 comparisons) After Fourth Pass {-11 -1 1 6 8 50} (2 = 3 – 1 comparisons) After Fifth Pass {-11 -1 1 6 8 50} (1 = 2 – 1 comparisons) Done This list has 6 elements and there are 5 + 4 + 3 + 2 + 1 comparisons.

3.3 Complexity of Algorithms In general, if the list has n elements, we will have to do (n-1) + (n-2) …. + 2 +1 = (n-1) n / 2 comparisons. Notice that the complexity here depends quadratically (degree-2) in n. So if list A is twice as long as list B, it will take four times longer to process A than B.

3.3 Complexity of Algorithms How about insertion sort?

3.3 Complexity of Algorithms Example { 1 –11 50 6 8 –1} Use Insertion Sort (sort in increasing order} After first pass {-11 1 50 6 8 -1} ( 1 comparison) After Second Pass {-11 1 50 6 8 -1} ( 2 comparisons) After Third Pass {--11 1 6 50 8 -1} ( 3 comparisons) After Fourth Pass {- 11 1 6 8 50 -1} ( 4 comparisons) After Fifth Pass {-11 -1 1 6 8 50} ( 2 comparisons, worst-case 5 comparisons) Done

3.3 Complexity of Algorithms In general, if the list has n elements, we will have to do 1+ 2+ … + (n-1) + (n-2) = (n-1) n / 2 comparisons (the result here is slightly different from the book). Notice that the complexity here depends quadratically (degree-2) in n. So if list A is twice as long as list B, it will take four times longer to process A than B.

Complexity Classes Linear search algorithm has linear complexity θ ( n ). Bubble sort has quadratic complexity θ ( n2 ). Binary search algorithm has logarithmic complexity θ ( log n ). The big-O and big- θ provide a language of describing the (time) complexity of algorithms.

Complexity Classes θ ( 1 ) Constant Complexity For problems with input of size n, the number of operations required by the algorithms. θ ( 1 ) Constant Complexity θ ( log n) Logarithmic Complexity θ ( n ) Linear Complexity θ ( n log n) n log n Complexity θ ( nb ) Polynomial Complexity θ ( bn ) Exponential Complexity θ ( n! ) Factorial Complexity

Complexity Classes How do we known if a given type of problems is too difficult to solve with computers? Of course, some problems simply cannot be solved. A problem that is solvable using an algorithm with polynomial worst-case complexity is called tractable. algorithm will produce the solution to the problem for reasonably sized input in a relative short time. It is intractable if it cannot be solved using an algorithm with worst-case polynomial time complexity. Even for a small input, the algorithm will not produce the solution in a relative short time.