Data Structures and Algorithms Introduction to Algorithms M. B. Fayek CUFE 2006

Agenda 1. Introduction to Algorithms 2. Algorithm Design Basics 3. Time Complexity of an Algorithm 4. Analysis using Asymptotic Notation 5. Basic Efficiency Classes

1. Introduction to Algorithms Why study Algorithms? Why study Algorithms? Heart of computer Heart of computer Promote analytical skills Promote analytical skills Donald Knuth: Donald Knuth: “ A person does not understand something until after teaching it to someone else. Actually: A person does not understand something until after he teaches it to a ……COMPUTER!”

1. Introduction to Algorithms What is an algorithm? What is an algorithm? “A sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.” “A sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.”

1. Introduction to Algorithms Algorithms are all around us in everyday life. Algorithms are all around us in everyday life. In the recipe of a cook book. In the recipe of a cook book. In assembling a toy. In assembling a toy. In setting the table. In setting the table. In preparing a cup of tea. In preparing a cup of tea. In calling your friend on the phone. In calling your friend on the phone. …. There are countless examples! …. There are countless examples!

Agenda 1. Introduction to Algorithms 2. Algorithm Design Basics 3. Time Complexity of an Algorithm 4. Analysis using Asymptotic Notation 5. Basic Efficiency Classes

2. Algorithm Design Basics Guidelines for Algorithm Designing and Analysis: Guidelines for Algorithm Designing and Analysis: 1. Understand the Problem (requirement analysis) 2. Select Data structure 3. Write Pseudo Code 4. Analyze Performance 5. Implement using suitable programming language 6. Test to resolve syntax and logic errors

2. Algorithm Design Basics Guidelines for Algorithm Designing and Analysis: Guidelines for Algorithm Designing and Analysis: 1. Understand the Problem (requirement analysis)  Gather data  Ask users  Carefully review any written requirements

2. Algorithm Design Basics Guidelines for Algorithm Designing and Analysis: Guidelines for Algorithm Designing and Analysis: 2. Select Data structure: To verify the appropriateness of the selected data structure:  Judge how well your data structure responds to user requirements (updates, questions)  Modify design as necessary

2. Algorithm Design Basics Guidelines for Algorithm Designing and Analysis: Guidelines for Algorithm Designing and Analysis: 3. Write Pseudo Code  Use pseudo code or flow chart  level of details of the pseudo code may vary

2. Algorithm Design Basics Guidelines for Algorithm Designing and Analysis: Guidelines for Algorithm Designing and Analysis: 4. Analyze Performance  Determine the feasibility of solution w.r.t. memory requirements, performance constraints … etc.  Manually review and validate pseudo code  Analyze the complexity of the algorithm using the big O notation to determine the complexity w.r.t. to time and storage.  Other criteria include: Clarity, Maintainability, Portability

2. Algorithm Design Basics Guidelines for Algorithm Designing and Analysis: Guidelines for Algorithm Designing and Analysis: 5. Implement using suitable programming language:  Is rather direct provided a good pseudo code is written, however erroneous mapping may occur!

2. Algorithm Design Basics Guidelines for Algorithm Designing and Analysis: Guidelines for Algorithm Designing and Analysis: 6. Test to resolve syntax and logic errors. Testing is divided into 2 main parts:  Trying to break the function of the program by entering unexpected data.  Debugging: It is concerned with finding out what is what caused the program to function in incorrectly.

Agenda 1. Introduction to Algorithms 2. Algorithm Design Basics 3. Time Complexity of an Algorithm 4. Analysis using Asymptotic Notation 5. Basic Efficiency Classes

3. Time Complexity of an Algorithm Time complexity is a main issue in evaluating an algorithm. Time complexity is a main issue in evaluating an algorithm. It reflects how the algorithm responds to the increase in data size (n) it handles, by measuring the corresponding increase in number of instructions to be performed. It reflects how the algorithm responds to the increase in data size (n) it handles, by measuring the corresponding increase in number of instructions to be performed. Time complexity is meant to classify algorithms into categories. Time complexity is meant to classify algorithms into categories.

3. Time Complexity of an Algorithm Steps to determine time complexity: Steps to determine time complexity: 1. Identify basic operations used for evaluating complexity. 2. Usually, loops and nested loops are the significant parts of a program. 3. One iteration of the loop is considered as a unit. It is then important to determine the order of magnitude of run time involved based on the number of iterations. 4. Parts concerned with initializations and reporting summary results are of secondary importance.

Example 1 Example1: count number of different chars in a file: // Input: string of characters // Output: array of 256 integers containing the count of the 256 chars // Output: array of 256 integers containing the count of the 256 chars For all 256 char do assign zero to counter end for while there are more chars do get next char increment count for this char end while N+1 Eof check + N count increment 257 assignment increment bound check Bookkeeping!

Example 1 Consider basic operation BOP = 1 Consider basic operation BOP = 1 Inc + 1 check for N= 500, integral BOP ≈ 500 for N= 500, integral BOP ≈ 500 bookkeeping ≈ 50 % For N =10000, BOP ≈ For N =10000, BOP ≈ bookkeeping = 7 % Total = 257 assign Inc check (bookkeeping) + N Inc + N+1 check (integral) Time grows linearly with n !

Example 2 Summing each of the rows of an N x N two- dimensional array A, storing row sums in a one-dimensional array Sum and the overall in GrandTotal. Summing each of the rows of an N x N two- dimensional array A, storing row sums in a one-dimensional array Sum and the overall in GrandTotal. // Input: 2-dimensional array A // Input: 2-dimensional array A // Output: 1- dimensional array Sum that contains the sum of 1 st, 2 nd,… row in 1 st, 2 nd, … element and GrandTotal that contains the total sum // Output: 1- dimensional array Sum that contains the sum of 1 st, 2 nd,… row in 1 st, 2 nd, … element and GrandTotal that contains the total sum

Example 2 First Algorithm : First Algorithm : GrandTotal = 0 For k = 1 to N do Sum [K] = 0 For J = 1 to N do Sum [K] = Sum [K] + A [K,J] GrandTotal = GrandTotal + A [K,J] EndForEndFor ► complexity = n 2 ! Basic Operation

Example 2 Second Algorithm : Second Algorithm : GrandTotal = 0 For k = 1 to N do Sum [K] = 0 For J = 1 to N do Sum [K] = Sum [K] + A [K,J] EndFor GrandTotal = GrandTotal + A [K,J] EndFor ► complexity = n 2 Basic Operation 1Basic Operation 2 ► complexity = n

Example 2 Total complexity= n 2 + n Total complexity= n 2 + n Hence Total Complexity mainly depends on n 2 Hence Total Complexity mainly depends on n 2 ►Note however that:  In the first approach the basic operation includes 2 additions  In the second approach each basic operation includes 1 addition  Hence, BOTH approaches are of order n 2, however, the second is better! …. Why?

Agenda 1. Introduction to Algorithms 2. Algorithm Design Basics 3. Time Complexity of an Algorithm 4. Analysis using Asymptotic Notation 5. Basic Efficiency Classes

4. Analysis using Asymptotic Notation Big Oh: Big Oh: If If Then there exists c such that Then there exists c such that

4. Analysis using Asymptotic Notation Find Big Oh Find Big Oh by finding f(n) such that for some real value

4. Analysis using Asymptotic Notation A Useful Property: A Useful Property: If If andThen

Agenda 1. Introduction to Algorithms 2. Algorithm Design Basics 3. Time Complexity of an Algorithm 4. Analysis using Asymptotic Notation 5. Basic Efficiency Classes

Common complexities (ordered in ascending order) are: Common complexities (ordered in ascending order) are:  log n nnnn  n log n n2n2n2n2 n3n3n3n3 2n2n2n2n Complexity

5. Basic Efficiency Classes Values of important asymptotic functions Values of important asymptotic functions

Example 1 Find the number of binary digits in the binary representation of a number Find the number of binary digits in the binary representation of a number // input: A is a positive number // output: number of bin digits that represent A count <-- 1 while n > 1 do count <-- count + 1 n <-- n/2 return count

Example 2: Element Uniqueness Element Uniqueness Problem: Element Uniqueness Problem: //Check whether all elements of array are unique //Check whether all elements of array are unique // input: array A[0,.. n-1] // input: array A[0,.. n-1] // output true or false // output true or false for i <-- 0 to n-2 do for j <-- i+1 to n-1 do if A[i] = A[j] return false return true ► Efficiency does not depend on n only! There is a best, worst and average case efficiency! There is a best, worst and average case efficiency! …… When do those cases occur? …… When do those cases occur?

Example 3: Fibonacci Numbers It is a series described by: It is a series described by: F i = F i-1 + F i-2 and F(0) =0, F(1) =1 // Input: A nonnegative integer n // Output: The n th Fibonacci number F[0] <-- 0; F[1] <-- 1 for i <-- 2 to n do F[i] <-- F[i-1] + F[i-2] return F[n] ►EXTRA ARRAY STORAGE, necessary?

Exercises Write an implementation of the fibonacci example without using and additional array! Write an implementation of the fibonacci example without using and additional array! Check if a number is prime or not Check if a number is prime or not Find all prime numbers in a specified range Find all prime numbers in a specified range Find the GCD of 2 numbers Find the GCD of 2 numbers

Website at