Complexity Theory

 Complexity theory is a problem can be solved? Given: ◦ Limited resources: processor time and memory space

Algorithm

 Definition: An algorithm is a finite step-by-step list of well- defined instructions for solving a particular problem.  Algorithm complexity: is a key task in comparing the efficiency of algorithms.  Two factors are considered: ◦ 1- the (running) timeTime Complexity ◦ 2- the space (usage)Space Complexity

 Time Complexity: estimates the running time of the algorithm. ◦ Measured by the number of key step of the basic steps used by the algorithm.  Basic Steps are: ◦ Arithmetic operations (+, -, *, /) ◦ Comparison operations (>, >=, <, <=, ==) ◦ Reading from a memory, file ◦ Writing to memory, file, (cin, fin)

 Key step: The most costly step of the algorithm ◦ Takes most of the running time ◦ Other steps is much less or at most proportional to the time of the key step.

 In worse-case analysis: ◦ We find the maximum value of the number of the key steps.  In Average-case analysis: ◦ We find the average value of the number of the key steps.

 Let f (n ) the complexity of an algorithm, i.e., ◦ # key operations ◦ For an instance of size n  A computer processor has a speed of 1 MIP ◦ Million Instructions Per (MIP) second

 Big Oh O() Notation is used ◦ To estimate the growth rate of f : N R + ◦ f O(g) ◦ There exist two positive integer k, n 0 :  f(n) ≤ k*g(n), for all n ≥ n 0  f grows not fast than g  f belongs to the complexity class O(g)

f(n) = 3 n+2= O(n) as 3 n +2 ≤ 4 n for all n ≥ 2. f(n) =100 n+6=O(n) as 100 n+6 ≤ 101 n for all n ≥ 6. f(n) =1000 n n - 6 = Ο(n 2 ) as 1000 n n - 6 ≤ 1001 n 2 for all n ≥ 100. f(n) = 6 2 n + n 2 = Ο (2 n ) as 6 2 n + n 2 ≤ 7 * 2 n for all n ≥ 4.

 Find the class complexity of the following functions  1. f(n) = n n 2 - 8n + 6  2. f(n) = (6n 7 +4) / (n 2 +1)  3. f(n) = 2 n + 3n n 4

 Common complexity classes  O(1)< O(log n) < O(n)< O(n log n)< O(n 2 )<O(n 3 )< O(2 n )< O(10 n )  O(n log n) is better than O(n 2 )  O(n) is better than O(n log n)

 Dropping Coefficient ◦ f(n) = k * n c f O(n c ) ◦ f(n) = 10 n 4 f O(n 4 )  Dominant Term ◦ f(n) = n c +n c-1 +…+ n 2 +n+1 f O(n c ) ◦ f(n) = (n 8 + 1) / (n 3 + 2n 2 – 1) f O(n 5 )

Introduction to Algorithms

What is Algorithm ??

 An algorithm : is a finite step-by-step list of well-defined instructions for solving a particular problem.  Algorithm complexity is a key task in comparing the efficiency of algorithms.  Two factors are considered: ◦ 1- the (running) timeTime Complexity ◦ 2- the space (usage)Space Complexity

 Sorting Algorithms ◦ Bubble Sort ◦ Insertion Sort ◦ And many other algorithms (Quick Sort, Selection Sort, …. )  Searching Algorithms ◦ Linear search (sequential search) ◦ Binary search ◦ And many other algorithms (BFS, DFS, … )  And There are many and many algorithms used in this fields and other fields.

Sorting Algorithms Bubble sort

Insertion Sort