Analysis of Non – Recursive Algorithms

Why Analysis of Algorithms? For a given problem there can be multiple solutions available for the same problem. Analysis of algorithm helps us to find which among them is better in terms of time and space. For Ex: If you have a stream of numbers coming and you would like to sort it. We might use Insertion sort ahead of bubble sort or selection sort. This decision can be taken if we would analyze the algorithm.

Goal of Analysis of Algorithms The goal of analysis of algorithms is to compare algorithms mainly in-terms of running time and memory.

What is running time analysis? It is the process of determining how processing time increases as size of the problem (input) increases. Input size is the number of elements in the input. Depending on the problem type the size of input may be different for different types.

How to Compare Algorithms Execution Time? Number of statements executed? Ideal Solution? Expressing the running time of an algorithm as a function of input size 'n'. (i.e. f(n) ). Compare these functions corresponding to running times. Independent of machine, programming language,etc..

Commonly used Rate of Growths What is Rate of Growth? The rate at which the running time increases as a function of input is called rate of growth. Commonly used Rate of Growths Time Complexity Name 1 Constant logn Logarithmic n Linear nlogn Linear Logarithmic n2 Quadratic n3 Cubic 2n Exponential

Types of Analysis Worst Case : Defines the input for which algorithm takes huge time. Input is the one for which algorithm runs slower. Best Case Defines the input for which algorithm takes lowest time. Input is the one for which algorithm runs faster. Average Case Provides a prediction about running time of algorithm Assumes that input is random Lower_Bound <= Average Time <= Upper_Bound

Asymptotic Notation It is a syntax that is used to represent the upper bound and lower bound. Big – O Notation Omega - Ω Notation Theta - Θ Notation

Big – O Notation This notation gives the tight upper bound of the given function. It is represented as, f(n) = O (g(n)). It means at large values of 'n', the upper bound of f(n) is g(n)

Big – O Notation c.g(n) f(n) 0<=f(n)<=c.g(n), for all n>=n0 RATE OF GROWTH 0<=f(n)<=c.g(n), for all n>=n0 g(n) is tight upper bound of f(n) n0 INPUT SIZE, n

Big – O Notation : Examples Ex 1 : Find the upper bound of f(n) = 3n + 8 Solution : 3n + 8 <= 4n, for all n>=8 c = 4, n0 = 8 3n+8 = O(n) Ex 2 : Find the upper bound of f(n) = n2 + 4 Solution : n2 + 4 <= 2n2, for all n>=2 c = 2, n0 = 2 n2 + 4 = O(n2)

Big – O Notation : Examples Ex 3 : Find the upper bound of f(n) = n4 + 100n2 + 50 Solution : n4 + 100n2 + 50 <= 2n4, for all n>=11 c = 2, n0 = 11 n4 + 100n2 + 50 = O(n4) Ex 4 : Find the upper bound of f(n) = n Solution : n<= n, for all n>=1 c = 1, n0 = 1 n = O(n) Ex 5: Find the upper bound of f(n) = 10. Answer : O(1)

Omega - Ω Notation It gives the tight lower bound of the given algorithm Represented as, f(n) = Ω(g(n)) It means for larger value of 'n' the tight lower bound of f(n) is g(n).

Omega - Ω Notation f(n) c.g(n) 0<=c.g(n)<=f(n), for all n>=n0 RATE OF GROWTH 0<=c.g(n)<=f(n), for all n>=n0 g(n) is tight lower bound of f(n) n0 INPUT SIZE, n

Omega - Ω Notation : Examples Ex 1 : Prove f(n) = 100n + 5 ≠ Ω (n2) Solution : By definition of Ω(g(n)) = f(n), We write 0<=c.g(n)<=f(n) 0 <=c.n2 <= 100n+5 ------------------ 1 100n+5<=100n+5n, for all n>=1 => 100n+5<=105n -------------------- 2 From 1 and 2, cn2<=105n cn2-105n<=0 cn<=105 n<=105/c

Theta - Θ Notation This notation decides whether the upper bound and lower bound of a given function is same or not. It is defined as, Θ(g(n)) = f(n), if there exists positive constants c1, c2 and n0 such that, 0 <= c1.g(n) <= f(n) <= c2.g(n), for-all, n>=n0

Theta - Θ Notation c2g(n) f(n) RATE OF GROWTH c1g(n) no INPUT SIZE, n

Theta - Θ Notation : Examples Ex 1: Find Θ bound for f(n) = n2/2 – n/2 Solution : n2/5 <= n2/2 – n/2 <= n2, for-all, n>=1 c1=1/5, c2=1. And g(n) = n2 f(n) = Θ(g(n)) Ex 2: Prove that n ≠ Θ(n2) c1n2 <= n <= c2n2, Holds true only for n<=1/c1 Hence, n ≠ Θ(n2)

Guidelines for Asymptotic Analysis Loops : The running time of a loop is at most, the running time of statements inside the loop. for(i=1; i<=n; i++) X = X + 2; //Constant Time, c Total Time = c*n = O(n)

Guidelines for Asymptotic Analysis Nested Loops : The total running time is the product of sizes of all loops. for(i=1; i<=n; i++){ X = X + 2; //Constant Time, c1 for(j=1; j<=n; j++) { M = M -2; //Constant Time, c2 } Total Time = c2*n*n = O(n2)

Guidelines for Asymptotic Analysis Consecutive statements : Add the time complexities of each statements. for(i=0;i<n;i++) printf(“Hello....\n”); //Constant Time, c for(i=1; i<=n; i++){ X = X + 2; //Constant Time, c1 for(j=1; j<=n; j++) { M = M -2; //Constant Time, c2 } Total Time = n*c + c2*n*n = O(n2)

Guidelines for Asymptotic Analysis Logarithmic Complexity for(i=1; i<=n; i=i*2) X = X + 2; //Constant Time, c Total Time = O(log2n)

Commonly used Logarithms and Summations logxy = ylogx logxy = log x + log y log logn = log(log n) log(x/y) = log(x) - log(y) alogbx = xlogba logk n=(logn)k 1+2+3+4+......+N = N*(N+1)/2 1+x+x2+x3+.......+xn = (xn+1 – 1) / (x-1) 1+1/2+1/3+.......+1/n ≈ logn

Analyse the following function void function(int n) { int i, count = 0; for(i=0; i*i <=n ; i++) count++; } Answer : O(√n)

What is the complexity of following function void function(int n) { int i, j, k, count=0; for(i=n/2; i<=n ; i++) for(j=1; j+n/2 <=n; j++) for(k=1;k<=n;k=k*2) count++; } Answer : O(n2logn)

What is the complexity of following function void function(int n) { int i, j, k, count=0; for(i=n/2; i<=n ; i++) for(j=1; j <=n; j=j*2) for(k=1;k<=n;k=k*2) count++; } Answer : O(nlog2n)

What is the complexity of following function void function(int n) { if(n==1) return; for(i=1; i<=n; i++) for(j=1; j<=n; j++) printf(“*”); break; } Answer : O(n)

What is the complexity of following function void function(int n) { float sum=0; for(i=1; i<=n; i++) for(j=1;j<=n;j=j*2) printf(“Hello World\n”); } Answer : O(nlogn)

What is the complexity of following function void function(int n) { float sum=0; for(i=1; i<=n/3; i++) for(j=1;j<=n;j=j+5) printf(“Hello World\n”); } Answer : O(n2)

What is the complexity of following function void function(int n) { int i=1; while(i<=n) int j = n; while(j>0) j = j / 2; i = i*2; } Answer : O(log2n)