Universidad Nacional Bogota Depto. De Sistemas nalysis of A lgorithms Universidad Nacional Bogota Depto. De Sistemas

OUTLINE Introduction What is an algorithm? Semantic analysis Correctness Analysis of complexity

What is an algorithm? Well-defined computational procedure that takes some value, or a set of values, as input and produces some value, or a set of values, as output. An algorithm is thus a sequence of computational of steps that transforms the input into the output. We can also view an algorithms as a tool for solving a well-specified computational problem[Cormen, 91]. A set of instructions, clearly specified, that are performed in order to solve a problem [Baase, 91]. Unambiguous specification of a sequence of steps performed to solve a problem [Aho, 95].

Algorithm A sequence of computational of steps for solving a well-specified computational problem The statement of the problem specifies in general terms the desired input/output relationship The algorithm describes a specific computational procedure for achieving that input/ouput relationship

Algorithm I: input / O:output I O steps

<a’1,a’2,a’3,...a’n> such that Sorting problem INPUT: A sequence of numbers <a1,a2,a3,...,an> OUTPUT: A permutation <a’1,a’2,a’3,...a’n> such that a’1  ai2  ai3  ...  ain

Instance of the sorting problem A sorting algorithms must return: A possible input sequence is a instance of the sorting problem <31,41,59,26,41,58> A sorting algorithms must return: <26,31,41,41,58,59>

Analysis of algorithms Semantic Analysis Correctness Complexity Time complexity Space complexity Analysis

Semantic Analysis What does an algorithm do? (Discovering its function?) Is the study of the function associated to an algorithm: I input space : the space of values of the input variables defined in the algorithm. O output space : the space of the values of the output variables defined in the algorithm. X state space : the space of the values of the variables defined in the algorithm. f : I  O  {} function associated to the algorithm

Example i  1 j  0 for k 1 to n do j  i + j i  j - i end-for * function mistery( n ) i  1 j  0 for k 1 to n do j  i + j i  j - i end-for * return j Value in the variables at the point * (n)  X0 X1 X2 X3 X4 X5 X6 X7 .… Xn  (j) k  0, 1, 2, 3, 4, 5, 6, 7,.…, j  0, 1, 1, 2, 3, 5, 8, 13,.…, i  1, 0, 1, 1, 2, 3, 5, 8,.…,

mistery( 0) = 0 mistery( 1) = 1 mistery( n ) = mistery( n - 1 ) + mistery( n - 2), for n > 2 . “mistery( n ) computess the n-th Fibonacci’s number” mistery( n ) =(n - n) /5 with =(1 + 5)/2 [Golden ratio]  =(1 - 5)/2

f : I  O  {} with I  I , O  O , Xi  X I O f (X) following an algorithm. ( I )  X0 (O0)  X1 (O1)  ...  Xm (Om)  ... If it stops in m steps the output is {O0, O1 ... Om } If it does not stop {O0, O1 ... Om … } f : I  O  {} with I  I , O  O , Xi  X I O f (X)

Given a computational problem and a algorithm A . Correctness Is an algorithm correct for a problem? (Does it do a given function?) Given a computational problem f : I  O  {} and a algorithm A . The algorithm A is correct for the problem f if the function that A calculates is f.

(n,T[1..n] )  min{T[i] | 1  i  n} Problem: Finding the minumun element in an array of n reals Input : T[1..n] : array of real Output : x  min{T[i] | 1  i  n} min : N x RN  R (n,T[1..n] )  min{T[i] | 1  i  n} “In order to prove that a program or algorithm is correct, we need to show that the program gives the correct answer for every possible input.”

function min( n , T[1..n] ) : real x  T[1] for i 2 to n do if T[i] < x then x  T[i] end_for return x end

function min( n , T[1..n] ) : real if n =1 then return T[1] else return MIN(min(n-1, T[1..n-1] ),T[n]) end function MIN(a,b ) : real if a < b then return a return b

function min( n , T[1..n] ) : real if n =1 then return T[1] else if n=2 then if T[1] < T[2] then else return T[2] return min(2,[min(n-1, T[1..n-1] ),T[n]]) end

Analysis of complexity How much resources are needed to run an algorithm as a function of the size of the problem it solves? Is the study of the amount of resources: time, [ time complexity] t(n) , memory, [ space complexity] s(n) , … [ other resources], that an algorithm will require to process an input of size n.

Complexity Time complexity t(n) computational steps Space complexity s(n) storage - memory Analysis

A measure of the amount of data that an algorithm takes as input Size of a problem n A measure of the amount of data that an algorithm takes as input Examples: Sorting n = number of elements in the input sequence Inverting a square matrix A n = number of elements in the matrix Finding the root of a function f(x) n = number of precise digits in the root

Multiplying two integers n = number of bits needed to represent the input Finding the shortest path in graph, in this case the size of the input is described by two numbers rather than one. n = number of vertex m = number of edges

Time Complexity t(n) Best case tb(n) Worst case tw(n) Number of computational steps Best case tb(n) Minimum execution time for any input of size n Worst case tw(n) Maximum execution time over all the inputs of size n Average case ta(n) Average execution time over all the inputs of size n

Worst-case analysis is appropriate for an algorithm whose response time is critical In a situation where an algorithm is to be used many times on many different instances, it may be more important to know the average execution time on instances of time n Sorting average time taken on the n! different ways of initially ordering n elements (assuming they are all different)

Elementary Operation An operation whose execution time can be bounded above by a constant depending only on the particular implementation used (machine, programming language etc). In general we use RAM elementary instructions as the model.

Elementary (primitive) instructions Arithmetic RAM Model Random Access Model Data types Integer Up to values n uses c log n bits. Floating point Bounded rage of values. Elementary (primitive) instructions Arithmetic Add +, substrac -, multiply *, divide /, remainder %, floor  , ceiling  . Data movement load, store, copy.

main – cache – virtual [demand paging] Control Conditional and unconditional branch, subroutine call, return. Special exponentiation 2k for k small  k-shift left. Each instructions takes a constant amount of time Memory-hierarchy effects main – cache – virtual [demand paging] are considered only in special cases

Example function mistery( n ) i  1 j  0 for k 1 to n do j  i + j i  j - i end-for return j If the operations are elementary the algorithm takes a time equal to a+bn, for appropriate constants a and b, i.e., time in O(n).

!!!But the operations are not elementary !!! For n=47 the addition “j  i + j” causes arithmetic overflow on a 32-bit machine 45.496 bits are needed to hold the result corresponding to n=65.536 We must attribute to the additions a cost proportional to the length of the operands time in O(n2)

Principle of Invariance “Two different implementations of the same algorithm will not differ in efficiency by more than some multiplicative constant”

Orden de complejidad de un algoritmo El orden de complejidad de un algoritmo Es una expresion matematica que indica como los recursos necesitados para correr un algoritmo se incrementan como se incrementa el tamaño del problema (que el resuelve).

Complejidad Lineal La complejidad de un algoritmo se dice que es lineal si: t(n)=a * n + b Con a y b son constantes

Example function min( T[1..n] ) : real x  T[1] /* b */ for i 2 to n do /* n times */ if T[i] < x then /* a */ x  T[i] end_for return x

Worst case time complexity compiler +code optimizations) tw(n) = a*n + b By the invariance principle we if we have two different implementations (computer + language + compiler +code optimizations) with worst case time complexity tw1(n) and tw2(n) then tw1(n) = c tw2(n)