1. Universidad Nacional Bogota Depto. De Sistemas
nalysis of A
lgorithms
Universidad Nacional Bogota
Depto. De Sistemas

2. OUTLINE Introduction What is an algorithm? Semantic analysis
Correctness
Analysis of complexity

3. What is an algorithm?
Well-defined computational procedure that takes some value, or a set of values, as input and produces some value, or a set of values, as output. An algorithm is thus a sequence of computational of steps that transforms the input into the output.
We can also view an algorithms as a tool for solving a well-specified computational problem[Cormen, 91].
A set of instructions, clearly specified, that are performed in order to solve a problem [Baase, 91].
Unambiguous specification of a sequence of steps performed to solve a problem [Aho, 95].

4. Algorithm
A sequence of computational of steps for solving a well-specified computational problem
The statement of the problem specifies in general terms the desired input/output relationship
The algorithm describes a specific computational procedure for achieving that input/ouput relationship

5. Algorithm
I: input / O:output I O
steps

6. <a’1,a’2,a’3,...a’n> such that
Sorting problem
INPUT:
A sequence of numbers
<a1,a2,a3,...,an>
OUTPUT:
A permutation
<a’1,a’2,a’3,...a’n> such that
a’1  ai2  ai3  ...  ain

7. Instance of the sorting problem A sorting algorithms must return:
A possible input sequence is a instance of the sorting problem
<31,41,59,26,41,58>
A sorting algorithms must return:
<26,31,41,41,58,59>

8. Analysis of algorithms
Semantic Analysis
Correctness
Complexity
Time complexity
Space complexity
Analysis

9. Semantic Analysis
What does an algorithm do? (Discovering its function?)
Is the study of the function associated to an algorithm:
I input space : the space of values of the input variables defined in the algorithm.
O output space : the space of the values of the output variables defined in the algorithm.
X state space : the space of the values of the variables defined in the algorithm.
f : I  O  {}
function associated to the algorithm

10. Example i  1 j  0 for k 1 to n do j  i + j i  j - i end-for *
function mistery( n )
i  1
j  0
for k 1 to n do
j  i + j
i  j - i
end-for *
return j
Value in the variables at the point *
(n)  X0 X1 X2 X3 X4 X5 X6 X7 .… Xn  (j)
k  0, 1, 2, 3, 4, 5, 6, 7,.…,
j  0, 1, 1, 2, 3, 5, 8, 13,.…,
i  1, 0, 1, 1, 2, 3, 5, 8,.…,

11. mistery( 0) = 0
mistery( 1) = 1
mistery( n ) = mistery( n - 1 ) +
mistery( n - 2), for n > 2 .
“mistery( n ) computess the n-th Fibonacci’s number”
mistery( n ) =(n - n) /5 with
=(1 + 5)/2 [Golden ratio]
 =(1 - 5)/2

12. f : I  O  {} with I  I , O  O , Xi  X I O f (X)
following an algorithm.
( I )  X0 (O0)  X1 (O1)  ...  Xm (Om)  ...
If it stops in m steps the output is {O0, O Om }
If it does not stop {O0, O Om … }
f : I  O  {} with I  I , O  O , Xi  X I O f
(X)

13. Given a computational problem and a algorithm A .
Correctness
Is an algorithm correct for a problem?
(Does it do a given function?)
Given a computational problem
f : I  O  {}
and a algorithm
A .
The algorithm A is correct for the problem f if the function that A calculates is f.

14. (n,T[1..n] )  min{T[i] | 1  i  n}
Problem: Finding the minumun element in an array of n reals
Input : T[1..n] : array of real
Output : x  min{T[i] | 1  i  n}
min : N x RN  R
(n,T[1..n] )  min{T[i] | 1  i  n}
“In order to prove that a program or algorithm is correct, we need to show that the program gives the correct answer for every possible input.”

15. function min( n , T[1..n] ) : real
x  T[1]
for i 2 to n do
if T[i] < x then
x  T[i]
end_for
return x
end

16. function min( n , T[1..n] ) : real
if n =1 then
return T[1]
else
return MIN(min(n-1, T[1..n-1] ),T[n])
end
function MIN(a,b ) : real
if a < b then
return a
return b

17. function min( n , T[1..n] ) : real
if n =1 then
return T[1]
else if n=2 then
if T[1] < T[2] then
else
return T[2]
return min(2,[min(n-1, T[1..n-1] ),T[n]])
end

18. Analysis of complexity
How much resources are needed to run an algorithm as a function of the size of the problem it solves?
Is the study of the amount of resources:
time, [ time complexity] t(n) ,
memory, [ space complexity] s(n) ,
… [ other resources],
that an algorithm will require to process an input of size n.

19. Complexity
Time complexity t(n)
computational steps
Space complexity s(n)
storage - memory
Analysis

20. A measure of the amount of data that an algorithm takes as input
Size of a problem n
A measure of the amount of data that an
algorithm takes as input
Examples:
Sorting
n = number of elements in the input sequence
Inverting a square matrix A
n = number of elements in the matrix
Finding the root of a function f(x)
n = number of precise digits in the root

21. Multiplying two integers
n = number of bits needed to represent the input
Finding the shortest path in graph, in this case the size of the input is described by two numbers rather than one.
n = number of vertex
m = number of edges

22. Time Complexity t(n) Best case tb(n) Worst case tw(n)
Number of computational steps
Best case tb(n)
Minimum execution time for any input of size n
Worst case tw(n)
Maximum execution time over all the inputs of size n
Average case ta(n)
Average execution time over all the inputs of size n

23. Worst-case analysis is appropriate for an algorithm whose response time is critical
In a situation where an algorithm is to be used many times on many different instances, it may be more important to know the average execution time on instances of time n
Sorting
average time taken on the n! different ways of
initially ordering n elements (assuming they are
all different)

24. Elementary Operation
An operation whose execution time can be bounded above by a constant depending only on the particular implementation used (machine, programming language etc). In general we use RAM elementary instructions as the model.

25. Elementary (primitive) instructions Arithmetic
RAM Model
Random Access Model
Data types
Integer
Up to values n uses c log n bits.
Floating point
Bounded rage of values.
Elementary (primitive) instructions
Arithmetic
Add +, substrac -, multiply *, divide /, remainder %, floor  , ceiling  .
Data movement
load, store, copy.

26. main – cache – virtual [demand paging]
Control
Conditional and unconditional branch, subroutine call, return.
Special exponentiation
2k for k small  k-shift left.
Each instructions takes a constant amount of time
Memory-hierarchy effects
main – cache – virtual
[demand paging]
are considered only in special cases

27. Example function mistery( n ) i  1 j  0 for k 1 to n do j  i + j
i  j - i
end-for
return j
If the operations are elementary the algorithm takes a time equal to a+bn, for appropriate constants a and b, i.e.,
time in O(n).

28. !!!But the operations are not elementary !!!
For n=47 the addition “j  i + j” causes arithmetic overflow on a 32-bit machine
bits are needed to hold the result corresponding to n=65.536
We must attribute to the additions a cost proportional to the length of the operands
time in O(n2)

29. Principle of Invariance
“Two different implementations of the same algorithm will not differ in efficiency by more than some multiplicative constant”

30. Orden de complejidad de un algoritmo
El orden de complejidad de un algoritmo
Es una expresion matematica que indica como los recursos necesitados para correr un algoritmo se incrementan como se incrementa el tamaño del problema (que el resuelve).

31. Complejidad Lineal
La complejidad de un algoritmo se dice que es lineal si:
t(n)=a * n + b
Con a y b son constantes

32. Example function min( T[1..n] ) : real x  T[1] /* b */
for i 2 to n do /* n times */
if T[i] < x then /* a */
x  T[i]
end_for
return x

33. Worst case time complexity compiler +code optimizations)
tw(n) = a*n + b
By the invariance principle we if we have two different implementations
(computer + language +
compiler +code optimizations)
with worst case time complexity tw1(n) and tw2(n) then
tw1(n) = c tw2(n)