1. Lecture 4. RAM Model, Space and Time Complexity

2. Recap
Time and space tradeoff factors should be focus during designing of algorithms,
Performance of algorithm should be measure rather than performance of its implemented program.
Switching between CPU and IO circuit should be in mind before writing the instructions of an algorithm.

3. The RAM Model Random Access Machine (not R.A. Memory)
An idealized notion of how the computer works
Each "simple" operation (+, -, =, if) takes exactly 1 step.
Each memory access takes exactly 1 step
Loops and method calls are not simple operations, but depend upon the size of the data and the contents of the method.
Measure the run time of an algorithm by counting the number of steps.

4. Random Access Machine A Random Access Machine (RAM) consists of:
a fixed program
an unbounded memory
a read-only input tape
a write-only output tape
Each memory register can hold an arbitrary integer (*).
Each tape cell can hold a single symbol from a finite alphabet s.
Program
Memory
...
output tape
input tape
Instruction set:
x  y, x  y {+, -, *, div, mod} z
goto label
if y {<, , =,  ,> , } z goto label
x  input, output  y
halt

5. Space Complexity
The amount of memory required by an algorithm to run to completion
The term memory leaks refer to the amount of memory required is larger than the memory available on a given system

6. Space Complexity (Cont !!!)
Fixed part: The size required to store certain data/variables, that is independent of the size of the problem:
Such as int a (2 bytes, float b (4 bytes) etc
Variable part: Space needed by variables, whose size is dependent on the size of the problem:
Dynamic array a[]

7. Space Complexity (cont’d)
S(P) = c + S(instance characteristics)
c = constant
Example:
void float sum (float* a, int n) {
float s = 0;
for(int i = 0; i<n; i++) {
s+ = a[i]; }
return s;
Constant Space:
one for n, one for a [passed by reference!], one for s, one for I , constant space=c=4

8. Running Time of Algorithms
depends on input size n
size of an array
polynomial degree
# of elements in a matrix
# of bits in the binary representation of the input
vertices and edges in a graph
number of primitive operations performed
Primitive operation
unit of operation that can be identified in the pseudo-code

9. Steps To determine Time Complexity
Step-1. Determine how you will measure input size. Ex:
N items in a list
N x M table (with N rows and M columns)
Two numbers of length N
Step-2. Choose the type of operation (or perhaps two operations)
Comparisons
Swaps
Copies
Additions
Note: Normally we don't count operations in input/output.

10. Steps To determine Time Complexity (Cont !!!)
Step-3. Decide whether you wish to count operations in the
Best case? - the fewest possible operations
Worst case? - the most possible operations
Average case? This is harder as it is not always clear what is meant by an "average case". Normally calculating this case requires some higher mathematics such as probability theory.
Step-4. For the algorithm and the chosen case (best, worst, average), express the count as a function of the input size of the problem.

11. Steps To determine Time Complexity (Cont !!!)
5. Given the formula that you have determined, decide the complexity class of the algorithm.
Q. What is the complexity class of an algorithm?
A. a complexity class is a set of problems of related resource-based complexity, such as P, NP classes
Q. Is there really much difference between 3n
5n + 20
and n -3
A. Yes but when n is large?

12. Primitive Operations in an algorithm
Assign a value to a variable (i.e. a=5)
Call a method (i.e. method())
Arithmetic operation (i.e. a*b, a-b*c)
Comparing two numbers ( i.e. a<=b, a>b &&a>c)
Indexing into an array (i.e. a[0]=5)
Following an object reference (i.e. Test obj)
Returning from a method (i.e. return I )

13. Assignment Statement
The running time of a an assignment is considered as constant
Examples:
A=5;
A[5]=7
C=a+b;
Note:- Running time of assignments can be considered as 1 or C (Refer to constant values)

14. for Loops
The running time of a for loop is at most the running time of the statements inside the for loop (including tests) times the number of iterations.
Example
For(i=0;i<=n-1;i++)
A[i]=0;
Let
running time of basic operations is constant C
and loop is iterated n times then
Total Running time will be
n*c
Note:
(number of basic steps in loop body) * (number of iterations)

15. Nested for Loops
The total running time of a statement inside a group of nested for loops is the running time of the statement multiplied by the product of the sizes of all the for loops.
Example
for(i=0;i<=n-1;i++)
for(j=0;j<=m-1;j++)
K++;
Let
running time of basic operations is constant C
Running time of outer loop (i.e. i) is n
And running time of Inner loop (i.e. j) is m
Total Running time will be
m*n*c

16. Consecutives for Loops
The total running time of consecutive loops is the sum of running of all consecutives loops
Example
for(i=0;i<=n-1;i++)
K++;
for(j=0;j<=m-1;j++)
Let
running time of basic operations is constant C
Running time of loop (i.e. i) is n
And running time of loop (i.e. j) is m
Total Running time will be
n*c + m * c

17. Conditional Statements
The running time of an if-else statement is never more than the running time of the test plus the larger of the running times of S1 and S2.
Example
if(Condition)
S1;
else
S2;
Let
Condition is true and path have one basic operation (i.e. S1) then Ruining time will be constant C1
Similarly, if condition is false and path have one basic operation (i.e. S2) then Ruining time will be constant C2
Note:
Number of basic steps on branch that is executed

18. Primitive Operations: Example -1
Count the number of primitive operations in this program :
Int method() {
i = 0;
a = 0;
for (i=1;i<=n;i++)
{ printf(“%d”,i);
a=a+i; }
return I }
Primitive Operations
Yes/No
Total
Assign a value to a variable
Call a method
Arithmetic operation
Comparing two numbers
Indexing into an array
Following an object reference
Returning from a method

19. Primitive Operations: Example -1 (Cont !!!)
Yes/No
Total
Assign a value to a variable
Yes 5
Call a method No
Arithmetic operation 2
Comparing two numbers 1
Indexing into an array
Following an object reference
Returning from a method

20. Primitive Operations - Example 2
Count the number of primitive operations in this program :
Int method()
{ i = 0;
a = 0;
for (i=1;i<=n;i++)
{ printf(“%d”, i);
for (a=1;a<=n;a++)
printf(“%d”,a); }
return I
Primitive Operations
Yes/No
Total
Assign a value to a variable
Call a method
Arithmetic operation
Comparing two numbers
Indexing into an array
Following an object reference
Returning from a method

21. Primitive Operations - Example 2 (Cont !!!)
Yes/No
Total
Assign a value to a variable
yes 6
Call a method No
Arithmetic operation 2
Comparing two numbers
Indexing into an array
Following an object reference
Returning from a method 1

22. Summary
RAM model is used to measure the run time of an algorithm by counting the number of steps.
Space complexity of an algorithm refer to the amount of memory required to run.
Time complexity of an algorithm refer to its running time, which depends on input size.
Primitive operations refer to the unit of operation that can be identified in the pseudo-code

23. In Next Lecturer
In next lecture, we will talk about the types of algorithm complexity and how to find running time of different pseudo codes.