1. Analysis of Algorithms: Time & Space
Salim Arfaoui
SJCNY-Brooklyn

10. What does ‘Space Complexity’ mean?
Space Complexity: The term Space Complexity is misused for Auxiliary Space at many places.
Auxiliary Space is the extra space or temporary space used by an algorithm.
Space Complexity of an algorithm is total space taken by the algorithm with respect to the input size. Space complexity includes both Auxiliary space and space used by input.

11. Program running time
When is the running time (waiting time for user) noticeable/important?

12. Program running time – Why?
When is the running time (waiting time for user) noticeable/important?
web search
database search
real-time systems with time constraints

13. Factors that determine running time of a program

14. Factors that determine running time of a program
problem size: n
basic algorithm / actual processing
memory access speed
CPU/processor speed
# of processors?
compiler/linker optimization?

15. Time Complexity measure of algorithm efficiency
has a big impact on running time.
Big-O notation is used.
To deal with n items, time complexity can be O(1), O(log n), O(n), O(n log n), O(n2), O(n3), O(2n), even O(nn).

16. Coding example #1
for ( i=0 ; i<n ; i++ ) m += i;

17. Coding example #2
for ( i=0 ; i<n ; i++ ) for( j=0 ; j<n ; j++ ) sum[i] += entry[i][j];

18. Coding example #3
for ( i=0 ; i<n ; i++ ) for( j=0 ; j<i ; j++ ) m += j;

19. Coding example #4
i = 1; while (i < n) { tot += i; i = i * 2; }

20. Example #4: equivalent # of steps?
i = n; while (i > 0) { tot += i; i = i / 2; }

21. Coding example #5
for ( i=0 ; i<n ; i++ ) for( j=0 ; j<n ; j++ ) for( k=0 ; k<n ; k++ ) sum[i][j] += entry[i][j][k];

22. Coding example #6
for ( i=0 ; i<n ; i++ ) for( j=0 ; j<n ; j++ ) sum[i] += entry[i][j][0]; for( k=0 ; k<n ; k++ ) sum[i] += entry[i][0][k];

23. Coding example #7
for ( i=0 ; i<n ; i++ ) for( j=0 ; j< sqrt(n) ; j++ ) m += j;

24. Coding example #8
for ( i=0 ; i<n ; i++ ) for( j=0 ; j< sqrt(995) ; j++ ) m += j;

25. Coding example #8 : Equivalent code
for ( i=0 ; i<n ; i++ ) { m += j; … m += j; // 31 times }

26. Coding example #9 int total(int n) main() for( i=0 ; i < n; i++)
  subtotal += i;
main()
 for ( i=0 ; i<n ; i++ )
  tot += total(i);

27. Coding example #9: Equivalent code
for ( i=0 ; i<n ; i++ ) { subtotal = 0; for( j=0 ; j < i; j++) subtotal += j; tot += subtotal; }

28. Important Functions Often appear in algorithm analysis: Constant  1
Logarithmic  log n
Linear  n
N-Log-N  n log n
Quadratic  n2
Cubic  n3
Exponential  2n

29. Important Functions Growth Rates
log(n)
nlog(n) n2 n3 2n 8 3 24 64
512
256 16 4
4096
65536 32 5
160
1024
32768
4.3x109 6
384
262144
1.8x1019
128 7
896
16384
3.4x1038
2048
1.2x1077

30. Growth Rates Illustration
Running Time in ms
(10-3 of sec)
Maximum Problem Size (n)
1000 ms ms *105 m
(1 second) (1 minute) (1 hour) n
1000
60,000
3,600,000 n2 32
245
1,897 2n 10 16 22

31. Practical Examples

32. Example #1: carry n items from one room to another room

33. Example #1: carry n items from one room to another room
How many operations?
n pick-ups, n forward moves, n drops and n reverse moves  4 n operations
4n operations = c. n = O(c. n) = O(n)
Similarly, any program that reads n inputs from the user will have minimum time complexity O(n).

34. Example #2: Sorting patients records in Doctor Office
What is the time complexity of Sort?

35. Example #2: Sorting patients records in Doctor Office
What is the time complexity of Sort?
Binary Sort algorithm at work
O(log n)
Sequential Sort?
O(n)

36. Example #3: Store manager gives gifts to first 10 customers
There are n customers in the queue.
Manager brings one gift at a time.

37. Example #3: Store manager gives gifts to first 10 customers
There are n customers in the queue.
Manager brings one gift at a time.
Time complexity = O(c. 10) = O(1)
Manager will take exactly same time irrespective of the line length.

38. Example #4: Thief visits a Doctor with Back Pain

39. Example #4: Thief visits a Doctor with Back Pain
Doctor asks a few questions:
Is there a lot of stress on the job?
Do you carry heavy weight?

40. Example #4: Thief visits a Doctor with Back Pain
Doctor asks a few questions:
Is there a lot of stress on the job?
Do you carry heavy weight?
Doctor says: Never carry > 50 kgs

41. Space complexity