1. Why do we study algorithms?

2. 2

3. First results are about bats and dolphins

4. Algorithmics: the Spirit of Computing As per David Harel (A writer on Algorithms) Algorithmics is more than a branch of computer science. It is the core of computer science, and, in all fairness, can be said to be relevant to most of science, business, and technology. [Har92, p. 6] 4

5. Donald Knuth, one of the most prominent computer scientists in the history of algorithmics, put it as follows: 5 A person well-trained in computer science knows how to deal with algorithms: how to construct them, manipulate them, understand them, analyze them. This knowledge is preparation for much more than writing good computer programs; it is a general-purpose mental tool that will be a definite aid to the understanding of other subjects, whether they be chemistry, linguistics, or music, etc. The reason for this may be understood in the following way: It has often been said that a person does not really understand something until after teaching it to someone else. Actually, a person does not really understand something until after teaching it to a computer, i.e., expressing it as an algorithm... An attempt to formalize things as algorithms leads to a much deeper understanding than if we simply try to comprehend things in the traditional way. [Knu96, p. 9]

6. Main Topics Covered Introduction to the course and the course objectives Why Algorithm Analysis Algorithm Analysis Techniques and Basic Mathematics Recurrence Divide and Conquer algorithm design strategy Sorting in linear time String Matching Dynamic Programming algorithm design strategy Greedy algorithms Backtracking Branch & Bound Graph Algorithms Guided Study of the state of art algorithms from the research papers 6

7. Algorithm an algorithm is a finite sequence of unambiguous instructions to perform a specific task An algorithm has a name, begins with a precisely specified input, and terminates with a precisely specified output Input and output are finite sequences of mathematical objects

8. Correct Algorithm An algorithm is said to be correct if given input as described in the input specifications: 1. the algorithm terminates in a finite time; 2.on termination the algorithm returns output as described in the output specifications.

9. At least three important questions need to be answered for each algorithm Is it correct? How much time and space does it take, as a function of n? And can we do better? Being problem solvers, we need to equip ourselves with the tools and techniques to answer these questions. We will first focus on point 2. 9

10. Time efficiency, also called time complexity, indicates how fast an algorithm in question runs Space efficiency, also called space complexity, refers to the amount of memory units required by the algorithm in addition to the space needed for its input and output 10

11. An Example Consider the problem of sorting numbersINPUT: Sequence of n numbers OUTPUT: Permutation (reordering) of the input sequence such that a1`<a2`<a3`<…..<an` Many algorithms are available. 11

12. An algorithm for sorting. Get a list of unsorted numbersSet a marker for the unsorted section at the front of the listRepeat steps 4 - 6 until one number remains in the unsorted sectionCompare all unsorted numbers in order to select the smallest oneSwap this number with the first number in the unsorted sectionAdvance the marker to the right one positionStop 12

13. Another algorithm for sorting Divide sequence of m elements into two sequences of m/2 elements Conquer both sub-sequences using Merge Sort (recursively) Combine two sorted sub-sequences using merge 13

14. How much time does each algorithm take, as a function of n? Implement both algorithmsNote down the execution time for each algorithmCompare the times 14

15. Possible steps? Understand the problemFormulate a solution / algorithmAnalyze the algorithm Design a program Implement the program Execute the code Measure the time See if the solution is okOtherwise try to come up with another solution and again start from step 1 15

16. A Typical Result for both the algorithms 16

17. Another view of the results 17

18. How to analyze an algorithm with minimum effort? Predict the resources that the algorithm requires – Memory – Communications Bandwidth – Logic gates etc – Most important is Computational Time 18

19. Possible steps with minimum effort? 1.Understand the problem 2.Formulate a solution / algorithm 3.Analyze the algorithm a)Design a program b)Implement the program c)Execute the code d)Measure the time 4.See if the solution is ok 5.Otherwise try to come up with another solution and again start from step 1 19

20. Important things before analysis – Model of the machine upon which the algorithms is executed. – Random Access Machine (RAM) (Sequential) to start with Running Time: – No. of primitive operations or “steps executed”. 20

21. 21 T(n) ≈ c op C(n) running time execution time for basic operation Number of times basic operation is executed input size Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size

22. How do we write algorithms? Pseudo Code: – Similar construct / keywords as in a high level programming languages, e.g. in C, Pascal etc. – Structured semantics of the high level languages without caring about the syntactic errors / grammatical rules 22

23. The language of algorithms Algorithm h algorithm name i INPUT: h input specification i OUTPUT: h output specification i ;... ; end;

24. Sample Pseudo Code Max-Subsequence-Sum(Array, N)//Where N is size of Array {int this-sum = 0, Max-sum = 0; for(int i = 0; i < N; i++) {for(int j = i; j < N; j++) { this-sum = 0; for(int k = i; k <= j; k++) this-sum = this-sum + Array[k]; if(this-sum > Max-sum) Max-sum = this-sum; } return(Max-sum); }

25. How much time each construct / keyword of a pseudo code takes to execute. Assume it takes ti (the ith construct)Sum / Add up the execution time of all the constructs / keywords.if there are m constructs then total time for all the constructs is 25

26. That will be the execution time for a given input size (say n) Running time as the function of the input size T(n) 26

27. What are the constructs / Keywords. Time for each construct Total Time Total time as a function of input size 27

28. Constructs: 1.Sequence 2.Selection 3.Iterations 4.Recursion 28

29. Sequence Statements: – Just add the running time of the statements Iteration is at most the running time of the statements inside the loop (including tests) times the number of iterations. 29

30. Example 1 30 Analyze Time for the following algorithm – Find the value of the largest element in a list of n numbers. MaxElement(A[0..n-1) maxVal = A[0]; for(I = 1; I < n; I++) if(A[I] > maxVal) maxVal = A[I]; return maxVal

31. If-Then-Else: – if (condition) S 1 else S 2 – Running time of the test plus the larger of the running times of S 1 and S 2. Nested Loops: Analyze these inside out. The total Running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the size of all the loops. Function Calls: Analyzing from inside to out. If there are function calls, these must be analyzed first. 31

32. Example 2 Check whether all the elements in a given array are distinct. UniqueElements(A[0..n-1]) for(I = 0; i<n-1; i++) for(j = i+1; j<n; j++) if(A[i] = A[j]) return false return true 32

33. Useful Formulas for the Analysis of Algorithms aqadir@jinnah.edu.pk 33

34. 34

35. 35

37. Analysis Example Max-Subsequence-Sum(Array, N)//Where N is size of Array {int this-sum = 0, Max-sum = 0; for(int i = 0; i < N; i++) {for(int j = i; j < N; j++) { this-sum = 0; for(int k = i; k <= j; k++) this-sum = this-sum + Array[k]; if(this-sum > Max-sum) Max-sum = this-sum; } return(Max-sum); }

38. Assignment: Analyze the algorithm 1

39. Assignment: Analyze the algorithm 2

40. Assignment: Analyze the algorithm 3

41. Assignment: Analyze the algorithm 4

42. Assignment: Analyze the algorithm 5

43. Assignment: Analyze the algorithm 6

44. Assignment: Analyze the algorithm 7