1. Design and Analysis of Algorithms Faculty Name : Ruhi Fatima Email: r.mazharuddin@mu.edu.sa Course Description This course provides techniques to prove the correctness and to analyze the running time of the basic algorithms for those classic problems in various domains; to apply the algorithms and design techniques to solve problems; and to analyze the complexities of various problems in different domains.

2. Introduction to Algorithm Topics Covered Study of Algorithms Algorithms as a technology Insertion sort Analyzing algorithm Designing algorithm Turing Machine

3. Donald E. Knuth stated “Computer Science is the study of algorithms” –Cornerstone of computer science. Programs will not exist without algorithms. Behind every successful programming project there is a good design. So if the underlying design is bad, then often no amount of fine-tuning is going to make a substantial difference Usually design of any algorithm is chosen from existing techniques that are discussed in data structure course and based upon the selection proper design steps are formed Help to develop the ability of analyzing and solving problems via computers To be sure that your design is the most suitable design for the underlying problem, you must develop the ability of analyzing the design you made. Why Study this Course?

4. Informally, an algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output. An algorithm is thus a sequence of computational steps that transform the input into the output. We can also view an algorithm as a tool for solving a well-specified computational problem. The statement of the problem specifies in general terms the desired input/output relationship. The algorithm describes a specific computational procedure for achieving that input/output relationship. What is an Algorithm? “computer” problem algorithm inputoutput

5. This course introduces students to the analysis and design of computer algorithms. Upon completion of this course, students will be able to do the following: –Analyze the asymptotic performance of algorithms. –Demonstrate a familiarity with major algorithms and data structures. –Apply important algorithmic design paradigms and methods of analysis. –Synthesize efficient algorithms in common engineering design situations. In this course we will cover the following topics: Understand foundations of algorithms && design and analysis various variants algorithms Accuracy, Efficiency, Comparing efficiencies Make use of skills to understand mathematical notations in algorithms and their simple mathematical proofs Gain familiarity with a number of classical problems that occur frequently in real-world applications Course Objectives

6. Understanding the problem –Asking questions, doing few examples by hand, think about special cases, etc. Deciding on –Exact vs. approximate problem solving, Appropriate data structure Design an algorithm Proving correctness Analyzing an algorithm –Time efficiency : how fast the algorithm runs –Space efficiency: how much extra memory the algorithm needs. Any algorithm can be analyzed when we study it as a pure mathematical entity, and so ignore issues such as programming language, machine, and operating system. Coding an algorithm Fundamentals of Algorithmic Problem Solving What kinds of Problems are solved by Algorithms Practical applications of algorithms are ubiquitous and include the following examples: The Human Genome Project has made great progress toward the goals of identifying all the 100,000 genes in human DNA, determining the sequences of the 3 billion chemical base pairs that make up human DNA, storing this information in databases, and developing tools for data analysis. Each of these steps requires sophisticated algorithms.

7. The Internet enables people all around the world to quickly access and retrieve large amounts of information. With the aid of clever algorithms, sites on the Internet are able to manage and manipulate this large volume of data. Examples of problems that make essential use of algorithms include finding good routes on which the data will travel and using a search engine to quickly find pages on which particular information resides. There are two characteristics that are common to many interesting algorithmic problems: 1. They have many candidate solutions, so finding one that is “best,” can present quite a challenge. 2. They have practical applications. Like, finding the shortest path provides the easiest examples. Or a routing node on the Internet may need to find the shortest path through the network in order to route a message quickly. Cont…

8. Algorithm as technology Computers may be fast, but they are not infinitely fast. And memory may be inexpensive, but it is not free. Computing time is therefore a bounded resource, and so is space in memory. So, these resources should be used wisely, and algorithms that are efficient in terms of time or space will help. Efficiency : Different algorithms devised to solve the same problem often differ dramatically in their efficiency. These differences can be much more significant than differences due to hardware and software. Example: Consider two algorithms for sorting. The first, insertion sort, takes time roughly equal to c 1 n 2 to sort n items, where c 1 is a constant that does not depend on n. The second, merge sort, takes time roughly equal to c 2 n lg n, and c 2 is another constant that also does not depend on n. Consider faster computer (computer A) running insertion sort against a slower computer (computer B) running merge sort. They each must sort an array of 10 million numbers. Computer A takes Computer B takes

9. Insertion sort Algorithms Input: A sequence of n numbers. Output: A permutation (reordering) of the input sequence such that a 1 ’ ≤ a 2 ’ ≤ …. ≤ a n ’

10. Merge Sort The divide-and-conquer paradigm involves three steps at each level of the recursion: Divide the problem into a number of subproblems that are smaller instances of the same problem. Conquer the subproblems by solving them recursively. If the subproblem sizes are small enough, however, just solve the subproblems in a straightforward manner. Combine the solutions to the subproblems into the solution for the original problem. The merge sort algorithm closely follows the divide-and-conquer paradigm. Divide: Divide the n-element sequence to be sorted into two subsequences of n=2 elements each. Conquer: Sort the two subsequences recursively using merge sort. Combine: Merge the two sorted subsequences to produce the sorted answer.

11. Cont… Analyzing divide-and-conquer algorithms When an algorithm contains a recursive call to itself, often its running time is described by a recurrence equation or recurrence, which describes the overall running time on a problem of size n in terms of the running time on smaller inputs. The mathematical tools can be used to solve the recurrence and provide bounds on the performance of the algorithm.

12. Cont… Analyzing divide-and-conquer algorithms : When an algorithm contains a recursive call to itself, often its running time is described by a recurrence equation or recurrence, which describes the overall running time on a problem of size n in terms of the running time on smaller inputs. The mathematical tools can be used to solve the recurrence and provide bounds on the performance of the algorithm. The recurrence is Where D(n ) time to divide the problem into subproblems and C(n ) time to combine the solutions to the subproblems into the solution to the original problem. The divide step just computes the middle of the subarray, which takes constant time. Thus, D(n )= Θ(1 ). Two subproblems are recursively solved, each of size n=2, which contributes 2T(n/2) to the running time. We have already noted that the MERGE procedure on an n-element subarray takes time Θ(n ), and so C(n )= Θ(n ).

13. Cont… Adding above functions to recurrence gives Master theorem, which can be used to show that T(n) is Θ (n lg n). With Merge sort Θ (n lg n) running time, outperforms insertion sort, whose running time is Θ (n 2 ), in the worst case. Let us rewrite recurrence as where the constant c represents the time required to solve problems of size 1 as well as the time per array element of the divide and combine steps. After recursively solving the above recurrence and ignoring the low-order term and the constant c gives the desired result of Θ (n lg n).

14. 0110111111011011000101 A TM consists of an infinite length tape, on which input is provided as a finite sequence of symbols. A head reads the input tape. The TM starts at start state s 0. On reading an input symbol it optionally replaces it with another symbol, changes its internal state and moves one cell to the right or left. Example: EG Successor Program Sample Rules: If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read , write 1, HALT! Let’s see how they are carried out on a piece of paper that contains the reverse binary representation of 47: Turing Machine

15. So the successor’s output on 111101 was 000011 which is the reverse binary representation of 48. 111101 011101 001101 000101 000001 000011