Design and Analysis of Algorithms Level : 7, Group : 31 Mohammad Shahnawaz Nasir College of Computer Science and Information System Jazan University , Jazan

Textbook Cover What does the cover include? Puzzles, Notions, and Stories of Algorithms.

Text Book and Reference Books Anany Levitin, Introduction to the Design and Analysis of Algorithms, Tsinghua University Press, 2003, 39RMB. Reference Books: M.H. Alsuwaiyel, Algorithms Design Techniques and Analysis, Publishing House of Electronics Industry, 2003, 40RMB. Thomas H. Cormen etc., Introduction to Algorithms (Second Edition), Higher Education Press & The MIT Press, 68RMB. Others, search on the website with keyword “Design and Analysis of algorithm”, you will find more...

Grading Schemes Total 100 Quizes: 10% Assignments: 5% Midterm Test: 20% Final Exam: 40% Lab Exam: 25% Bonus is possible for those positive and active students

Why Study this Course? Donald E. Knuth stated “Computer Science is the study of algorithms” Cornerstone of computer science. Programs will not exist without algorithms. Closely related to our lives Help to guide how others analyze and solve problems Help to develop the ability of analyzing and solving problems via computers Very interesting if you can concentrate on this course

Course Prerequisite Data Structure Discrete Structure C, Java or other programming languages Advanced Mathematics

Algorithm: A brief History Muhammad ibn Musa al-Khwarizmi, one of the most influential mathematicians in 9th century in Baghdad, wrote a textbook in Arabic about adding, multiplying, dividing numbers, and extracting square roots and computing π. www.lib.virginia.edu/science/parshall/khwariz.html Many centuries later, decimal system was adopted in Europe, and the procedures in Al Khwarizmi’s book were named after him as “Algorithms”. (image of Al Khwarizmi from http://jeff560.tripod.com/)

Notion: Algorithms problem algorithm “computer” output input An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. “computer” problem algorithm input output

Notion: Algorithm More precisely, an algorithm is a method or process to solve a problem satisfying the following properties: Finiteness - terminates after a finite number of steps Definiteness - Each step must be rigorously and unambiguously specified. Input - Valid inputs must be clearly specified. Output - can be proved to produce the correct output given a valid input. Effectiveness - Steps must be sufficiently simple and basic.

Examples Is the following a legitimate algorithm? i 1 While (i <= 10) do a  i + 1 Print the value of a End of loop Stop

Examples of Algorithms – Computing the Greatest Common Divisor of Two Integers The greatest common divisor of two non-negative, not-both-zero integers m and n, denoted gcd(m, n), is defined as the largest integer that divides both m and n evenly, i.e, with a remainder to zero. Euclid’s algorithm: gcd(m, n) = gcd(n, m mod n) For example gcd(60, 24) can be computed as follows: gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12 Gcd(70, 25) = gcd(25, 20) = gcd(20, 5) = gcd (5, 0) = 5 Gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12

Computing the Greatest Common Divisor of Two Integers Here is more structured description of this algorithm: Euclid’s Algorithm for computing gcd(m, n) Step1: If n = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2. Step2: Divide m by n and assign the value of the remainder to r. Step 3: Assign the value of n to m and the value of r to n. Go to Step 1.

Pseudocode of Euclid’s Algorithm Algorithm Euclid(m, n) //Computes gcd(m, n) by Euclid’s algorithm //Input: Two nonnegative, not-both-zero integers m and n //Output: Greatest common divisor of m and n while n ≠ 0 do r  m mod n m  n n  r return m .

Second Try for gcd(m, n) Consecutive integer checking algorithm for computing gcd(m, n) Step1: Assign the value of min{m, n} to t. Step2: Divide m by t. If the remainder of this division is 0, go to Step3;otherwise, go to Step 4. Step3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step4. Step4: Decrease the value of t by 1. Go to Step2. For example, for number 60 and 24, the algorithm will try first 24, then 23, and so on until it reaches 12, where it stops.

Third try for gcd(m, n) Middle-school procedure for computing gcd(m, n) Step1: Find the prime factors of m. Step2: Find the prime factors of n. Step3: Identify all the common factors in the two prime expansions found in Step1 and Step2. (If p is a common factor occurring Pm and Pn times in m and n, respectively, it should be repeated min{Pm, Pn} times.) Step4: Compute the product of all the common factors and return it as the gcd of the numbers given. Thus, for the numbers 60 and 24, we get 60 = 2.2.3.5 24 = 2.2.2.3 gcd(60,24) = 2.2.3 = 12

What can we learn from the previous 3 examples? Each step of an algorithm must be unambiguous. The same algorithm can be represented in several different ways. (different pseudo-codes) There might exists more than one algorithm for a certain problem. Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds.

Fundamentals of Algorithmic Problem Solving We can consider algorithms to be a procedural solutions to problems. These solutions are not answers but rather specific instructions for getting answers. The sequence of steps one typically goes through in designing and analyzing an algorithm: Understanding the problem: understand completely the problem given, ask questions if you have doubt, do a few examples by hand, think about special cases, etc. Deciding on: - you need to ascertain the capabilities of the computational device the algorithm is intended for. - Decision to choose between solving the problem exactly or solving it approximately. - Appropriate data structure

Fundamentals of Algorithmic Problem Solving Design an Algorithm: using a natural language has an obvious appeal. a pseudo-code is a mixture of natural language and programming language constructs. Proving an Algorithm’s Correctness: that is, you have to prove that the algorithm yields a required result for every legitimate input in a finite amount of time. Analyzing an algorithm: - Time efficiency : how fast the algorithm runs - Space efficiency: how much extra memory the algorithm needs. Coding an algorithm: ultimately implemented as computer programs.

Algorithm Design and Analysis Process

Two main issues related to algorithms How to design algorithms How to analyze algorithm efficiency

Algorithm Design Techniques/Strategies Brute force Divide and conquer Decrease and conquer Transform and conquer Space and time tradeoffs Greedy approach Dynamic programming Backtracking Branch and bound

Analysis of Algorithms How good is the algorithm? time efficiency space efficiency Does there exist a better algorithm? lower bounds optimality

Example: Fibonacci Number Fibonacci’s original question: Suppose that you are given a newly-born pair of rabbits, one male, one female. Rabbits are able to mate at the age of one month so that at the end of its second month a female can produce another pair of rabbits. Suppose that our rabbits never die. Suppose that the female always produces one new pair (one male, one female) every month. Question: How many pairs will there be in one year? 1. In the beginning: (1 pair) 2. End of month 1: (1 pair) Rabbits are ready to mate. 3. End of month 2: (2 pairs) A new pair of rabbits are born. 4. End of month 3: (3 pairs) A new pair and two old pairs. 5. End of month 4: (5 pairs) ... 6. End of month 5: (8 pairs) ... 7. after 12 months, there will be 233 rabbits (image of Leonardo Fibonacci from http://www.math.ethz.ch/fibonacci)

Recurrence Relation of Fibonacci Number fib(n): {0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …}

Algorithm: Fibonacci Problem: What is fib(200)? What about fib(n), where n is any positive integer? Questions that we should ask ourselves. 1. Is the algorithm correct? 2. What is the running time of our algorithm? 3. Can we do better? Algorithm 1 fib(n) if n = 0 then return (0) if n = 1then return (1) return (fib(n − 1) + fib(n − 2))

Answer of Questions Is the algorithm correct? Yes, we simply follow the definition of Fibonacci numbers How fast is the algorithm? If we let the run time of fib(n) be T(n), then we can formulate T(n) = T(n − 1) + T(n − 2) + 3  1.6n T(200)  2139 The world fastest computer BlueGene/L, which can run 248 instructions per second, will take 291 seconds to compute. (291 seconds = 7.85 × 1019 years ) Can Moose’s law, which predicts that CPU get 1.6 times faster each year, solve our problem? No, because the time needed to compute fib(n) also have the same “growth” rate if we can compute fib(100) in exactly a year, then in the next year, we will still spend a year to compute fib(101) if we want to compute fib(200) within a year, we need to wait for 100 years.

Improvement Can we do better? Algorithm 2: fib(n) Yes, because many computations in the previous algorithm are repeated. Algorithm 2: fib(n) comment: Initially we create an array A[0: n] A[0] ← 0,A[1] ← 1 for i = 2 to n do A[i] = A[i − 1] + A[i − 2] return (A[n])

Important problem types sorting searching string processing graph problems combinatorial problems geometric problems numerical problems

Fundamental data structures list array linked list string stack queue priority queue graph tree set and dictionary