1. Advanced Algorithms Analysis and Design By Dr. Nazir Ahmad Zafar Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

2. Lecture No. 45 Review Lecture Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

3. Analysis independent of the variations in machine, operating system, language, compiler, etc. Our model was an abstraction of a standard generic single-processor machine, called a random access machine RAM –infinitely large random-access memory, –instructions execute sequentially Every instruction, a basic operation taking unit time. We identified some weaknesses in our model of computation but finally we proved that with all these weaknesses, our model is not so bad because it fulfils our needs in design and analysis of algorithms Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Lecture No 1: Model of Computation

4. A Sequence of Mathematical Tools Sets, Sequences, Cross Product, Relation, Functions, Operators over above structures Logic and Proving Techniques Propositional Logic, Predicate Logic Proofs Logical Equivalences, Contradiction, Rule of Inference Mathematical Induction Simple Induction Strong Induction Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Lecture No 2, 3, 4 & 5: Mathematical Tools

5. Fibonacci Sequences Recursion? Recursive Mathematical Models First, second, higher order Linear Homogenous Recurrences with Constant Coefficients General Homogenous Recurrence when –Roots distinct, repeated, multiplicity of root is k –many roots with different multiplicities Non-homogenous Recurrence, Characteristics and solution Recursive Tree methods Substitution Method Proof of Master Theorem Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Lecture 6, 7, 8, & 9: Recursion

6. Major Factors in Algorithms Design Complexity Analysis Growth of Functions Asymptotic Notations Usefulness of Notations Reflexivity, Symmetry, Transitivity Relations over , , O,  and o Relation between ,  and O Various Examples Explaining each concept Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Lecture 10 & 11: Asymptotic Notations

7. Brute Force Approach, Checking primality Sorting sequence of numbers Knapsack problem Closest pair in 2-D, 3-D and n-D Finding maximal points in n-D Divide and Conquer? Merge Sort algorithm Finding Maxima in 1-D, and 2-D Finding Closest Pair in 2-D Lecture 12, 13 & 14 Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

8. Optimizations Problems and Dynamic Programming 1.Chain-Matrix Multiplication 2.Assembly Line Scheduling Problem 3.Generalization to n-Line Assembly Problem 4.0-1 Knapsack Problem 5.Optimal Weight Triangulation 6.Longest Common sub-sequence problem 7.Optimal Binary Search Trees Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Lecture 15 - 24: Dynamic Programming

9. Statement: The chain-matrix multiplication problem can be stated as below: Given a chain of [A 1, A 2,..., A n ] of n matrices for i = 1, 2,..., n, matrix A i has dimension p i-1 x p i, find the order of multiplication which minimizes the number of scalar multiplications. Objective Function Let m[i, j] = minimum number of multiplications needed to compute A i..j, for 1 ≤ i ≤ j ≤ n Objective function = finding minimum number of multiplications needed to compute A 1..n i.e. to compute m[1, n] Dr Nazir A. Zafar Advanced Algorithms Analysis and Design 1.Chain Matrix Multiplication

10. There are two assembly lines each with n stations The jth station on line i is denoted by S i, j The assembly time at that station is a i,j. An auto enters factory, goes into line i taking time e i After going through the jth station on a line i, the auto goes on to the (j+1)st station on either line There is no transfer cost if it stays on the same line It takes time t i,j to transfer to other line after station S i,j After exiting the nth station on a line, it takes time x i for the completed auto to exit the factory. Problem is to determine which stations to choose from lines 1 and 2 to minimize total time through the factory. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design 2.Assembly-Line Scheduling

11. Let f i [j] = fastest time from starting point station S i, j Objective function = f* = min(f 1 [n] + x 1, f 2 [n] + x 2 ) l* = line no. whose n th station is used in fastest way. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design 2.Assembly-Line Scheduling Problem S1 j-1 85657 In Out 2 4 3 2 S i,j a i,j t 1,j-1

12. There are n assembly lines each with m stations The jth station on line i is denoted by S i, j The assembly time at that station is a i,j. An auto enters factory, goes into line i taking time e i After going through the jth station on a line i, the auto goes on to the (j+1)st station on either line It takes time t i,j to transfer from line i, station j to line i’ and station j+1 After exiting the nth station on a line i, it takes time x i for the completed auto to exit the factory. Problem is to determine which stations to choose from lines 1 to n to minimize total time through the factory. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design 3.n Line Assembly Scheduling Problem

13. Let f i [j] = fastest time from starting point to station S i, j l i [j] = Line no. 1 to n, for station j-1 used in fastest way t i [j-1] = transfer time from station S i, j-1 to station S i,, j a[i, j] = time of assembling at station S i, j f* = is minimum time through any way l* = line no. whose m th station is used in a fastest way Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design 3.n-Line Assembly Problem

14. Assumption Each item must be put entirely in the knapsack or not included at all that is why the problem is called 0-1 knapsack problem Remarks Because an item cannot be broken up arbitrarily, so it is its 0-1 property that makes the knapsack problem hard. If an item can be broken and allowed to take part of it then algorithm can be solved using greedy approach optimally 4.0-1 Knapsack Problem Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

15. A triangulation of a convex polygon is a maximal set T of pair-wise non-crossing chords. It is easy to see that such a set subdivides interior of polygon into a collection of triangles, pair-wise disjoint Problem Statement Given a convex polygon, determine a triangulation that minimizes sum of the perimeters of its triangles Dr Nazir A. Zafar Advanced Algorithms Analysis and Design 5.Optimal Weight Triangulation Problem

16. Statement: In the longest-common-subsequence (LCS) problem, we are given two sequences X = and Y = And our objective is to find a maximum-length common subsequence of X and Y. Note: This LCS problem can be solved using brute force approach as well but using dynamic programming it will be solved more efficiently. 6.Longest Common Subsequence Problem Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

17. If X = (x 1, x 2,..., x m ), and Y = (y 1, y 2,..., y n ) be sequences and let us suppose that Z = (z 1, z 2,..., z k ) be a longest common sub-sequence of X and Y 1.if x m = y n, then z k = x m and Z k – 1 is LCS of X m – 1, Y n-1. 2.If x m  y n, then z k  x m implies that Z is LCS of X m – 1 and Y 3.If x m  y n then z k  y n implies Z is LCS of X and Y n – 1 Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design Optimal Substructure of an LCS

18. A translator from English to, say, Urdu. Use a binary search tree to store all the words in our dictionary, together with their translations. The word “the” is much more likely to be looked up than the word “ring” So we would like to make the search time for the word “the” very short, possibly at the expense of increasing the search time for the word “ring.” Problem Statement: We are given a probability distribution that determines, for every key in the tree, the likelihood that we search for this key. Objective is to minimize expected search time of tree. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design 7.Optimal Binary Search Trees

19. List of Greedy Algorithms discussed in this Course 1.Activity Selection Problem 2.Fractional Knapsack Problem 3.Coin Change Making Problem 4.Huffman Problem 5.Road Trip Problem Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Lecture 25-26: Greedy Algorithms

20. We went through the following steps in the above problem: 1.Determine the suboptimal structure of the problem. 2.Develop a recursive solution. 3.Prove that at any stage of the recursion, one of the optimal choices is the greedy choice. Thus, it is always safe to make the greedy choice. 4.Show that all but one of the sub-problems induced by having made the greedy choice are empty. 5.Develop a recursive algorithm that implements the greedy strategy. 6.Convert this recursive algorithm to an iterative one. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design Steps Designing Greedy Algorithms

21. In Huffman coding, variable length code is used Data considered to be a sequence of characters. Huffman codes are a widely used and very effective technique for compressing data –Savings of 20% to 90% are typical, depending on the characteristics of the data being compressed. Huffman ’ s algorithm uses table of frequencies of occurrence of characters to build up an optimal way of representing each character as a binary string. Objective in Huffman coding is to develop a code that represents given text as compactly as possible Huffman Codes Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

22. Graph Theoretic Algorithms Graph Concepts and types of graphs Representation of graphs Searching Algorithms Backtracking, Branch and Bound Algorithms Applications of Searching Algorithm Minimal Spanning Tree Algorithms –Kruskal’s Algorithm –Prim’s Algorithm Shortest Path Algorithms Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Lecture 27-37: Graph Theoretic Algorithms

23. Given a graph G = (V, E) such that –G is connected and undirected –w(u, v) weight of edge (u, v) T is a Minimum Spanning Tree (MST) of G if –T is acyclic subset of E (T  E) –It connects all the vertices of G and –Total weight, w(T) = is minimized. Minimum Spanning Tree Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

24. Shortest Path Problems Single-source shortest path –The Bellman-Ford Algorithm –Shortest Path in directed acyclic graphs –Dijkstra’s Algorithm Single-destination shortest path Single-pair shortest path All-pairs shortest-paths –Matrix Multiplication –The Floyd-Warshall Algorithm –Johnson’s Algorithm

25. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design The Bellman-Ford Algorithm BELLMAN-FORD (G, w, s) 1 INITIALIZE-SINGLE-SOURCE (G, s) 2 for i ← 1 to |V [G]| - 1 3 do for each edge (u, v)  E[G] 4 do RELAX (u, v, w) 5 for each edge (u, v)  E[G] 6 do if d[v] > d[u] + w(u, v) 7 then return FALSE 8 return TRUE  (V) }  (V.E) } O(E) Total Running Time = O(V.E) Contd..

26. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design Algorithm : Shortest Path (dag) DAG-SHORTEST-PATHS (G, w, s) 1 topologically sort the vertices of G 2INITIALIZE-SINGLE-SOURCE (G, s) 3 for each vertex u, taken in topologically sorted order 4 do for each vertex v  Adj[u] 5 do RELAX (u, v, w)  (V+E) Each iteration of for loop takes  (1) Total Running Time =  (V+E)  (V)  (V)  (E)

27. Input Given graph G(V, E) with source s, weights w Assumption Edges non-negative, w(u, v) ≥ 0,  (u, v)  E Directed, if (u, v)  E then (v, u) may or may not  E Objective: Find shortest paths from s to every u  V Approach Maintain a set S of vertices whose final shortest-path weights from s have been determined Repeatedly select, u  V – S with minimum shortest path estimate, add u to S, relax all edges leaving u. Greedy, always choose light vertex in V-S, add to S Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design Dijkstra’s Algorithm

28. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design Lecture 38-40: Number Theoretic Algorithms Applications of Number Theory Some Important Concepts and Fats useful in number theoretic Algorithms Modular Arithmetic Finding GCD Euclid’s Algorithm Extended Euclid’s Algorithm Residues and Reduced set of Residues Chinese Remainder Theorem RSA Cryptosystem

29. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design 1.Choose two distinct large random prime numbers p and q such that p  q 2.Compute n by n = pq, n is used as modulus 3.Compute the totient function  (n) 4.Choose an integer e such that 1 < e <  (n) and e and  (n) share no factors other than 1 5.Compute d to satisfy the congruence relation; de ≡ 1 mod  (n) i.e. de = 1 + k  (n) for some integer k 6.Publish the pair P =(e, n) as his RSA public Key 7.Keep secret pair S =(d, n) as his RSA secret Key The RSA Public Key Cryptosystem

30. Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design Lecture 41-44: Further Topics String Matching Problem –Naïve approach –Rabin Karp algorithm –String Matching using Finite automata Polynomials and Fast Fourier Transform NP Completeness