1. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Design and Analysis of Algorithms Khawaja Mohiuddin Assistant Professor, Department of Computer Sciences Bahria University, Karachi Campus, Contact: Khawaja.mohiuddin@bimcs.edu.pk Lecture # 11 – Divide And Conquer

2. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 2  What is Divide And Conquer Strategy?  The most-well known algorithm design strategy: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances recursively 3. Obtain solution to original (larger) instance by combining these solutions

3. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 3  What is Divide And Conquer Strategy?

4. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 4  Example : Multiplication of Large Integers  Consider the problem of multiplying two (large) N-digit integers represented by arrays of their digits such as: A = 12345678901357986429 B = 87654321284820912836 The grade-school algorithm: a 1 a 2 … a n b 1 b 2 … b n a 1 b n a 2 b n … a n b n a 1 b n-1 a 2 b n-1 … a n b n-1 … … … … … … … … + a 1 b 1 a 2 b 1 … a n b n1 Runtime: O(n 2 ) single-digit multiplications

5. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 5  Example : Multiplication of Large Integers – First Algorithm  In general, if A = A 1 A 2 and B = B 1 B 2 (where A and B are n-digit, A 1, A 2, B 1, B 2 are n/2-digit numbers), A  B = A 1  B 1 ·10 n + (A 1  B 2 + A 2  B 1 ) ·10 n/2 + A 2  B 2  Example: A  B where A = 2135 and B = 4014 A = (21·10 2 + 35), B = (40 ·10 2 + 14) So, A  B = (21 ·10 2 + 35)  (40 ·10 2 + 14) = 21  40 ·10 4 + (21  14 + 35  40) ·10 2 + 35  14  Recurrence for the number of one-digit multiplications M(N): M(N) = 4M(N/2), M(1) = 1 Solution: M(N) = 4 log 2 N = N log 2 4 = N 2 Runtime = O(N 2 )

6. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 6  Example : Multiplication of Large Integers- Analysis  This procedure can be used recursively to reduce multiplication of two integers of any length to a sequence of single-digit multiplication  This certainly makes the algorithm easier, but it does not reduce the computational complexity  At each stage in the recursion, the number of multiplications increases by a factor of four and the size of the numbers being multiplied decreases by a factor of two – so the number of single-digit multiplications in each of the multiplications decreases by a factor of four  These effects cancel and there is no reduction in the number of single-digit multiplications. It is still of order O(N 2 )

7. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 7  Example : Multiplication of Large Integers- Second Algorithm A  B = A 1  B 1 ·10 n + (A 1  B 2 + A 2  B 1 ) ·10 n/2 + A 2  B 2 The idea is to decrease the number of multiplications from 4 to 3: (A 1 + A 2 )  (B 1 + B 2 ) = A 1  B 1 + (A 1  B 2 + A 2  B 1 ) + A 2  B 2, i.e., (A 1  B 2 + A 2  B 1 ) = (A 1 + A 2 )  (B 1 + B 2 ) - A 1  B 1 - A 2  B 2, which requires only 3 multiplications at the expense of (4-1) extra add/sub. Recurrence for the number of multiplications M(n): M(N) = 3M(N/2), M(1) = 1 Solution: M(n) = 3 log 2 N = N log 2 3 ≈ N 1.585

8. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 8  Example : Multiplication of Large Integers- Second Algorithm Example  To compute the product of 12345 and 6789, choose B = 10 and m = 3. Then we decompose the input operands using the resulting base (B m = 1000), as:  12345 = 12 · 1000 + 345 6789 = 6 · 1000 + 789 Only three multiplications, which operate on smaller integers, are used to compute three partial results:  z 2 = 12 × 6 = 72 z 0 = 345 × 789 = 272205 z 1 = (12 + 345) × (6 + 789) − z 2 − z 0 = 357 × 795 − 72 − 272205 = 283815 − 72 − 272205 = 11538 We get the result by just adding these three partial results, shifted accordingly (and then taking carries into account by decomposing these three inputs in base 1000 like for the input operands):  result = z 2 · B 2m + z 1 · B m + z 0, i.e. result = 72 · 1000 2 + 11538 · 1000 + 272205 = 83810205.

9. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 9  Example : Conventional Matrix Multiplication  Brute Force Algorithm  Total : 8 Multiplications  Runtime: O(N 3 )

10. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 10  Example : Strassen’s Matrix Multiplication  Divide each of the matrices into four sub-matrices, each of dimensions N/2 x N/2  Strassen derived the following result  7 Multiplications

11. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 11  Example : Strassen’s Matrix Multiplication  M 1 = (a 00 + a 11 ) * (b 00 + b 11 )  M 2 = (a 10 + a 11 ) * b 00  M 3 = a 00 * (b 01 - b 11 )  M 4 = a 11 * (b 10 - b 00 )  M 5 = (a 00 + a 01 ) * b 11  M 6 = (a 10 - a 00 ) * (b 00 + b 01 )  M 7 = (a 01 - a 11 ) * (b 10 + b 11 )

12. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 12  Example : Strassen’s Matrix Multiplication - Analysis  If N is not a power of 2, matrices can be padded with zeros. Number of multiplications: M(N) = 7M(N/2), M(1) = 1 Solution: M(N) = 7 log 2 N = N log 2 7 ≈ N 2.807 vs. N 3 of brute-force algorithm Algorithms with better asymptotic efficiency are known but they are even more complex and not used in practice

13. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 13  Triangulation  Triangulation is a process that takes a region of space and divides it into sub-regions  The space may be of any dimensions (e.g. 2D), and the sub-regions typically are shaped as triangles (2D), tetrahedrons (3D), and so on.  The goal of triangulation is to produce a sub-division that obeys certain “nice” properties  For example: one in which the output triangles are nearly equilateral

14. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 14  Triangulation  Triangulation is used in many real-world applications  One such application is the finite element simulation, in which a partial differential equation (PDE) is simulated over a continuous surface or space by first using a triangulation to break the space into small, discrete element, then simulating the PDE discretely over the elements  Finite element simulations are common in fluid flow applications and similar problems  Other important applications of triangulation include:

15. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 15  Triangulation  Surface Approximation: A continuous surface is approximated by a mesh of triangles. This is used in graphic applications (example, face representation), compression of surface representations, interpolation of surfaces, and construction applications (example, optimizing dirt movement to contour a region for new construction)  Nearest Neighbour Structure Identification: The triangulation establishes data identifying the nearest neighbours of each point in the triangulated data set. This is potentially useful for clustering applications

16. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 16  Convex Hulls  Given a set of points S, the convex hull problem is to find the smallest convex region that contains all points in S  The boundary of the region is called the convex hull of S, and is usually specified by a set of points and their neighbour relation  Convex hulls are well-defined in any dimension, a 2-dimensional example is given in Figure 11.1  The following is a summary of the algorithms for finding convex hulls based on D&C strategy  Mergehull: Time Complexity: T(sort) + O(N)  Quickhull: Time Complexity: O(N) best, O(N 2 ) Worst, O(NLogN) Average  Kirkpatrick-Seidel: Time Complexity: O(Nlog H)

17. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 17  Example – Convex Hulls Convex Hulls Figure 11.1: Example of 2-D Convex Hull Extreme Point

18. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 18  Quickhull Algorithm  The Quickhull algorithm is one of the algorithms based on the recursive divide-and- conquer method for finding convex hulls  It is similar in structure to the Quicksort algorithm  Quickhull works in (2-D) by locating three points A, B, and C on the convex hull of its input S (with C above both A and B), then recursively refine the hull between AC and CB  When the algorithm completes, the upper half-hull between A and B will be complete

19. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 19  Quickhull Algorithm 1. Find the farthest point C from the line joining A and B. To do this is the line-side test can be used, as it returns the area of the triangle ABC, and the triangle with largest area is the one that has the farthest point C. Note that, when building the upper half-hull, C should be above AB, whereas C should be above the inverse line BA when building the lower half-hull 2. Discard all points in triangle ABC, as they will not be on the boundary of the final hull (since ABC is contained within the hull) 3. Recurse on (A,C) and (C,B) to fully refine the hull between A and C, and between C and B

20. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 20  Quickhull Algorithm 4. Connect the two resulting hulls at C 5. Repeat for the lower half-hull, and connect it at A and B  The worst case running time of the Quickhull algorithm is O(N 2 ), for if the splits (choices of C) are bad, the algorithm may only eliminate one point on each recursion, resulting in a recursion depth of N and a per-recursion cost of N.  Despite this poor worst-case bound, Quickhull runs in O(NlogN) time for most “realistic” point sets  In practice, it can run as fast as O(N), since the algorithm discards points in Step 2, reducing the size of the input set for further recursions

21. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 21  Where Divide & Conquer Fails?  All divide & conquer (D & C) problems do not exhibit optimal problem division  Consider the problem of visiting all the vertices of a graph in a simple cycle, starting from a chosen vertex  The problem is to find the cycle that yields the smallest total edge-cost  We divide the problem into two parts – the set of vertices to be visited before the last vertex, and the last vertex  The optimal tour is A→B→C→D→E→A, but when the last vertex E is removed, the optimal tour for sub-problems (ABCD) is A→B→C→D→A, which will not lead to complete optimal solution.

22. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Divide And Conquer 22  Where Divide & Conquer Fails?  The type of problems for which divide-and-conquer is not suitable are: 1. Where the solution of size n depends upon n sub-solutions, each of size (n – 1) 2. Overlapping sub-problems

23. CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Summary 23  What is Divide and Conquer Strategy?  A Multiplication Algorithm and Its Analysis  Triangulation  Convex Hulls  Where Divide And Conquer Fails?