1. 242-535 ADA: 4.5. Matrix Mult.1 Objective o an extra divide and conquer example, based on a question in class Algorithm Design and Analysis (ADA) 242-535, Semester 1 2015-2016 4.5. Divide and Conquer: Matrix Multiplication

2. 242-535 ADA: 4.5. Matrix Mult.2 Input: A = [a i j ], B = [b i j ] Output: C = [c i j ] = A×B Matrix Multiplication i, j = 1, 2,…, n.

3. 242-535 ADA: 4.5. Matrix Mult.3 void multiply(int[][] A, int[][] B, int[][] C, int n) { for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { C[i][j] = 0; for (int k = 0; k < n; k++) C[i][j] += A[i][k] * B[k][j]; } } } Running time = O(n 3 ) Standard Algorithm

4. 242-535 ADA: 4.5. Matrix Mult.4 Matrix multiplication: T(n) is O(n 3 ) Matrix addition: T(n) is O(n 2 ) Can we use divide and conquer to speed up matrix multiplication? Can we replace some multiplications by additions? Observations / Questions

5. 242-535 ADA: 4.5. Matrix Mult.5 n*n matrix = 2*2 matrix of (n/2) *(n/2) submatrices: Divide and Conquer Alg. 8 mults of (n/2) *(n/2) submatrices 4 adds of (n/2) *(n/2) submatrices CAB recursive

6. 242-535 ADA: 4.5. Matrix Mult.6 T(n) = 8 T(n/2) + O(n 2 ) n log b a = n log 2 8 = n 3 Master method, case 1: T(n) is O(n 3 ) No better than the ordinary algorithm Analysis of Algorithm no. of submatricies submatrix size work adding submatricies

7. 242-535 ADA: 4.5. Matrix Mult.7 Multiply 2*2 matrices with only 7 mults, not 8. Volker Strassen’s Idea (1969) Total calculations are: 7 mults, 18 adds/subs Special submatricies

8. 242-535 ADA: 4.5. Matrix Mult.8 Pictorial Explanation from wikipedia means "-M2" means "-M5" = = = = sum the Ms

9. 242-535 ADA: 4.5. Matrix Mult.9 The sum of a row shows how a Cxy is calculated from the M's o requires one multiplication for each red/green "1" o green means "add the result" o red means "subtract the result" Each of the M columns is how one of the special submatricies are made close-ups:

10. 242-535 ADA: 4.5. Matrix Mult.10 1. Divide : Partition A and B into (n/2) x (n/2) submatrices. o Form terms to be multiplied using + and - 2. Conquer : Perform 7 multiplications of (n/2) x (n/2) submatrices recursively. 3. Combine : Form product matrix C using + and - on (n/2) x (n/2) submatrices. T(n) = 7 T(n/2) + O(n 2 ) Strassen’s algorithm

11. 242-535 ADA: 4.5. Matrix Mult.11 T(n) = 7 T(n/2) + O(n 2 ) n log b a = n log 2 7 ≈ n 2.81 Master method case 1: T(n) is O(n log 2 7 ) or O(n 2.81 ) 2.81 may not seem much smaller than 3, but the difference is in the exponent. The impact on running time is significant. Strassen’s algorithm beats the ordinary algorithm on today’s machines for n ≥ 32 or so Coppersmith-Winograd, 1987 : O(n 2.3755 ) Williams & Le Gall, 2014 : O(n 2.3729 ) Analysis of Strassen not practical for real n not practical for real n

12. 242-535 ADA: 4.5. Matrix Mult.12 Cormen et al., ch 4, section 4.2 https://en.wikipedia.org/wiki/ Strassen_algorithm Java code: http://massivealgorithms.blogspot.com/2014/07 /divide-and-conquer-set-5-strassens.html More Information