Notes on the analysis of multiplication algorithms.. Dr. M. Sakalli, Marmara University

3-1 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Integer Multiplication MIT notes and wikipedia Example.. Classic High school math.. Let g = A|B and h = C|D where A,B,C and D are n/2 bit integers Simple Method: gh = (2 n/2 A+B)(2 n/2 C+D) same as given above. 4 multiplication routines. XY = (2 n )AC+2 n/2 (AD+BC) + BD and carriages c. Long multiplication: r j = c + Σ k = i-j g j h k Running Time Recurrence T(n) < 4T(n/2) + 100n, 100 multiplications.??, In-place??.. T(n) =  (n 2 ) Provided that neither c nor the total sum exceed log space, indeed, a simple inductive argument shows that the carry c and the total sum for r i can never exceed n and 2 n : <<?? 2lg n respectively. Space efficiency: S(n)=O(loglog(N)),  (loglog(N)). N= gh.

3-2 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Integer Multiplication MIT notes and wikipedia Pseudo code: Log space multiplication algorithm, multiply( g [0..n-1], h [0..n-1]) // Arrays representing to the binary representations x ← 0 for i= 0 : 2n-1 for j= 0 : i for j= 0 : i k ← i - j k ← i - j x ← x + ( g [j] × h [k]) x ← x + ( g [j] × h [k]) r[i] ← x mod 2 r[i] ← x mod 2 x ← floor(x/2) //I think this is carriage return. Last bit if 1.. x ← floor(x/2) //I think this is carriage return. Last bit if 1.. end endend Lattice method, Muhammad ibn Musa al-Khwarizmi. Gauss's complex multiplication algorithm. Muhammad ibn Musa al-KhwarizmiMuhammad ibn Musa al-Khwarizmi

3-3 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Integer Multiplication MIT notes and wikipedia Karatsuba’s algorithm: Polynomial extensions.. g = g 1 10 n/2 + g 2 h = h 1 10 n/2 + h 2 g h = g 1 h 1 10 n + (g 1 h 2 + g 2 h 1 )10 n/2 + g 2 h 2 (g 1 h 2 + g 2 h 1 ) = (g 1 + h 1 )(g 2 + h 2 ) - (g 2 h 2 + g 1 h 1 ), f(n) = 4sums+1 more final sum = 5n, n>2, suppose it is a constant 100n, and some carriages. XY = (2 n/2 +2 n )AC+2 n/2 (A-B)(C-D) + (2 n/2 +1) BD A(n) = 3A(n/2)+5n, A(n) < O(n lg 3 ) ≈  (n 1.6 ) Base value 7, when n<2,

3-4 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Karatsuba (g, h : n-digit integer; n : integer) // return (2n)-digit integer is // return (2n)-digit integer is a, b, c, d;// (n/2)-digit integer U, V, W; //n-digit integer; begin if n == 1 then if n == 1 then return  g(0)*h(0); ???? return  g(0)*h(0); ???? else else g1  g(n-1)... g(n/2); g1  g(n-1)... g(n/2); g2  g(n/2-1)... g(0); g2  g(n/2-1)... g(0); h1  h(n-1)... h(n/2); h1  h(n-1)... h(n/2); h2  h(n/2-1)... h(0); h2  h(n/2-1)... h(0); U  Karatsuba ( g1, h1, n/2 ); U  Karatsuba ( g1, h1, n/2 ); V  Karatsuba ( g2, h2, n/2 ); V  Karatsuba ( g2, h2, n/2 ); W  Karatsuba ( g1+g2, h1+h2, n/2 ); W  Karatsuba ( g1+g2, h1+h2, n/2 ); return  U*10 n + (W-U-V)*10^n/2 + V; return  U*10 n + (W-U-V)*10^n/2 + V; end if; end if; end Karatsuba; end Karatsuba; FFT and Fast Matrix multiplication.

3-5 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Quarter square multiplier 1980, Everett L. Johnson: gh = {( g + h ) 2 - ( g - h ) 2 }/4= {( g hg + h 2 ) - ( g hg + h 2 ) }/4 Think hardware implementation, with a lookup table (converter), the difficulty is that summation of the two numbers each 8bits, will require at least 9 bits, when squared, 18 bits wide.. But if divided by 2 before squared, (discarding remainder when n is odd). Table lookup from 0 to.. 9+9, from 0 … to 81. O(3n), working S(n) =  (n). i.e. 7 by 3, observe that the sum and difference are 10 and 4 respectively. Looking both of those values up on the table yields 25 and 4, the difference of which is 21.

3-6 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Russian (Egyptian) Peasant’s binary multiplication Shift and add.. In-place algorithm, may be implemented and 2n space.. Try complex examples. 11 3, in binary , , , = T(n) =  (n)+O(n 2 ), think about this?.. Why.. S(n) =  (loglog(n)) which is the carriage. If invertible? Division potential question.

3-7 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Matrix multiplication, 8 multiplications, O(n 3 ) Pseudo code for MM. MM(A, B) for i ← 1 : N for j ← 1 : N for j ← 1 : N C(i, j) ← 0; C(i, j) ← 0; for k ← 1 : N for k ← 1 : N C(i, j) ← C(i, j) + A(i, k) * B(k, j) C(i, j) ← C(i, j) + A(i, k) * B(k, j) end, end, end Time complexity of this algo is n 3 multiplications and additions. Can we do better using divide and conquer?.. Subdividing matrices into four sub-matrices. T(n) = b, n  2, T(n) = 8T(n/2) + c n 2, n>2, which has T(n) = O(n??)

3-8 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Strassen’s Algorithm Strassen: 7 multiplies, 18 additions T(n) = b, n  2, T(n) = 7T(n/2) + (7m+18s) n 2, n>2, which has T(n) = O(n 2.81 ) 7n 2 (1/4+1/16+…) Strassen-Winograd: 7 multiplies, 15 additions Coppersmith-Winograd, O(n ) (not easily implementable) In practice faster (not large hidden constants) for relatively smaller n~64, and stable but demonstrated that for some matrices (Strassen and Strassen-Winograd) are too unstable.