1 Mathematical Algorithms 1. Arithmetic, (Pseudo) Random Numbers and all that 2. Evaluation & Multiplication of Polynomials I 3. Multiplication of Large Integers 4. Strassen's algorithm for large Matrices 5. Zeros of Polynomials: Bisection vs Newton's method 6. Evaluation & Multiplication of Polynomials II 7. Fast Fourier Transforms 8. Interpolation/splines etc. 9. SHOULD BE A SEPARATE COURSE!!! Ari’s class: PHO 211 – 4/13/06

2 Adding, Multiplying and Evaluating Polynomials Why are polynomials so interesting? P N (x) = a 0 + a 1 x + a 2 x 2  a N x N Fit data d i = P N (x i ) by picking a i ’s Integer arithmetic base B is polynomial with x = B: e.g Integer decimal base (B=10) P N-1 (10) = a 0 + a a  a N-1 10 N-1 Fourier transform is P N-1 (x k ) with x k =  k N = exp[i k 2  /N] y k ´  l e ik n 2  /N a n ETC

3 Evaluation of p(x) = a 0 + a 1 x a N x N Method 1:  Compute x, x 2, x 3,...x N with N-1-mults plus N mults and N adds or 2N-1 mults and N adds Horner’s Method 2:  a 0 + x(a 1 + x (a 2 + x (a 3 +.a 4 x) ) ) for N = 4 N adds and N multiplies!  Evaluating it at N points is O(N 2 ) Special case: p(x) = x N do in Log(N) steps!  Consider N in binary base eg Cal x, x 2, x 4, x by repeated squaring and then multiply to get x N in O(Log 2 (N)) steps.

4 Polynomial Multiplication P(x) = a 0 + a 1 x a N x N and Q(x) = b 0 + b 1 x b N x N P(x) Q(x) = a 0 b 0 + (a 0 b 1 + a 1 b 0 )x + (a 0 b 2 + a 1 b 1 + a 2 b 0 )x a N b N x 2N  This is O(N 2 ): N-1 to get to x N and similarly for x N+1 to N 2N Try devide and conquer: P N (x) = p L (x) + x N /2 p’ H (x) p L (x) = a 0 + a 1 x +... a N/2 x N/2 p H (x) = a N/ a N/2 x N/2 P(x) Q(x) = p L (x) q L (x) + (p L (x) q H (x) + p H (x) q L (x)) x N/2 + p H (x) q H (x) x N T(N) = 4 T(N/2) + c 0 N  T(N) = O(N 2 ) Better: define (p L (x) + p H (x))(q L (x) + q H (x)) – p L (x) q L (x) – p H (x) p H (x) T(N) = 3 T(N/2) + c 0 N  T(N) = O(N  )

5 Large Integer Multiply: Z = X * Y Set: X & Y has at most N Digits:  X= 137 = 1* * Y= 025 = 0* * Steps: Single Digit Multipliers and Adders with carry. e.g. Standard method:  Complexity: T(N) = const * N 2  Is this best algorithm?  Is there a best upper bound A x B C

6 Beating O(N ) by “Divide & Conquer y Split each N digit integer into two N/2 integers:  X = XH * 10 N/2 + XL and Y = YH * 10 N/2 + YL  X*Y = XH*YH 10 N + (XH*YL + XL*YH) 10 N/2 + XL*YL  Number of Multiplies: 4 * (N/2) 2 = N 2 NO CHANGE ? BUT: XH*YL + XL*YH = (XH + XL)*(YH + YL) – XH*YH – XL*YL “Really 3 times half problems or 3 (N/2) 2 = 3N 2 /4 RECURSIVELY this gives T(N) = const N log(3)/log(2) = N 1.59… 2 y See Weiss Sec page 419

7 Large Integer Multiplication: T(N) = 3 T(N/2) + c N using: adding:

8 Integer Multiplication: T(N) = ? FunctionValueComplexity XL 6143Input O(N) XH8521Input O(N) YL9473Input O(N) YH6407Input O(N) D1 = XH + XL14664N adds D2 = YH + YL15880N adds XL*YL T(N/2) XH*YH T(N/2) D1* D T(N/2) D3 = D1*D2 - XH*YH - XL*YL N adds XH*YH + D3* XH*YH Add/Output O(N) Direct Method: T(N) = c1 N 2 + c2 N O(c1 N 2 + c2 N) = O(N 2 ) One Level: T(N) = 3 * (c1 (N/2) 2 + c2 N/2) O(0.75 c1 N 2 ) = O(N 2 ) Naive Recursion: T(N) = 4 T(N/2) + c N O(N 2 ) Smart Recursion: T(N) = 3 T(N/2) + c O( N l l og(3)/log(2) )

9 Strassen’s Algorithm O(N 2.81 ), p. 422 where: T(N) = 7 T(N/2) + c N 2 Naïve T(N) = c N ^3

10 Zeros of P N (x): Bisection vs Newton’s Methods P_N(x) has N complex roots. No closed form for roots for Quintic (N=5) & above Real roots by bisection: accuracy  = 1/N  O(log(N)) Newton’s Method: accuracy  = 1/N  O(log(N))

11 Multiplying Polynomials and FFTFFT Multiply P N/2 (x), Q N/2 (x) 1. Evaluate at 2N points (k=0,  N-1) O(N 2 ) 2. W(x k ) = Q(x k ) P(x k ) O(N) 3. Interpolate to find c k, n = 0,...,N-1 O(N 2 ) a i, b j c _k Q(x _k ) P(x _k ) W(x k ) = Q(x k ) P(x k ) O(N 2 ) O(N log(N)) O(N) O(N log(N)) FFT convolution multiply

12 FFT: x k =  k N = exp[i 2  k/N] spit polynomial into low/high pieces: even k odd k  One N = Two N/2 Fourier transforms

13 Thus T(N) = 2 T(N/2) + c o N: T(N) » N Log(N) Insert butterfly and discuss bit reversal

14 End Ari’s Class