1. Applied Symbolic Computation
September 4, 1997
Applied Symbolic Computation (CS 680/480) Lecture 6: Integer Multiplication, Interpolation, and the Chinese Remainder Theorem
Jeremy R. Johnson
May 9, 2001
May 9, 2001
Applied Symbolic Computation

2. Applied Symbolic Computation
September 4, 1997
Introduction
Objective: To show the relationship between the Chinese Remainder Theorem and Interpolation. To derive a series of fast integer multiplication algorithms using interpolation.
Karatsuba’s Algorithm
Polynomial algebra
Polynomial version of the Chinese Remainder Theorem
Interpolation
Vandermonde Matrices
Toom-Cook algorithm
Polynomial multiplication using interpolation
References: Lipson, Cormen et al.
May 9, 2001
Applied Symbolic Computation

3. Karatsuba’s Algorithm
Using the classical pen and paper algorithm two n digit integers can be multiplied in O(n2) operations. Karatsuba came up with a faster algorithm.
Let A and B be two integers with
A = A110k + A0, A0 < 10k
B = B110k + B0, B0 < 10k
C = A*B = (A110k + A0)(B110k + B0)
= A1B1102k + (A1B0 + A0 B1)10k + A0B0
Instead this can be computed with 3 multiplications
T0 = A0B0
T1 = (A1 + A0)(B1 + B0)
T2 = A1B1
C = T2102k + (T1 - T0 - T2)10k + T0
May 9, 2001
Applied Symbolic Computation

4. Complexity of Karatsuba’s Algorithm
Let T(n) be the time to compute the product of two n-digit numbers using Karatsuba’s algorithm. Assume n = 2k. T(n) = (nlg(3)), lg(3)  1.58
T(n)  3T(n/2) + cn
 3(3T(n/4) + c(n/2)) + cn = 32T(n/22) + cn(3/2 + 1)
 32(3T(n/23) + c(n/4)) + cn(3/2 + 1)
= 33T(n/23) + cn(32/22 + 3/2 + 1) …
 3iT(n/2i) + cn(3i-1/2i-1 + … + 3/2 + 1)
...
 cn[((3/2)k - 1)/(3/2 -1)] Assuming T(1)  c
 2c(3k - 2k)  2c3lg(n) = 2cnlg(3)
May 9, 2001
Applied Symbolic Computation

5. Divide & Conquer Recurrence
Assume T(n) = aT(n/b) + (n)
T(n) = (n) [a < b]
T(n) = (nlog(n)) [a = b]
T(n) = (nlogb(a)) [a > b]
May 9, 2001
Applied Symbolic Computation

6. Applied Symbolic Computation
Polynomial Algebra
Let F[x] denote the set of polynomials in the variable x whose coefficients are in the field F.
F[x] becomes an algebra where +, * are defined by polynomial addition and multiplication.
May 9, 2001
Applied Symbolic Computation

7. Polynomial Algebra mod a Polynomial
A(x)  B(x) (mod f(x))  f(x)|(A(x) - B(x))
This equivalence relation partitions polynomials in F[x] into equivalence classes where the class [A(x)] consists of the set {A(x) + k(x)f(x), where k(x) and f(x) are in F[x]}.
Choose a representative for [A(x)] with degree < deg(f(x)). Can choose rem(A(x),f(x)).
Arithmetic
[A(x)] + [B(x)] = [A(x) + B(x)]
[A(x)] * [B(x)] = [A(x)B(x)]
The set of equivalence classes with arithmetic defined like this is denoted by F[x]/(f(x))
May 9, 2001
Applied Symbolic Computation

8. Applied Symbolic Computation
Modular Inverses
Definition: B(x) is the inverse of A(x) mod f(x), if A(x)B(x)  1 (mod f(x))
The equation A(x)B(x)  1 (mod f(x)) has a solution iff gcd(A(x),f(x)) = 1.
In particular, if f(x) is irreducible, then F[x]/(f(x)) is a field.
By the Extended Euclidean Algorithm, there exist u(x) and v(x) such that A(x)u(x) + B(x)v(x) = gcd(A(x),f(x)). When gcd(A(x),f(x)) = 1, we get A(x)v(x) + f(x)v(x) = 1. Taking this equation mod f(x), we see that A(x)v(x)  1 (mod f(x))
May 9, 2001
Applied Symbolic Computation

9. Polynomial Version of the Chinese Remainder Theorem
Theorem: Let f(x) and g(x) be polynomials in F[x] (coefficients in a field). Assume that gcd(f(x),g(x)) = 1. For any A1(x) and A2(x) there exist a polynomial A(x) with A(x)  A1(x) (mod f(x)) and A(x)  A2(x) (mod g(x)).
Theorem: F[x]/(f(x)g(x))  F[x]/(f(x))  F[x]/(g(x)). I.E. There is a 1-1 mapping from F[x]/(f(x)g(x)) onto F[x]/(f(x))  F[x]/(g(x)) that preserves arithmetic.
A(x)  (A(x) mod f(x), A(x) mod g(x))
May 9, 2001
Applied Symbolic Computation

10. Constructive Chinese Remainder Theorem
Theorem: If gcd(f(x),g(x)) = 1, then there exist Ef(x) and Eg(x) (orthogonal idempotents)
Ef(x)  1 (mod f(x))
Ef(x)  0 (mod g(x))
Eg(x)  0 (mod f(x))
Eg(x)  1 (mod g(x))
It follows that A1(x) Ef(x) + A2(x) Eg(x)  A1(x) (mod f(x)) and  A2(x) (mod g(x)).
Proof.
Since gcd(f(x),g(x)) = 1, by the Extended Euclidean Algorithm, there exist u(x) and v(x) with f(x)u(x) + g(x)v(x) = 1. Set Ef(x) = g(x)v(x) and Eg(x) = f(x)u(x)
May 9, 2001
Applied Symbolic Computation

11. Applied Symbolic Computation
Interpolation
A polynomial of degree n is uniquely determined by its value at (n+1) distinct points.
Theorem: Let A(x) and B(x) be polynomials of degree m. If A(i) = B(i) for i = 0,…,m, then A(x) = B(x).
Proof.
Recall that a polynomial of degree m has m roots.
A(x) = Q(x)(x- ) + A(), if A() = 0, A(x) = Q(x)(x- ), and deg(Q) = m-1
Consider the polynomial C(x) = A(x) - B(x). Since C(i) = A(i) - B(i) = 0, for m+1 points, C(x) = 0, and A(x) must equal B(x).
May 9, 2001
Applied Symbolic Computation

12. Lagrange Interpolation Formula
Find a polynomial of degree m given its value at (m+1) distinct points. Assume A(i) = yi
Observe that
May 9, 2001
Applied Symbolic Computation

13. Polynomial Evaluation, Interpolation and the CRT
Since A(x) = Q(x)(x- ) + A(), A(x)  A() (mod (x- ))
If   , then gcd((x- ),(x- )) = 1. Therefore, we can apply the CRT to find a quadratic polynomial A(x) such that A() = y and A() = z.
A(x) = A() E(x) + A() E(x), where
E(x) = (x - )/( - )
E(x) = (x - )/( - )
Observe that
E() = 1 and E() = 0, so that E(x)  1 (mod (x- )) and E(x)  0 (mod (x- )). The equivalent results hold for E(x)
May 9, 2001
Applied Symbolic Computation

14. Applied Symbolic Computation
Multifactor CRT
The CRT can be generalized to the case when we have n pairwise relatively prime polynomials. If f1(x),…,fn(x) are pairwise relatively prime, i.e. gcd(fi(x),fj(x)) = 1 for i  j, then given A1(x),…,An(x) there exists a polynomial A(x) such that A  Ai(x) (mod fi(x)).
Moreover, there exist a system of orthogonal idempotents: E1(x),…,En(x), such that Ei(x)  1 (mod fi(x)) and Ei(x)  0 (mod fj(x)) for i  j.
A(x) = A1(x)E1(x) + … + An(x)En(x)
May 9, 2001
Applied Symbolic Computation

15. Lagrange Interpolation and the CRT
Assume that 0, 1,…, n are distinct and let fi(x) = (x- i). Then gcd(fi(x),fj(x)) = 1 for i  j.
Let
Then Ei(x)  1 (mod fi(x)) and Ei(x)  0 (mod fj(x)) for i  j, and Lagrange’s interpolation formula is
A(x) = A (0)E0(x) + … + A(n )En(x)
May 9, 2001
Applied Symbolic Computation

16. Matrix Version of Polynomial Evaluation
Let A(x) = a3x3 + a2x2 + a1x + a0
Evaluation at the points , , ,  is obtained from the following matrix-vector product
May 9, 2001
Applied Symbolic Computation

17. Applied Symbolic Computation
Vandermonde Matrix
V(0,…, n) is non-singular when 0,…, n are distinct.
May 9, 2001
Applied Symbolic Computation

18. Matrix Interpretation of Interpolation
Let A(x) = anxn + … + a1x +a0 be a polynomial of degree n. The problem of determining the (n+1) coefficients an,…,a1,a0 from the (n+1) values A(0),…,A(n) is equivalent to solving the linear system
May 9, 2001
Applied Symbolic Computation

19. Matrix Interpretation of Interpolation
The previous system has a solution when the 0,…, n are distinct. The solution can be obtained using Lagrange interpolation. In fact, the inverse of V(0,…, n) is obtained from the idempotents
May 9, 2001
Applied Symbolic Computation

20. Polynomial Multiplication using Interpolation
Compute C(x) = A(x)B(x), where degree(A(x)) = m, and degree(B(x)) = n. Degree(C(x)) = m+n, and C(x) is uniquely determined by its value at m+n+1 distinct points.
[Evaluation] Compute A(i) and B(i) for distinct i, i=0,…,m+n.
[Pointwise Product] Compute C(i) = A(i)*B(i) for i=0,…,m+n.
[Interpolation] Compute the coefficients of C(x) = cnxm+n + … + c1x +c0 from the points C(i) = A(i)*B(i) for i=0,…,m+n.
May 9, 2001
Applied Symbolic Computation

21. Interpolation and Karatsuba’s Algorithm
Let A(x) = A1x + A0, B(x) = B1x + B, C(x) = A(x)B(x) = C2x2 + C1x + C0
Then A(10k) = A, B(10k) = B, and C = C(10k) = A(10k)B(10k) = AB
Use interpolation based algorithm:
Evaluate A(), A(), A() and B(), B(), B() for  = 0,  = 1, and  =.
Compute C() = A()B(), C() = A() B(), C() = A()B()
Interpolate the coefficients C2, C1, and C0
Compute C = C2102k + C110k + C0
May 9, 2001
Applied Symbolic Computation

22. Matrix Equation for Karatsuba’s Algorithm
Modified Vandermonde Matrix
Interpolation
May 9, 2001
Applied Symbolic Computation

23. Integer Multiplication Splitting the Inputs into 3 Parts
Instead of breaking up the inputs into 2 equal parts as is done for Karatsuba’s algorithm, we can split the inputs into three equal parts.
This algorithm is based on an interpolation based polynomial product of two quadratic polynomials.
Let A(x) = A2x2 + A1x + A0, B(x) = B2x2 + B1x + B, C(x) = A(x)B(x) = C4x4 + C3x3 + C2x2 + C1x + C0
Thus there are 5 products. The divide and conquer part still takes time = O(n). Therefore the total computing time T(n) = 5T(n/3) + O(n) = (nlog3(5)), log3(5)  1.46
May 9, 2001
Applied Symbolic Computation

24. Asymptotically Fast Integer Multiplication
We can obtain a sequence of asymptotically faster multiplication algorithms by splitting the inputs into more and more pieces.
If we split A and B into k equal parts, then the corresponding multiplication algorithm is obtained from an interpolation based polynomial multiplication algorithm of two degree (k-1) polynomials.
Since the product polynomial is of degree 2(k-1), we need to evaluate at 2k-1 points. Thus there are (2k-1) products. The divide and conquer part still takes time = O(n). Therefore the total computing time T(n) = (2k-1)T(n/k) + O(n) = (nlogk(2k-1)).
May 9, 2001
Applied Symbolic Computation

25. Asymptotically Fast Integer Multiplication
Using the previous construction we can find an algorithm to multiply two n digit integers in time (n1+ ) for any positive .
logk(2k-1) = logk(k(2-1/k)) = 1 + logk(2-1/k)
logk(2-1/k)  logk(2) = ln(2)/ln(k)  0.
Can we do better?
The answer is yes. There is a faster algorithm, with computing time (nlog(n)loglog(n)), based on the fast Fourier transform (FFT). This algorithm is also based on interpolation and the polynomial version of the CRT.
May 9, 2001
Applied Symbolic Computation