1. Advanced Algorithms Analysis and DesignBy
Dr. Nazir Ahmad Zafar
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

2. Lecture No. 43 Polynomials and Fast Fourier Transform
Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

3. Definitions
A Field is a set F with two binary operations + : F × F → F and * : F × F → F such that
(F, +) is an abelian group with identity element 0
(F\{0}, *) is an abelian group with identity element 1
Multiplication distributes over addition
a*(b + c) = (a*b) + (a*c)
(a + b)*c = (a*c) + (b*c)
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

4. Definitions Polynomial
A polynomial in the variable x over an algebraic field F is a representation of a function A(x) as a formal sum
A(x) = a0 + a1x1 + a2x anxn
Coefficients
Values a0, a1,..., an are coefficients of polynomial, and drawn from a field F, typically set of complex numbers.
Degree
A polynomial A(x) is said to have degree n if its highest coefficient an is nonzero
Degree Bound
Any integer strictly greater than the degree of a polynomial is a degree-bound of that polynomial.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

5. Addition of two Polynomials: Brute Force
Addition of two polynomials of degree n takes Θ(n) time,
Example 1
A (x) = a0 + a1x1 + a2x anxn
B (x) = b0 + b1x1 + b2x bnxn
C (x) = (a0 + b0) + (a1 + b1) x (an + bn)xn
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

6. Multiplication of Two Polynomial: Brute Force
Multiplication of two polynomials of degree n takes Θ(n2)
Example 2
A (x) = a0 + a1x1 + a2x anxn
B (x) = b0 + b1x1 + b2x bnxn
a0b0 + a1b0x (anb0)xn
a0b1x1 + a1b1x (anb1)xn+1
. . .
a0bn xn + a1bnxn (anbn)xn+n
C (x) = (a0b0) + (a1b0 + a0b1)x (anbn)xn+n
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

7. Polynomial Representation
The Coefficient Representation
Point Value Presentation
Note
The method for multiplying polynomials equations as above take (n2) time when the polynomials are represented in coefficient form
But (n) time when represented in point value form
We can multiply using coefficient representation in only (n log n) time converting between two forms
This lecture make much use of complex numbers, the symbol i has same meaning, you know sqr(-1)
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

8. 1. Coefficient Representation
A coefficient representation of a polynomial degree bound n is a vector of coefficients: a = (a0, a1,…, an-1)
Vectors as Column
In this lecture, we will treat vector as column vector
Convenient Way
The coefficients representation is convenient for certain operations on polynomials
Example:
Computing A(x) at x0
Operation of evaluating polynomials A(x) at given point x0 consists of computing value of A(x0).
Evaluation takes time (n) using Horner’s rule
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

9. Evaluation and Addition using Coefficient Form
Operation 1: Horner’s Rule
A(x0)= a0 + x0( a1 + x0( a2 +…x 0(an-2 +x0(an-1))… )
Operation 2: Addition of Two Polynomials
Similarly adding two polynomials represented by the coefficient vectors:
a = (a0, a1 ,…, an-1 ) and
b = (b0, b1, … ,bn-1) takes (n) times
We just produce the coefficient vector:
c = (c0, c1,…, c n-1) where
cj = aj + bj,  j = 1 ,…, n - 1
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

10. Multiplication using Coefficient Representation
Operation 3: Multiplication of Two Polynomials
Consider multiplication of A(x) and B(x), with degree bounds n, represented in coefficient form
If we use the method described above polynomials multiplication takes time O(n2).
Since each coefficient in vector a must be multiplied by each coefficients in the vector b.
Operation of multiplying polynomials in coefficient form seems to be considerably more difficult than that of evaluating or adding two polynomials.
The resulting coefficient vector c, also called the convolution of the input vectors a and b.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

11. 2. Point–value Representation
A point value representation of a polynomial A(x) of degree bound n is a set of n point value pairs.
{ (x0, y0), (x1, y1),. . . ,(x n-1, y n-1) },
all of the x k are distinct and y k = A(x k), for k = 0,1, . . ., n-1.
Polynomial has various point value representations, since any set of n distinct points x0 ,x1 ,…,x n-1 can be used as a basis for the representation.
Conversion: From a Coefficient Form to Point Value Form
Computing a point value representation for a polynomials given in coefficient form is in principle straight forward ,
This is because select n distinct points x0 ,x1 ,…,x n-1 and then evaluate A(x k) for k = 0,1 ,…, n-1.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

12. 2. Point–value Representation
With Horner’s rule, n-point evaluation takes q(n2).
This is because for x = x0, evaluation cost is q(n). And since there are n number of points, hence there will be q(n2) cost for evaluating all of the n number of points using Horner’s rule.
Clever Choose of xk
We shall see that if we choose xk cleverly, this computation can be accelerated to run in q(n log n)
Inverse of evaluating coefficient form of polynomial from point value representation called interpolation.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

13. Theorem Uniqueness of an interpolating polynomial
For any set { (x0, y0),(x1, y1),. . ., (x n-1, y n-1) } of n point–value pairs such that all xk values distinct, there is a unique polynomial A(x) of degree bound n such that yk = A(xk) for k = 0,1, . . ., n-1.
Proof
Proof is based on existence of inverse of a matrix.
Let us suppose that A(x) is required polynomial
A(x) = a0 + a1x1 + a2x anxn
Equation: yk = A(xk) is equivalent to the matrix equation given in the next slide.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

14. Theorem
1 x0 x0 … x0 a0 y0
1 x1 x1 … x1 a1 y1 …
1 x x … x a y
This matrix on the left side is called vander-monde matrix and is denoted V(x0, x1, …..xn-1)
The determinant of this this matrix is
If xk are distinct then it is nonsingular. The coefficient aj can be uniquely determined a = V(x0, x1, …..xn-1)-1 y
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

15. Representation of polynomials
Solving The Equation in Proof of Theorem
Using LU decomposition algorithms, we can solve these equation in O(n3)
A faster algorithm, in Θ(n2), for n-point interpolation is based on Lagrange's formula:
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

16. Representation of polynomials
Addition using Point Value Form
The point-value representation is quite convenient for many operations on polynomials.
For addition:
C(x) = A(x) + B(x)  C(xk) = A(xk) + B(xk)
More precisely, if point value representation for A
{(x0, y0), (x1, y1),. . ., (xn-1, yn-1)},
And for B: {(x0, y’0), (x1, y’1),. . ., (xn-1, y’n-1)},
Then a point-value representation for C is
{(x0, y0+y’0), (x1, y1+y’1),. . ., (xn-1, yn-1+y’n-1)},
Thus, the time to add two polynomials of degree-bound n in point-value form is Θ(n).
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

17. Multiplication using Point Value Form
Similarly, point-value representation is convenient for multiplying polynomials as well.
C(x) = A(x) B(x)  C(xk) = A(xk)B(xk) for any xk,
We can multiply a point value representations for A and B to obtain a point-value representation for C.
A standard point-value representation for A and B consists of n point-value pairs for each polynomial
Multiplying these, we must extended point-value representations for A and B of 2n point-value each.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

18. Multiplication using Point Value Form
Given an extended point-value representation for A,
{(x0, y0), (x1, y1),..., (x2n-1, y2n-1)},
And extended point-value representation for B,
{(x0, y’0), (x1, y’1),..., (x2n-1, y’2n-1)},
Then a point-value representation for C is
{(x0, y0 y’0), (x1, y1 y’1),..., (xn-1, yn-1 y’n-1)}
Finally, we consider how to evaluate a polynomial given in point-value form at a new point.
Apparently no simpler approach than converting polynomial to coefficient form, and then evaluating it
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

19. Discrete Fourier Transform
We can use any points as evaluation points, but by choosing evaluation points carefully, we can convert between representations in only Θ(n lg n) time.
If we take “complex roots of unity” evaluation points, we can produce a point-value representation taking Discrete Fourier Transform of coefficient vector.
The inverse operation, interpolation, can be performed by taking “inverse DFT” of point-value pairs, yielding a coefficient vector.
We will show how FFT performs the DFT and inverse DFT operations in Θ(n lg n)
Multiplication procedure is shown in the next slide
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

20. Fast multiplication of polynomials in coefficient form
representation
Point-value
representation
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

21. Procedure: Multiplication of Polynomials in n lg n
We assume n is a power of 2; this requirement can always be met by adding zero coefficients.
Double degree-bound:
Create coefficient representations of A(x) and B(x) as degree bound 2n polynomials by adding n high-order zero coefficients to each.
Evaluate:
Compute point-value representations of A(x) and B(x) of length 2n through two applications of FFT of order 2n. These representations contain the values of the two polynomials at the (2n)th roots of unity.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

22. Multiplication of Polynomials in n lg n
Point wise multiply:
Compute point-value form for polynomial C(x) = A(x)B(x) by multiplying these together point wise. This representation contains the value of C(x) at each (2n)th root of unity.
Interpolate:
Create coefficient representation of C(x) through a single application of an FFT on 2n point-value pairs to compute inverse DFT.
Steps (1) and (3) take time Θ(n), and steps (2) and (4) take time Θ(n lg n).
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

23. Complex Roots of Unity and Their Properties
We claimed that if we use complex roots of unit we can evaluate and interpolate polynomials in Θ(nlgn) time.
Here, we define complex roots of unity and study their properties.
Define the DFT,and then show how the FFT computes the DFT and its inverse in just Θ(nlgn) time.
Complex root of unity
A complex nth root of unity is a complex number ω such that
ωn =1
There are exactly n complex nth roots of unity: e2πik/n for k=0,1,…,n-1
eiu =cos(u) + isin(u).
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

24. Complex Roots of Unity and Their Properties
Values of ω08 , ω18 ,. . ., ω78 in complex plane are shown
where ω8 = e2πi/8 is the principal 8th root of unity.
Complex roots of unity form a cyclic group
Complex roots of unity have interesting properties
Some of those are discussed in the next
ω18
ω48
ω38
ω28
ω68
ω58
ω78
ω08 =ω88 i -i -1 1
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

25. Properties: Complex Roots of Unity
Lemma 1 (cancellation lemma)
For any integers n ≥ 0,k ≥0,and d > 0,
ωdkdn = ωkn .
Proof : the lemma follows directly from ωn =e2πi/n ,since
ωdkdn = (e2πi/dn)dk
=(e2πi/n)k
= ωkn
Corollary 1 (cancellation lemma)
For any even integer n > 0,
ωn/2n = -1
Proof: We know that ωn =e2πi/n
Now ωn/2n = ωn/22.n/2 = ω2 = ωn =e2πi/2 = ωn =eπi = -1
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

26. Properties: Complex Roots of Unity
Lemma 2 (Halving Lemma)
If n > 0 is even, then squares of n complex nth root of unity are the n/2 complex (n/2)th root of unity.
Proof:
By the cancellation lemma, we have: (ωnk )2 = ωkn/2
For any nonnegative integer k, note that if we square all of complex nth root of unity, then each (n/2)th root of unity is obtained exactly twice, since
(ωnk +n/2)2 = ω2k+nn/2
= ω2knωnn = ω2kn
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

27. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
Continue
Halving Lemma is Essential in Reducing Cost
Thus ωkn and ωk+n/2n/2 have the same square.
This property can also be proved using corollary,
ωnn/2 = ω2 = -1
Since ωn/2n = -1 implies ωk+n/2n = - ωkn and thus
(ωnk +n/2)2 = (ωnk)2
As we shall see, the halving lemma is essential to our divide-and-conquer approach of converting between coefficient and point-value representation of polynomials
Since it guarantees that the recursive sub problems are only half as large.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

28. Properties: Complex Roots of Unity
Lemma 3 (Summation Lemma)
For any integer n ≥ 1 and nonnegative integer k not divisible by n,
n-1
∑ (ωkn)j = 0
j=0
Proof:
∑ (ωkn)j = (ωnk)n-1
j= ωnk-1
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

29. Properties: Complex Roots of Unity
= (ωnn)k-1
ωnk-1
= (1)k-1
= 0
Requiring that k not be divisible by n ensures that the denominator is not 0, since ωnk = 1 only when k is divisible by n.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

30. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
The DFT
Recall that we wish to evaluate a polynomial
n-1
A(x) = ∑ ajxj
j=0
of degree-bound n at ωn0 ,ωn1, ωn2, ……..ωnn-1 ( that is, at the n complex nth roots of unity).
Without loss of generality, we assume that n is a power of 2, since a given degree-bound can always be raised we can always add new high-order zero coefficients as necessary.
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

31. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
The DFT
We assume that A is given in coefficient from a = ( a0, a1,. . ., an-1).
Let us define the results yk for k = 0,1,. . ., n-1, by
yk = A(ωkn)
n-1
= ∑ aj ωk jn
j=0
The vector y = (y0, y1,. . ., yn-1) is Discrete Fourier Transform (DFT) of the coefficient vector
a = ( a0, a1,. . ., an-1).
We can also write y = DFTn(a)
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

32. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
The FFT
And hence
(A)
(A)
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

33. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design
The FFT
For k = 0, , n/2-1
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

34. FFT Recursive Algorithm
Dr Nazir A. Zafar Advanced Algorithms Analysis and Design