1. Chapter 2 Divide-and-Conquer algorithms
Break up problem into several parts.
Solve each part recursively.
Combine solutions to sub-problems into overall solution.
Most common usage.
Break up problem of size n into two equal parts of size ½n.
Solve two parts recursively.
Combine two solutions into overall solution in linear time.
Consequence.
Brute force: n2.
Divide-and-conquer: n log n.

2. Example 1 Integer Multiplication
"Divide and conquer", Caesar's other famous quote "I came, I saw, I conquered"
Divide-and-conquer idea dates back to Julius Caesar.
Favorite war tactic was to divide an opposing army in two halves, and then assault one half with his entire force.

3. Integer Arithmetic
Add. Given two n-digit integers a and b, compute a + b.
O(n) bit operations.
Multiply. Given two n-digit integers a and b, compute a × b.
Brute force solution: (n2) bit operations. 1 *
Multiply 1 +
Add

4. Divide-and-Conquer Multiplication: Warmup
To multiply two n-digit integers:
Multiply four ½n-digit integers.
Add two ½n-digit integers, and shift to obtain result.
assumes n is a power of 2

5. Karatsuba’s algorithm
To multiply two n-digit integers:
Add two ½n digit integers.
Multiply three ½n-digit integers.
Add, subtract, and shift ½n-digit integers to obtain result.
Theorem. [Karatsuba-Ofman, 1962] Can multiply two n-digit integers in O(n1.585) bit operations.
Karatsuba: also works for multiplying two degree N univariate polynomials A B A C C

6. Recursion Tree . . . . . . T(n) n T(n/2) T(n/2) T(n/2) 3(n/2) T(n/4)
T(n / 2k)
3k (n / 2k)
. . .
. . .
T(2)
T(2)
T(2)
T(2)
T(2)
T(2)
T(2)
T(2)
3 lg n (2)

7. Example 2: Mergesort
"Divide and conquer", Caesar's other famous quote "I came, I saw, I conquered"
Divide-and-conquer idea dates back to Julius Caesar.
Favorite war tactic was to divide an opposing army in two halves, and then assault one half with his entire force.

8. Sorting Sorting. Given n elements, rearrange in ascending order.
Obvious sorting applications.
List files in a directory.
Organize an MP3 library.
List names in a phone book.
Display Google PageRank results.
Problems become easier once sorted.
Find the median.
Find the closest pair.
Binary search in a database.
Identify statistical outliers.
Find duplicates in a mailing list.
Non-obvious sorting applications.
Data compression.
Computer graphics.
Interval scheduling.
Computational biology.
Minimum spanning tree.
Supply chain management.
Simulate a system of particles.
Book recommendations on Amazon.
Load balancing on a parallel computer

9. Mergesort Mergesort. Divide array into two halves.
Recursively sort each half.
Merge two halves to make sorted whole.
Jon von Neumann (1945) A L G O R I T H M S
divide
O(1)
sort
2T(n/2)
merge
O(n)

10. Merging Merging. Combine two pre-sorted lists into a sorted whole.
How to merge efficiently?
Linear number of comparisons.
Use temporary array.
Challenge for the bored. In-place merge. [Kronrud, 1969] A G L O R H I M S T A G H I
using only a constant amount of extra storage

11. A Useful Recurrence Relation
Def. T(n) = number of comparisons to mergesort an input of size n.
Mergesort recurrence.
Solution. T(n) = O(n log2 n).
Assorted proofs. We describe several ways to prove this recurrence. Initially we assume n is a power of 2 and replace  with =.

12. Proof by Recursion Tree
T(n) n
T(n/2)
T(n/2)
2(n/2)
T(n/4)
T(n/4)
T(n/4)
T(n/4)
4(n/4)
log2n
. . .
T(n / 2k)
2k (n / 2k)
. . .
T(2)
T(2)
T(2)
T(2)
T(2)
T(2)
T(2)
T(2)
n/2 (2)
n log2n

13. Proof by Telescoping
Claim. If T(n) satisfies this recurrence, then T(n) = n log2 n.
Pf. For n > 1:
assumes n is a power of 2

14. Proof by Induction
Claim. If T(n) satisfies this recurrence, then T(n) = n log2 n.
Pf. (by induction on n)
Base case: n = 1.
Inductive hypothesis: T(n) = n log2 n.
Goal: show that T(2n) = 2n log2 (2n).
assumes n is a power of 2

15. Analysis of Mergesort Recurrence
Claim. If T(n) satisfies the following recurrence, then T(n)  n lg n.
Pf. (by induction on n)
Base case: n = 1.
Define n1 = n / 2 , n2 = n / 2.
Induction step: assume true for 1, 2, ... , n–1.
log2n

19. Randomized Selection Algorithm
Input: An array A, low, high, k
Output: the k-th smallest element in A[low], A[low+1] … A[high]
Assume: 1<= k <= high – low + 1
Algorithm RandomSelect(A, low, high, k):
Step 1: m = partition(A, low, high);
Step 2: if (m – low + 1 == k) return A[m]
else if (m – low + 1 > k)
return RandomSelect(A, low, m, k)
else
return RandomSelect(A, m+1, high, _________ )
End.

22. The Fast Fourier Transform
Cryptography
The Fast Fourier Transform
6/12/2018 2:29 PM
FFT

23. Outline and Reading Polynomial Multiplication Problem
Primitive Roots of Unity (page 63)
The FFT Algorithm (page 64)
Integer Multiplication ([AHU] Section 8.1)
Java FFT Integer Multiplication
FFT

24. Polynomials
Polynomial:
In general,
FFT

25. Polynomial Evaluation
Horner’s Rule:
Given coefficients (a0,a1,a2,…,an-1), defining polynomial
Given x, we can evaluate p(x) in O(n) time using the equation
Eval(A,x): [Where A=(a0,a1,a2,…,an-1)]
If n=1, then return a0
Else,
Let A’=(a1,a2,…,an-1)
return a0+x*Eval(A’,x)

26. Polynomial Multiplication Problem
Given coefficients (a0,a1,a2,…,an-1) and (b0,b1,b2,…,bn-1) defining two polynomials, p() and q(), and number x, compute p(x)q(x).
We need to compute the coefficients ci
where
A straightforward evaluation would take O(n2) time. FFT will do it in O(n log n) time.
FFT

27. Polynomial Interpolation & Polynomial Multiplication
Given a set of n points in the plane with distinct x-coordinates, there is exactly one (n-1)-degree polynomial going through all these points.
Alternate approach to computing p(x)q(x):
Calculate p() on 2n x-values, x0,x1,…,x2n-1.
Calculate q() on the same 2n x values.
Find the (2n-1)-degree polynomial that goes through the points {(x0,p(x0)q(x0)), (x1,p(x1)q(x1)), …, (x2n-1,p(x2n-1)q(x2n-1))}.
Unfortunately, a straightforward evaluation would still take O(n2) time, as we would need to apply an O(n)-time Horner’s Rule evaluation to 2n different points.
FFT will do it in O(n log n) time, by picking 2n points that are easy to evaluate…
FFT

28. Primitive Roots of Unity
A number w is a primitive n-th root of unity, for n > 1, if
wn = 1
The numbers 1, w, w2, …, wn-1 are all distinct
Example 1:
Z*11:
2, 6, 7, 8 are 10-th roots of unity in Z*11
22=4, 62=3, 72=5, 82=9 are 5-th roots of unity in Z*11
2-1=6, 3-1=4, 4-1=3, 5-1=9, 6-1=2, 7-1=8, 8-1=7, 9-1=5
Example 2: The complex number e2pi/n is a primitive n-th root of unity, where

29. Properties of Primitive Roots of Unity
Inverse Property: If w is a primitive root of unity, then w -1=wn-1
Proof: wwn-1=wn=1
Cancellation Property: For non-zero -n<k<n,
Proof:
Reduction Property: If w is a primitive (2n)-th root of unity, then w2 is a primitive n-th root of unity.
Proof: If 1,w,w2,…,w2n-1 are all distinct, so are 1,w2,(w2)2,…,(w2)n-1
Reflective Property: If n is even, then wn/2 = -1.
Proof: By the cancellation property, for k=n/2:
Corollary: wk+n/2= -wk.
FFT

30. The Discrete Fourier Transform
Given coefficients (a0,a1,a2,…,an-1) for an (n-1)-degree polynomial p(x)
The Discrete Fourier Transform is to evaluate p at the values
1,w,w2,…,wn-1
We produce (y0,y1,y2,…,yn-1), where yj=p(wj)
That is,
Matrix form: y=Fa, where F[i,j]=wij.
The Inverse Discrete Fourier Transform recovers the coefficients of an (n-1)-degree polynomial given its values at 1,w,w2,…,wn-1
Matrix form: a=F -1y, where F -1[i,j]=w-ij/n.
FFT

31. Correctness of the inverse DFT
The DFT and inverse DFT really are inverse operations
Proof: Let A=F -1F. We want to show that A=I, where
If i=j, then
If i and j are different, then
FFT

32. Convolution
The DFT and the inverse DFT can be used to multiply two polynomials
So we can get the coefficients of the product polynomial quickly if we can compute the DFT (and its inverse) quickly…
FFT

33. The Fast Fourier Transform
The FFT is an efficient algorithm for computing the DFT
The FFT is based on the divide-and-conquer paradigm:
If n is even, we can divide a polynomial
into two polynomials
and we can write
FFT

34. The FFT Algorithm
The running time is O(n log n). [inverse FFT is similar]
FFT

35. The FFT Algorithm
FFT

36. FFT

37. FFT

38. Non-recursive FFT There is also a non-recursive version of the FFT
Performs the FFT in place
Precomputes all roots of unity
Performs a cumulative collection of shuffles on A and on B prior to the FFT, which amounts to assigning the value at index i to the index bit- reverse(i).
The code is a bit more complex, but the running time is faster by a constant, due to improved overhead
FFT

39. Experimental Results
Log-log scale shows traditional multiply runs in O(n2) time, while FFT versions are almost linear
FFT