The Fast Multipole Method: It’s All about Adding Functions Alan Edelman MIT: Dept of Mathematics, Lab for Computer Science FOCM 2002 Saturday, August 10 11/15/2018

Outline Multipole: What are we adding? The exclusive add without the multipole Representing functions on a computer “Adding” functions and the Pascal Matrix Research Opportunities 11/15/2018

The matrix-vector viewpoint: f =V(y, x) q potentials at y1 , ..., yN charges at x1 , ..., xN Vij = r(yi - xj) qj ||yi-xj|| fi=S Example: j Direct evaluation: O(N2) work Greengard, Rokhlin FMM (1987): (N) 11/15/2018

Some applications: N-body Problems Potential Evaluation Future Fast Fourier Transform Divide and Conquer Eigenvalue Solvers Polynomial Roots More waiting to be found …. 11/15/2018

Outline Multipole: What are we adding? The exclusive add without the multipole Representing functions on a computer “Adding” functions and the Pascal Matrix Research Opportunities 11/15/2018

It’s all about adding functions: fi=S qj ||yi-xj|| j f(y)=S fj(y) j fj(y)= qj ||y-xj|| 11/15/2018

What might these functions look like? f(y)=S fj(y) j f1(y) f2(y) 11/15/2018

Outline Multipole: What are we adding? The exclusive add without the multipole Representing Functions on a computer “Adding” functions and the Pascal Matrix Research Opportunities 11/15/2018

The Exclusive Add Why not sum and subtract each element? sum(x)-x 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 y = x yi = ji xj Why not sum and subtract each element? sum(x)-x What if NaN’s or numbers on different scales are present? 11/15/2018

The Exclusive Add A Good Algorithm is worth seeing five ways! 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 y = x yi = ji xj A Good Algorithm is worth seeing five ways! 1. Pseudocode 2. MATLAB 3. On a tree 4. Matrix Notation 5. Kronecker Product Notation 11/15/2018

Exclusive Add (pseudocode) yi = ji xj 1 2 3 4 5 6 7 8 3 7 11 15 33 29 25 21 35 34 33 32 3130 29 28 1)Pairwise Sums 2)Recursive Excl Add 3)Update “evens & odds” 11/15/2018

Exclusive Add in MATLAB yi = ji xj 1 2 3 4 5 6 7 8 3 7 11 15 33 29 25 21 35 34 33 32 3130 29 28 1)Pairwise Sums 2)Recursive Excl Add 3)Update “evens & odds” function y=exclude(v) n=length(v); if n==1, y=0*v; else w=v(1:2:n)+v(2:2:n); % Pairwise adds w=exclude(w); % Recur y(1:2:n)=w+v(2:2:n); y(2:2:n)=w+v(1:2:n); % Update Adds end 11/15/2018

Exclusive Add on a Tree yi = ji xj 36 10 26 3 7 11 15 1 2 3 4 5 6 7 8 26 10 33 29 25 21 35 34 33 32 3130 29 28 Pairwise sums “up the tree” Modify evens/odds down 11/15/2018

With Matrices T 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 1 1 1 1 0 1 1 1 1 0 1 1 1 1 0 + = 11/15/2018

Kronecker product Approach 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 1 1 1 1 0 1 1 1 1 0 1 1 1 1 0 + = A8 =(I4( )) A4 (I4( ))T + I4  ( ) 11 11 0 1 1 0 1)Pairwise Sums 2)Recursive Excl Add 3)Update “evens & odds” 11/15/2018

Kronecker product Approach 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 1 1 1 1 0 1 1 1 1 0 1 1 1 1 0 + = A8 =(I4( )) A4 (I4( ))T + I4  ( ) 11 11 0 1 1 0 1)Pairwise Sums 2)Recursive Excl Add 3)Update “evens & odds” 11/15/2018

Kronecker product Approach 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 1 1 1 1 0 1 1 1 1 0 1 1 1 1 0 + = A8 =(I4( )) A4 (I4( ))T + I4  ( ) 11 11 0 1 1 0 1)Pairwise Sums 2)Recursive Excl Add 3)Update “evens & odds” 11/15/2018

Kronecker product Approach 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 1 1 1 1 0 1 1 1 1 0 1 1 1 1 0 + = A8 =(I4( )) A4 (I4( ))T + I4  ( ) 11 11 0 1 1 0 1)Pairwise Sums 2)Recursive Excl Add 3)Update “evens & odds” 11/15/2018

The Family Directions Left Right Left/Right Inclusive Exc=0 Prefix All Algorithms 1)Pairwise Sums 2)Recursive “foo” 3)Update Directions Left Right Left/Right Inclusive Exc=0 Prefix Suffix Reduce Exclusive Exc=1 Exc Prefix Exc Suffix Exc Add Neighbor Exc Exc=2 Left Multipole Right " " " Multipole 11/15/2018

Multipole is a doubly exclusive add 0 0 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 We add functions with poles not numbers We exclude ourselves and our nearest neighbors For reasons analogous to the floating point example: The function can not be represented accurately near the singularities. This is 1d, higher dimensions analogous Algorithm: Pairwise Add, Recursion, Update Missing Pieces 11/15/2018

Outline Multipole: What are we adding? The exclusive add without the multipole Representing Functions on a computer “Adding” functions and the Pascal Matrix Research Opportunities 11/15/2018

“Rounding” functions to p coefficients Best you can do: 1)Pick an interval 2)Pick a function space inside or outside the interval. 3)Find the best approximation on that function space. 11/15/2018

“Rounding” functions to p coefficients Ideal polynomial Approximation Truncated Taylor Series f(x) =  ak(x-c)k Theory: Converges in a disk from c to nearest singularity Practical: Accurate where smooth, far from the singularity Alternative Polynomial: Interpolate rather than Taylor 11/15/2018

“Rounding” functions to p coefficients Ideal Multipole Approximation Truncated Multipole Series f(x) =  ak/(x-c)k+1 Theory: Converges outside a disk from c to nearest singularity Practical: Accurate where smooth, far from the singularity Alternative Multipole: Interpolate 11/15/2018

“Rounding” functions to p coefficients Examples R multipole/Taylor Gu R Chebyshev interpolation Dutt, Gu, Rokhlin S1 Chebyshev interpolation Dutt, Rokhlin R singular funs of int ops Yarvin, Rokhlin C multipole/Taylor Greengard, Rokhlin C discretized Poisson form Anderson C singular funs of int ops Hrycak, Rokhlin R3 multipole/Taylor Greengard R3 discretized Poisson Anderson R3 singular funs of int ops Greengard, Rokhlin Any Virtual Charges E, McCorquodale 11/15/2018

Finite Precision Arithmetic Idea adding real numbers We all know what this means x=1+1 x=e+ On a computer: Storage: must round to d-bit precision Arithmetic: need a finite precision algorithm 11/15/2018

Finite Precision Arithmetic Idea adding functions We all know what this means f(x)=sin2(x)+cos2(x) = 1 f(x)=log(x)+log(x) = log(x2) On a computer: Storage: must round to d-bit precision Arithmetic: need a finite precision algorithm 11/15/2018

Functions to add Slowly growing singularity f(x)=q/(x-c) f(x)=q*log(x-c) f(x)=q*cot(x-c) but not f(x)=exp(-(x-c)2) 11/15/2018

Outline Multipole: What are we adding? The exclusive add without the multipole Representing Functions on a computer “Adding” functions and the Pascal Matrix Research Opportunities 11/15/2018

Adding Functions Issues with adding polynomials common center accuracy Same issues appear with multipole 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 Gil Strang’s latest favorite matrix i+j i 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 i+j i 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 i+j i 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 i+j i 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 i+j i 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 i+j i 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 i+j i 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 i+j i 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 i+j i 11/15/2018

The Pascal Matrix P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 i+j i 11/15/2018

Pascal and Cholesky(Pascal) P=pascal(5) 1 1 1 1 1 1 2 3 4 5 1 3 6 10 15 1 4 10 20 35 1 5 15 35 70 Pij=( ) i,j = 0,…,n-1 L=chol(pascal(5)) 1 0 0 0 0 1 1 0 0 0 1 2 1 0 0 1 3 3 1 0 1 4 6 4 1 Lij=( ) i,j = 0,…,n-1 i j i+j i P=LL’ ( )=  ( )( ) i+j j i k j j-k 11/15/2018

Binomial Identities (1-x)-(j+1) = i=0 ( ) xi  P i+j i  (1+x) j = i=0 ( ) xi  L’ j j i (x-1)-(j+1) = i=j ( ) x-(i+1)  L  i j 11/15/2018

“Flipping” (multipoletaylor) w = F v  wi (x-d)i =  vj / (x-c) j+1 i=0 j=0 Fij = (c-d)-i Pij (d-c)-(j+1) Proof: Differentiate i times then evaluate at x=d or cleverly use (1-x)-(j+1) = ( )xi i+j i 11/15/2018

“Shifting” (multipolemultipole) w = S v  wi /(x-d) i+1 =  vj / (x-c) j+1 i=0 j=0 Sij = (c-d) (i+1) Lij (c-d)-(j+1) 11/15/2018

“Shifting” (taylortaylor) w = S v  wi (x-d)i =  vj(x-c)j i=0 j=0 T Sij = (d-c)-i Lij (d-c) j 11/15/2018

Outline Multipole: What are we adding? The exclusive add without the multipole Representing Functions on a computer “Adding” functions and the Pascal Matrix Research Opportunities 11/15/2018

Group Representations Let V be a vector space of functions of x e.g. polynomials, rational functions, etc. Let G be a group acting on {x}, e.g. rotations, non-singular matrices. Clearly the map  from f(x) to h(x)=f(g-1x) is linear. Clearly (gh)= (g) (h). We say that  is a representation of G. 11/15/2018

Group Representations Research Generalize Fast Multipole to Arbitrary Representations! Algebraic Multipole Theory??? 11/15/2018