Progress in the method of Ghosts and Shadows for Beta Ensembles Alan Edelman (MIT) Alex Dubbs (MIT) and Plamen Koev (SJS) Aug 10, 2012 IMS Singapore Workshop 1

Wishart Matrices (arbitrary covariance) G=mxn matrix of Gaussians Σ=mxn semidefinite matrix G’G Σ is similar to A=Σ ½ G’GΣ -½ For β=1,2,4, the joint eigenvalue density of A has a formula: Known for β=2 in some circles as Harish-Chandra- Itzykson-Zuber 2

Main Purpose of this talk 3 Eigenvalue density of G’G Σ ( similar to A=Σ ½ G’GΣ -½ ) Present an algorithm for sampling from this density Show how the method of Ghosts and Shadows can be used to derive this algorithm Further evidence that beta=1,2,4 need not be special

Eigenvalues of GOE (β=1) Naïve Way: MATLAB®: A=randn(n); S=(A+A’)/sqrt(2*n);eig(S) R: A=matrix(rnorm(n*n),ncol=n);S=(a+t(a))/sqrt(2*n);eigen(S,symmetric=T,only.values=T)$values; Mathematica: A=RandomArray[NormalDistribution[],{n,n}];S=(A+Transpose[A])/Sqrt[n];Eigenvalues[s] 4

Eigenvalues of GOE/GUE/GSE and beyond… Real Sym Tridiagonal: β=1:real, β=2:complex, β=4:quaternion General β>0 Computations significantly faster Diagonals N(0,2), Off-diagonals “chi” random-variables 5

Introduction to Ghosts G 1 is a standard normal N(0,1) G 2 is a complex normal (G 1 +iG 1 ) G 4 is a quaternion normal (G 1 +iG 1 +jG 1 +kG 1 ) G β (β>0) seems to often work just fine 6

Chi-squared Defn: χ β is the sum of β iid squares of standard normals if β=1,2,… Generalizes for non-integer β as the “gamma” function interpolates factorial χ β is the sqrt of the sum of squares (which generalizes) (wikipedia chi-distriubtion) |G 1 | is χ 1, |G 2 | is χ 2, |G 4 | is χ 4 So why not |G β | is χ β ? I call χ β the shadow of G β 2 7

Linear Algebra Example Given a ghost Gaussian, G β, |G β |is a real chi- beta variable χ β variable. Gram-Schmidt:    or = [Q] * GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ χ3βχ3β GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ χ3βχ3β GβGβ GβGβ χ 2β GβGβ GβGβ χ3βχ3β GβGβ GβGβ GβGβ ΧβΧβ H3H3 H2H2 H1H1 GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ χ3βχ3β GβGβ GβGβ GβGβ ΧβΧβ 8

Linear Algebra Example Given a ghost Gaussian, G β, |G β |is a real chi- beta variable χ β variable. Gram-Schmidt:    or = [Q] * GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ χ3βχ3β GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ χ3βχ3β GβGβ GβGβ χ 2β GβGβ GβGβ χ3βχ3β GβGβ GβGβ GβGβ ΧβΧβ H3H3 H2H2 H1H1 GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ χ3βχ3β GβGβ GβGβ GβGβ ΧβΧβ Q has β - Haar Measure! We have computed moments! 9

Tridiagonalizing Example GβGβ GβGβ GβGβ …GβGβ GβGβ GβGβ GβGβ …GβGβ GβGβ GβGβ GβGβ …GβGβ GβGβ GβGβ GβGβ GβGβ GβGβ  Symmetric Part of 10 (Silverstein, Trotter, etc)

Histogram without Histogramming: Sturm Sequences Count #eigs < 0.5: Count sign changes in Det( (A-0.5*I)[1:k,1:k] ) Count #eigs in [x,x+h] Take difference in number of sign changes at x+h and x Mentioned in Dumitriu and E 2006, Used theoretically in Albrecht, Chan, and E

A good computational trick is a good theoretical trick! Finite Semi-Circle Laws for Any Beta! Finite Tracy-Widom Laws for Any Beta! 12

Stochastic Differential Eigen-Equations Tridiagonal Models Suggest SDE’s with Brownian motion as infinite limit: – E: 2002 – E and Sutton 2005, 2007 – Brian Rider and (Ramirez, Cambronero, Virag, etc.) (Lots of beautiful results!) – (Not today’s talk) 13

“Tracy-Widom” Computations Eigenvalues without the whole matrix! E: 2003 Persson and E: 2005 Never construct the entire tridiagonal matrix! Just say the upper 10n 1/3 by 10n 1/3 Compute largest eigenvalue of that, perhaps using Lanczos with shift and invert strategy! Can compute for n amazingly large!!!!! And any beta. 14

Other Computational Results Infinite Random Matrix Theory – The Free Probability Calculator (Raj) Finite Random Matrix Theory – MOPS (Ioana Dumitriu) (β: orthogonal polynomials) – Hypergeometrics of Matrix Argument (Plamen Koev) (β: distribution functions for finite stats such as the finite Tracy-Widom laws for Laguerre) Good stuff, but not today. 15

Scary Ideas in Mathematics Zero Negative Radical Irrational Imaginary Ghosts: Something like a commutative algebra of random variables that generalizes random Reals, Complexes, and Quaternions and inspires theoretical results and numerical computation 16

Did you say “commutative”?? Quaternions don’t commute. Yes but random quaternions do! If x and y are G 4 then x*y and y*x are identically distributed! 17

RMT Densities Hermite: c ∏|λ i - λ j | β e -∑λ i 2 /2 (Gaussian Ensemble) Laguerre: c ∏|λ i -λ j | β ∏λ i m e -∑λ i (Wishart Matrices) Jacobi: c ∏|λ i -λ j | β ∏λ i m 1 ∏(1-λ i ) m 2 (Manova Matrices) Fourier: c ∏|λ i -λ j | β (on the complex unit circle) “Jack Polynomials” Traditional Story: Count the real parameters β=1,2,4 for real, complex, quaternion Applications: β=1: All of Multivariate Statistics β=2:Tons, β=4: Almost nobody cares Dyson 1962 “Threefold Way” Three Associative Division Rings 18

β-Ghosts β=1 has one real Gaussian (G) β=2 has two real Gaussians (G+iG) β=4 has four real Gaussians (G+iG+jG+kG) β≥1 has one real part and (β-1) “Ghost” parts 19

Introductory Theory There is an advanced theory emerging – A β-ghost is a spherically symmetric random variable defined on R β – A shadow is a derived real or complex quantity 20

Wishart Matrices (arbitrary covariance) G=mxn matrix of Gaussians Σ=mxn semidefinite matrix G’G Σ is similar to A=Σ ½ G’GΣ -½ For β=1,2,4, the joint eigenvalue density of A has a formula: 21

Joint Eigenvalue density of G’G Σ The “0F0” function is a hypergeometric function of two matrix arguments that depends only on the eigenvalues of the matrices. Formulas and software exist. 22

Generalization of Laguerre Laguerre: density = c ∏λ i m ∏|λ k -λ j | β e -∑λk vs 23

General β? The joint density : is a probability density for all β>0. Goals: Algorithm for sampling from this density Get a feel for the density’s “ghost” meaning 24

Main Result An algorithm derived from ghosts that samples eigenvalues A MATLAB implementation that is consistent with other beta-ized formulas – Largest Eigenvalue – Smallest Eigenvalue 25

Working with Ghosts Real quantity 26

More practice with Ghosts 27

Bidiagonalizing Σ=I Z’Z has the Σ=I density giving a special case of 28

The Algorithm for Z=GΣ ½ 29

The Algorithm for Z=GΣ ½ 30

Removing U and V 31

Algorithm cont. 32

Completion of Recursion 33

Numerical Experiments – Largest Eigenvalue Analytic Formula for largest eigenvalue dist E and Koev know have software to compute 34

35

36

37

Smallest Eigenvalue as Well 38 The cdf of the smallest eigenvalue

Cdf’s of smallest eigenvalue 39

Goals Continuum of Haar Measureas generalizing orthogonal, unitary, symplectic New Definition of Jack Polynomials generalizing the zonals Computations! E.g. Moments of the Haar Measures Place finite random matrix theory “β”into same framework as infinite random matrix theory: specifically β as a knob to turn down the randomness, e.g. Airy Kernel –d 2 /dx 2 +x+(2/β ½ )dW  White Noise 40

Formally Let S n =2π/Γ(n/2)=“suface area of sphere” Defined at any n= β>0. A β-ghost x is formally defined by a function f x (r) such that ∫ ∞ f x (r) r β-1 S β-1 dr=1. Note: For β integer, the x can be realized as a random spherically symmetric variable in β dimensions Example: A β-normal ghost is defined by f(r)=(2π) -β/2 e -r 2 /2 Example: Zero is defined with constant*δ(r). Can we do algebra? Can we do linear algebra? Can we add? Can we multiply? r=0 41

A few more operations ΙΙxΙΙ is a real random variable whose density is given by f x (r) (x+x’)/2 is real random variable given by multiplying ΙΙxΙΙ by a beta distributed random variable representing a coordinate on the sphere 42

Addition of Independent Ghosts: Addition returns a spherically symmetric object Have an integral formula Prefer: Add the real part, imaginary part completed to keep spherical symmetry 43

Multiplication of Independent Ghosts Just multiply ΙΙzΙΙ’s and plug in spherical symmetry Multiplication is commutative – (Important Example: Quaternions don’t commute, but spherically symmetric random variables do!) 44

Understanding ∏|λ i -λ j | β Define volume element (dx)^ by (r dx)^=r β (dx)^ (β-dim volume, like fractals, but don’t really see any fractal theory here) Jacobians: A=QΛQ’ (Sym Eigendecomposition) Q’dAQ=dΛ+(Q’dQ)Λ- Λ(Q’dQ) (dA)^=(Q’dAQ)^= diagonal ^ strictly-upper diagonal = ∏dλ i =(dΛ)^ off-diag = ∏ ( (Q’dQ) ij (λ i -λ j ) ) ^=(Q’dQ)^ ∏|λ i -λ j | β 45

Haar Measure β=1: E Q (trace(AQBQ’) k )=∑C κ (A)C κ (B)/C κ (I) Forward Method: Suppose you know C κ ’s a-priori. (Jack Polynomials!) Let A and B be diagonal indeterminants (Think Generating Functions) Then can formally obtain moments of Q: Example: E(|q 11 | 2 |q 22 | 2 ) = (n+α-1)/(n(n-1)(n+α)) α:=2/ β Can Gram-Schmidt the ghosts. Same answers coming up! 46

Further Uses of Ghosts Multivariate Othogonal Polynomials Largest Eigenvalues/Smallest Eigenvalues Expect Lots of Uses to be discovered… 47