1. Simulation of the matrix Bingham-von Mises- Fisher distribution, with applications to multivariate and relational data Discussion led by Chunping Wang ECE, Duke University July 10, 2009 Peter D. Hoff to appear in Journal of Computational and Graphical Statistics

2. Outline Introduction and Motivations Sampling from the Vector Von Mises-Fisher (vMF) Distribution (existing method) Sampling from the Matrix Von Mises-Fisher (mMF) Distribution Sampling from the Bingham-Von Mises-Fisher (BMF) Distribution One Example Conclusions 1/21

3. Introduction The matrix Bingham distribution – quadratic term The matrix von Mises-Fisher distribution – linear term The matrix Bingham-von Mises-Fisher distribution Stiefel manifold: set of rank- orthonormal matrices, denoted 2/21

4. Motivations Sampling orthonormal matrices from distributions is useful for many applications. Examples: Factor analysis latent Given uniform priors over Stiefel manifold, observed matrix 3/21

5. Motivations Principal components observed matrix, with each row Eigen-value decomposition Likelihood Posterior with respect to uniform prior with 4/21

6. Motivations Network data, symmetric binary observed matrix, with the 0-1 indicator of a link between nodes i and j. Posterior with respect to uniform prior E: symmetric matrix of independent standard normal noise 5/21

7. Sampling from the vMF Distribution (wood, 1994) the modal vector; constant distribution for any given angle, concentration parameter A distribution on the -sphere in defines the modal direction. 6/21

8. Sampling from the vMF Distribution (wood, 1994) ( Proposal envelope ) (1) A simple direction For a fixed orthogonal matrix, 7/21 (2) An arbitrary direction

9. Sampling from the mMF Distribution Rejection sampling scheme 1: uniform envelope Acceptance region rejection region accept Sample when a bound Extremely inefficient 8/21

10. Sampling from the mMF Distribution Rejection sampling scheme 2: based on sampling from vMF Y Y Y 9/21 Proposal samples are drawn from vMF density functions with parameter, constrained to be orthogonal to other columns of.

11. Sampling from the mMF Distribution Rejection sampling scheme 2: based on sampling from vMF Y Y Y 9/21 Proposal samples are drawn from vMF density functions with parameter, constrained to be orthogonal to other columns of.

12. Sampling from the mMF Distribution Rejection sampling scheme 2: based on sampling from vMF Y Y Y Rotate the modal direction 9/21 Proposal samples are drawn from vMF density functions with parameter, constrained to be orthogonal to other columns of.

13. Sampling from the mMF Distribution Rejection sampling scheme 2: based on sampling from vMF Y Y Y Rotate the sample to be orthogonal to the previous columns 9/21 Proposal samples are drawn from vMF density functions with parameter, constrained to be orthogonal to other columns of.

14. Sampling from the mMF Distribution Rejection sampling scheme 2: based on sampling from vMF Proposal samples are drawn from vMF density functions with parameter, constrained to be orthogonal to other columns of. Y Y Y Proposal distribution 9/21

15. Sampling from the mMF Distribution Rejection sampling scheme 2: based on sampling from vMF Sample scheme: 10/21

16. Sampling from the mMF Distribution A Gibbs sampling scheme Sample iteratively Note that. When. remedy: sampling two columns at a time Non-orthogonality among the columns of add to the autocorrelation in the Gibbs sampler. remedy: performing the Gibbs sampler on 11/21

17. Sampling from the BMF Distribution The vector Bingham distribution 12/21

18. Sampling from the BMF Distribution The vector Bingham distribution 12/21

19. Sampling from the BMF Distribution The vector Bingham distribution Better mixing 12/21

20. Sampling from the BMF Distribution The vector Bingham distribution From variable substitution, rejection sampling or grid sampling 12/21

21. Sampling from the BMF Distribution The vector Bingham distribution The density is symmetric about zero 12/21

22. Sampling from the BMF Distribution The vector Bingham-von Mises-Fisher distribution The density is not symmetric about zero any more, is no longer uniformly distributed on. The update of and should be done jointly. The modified step 2(b) and 2(c) are: 13/21

23. Sampling from the BMF Distribution The matrix Bingham-von Mises-Fisher distribution 14/21 Rewrite

24. Sampling from the BMF Distribution The matrix Bingham-von Mises-Fisher distribution 15/21 Sample two columns at a time Parameterize 2-dimensional orthonormal matrices as Uniform pairs on the circle Uniform

25. Sampling from the BMF Distribution The matrix Bingham-von Mises-Fisher distribution 16/21

26. Example: Eigenmodel estimation for network data 17/21

27. Example: Eigenmodel estimation for network data indicator of a link between nodes i and j. Posterior with respect to uniform prior, symmetric binary observed matrix, with the 0-1 18/21 E: symmetric matrix of independent standard normal noise BMF distribution with

28. Samples from two independent Markov chains with different starting values Example: Eigenmodel estimation for network data 19/21

29. Example: Eigenmodel estimation for network data 20/21

30. Conclusions The sampling scheme of a family of exponential distributions over the Stiefel manifold was developed; This enables us to make Bayesian inference for those orthonormal matrices and incorporate prior information during the inference; The author mentioned several application and implemented the sampling scheme on a network data set. 21/21

31. References Andrew T. A. Wood. Simulation of the von Mises Fisher distribution. Comm. Statist. Simulation Comput., 23:157- 164, 1994 G. Ulrich. Computer generation of distributions on the m-sphere. Appl. Statist., 33, 158-163, 1984 J. G. Saw. A family of distributions on the m-sphere and some hypothesis tests. Biometrika, 65, 69-74, 1978