1. Fast N-Body Learning Nando de Freitas University of British Columbia

2. Historical Perspective Non-iterative or “direct” methods for eigenvalue problems and linear systems of equations require O(N 3 ) operations. Let's look at the history of what has been regarded as large N: 1950: N=20 1965: N=200 1980: N=2000 1995: N=20000 So over the course of 45 years N has increased by a factor of 10 3. However, the speed of computers has increased by a factor of 10 9. From this the O(N 3 ) bottleneck is evident. If only we could reduce the cost to O(N) – sigh!

3. Krylov for Eigen-Problems

4. Krylov for Systems of Equations

5. N-Body Problems in Learning Sum-kernel problem: Max-kernel problem:

6. N-Body Problems

7. Obvious applications of N-body Learning Exact and approximate message propagation. Markov chain Monte Carlo Gaussian processes, Wishart processes and Laplace processes. Spectral learning: eigenmaps, SNE, NCUTS, ranking on manifolds, … (even if using Nystrom) Reinforcement learning. The E step.

8. Kernel-(fill in your favourite name). Rao-Blackwellised Monte Carlo. Nearest neighbour methods. Some types of boosting. Computer graphics. EM, fluid dynamics, gravitation, quantum systems. … and much more ! Obvious applications of N-body Learning

9. Illustrative Example: Zhu, Lafferty & Zoubin

10. Illustrative Example Energy function using the graph Laplacian: Easy, but … a big linear system: Naïve iterative solution:

11. Illustrative Example We have solved a Gaussian process (where the covariance is the inverse graph Laplacian in O(N).

12. Illustrative Example

13. Message propagation Whether it’s exact: … or approximate:

14. Fast Methods in this Workshop

15. Fast Multipole Methods

16. Recursive Tree Structures

18. Distance Transform m(j) = min ( w(i) + d(i,j) ) i

19. In this workshop You’ll encounter tutorials on fast methods from the people who’ve been developing them. You’re likely to see encounter people arguing over error bounds, implementation strategies, applications and many more things. You’ll see statistics, learning, data structures and numerical computation come together. You’ll dream of the powder up on the hill.