Stochastic approximate inference Kay H. Brodersen Computational Neuroeconomics Group Department of Economics University of Zurich Machine Learning and Pattern Recognition Group Department of Computer Science ETH Zurich

2 When do we need approximate inference? sample from an arbitrary distribution compute an expectation

3 Deterministic approximations through structural assumptions  application often requires mathematical derivations (hard work)  systematic error  computationally efficient  efficient representation  learning rules may give additional insight Which type of approximate inference? Stochastic approximations through sampling  computationally expensive  storage intensive  asymptotically exact  easily applicable general-purpose algorithms

4 Themes in stochastic approximate inference z z sampling summary

Transformation method 1

6 transformation

7 Transformation method: algorithm

8 Transformation method: example

9  Discussion  yields high-quality samples  easy to implement  computationally efficient  obtaining the inverse cdf can be difficult Transformation method: summary

Rejection sampling and importance sampling 2

11 Rejection sampling

12 Rejection sampling: algorithm

13 Importance sampling

14 Importance sampling

Markov Chain Monte Carlo 3

16  Idea: we can sample from a large class of distributions and overcome the problems that previous methods face in high dimensions using a framework called Markov Chain Monte Carlo. Markov Chain Monte Carlo (MCMC)

17 Background on Markov chains

18 Background on Markov chains Equlibrium distribution Markov chain: state diagram 0.5

19 Background on Markov chains Equlibrium distribution Markov chain: state diagram

20 The idea behind MCMC z z Empirical distribution of samples Desired target distribution

21 Metropolis algorithm

22 Metropolis-Hastings algorithm

23 Metropolis: accept or reject? Inrease in density: Decrease in density: p(z*) p(z τ ) p(z*) zτzτ zτzτ z*

24 Gibbs sampling

25 Gibbs sampling

26  Throughout Bayesian statistics, we encounter intractable problems. Most of these problems are: (i) evaluating a distribution; or (ii) computing the expectation of a distribution.  Sampling methods provide a stochastic alternative to deterministic methods. They are usually computationally less efficient, but are asymptotically correct, broadly applicable, and easy to implement.  We looked at three main approaches: € Transformation method: efficient sampling from simple distributions € Rejection sampling and importance sampling: sampling from arbitrary distributions; direct computation of an expected value € Monte Carlo Markov Chain (MCMC): efficient sampling from high-dimensional distributions through the Metropolis-Hastings algorithm or Gibbs sampling Summary

27 Supplementary slides

28 Transformation method: proof

29 Background on Markov chains