1. Bayesian inference Lee Harrison York Neuroimaging Centre 01 / 05 / 2009

2. Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

3. Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

4. Generation Recognition Introduction time

5. Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

6. Curve fitting without Bayes DataOrdinary least squares

7. Curve fitting without Bayes DataOrdinary least squares

8. Curve fitting without Bayes Ordinary least squares Bases (explanatory variables)Sum of squared errors Data and model fit Bases (explanatory variables)Sum of squared errors

9. Curve fitting without Bayes Data and model fitOrdinary least squares Bases (explanatory variables)Sum of squared errors

10. Curve fitting without Bayes Data and model fitOrdinary least squares Bases (explanatory variables)Sum of squared errors

11. Curve fitting without Bayes Data and model fit Bases (explanatory variables)Sum of squared errors Over-fitting: model fits noise Inadequate cost function: blind to overly complex models Solution: incorporate uncertainty in model parameters Ordinary least squares

12. Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

13. Bayesian Paradigm: priors and likelihood Model:

14. Bayesian Paradigm: priors and likelihood Model: Prior:

15. Sample curves from prior (before observing any data) Mean curve Bayesian Paradigm: priors and likelihood Model: Prior:

16. Bayesian Paradigm: priors and likelihood Model: Prior: Likelihood:

17. Bayesian Paradigm: priors and likelihood Model: Prior: Likelihood:

18. Bayesian Paradigm: priors and likelihood Model: Prior: Likelihood:

19. Bayesian Paradigm: posterior Model: Prior: Likelihood: Bayes Rule: Posterior:

20. Bayesian Paradigm: posterior Model: Prior: Likelihood: Bayes Rule: Posterior:

21. Bayesian Paradigm: posterior Model: Prior: Likelihood: Bayes Rule: Posterior:

22. Bayesian Paradigm: model selection Cost function Bayes Rule: normalizing constant Model evidence:

23. Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

24. Hierarchical models recognition time space generation

25. Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

26. Variational methods: approximate inference True posterior Initial guess and iteratively improve to approximate true posterior L F KL fixed Can compute Maximize minimize KL Difference btw approx. and true posterior But cannot compute as do not know But how?

27. Variational methods: approximate inference accuracy complexity If you assume posterior factorises then F can be maximised by letting where

28. Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

29. AR coeff (correlated noise) prior precision of AR coeff VB estimate of W ML estimate of W aMRI smoothed W (RFT) fMRI time series analysis with spatial priors observations GLM coeff prior precision of GLM coeff prior precision of data noise Penny et al 2005 degree of smoothnessSpatial precision matrix

30. AR coeff (correlated noise) prior precision of AR coeff VB estimate of W ML estimate of W aMRI smoothed W (RFT) fMRI time series analysis with spatial priors observations GLM coeff prior precision of GLM coeff prior precision of data noise Penny et al 2005

31. Mean (Cbeta_*.img) Std dev (SDbeta_*.img) PPM (spmP_*.img) activation threshold Posterior density q(w n ) Probability mass p n fMRI time series analysis with spatial priors: posterior probability maps probability of getting an effect, given the data mean: size of effect covariance: uncertainty Display only voxels that exceed e.g. 95%

32. fMRI time series analysis with spatial priors: single subject - auditory dataset 0 2 4 6 8 0 50 100 150 200 250 Active > Rest Active != Rest Overlay of effect sizes at voxels where SPM is 99% sure that the effect size is greater than 2% of the global mean Overlay of 2 statistics: This shows voxels where the activation is different between active and rest conditions, whether positive or negative

33. fMRI time series analysis with spatial priors: group data – Bayesian model selection Compute log-evidence for each model/subject model 1 model K subject 1 subject N Log-evidence maps

34. fMRI time series analysis with spatial priors: group data – Bayesian model selection BMS maps PPM EPM model k Joao et al, 2009 (submitted) Compute log-evidence for each model/subject model 1 model K subject 1 subject N Log-evidence maps Probability that model k generated data

35. Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

36. Distributed Source model MEG/EEG Source Reconstruction Forward model (generation) Inversion (recognition) - under-determined system - priors required Mattout et al, 2006 [n x t][n x p] [n x t] [p x t] n : number of sensors p : number of dipoles t : number of time samples

37. Overview Probabilistic modeling and representation of uncertainty –Introduction –Curve fitting without Bayes –Bayesian paradigm –Hierarchical models Variational methods (EM, VB) SPM applications –fMRI time series analysis with spatial priors –EEG source reconstruction –Dynamic causal modelling

38. Neural state equation: Electric/magnetic forward model: neural activity EEG MEG LFP Neural model: 1 state variable per region bilinear state equation no propagation delays Neural model: 8 state variables per region nonlinear state equation propagation delays fMRIERPs inputs Hemodynamic forward model: neural activity BOLD Dynamic Causal Modelling: generative model for fMRI and ERPs

39. Thank-you