Presentation is loading. Please wait.

Presentation is loading. Please wait.

BIRS 2016: Opening the analysis black box: Improving robustness and interpretation Matthew Brown, PhD University of Alberta, Canada.

Similar presentations


Presentation on theme: "BIRS 2016: Opening the analysis black box: Improving robustness and interpretation Matthew Brown, PhD University of Alberta, Canada."— Presentation transcript:

1 BIRS 2016: Opening the analysis black box: Improving robustness and interpretation Matthew Brown, PhD University of Alberta, Canada

2 Overview 1.About us 2.Preprocessing quality assurance 3.Interpretation of group vs. individual differences 4.Trial type fMRI signatures

3 Dept. Psychiatry Dept. Computing Science Computational Psychiatry Group

4 Diagnosis – What disease? Prognosis – Predict patient response to treatment options Clinical decision-making

5 What are we detecting? 10 psychosis patients, 10 controls, fMRI Highly diagnostic Fourier power distribution from voxels IN THE EYES Eye movement disturbances in psychosis

6 ADHD-200 and ABIDE datasets n=1000 approx. ADHD patients or autism patients Structural MRI, resting state fMRI Simple diagnosis – Classify patients vs. controls – Accuracy 50-70% in various papers Some papers reported higher 75%+ accuracy BUT cherry-picking sites?

7 ADHD-200 Global Competition Best-performing algorithm, but did not win Used only non-imaging features: – Age, gender, handedness, IQ, site of scan – 3-class classification (ADHD-c, ADHD-i, control) – 63% hold-out accuracy (vs. 54% chance) Using non-imaging features Brown et al. 2012 Chance accuracy Validation Accuracy (%)

8 Histogram of oriented gradient (HOG) features Image from Ghiassian et al. under review. Also see Dalal and Triggs 2005. IEEE Computer Society Conference on. vol. 1. IEEE, p.886–893.

9 ADHD-200 and ABIDE datasets Ghiassian et al. under review State of the art (as of 1.5 years ago) 2-class classification (patients vs. controls) ADHD-200ABIDE Chance55%51% Non-imaging69%60% Non-imaging + Structural MRI 70%64% Non-imaging + Functional MRI 64%65%

10 Overview 1.About us 2.Preprocessing quality assurance 3.Interpretation of group vs. individual differences 4.Trial type fMRI signatures

11 Registration failure Subject 1Subject 15 Fixed -> Standard preprocessing methods failed for 1 of 21 subjects.

12 Inter-site variability Sen et al. in preparation PCA Component 1 PCA Component 2 ADHD-200 Subjects Projected onto PCA component space Each colour is a different scanning site. Even with standard normalization procedures, inter-site structure remains in the data.

13 Overview 1.About us 2.Preprocessing quality assurance 3.Interpretation of group vs. individual differences 4.Trial type fMRI signatures

14 Clinical research Huntington’s Image from Wikipedia Healthy One goal: Associate disease with biological features

15 ADHD-200 resting state fMRI functional connectivity analysis ICA Brown et al. 2012

16 ADHD patients vs. controls “Default mode” networkPatients vs. controls Brown et al. 2012 “Desired” simple interpretation: “Patients are different from controls. This difference tells us something about the disease.”

17 Group vs. individual differences Patients Controls Statistically significant group differences, but substantial overlap between individual patients and controls. Brown et al. 2012

18 Interpretation Simple interpretation “patients are different from controls” Overlap precludes simple interpretation Yet many papers provide precisely and only the simple interpretation Patients Controls Brown et al. 2012

19 Overview 1.About us 2.Preprocessing quality assurance 3.Interpretation of group vs. individual differences 4.Trial type fMRI signatures

20 Black box analysis Analysis Software Analysis Software

21 General linear model regression Model voxel i’s timecourse Model matrix for trial type k

22 Two different models for hemodynamic response function SPM canonical model Finite impulse response model

23 Check deconvolved timecourses Basically agree on shape (but not statistical differences in this case) SPM canonical model Finite impulse response model, same region

24 Check deconvolved timecourses SPM canonical model Finite impulse response model, same region Noise in deconvolved timecourses

25 Another example SPM canonical model Finite impulse response model, same region Noise in deconvolved timecourses

26 GLM analysis Check deconvolved timecourses What is the model fitting – Noise vs. signal Model selection – regularization

27 Summary Quality check everything Visualization Intermediate steps and final results Particularly important for non-technical end-users

28 Acknowledgements People: Azad, Benoit, Dursun, Ghiassian, Greenshaw, Greiner, Juhas, Purdon, Ramasubbu, Rish, Sen, Silverstone Funding: AICML, AIHS, CIHR, Norlien Foundation, AHS, AMHB, UAlberta Questions?

29 Invitation Continue informing other researchers about analysis pitfalls and caveats. Questions?

30 Title


Download ppt "BIRS 2016: Opening the analysis black box: Improving robustness and interpretation Matthew Brown, PhD University of Alberta, Canada."

Similar presentations


Ads by Google