1. Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by the Human Brain Project NIDCD NIMH

2. Goals An information framework for brain mapping Organized around structure –Spatial –Symbolic Implemented in a distributed, web- accessible system Optimized for language mapping

3. Photo

4. Visual Brain Mapper (VBM)

5. Requirements for visualization- based brain mapping Accurate reconstruction of brain surface as seen at neurosurgery Different problem than surface reconstruction for unfolding, flattening, analysis

6. Reconstructing the surface Isosurface following Needs cleanly segmented cortex: the skull- stripping problem

7. Approaches to skull-stripping Manual Deformable models 3-D region growing/mathematical morphology

8. A shape-based approach Represent “knowledge” of expected shape and range of variation of cortical envelope Use to guide the search for gray-CSF boundary Infer the boundary where none is present Allow user corrections

9. Representing shape Geometric constraint networks Set of local constraints Interact together to generate a global shape description

10. Constraint Network V i (V i1, V i2,…) V j (V j1, V j2,…) C ij

11. Geometric Constraint Network V i (V i1, V i2,…) V j (V j1, V j2,…) C ij

12. Radial contour model Structural component R i [R i,inner … R i,outer ] R j [R j,inner, R j,outer ] C ij

13. Radial contour model Shape component RiRi RjRj C ij = [Ri/Rj min … Ri/Rj max ]

14. Constraint Propagation

15. Radial surface model

16. Radial surface model neighbors

17. Shape-based skull stripping A module of VBM Load previously trained shape model Load MRI, MRV, MRA volumes Find cortical envelope Mask skull Find cortical surface Find veins and arteries

18. Mid-sagittal plane

19. Anterior Commissure

20. Posterior commissure

21. Cortical bounding box

22. Initial model

23. Initial model - end-on

24. Initial model - lateral view

25. Propagation from poles

26. Define one radial

27. Propagation from one radial

28. Error

29. Correct error

30. Generate mask

32. Error in mask

33. Generate mask

34. Generate isosurface

35. Extract veins

36. Mask veins

37. Vein isosurface

38. Extract arteries

39. Mask arteries

40. Artery isosurface

41. Vessels

42. All

43. Map

44. Workflow Select Patient Align Segment Cortex Segment Veins Segment Arteries Map Find Envelope Mask Volume Iso- Surface Select Model Define Landmarks Propagate & Search

45. Workflow Architecture Select Patient Align Segment Cortex Segment Veins Segment Arteries Map

46. Spatial Knowledge 3-D Models Images 3-D Maps Symbolic Model Builder Visual Brain Mapper Authoring Programs Spatial Information Internet Experiment Manager End User Web Interfaces Brain Map Visualizer Analyzer Graphics Server Knowledge Server Structural Information Servers Repository Manager Symbolic Knowledge Metadata Symbolic Information

47. Contributors Kevin Hinshaw Jeff Prothero Andrew Poliakov Richard Martin David Corina Ken Maravilla George Ojemann Rex Jakobovits Kate Mulligan Hector Lettich Linda Shapiro Cornelius Rosse