1. Visualization for astronomy, via medical imaging Michael Halle Initiative in Innovative Computing at Harvard & Surgical Planning Lab, Brigham and Women’s Hospital http://am.iic.harvard.edu Michael Halle Initiative in Innovative Computing at Harvard & Surgical Planning Lab, Brigham and Women’s Hospital http://am.iic.harvard.edu IIC Astronomical Medicine Project http://am.iic.harvard.edu Image: M. Borkin

2. IIC Astronomical Medicine Project http://am.iic.harvard.edu Outline The data analysis (“visualization”) pipeline Medical imaging and astronomy Algorithms, libraries, frameworks, apps The AM project

3. IIC Astronomical Medicine Project http://am.iic.harvard.edu Welcome to my world… …your quick introduction to medical imaging, and thereby astronomical visualization.

4. IIC Astronomical Medicine Project http://am.iic.harvard.edu Wayback machine: early 1990’s Radiologists: special training Slice by slice inspection of CT or MRI “We do it in our head” Not prepared for large datasets “Cine loop”: not much better

5. IIC Astronomical Medicine Project http://am.iic.harvard.edu Today 3D visualization widespread Complex structures Advanced image processing

6. IIC Astronomical Medicine Project http://am.iic.harvard.edu Example: Diffusion Tensor Imaging A. Mewes

7. IIC Astronomical Medicine Project http://am.iic.harvard.edu The jump to 3D, with new semantics A. Mewes

8. IIC Astronomical Medicine Project http://am.iic.harvard.edu A. Mewes

9. IIC Astronomical Medicine Project http://am.iic.harvard.edu From scanner to display segmentregisteracquire visualize display & interact enhance

10. IIC Astronomical Medicine Project http://am.iic.harvard.edu Acquisition

11. IIC Astronomical Medicine Project http://am.iic.harvard.edu Enhancement C.F. Westin et. al.

12. IIC Astronomical Medicine Project http://am.iic.harvard.edu Registration Align multiple images Different…  positions  timepoints  subjects May be warped In medicine, no absolute coordinates S. Pieper

13. IIC Astronomical Medicine Project http://am.iic.harvard.edu Segmentation T. Kapur. Guttmann et. al Single channel Multi-channel

14. IIC Astronomical Medicine Project http://am.iic.harvard.edu Visualization

15. IIC Astronomical Medicine Project http://am.iic.harvard.edu Display and interaction B. Davis, S. Barre, Y. Yuan, W. Schroeder, P. Golland, K. Pohl

16. IIC Astronomical Medicine Project http://am.iic.harvard.edu Not a straight-line process! Interaction can drive:  image processing  registration  segmentation  visualization Iteration is powerful S. Pieper

17. IIC Astronomical Medicine Project http://am.iic.harvard.edu So what is “visualization” ? Computer graphics and computation - plus - Generic structures and techniques  volumes  isosurfaces  streamlines  visual controls, etc. - plus - Application semantics - plus - Human perception Pieper, Fallon (UCI)

18. IIC Astronomical Medicine Project http://am.iic.harvard.edu Example: isosurfaces vs. volume rendering Isosurface renderingVolume rendering representation:surfacesvolumes primitives:continuousdiscrete speed:faster, grows linearly w/complexityslower, stays constant w/complexity segmentation:your choiceglobal transfer functions SCI Institute, Univ. of Utah Talos et. al.

19. IIC Astronomical Medicine Project http://am.iic.harvard.edu How is this like astronomy? Multi-dimensional image data Analysis pipeline:  s/human/sky/g  s/scanner/telescope/g - Most of analysis pipeline the same - Differences:  WCS  Often only partially spatial  Second order image/noise characteristics  Interchange issues Borkin

20. IIC Astronomical Medicine Project http://am.iic.harvard.edu “Astronomical Medicine” Bootstrap astronomical visualization & discover Dual-purpose development Astronomy-specific functionality Cross-pollenate segmentation techniques Borkin Rosolowsky, Kauffmann et.al.

21. IIC Astronomical Medicine Project http://am.iic.harvard.edu Different layers of software Applications  3D Slicer, OsiriX, ParaView, VolView, Amira…  Apps you’ll see here Frameworks/libraries  Visualization: VTK  Segmentation and registration: ITK Rendering and interaction libraries  OpenInventor/Coin  Java3D  OpenGL Borkin

22. IIC Astronomical Medicine Project http://am.iic.harvard.edu Applications: 3D Slicer Medical image analysis, segmentation, and visualization platform Cross platform, open source Surface rendering (volume rendering to come) Borkin

23. IIC Astronomical Medicine Project http://am.iic.harvard.edu Applications: OsiriX Originally designed for radiology image databases Mac only, open-source Volume rendering (some isosurfaces as well) Borkin

24. IIC Astronomical Medicine Project http://am.iic.harvard.edu Research prototypes Dendrogram visualization AMView astronomy data viewer

25. IIC Astronomical Medicine Project http://am.iic.harvard.edu Stereoscopic imaging and 3D See Nick Holliman’s talk and demos! Stereoscopic imaging and 3D

26. IIC Astronomical Medicine Project http://am.iic.harvard.edu Parting thoughts Augment, not replace existing tools Working together to solve, create and discover New “golden age” of astronomy visualization? Borkin

27. IIC Astronomical Medicine Project http://am.iic.harvard.edu Thanks Astronomical Medicine Project Alyssa Goodman, Michelle Borkin, Jens Kauffmann, Doug Alan, Nick Holliman Harvard IIC Rita Tavilla, Emily Lohmann Surgical Planning Lab Ron Kikinis, Steve Pieper, Tina Kapur, and many others Harvard-Smithsonian Center for Astrophysics The CfA COMPLETE team, including Erik Rosolowsky