1. Liz Marai 01/30/09 1 Computational Modeling and Visualization for Science Liz Marai Computer Science http://vis.cs.pitt.edu

2. Liz Marai 01/30/09 2 What is Computer Science? (… the study of computers?) (… the art of programming?) Edsger DijkstraEdsger Dijkstra: "Computer science is no more about computers than astronomy is about telescopes." Hint (early ‘computer scientist’ names): turingineer, turologist, flow- charts-man, applied meta-mathematician, comptologist, datalogist, computics specialist, informatik specialist

3. Liz Marai 01/30/09 3 Computer Science “the study of information and computation, and their implementation and application in computer systems“ [collective wisdom of Wikipedia] sub-areas emphasize: – the computation of specific results (e.g., computer graphics)computer graphics – properties of computational problems (e.g., computational complexity theory)computational complexity theory – the challenges in implementing computations (e.g., programming language theory, human-computer interaction)programming language theoryhuman-computer interaction in a nutshell, the study of computation

4. Liz Marai 01/30/09 4 The Study of Computation “Computer Science is a science of abstraction - creating the right model for a problem and devising the appropriate mechanizable techniques to solve it.” A. Aho and J. Ullman, 1992 “Computer Scientists are engineers of abstract objects” H. Zemanek, 1975 “The two A-s of computation: abstraction (i.e., modeling) - e.g., 115 pebbles  the natural number 115 -> the string (array) of characters ‘115’ or ‘CXV’ automation (mechanizing the abstraction) Computing is the automation of abstractions.” J. Wing, 2008 Director CISE at NSF

5. Liz Marai 01/30/09 5 Examples MySpace, You Tube are social networks DNA sequences are strings (that can be matched) Cells as a self-regulatory system are like electronic circuits Astronomy multi-dimensional data are KD-trees Abstraction: graph Automation: data structures and algorithms stack queuetree (upside- down)

6. Liz Marai 01/30/09 6 Abstraction and Automation Note: Neither abstraction nor automation are unique to Computer Science E.g. abstractions in other fields: Schroedinger’s equation in physics, chemistry; natural numbers, sets & tables in math etc. E.g. automation in other fields: algorithms for long division or factoring in math automated processes in engineering (not surprising! Cca 1960: math + electrical engineering -> CS) But implementing the “automation of abstractions” process as well as studying the properties of this process are traits of computer science.

7. Liz Marai 01/30/09 7 Computer Graphics Computer graphics generally means creation, storage and manipulation of geometrical models and their images Such models come from diverse, often non-CS fields including physical, mathematical, artistic, biological, and even conceptual (abstract) structures Frame from animation by William Latham, shown at SIGGRAPH 1992. Latham uses rules that govern patterns of natural forms to create his artwork.

8. Liz Marai 01/30/09 8 Keyframing smoke Adrien Treuille (UW)

9. Liz Marai 01/30/09 9

10. 10 Simulating the air flow around a bat wing M. Kostandov (Brown)

11. Liz Marai 01/30/09 11 Donald Burke, Pitt Public Health

12. Liz Marai 01/30/09 12 Pitt Visualization Research Lab

13. Liz Marai 01/30/09 13 Interdisciplinary Visualization Observe Hypothesize (across disciplines) Visualize Validate Evaluate Explore (across disciplines) Measure Model Simulate Insight [Laidlaw 2005]

14. Liz Marai 01/30/09 14 Example projects Motion tracking Predictive orthopaedics modeling The Chinese Room: collaborative machine translation

15. Liz Marai 01/30/09 15 Motion tracking Joint work with Yinglin Sun, MD. Abedul Haque, Scott Tashman, Bill Anderst

16. Liz Marai 01/30/09 16 (not too many sample poses – radiation concerns)

17. Liz Marai 01/30/09 17 UPMC: Orthopaedic Biodynamics Laboratory A consecutive sequence of 2-D radiographs

18. Liz Marai 01/30/09 18 Tracking motion: problem imaging artifacts -> limited tracking accuracy -> bone collisions 2 2 1 134 3 4 Grey-value matching

19. Liz Marai 01/30/09 19 Tracking motion: solution (Marai et al.,TMI'06) Step 1: extract bone outline from one volume image Step 2: use tissue-classification (neighborhood) to emphasize the bone boundary

20. Liz Marai 01/30/09 20

21. Liz Marai 01/30/09 21 Tracking motion:solution Step 3: optimize outline position & orientation until it matches the tissue-classified image (illustrated here in 2D)

22. Liz Marai 01/30/09 22 Tracking motion: results grey-value tissue-classif.vs. collision no collision 43% error-decrease compared to grey-value matching Results on marked cadaver data (motion error relative to ground truth, 0 is good)

23. Liz Marai 01/30/09 23 Tracking motion: summary ● sub-voxel accurate method for tracking bone-motion from sequences of CT scans ● 43% error-decrease from state-of-the-art technique ● 12 volume images in 1.5 hours on 40 processor cluster ● enables the analysis of soft-tissue deformation with motion ● results in a wrist motion database of unprecedented detail

24. Liz Marai 01/30/09 24 Inverse-imaging biological structures Joint work with David Laidlaw, Trey Crisco

25. Liz Marai 01/30/09 25 Computational modeling: joint-spacing and cartilage Idea: cartilage correlates with bone proximity parameter: p the proximity threshold

26. Liz Marai 01/30/09 26 Cartilage maps: results

27. Liz Marai 01/30/09 27 1.05mm1.21mmMax 0.276mm0.275mmMin 0.596mm ± 0.20mm0.601mm ± 0.21mmMean±Std.dev. Non-invasively (kinem.- generated) Invasively (µCT-imaged) Cartilage thickness

28. Liz Marai 01/30/09 28 Predictive orthopaedic systems Joint work with David Laidlaw, Trey Crisco, Douglas Moore

29. Liz Marai 01/30/09 29 1 2 images bone surfaces & motion anatomy book knowledge 3 soft tissue geometry & behavior visualization & quantification + …

30. Liz Marai 01/30/09 30 The push-up debate (Alexis vs. Crystal) on your knuckles or not?

31. Liz Marai 01/30/09 31 The push-up debate (Crystal wins) previously: computationally intractable CT volume images of one individual 7 different poses (knuckle-pose included) computed cartilage contact & ligament lengthening in each pose ~48 hrs, single processor knuckle pose yields maximum contact

32. Liz Marai 01/30/09 32 The push-up debate: knuckle-walkers

33. Liz Marai 01/30/09 33 DRUJ malunion Distal radioulnar joint (DRUJ)

34. Liz Marai 01/30/09 34 DRUJ malunion

35. Liz Marai 01/30/09 35 The Chinese Room Joint work with Josh Albrecht & Rebecca Hwa

36. Liz Marai 01/30/09 36 What does this say? Machine translations: “He utter eyes and not the slightest attention As leakage.” “He Zhengzhao eyes, eyes can no leakage.”

37. Liz Marai 01/30/09 37 A collaborative approach

38. Liz Marai 01/30/09 38 Chinese Room: results Example: MT: “He utter eyes and not the slightest attention As leakage.” Chinese Room MT: “His eyes were placed wide-apart; nothing escaped their attention.” MT-quality improved on average from 0.35 to 0.53 the gap between MT and pro bilingual translations reduced by 36.9%

39. Liz Marai 01/30/09 39 CS 2620 Interdisciplinary Modeling and Visualization Pitt CS Teaching Award ’08 offered Spring’09 Mon/Wed 11am visualization nuts and bolts 2 nd half: work in small multidisciplinary groups Image credits: cs2620 alumni J.Albrecht, M.Grabmair, Yl.Sun, J.D.Park, M.Fagerburg

40. Liz Marai 01/30/09 40 Contact http://vis.cs.pitt.edu marai@cs.pitt.edu SENSQ 5423

41. Liz Marai 01/30/09 41