Speaker: Shau-Shiang Hung( 洪紹祥 ) advisor :Shu-Chen Cheng( 鄭淑真 ) Date : 2010/4/8 Computer Graphics and Applications, IEEE Publication Date : March-April 2005 Volume : 25, Issue:2 On page(s): Author : Alexandre Gillet, Michel Sanner, Daniel Stoffler, and Arthur Olson

Outline Introduction Design of physical models Augmented reality interface Implementation Examples Evaluation Conclusions

Introduction 1/2 With the prevalence of structural and genomic data, molecular biology has become a human-guided, computer-assisted endeavor Exploring scientific date Test scientists hypotheses As databases grow, as structure and process,and as software methods become more diverse Access and manipulation of digital information is increasingly a critical issue

Introduction 2/2 Develop an augmented reality (AR) system Visual 3D representations Molecular structure 、 properties With tangible interaction environment User can intuitively manipulate molecular models and interactions Easily change the representation shown Access information about molecular properties

Design of physical models 1/2 Use Python Molecular Viewer (PMV) Create virtual objects Design tangible models Simplifying the integration of the models with the virtual environment. PMV is software framework Molecular surfaces Extruded volumes Backbone ribbons Atomic ball-and-stick Show molecular shape in interaction Atomic details

Design of physical models 2/2 PMV includes Generic 3D visualization component A high level interface to the OpenGL library and its geometry viewing application Add all molecular modeling and visualization functionality Output formats (.stl and.vrml) for the solid printers serve as input

Augmented reality interface 1/2 Use computer-based spatial tracking and rendering method to enhance the physical models’ semantic content and show dynamic properties. Combines real-world user and physical model presence with computation models and data Approach is based on the widely used ARToolKit Calculate the real camera position and orientation relative to physical marker in real time, allowing overlay of virtual objects relative to the physical markers.

Augmented reality interface 2/2 Add PyARTK Integrate ARToolKit with PMV to manage markers Advantage: Streamline the design and display of models within the same environment. Assigns the geometries, animations, and masks to specific AR markers or sets of markers The interface comes with controls for : Computer graphic objects Camera operations Clipping Lighting controls Marker

Implementation 1/2 Integrating the physical models with the virtual augmentation requires superpositioning the two worlds into the same perceptual space There are a number of possible ways using a head-mounted display the user sees video of the real-world scene and the superimposed computer graphic information. Projecting the computer information onto the physical model. But expense and imperfect performance

Implementation 2/2 A simpler two-view Solution This configuration has proven to be an effective, inexpensive, and portable solution. Physical model Computer –augmented scene Camera

Examples 1/2 HIV protease

Examples 2/2 Superoxide dismutase (SOD)

Evaluation 1/1 The first pilot test involved high school students from the Biotech Academy program in Seattle The second pilot study with students from a college- level biochemistry class at the University of Washington Produced a number of models for colleagues in our institute for applications ranging from drug design to assembly of large biomolecular complexes Received uniformly positive comments for enhanced comprehension and communication of structural characteristics.

Conclusions 1/1 In our experience, we found that tangible molecular models can provide several advantages over computer visualizations alone. They produce a multisensory engagement that includes visual, tactile, and proprioceptive perceptual pathways for learning and memory. Using PMV along with our system has proven to be a fast and efficient approach to develop and test new ideas.