Information Enhancement of Synthetic Environments Nicholas F. Polys Virginia Bioinformatics Institute Virginia Tech Computer Science & Center for Human Computer Interaction

Presentation Introduction HCI & Usability Engineering Integrated Information Spaces Research: –CML -> X3D –CML -> Snap + Diverse –Biological Simulation in PathSim

What is HCI? A multidisciplinary science of the interface: psychology, design & media, human factors, sociology, computer science Experimental methods to rationalize UI features, design, and software architecture

Usability Engineering Where the rubber meets the road… Scenario-Based Design: –Activities –Information –Interaction –Claims analysis See: ROSSON, M.B. AND CARROLL, J Usability Engineering: Scenario based development of Human-Computer Interaction. NY, Morgan Kaufmann.

Problem: Integrated Information Spaces Researchers, designers, students, and analysts need to explore, search, recognize, and compare patterns across a wide variety of data types and applications. Some representations are better suited to different data types and questions…

Fundamental Information Types –Abstract information -> Information Visualizations –Perceptual or spatial information -> Virtual Environments –Temporal Information

User Problem: Perceptual, environmental phenomena alone do not guarantee accurate mental models Interfaces for information and controls are consistently under-represented and poorly integrated in VEs

Developer Problem: Increasingly, objects rendered in VEs have metadata and annotations associated with them. These may change over time. Lack of design guidelines and information architectures for efficient management of Integrated Information Spaces

Approach: Information-Rich Virtual Environments (IRVEs) IRVEs are at the union of VE and Info Vis: perceptual + abstract information is simultaneously accessible and linked Define the issues and the task space for this class of applications Define the design space to solve and support them Apply usability engineering methods to identify tradeoffs and enumerate design guidelines

Some Research Questions: Where, how, and when should the abstract information be displayed in the VE? In what form should the information be displayed? How can interrelationships between the spatial and abstract information be represented?

IRVE Activity Design See also : Bowman, Doug, North, Chris, Chen, J., Polys, N., Pyla, P., Yilmaz, U., (2003) “Information-Rich Virtual Environments: Theory, Tools, and Research Agenda”. Proc. Virtual Reality Software and Technology (VRST) ActivityTaskExample ExploringOverview data loading, choosing the data and display method FindingDetails-on- demand pop-ups, queries, lenses, zooming via 3D navigation etc. Pattern Recognition Relating, Comparing rendering associations; brushing and linking

Task-Knowledge Structure e.g. for Situational Awareness and decision support systems… [Entity Relationship diagrams per task and media] SUTCLIFFE, A. AND FARADAY, P “Designing Presentation in Multimedia Interfaces”, Proc. CHI '94,

Research Thrust: Data Models behind IRVEs

CML via XSLT

CML is a good example of an IRVE database POLYS, NICHOLAS F “Stylesheet Transformations for Interactive Visualization: Towards a Web3D Chemistry Curricula”, Proceedings of the Web3D 2003 Symposium, ACM SIGGRAPH.

Research Thrust: Interactions and Architectures for IRVEs

Snap2Diverse: Coordinating Information Visualizations and Virtual Environments DIVERSE is an multi-platform open-source VR toolkit [diverse.sourceforge.net] Snap-Together Visualization is a free toolkit for information visualization [infovis.cs.vt.edu]

Multiple, Coordinated Views for IRVEs Users can index to and from perceptual and abstract information through ‘brushing and linking’ –Diverse as a Snap-able component –Unique Ids –Event communication (e.g. select, load)

System overview

Application: Cheminformatics

Snap2Diverse Video POLYS, N. F., NORTH, C., BOWMAN, D. A., RAY, A., MOLDENHAUER, M., AND DANDEKAR, C “Snap2Diverse: Coordinating Information Visualizations and Virtual Environments”. Proc. SPIE Conference on Visualization and Data Analysis.

Usability Evaluation Summary Snap2Diverse as tested is: –high-threshold for learnability –high-ceiling for functionality Users were able to brush&link to solve tasks In most cases, users chose suitable visualizations to recover the information required for the finding and comparing tasks Given a choice, users tended to recover information from the VE instead of InfoVis Xwand is a moded interface and switching between 2D and 3D navigation and selection was not intuitive.

Lessons Learned Simple events provide ‘glue’ to support complex functionality: Benefits of exposing VE scenegraphs to external events; enabling technology must be extended Benefits of composable and integrated information spaces Designers must consider tasks and the knowledge required for completion (ie Sutcliffe and Faraday, Shneiderman)

Research Thrust: Objects and Patterns for IRVEs X3D Event Utilities MFSequencers Information Panels Semantic Objects HUDs

PathSim Demo Paper to be presented at Web3D 2004: Polys, N., Bowman, D., North, C., Laubenbacher, R., Duca, K., (2004). “PathSim Visualizer: An Information-Rich Virtual Environment for Systems Biology” Proceedings of Web3D 2004, ACM SIGGRAPH.

Summary There is an increasing demand for information-rich functionality across a wide variety of domains A systematic HCI approach to design problems of Integrated Information Spaces is required Goal is to understand tradeoffs of design and software architectures to build better interfaces

Future Work IRVE Testbed construction Controlled experiments and usability evaluations to identify the significant design dimensions and techniques for integrated information spaces. Identify tradeoffs, guidelines, and design patterns for this class of problems. Propose future X3D standard components: –Annotation (labels, metadata) –Compositing (HUD, ApplicationTexture)

Thanks and happy hacking!