1. Usability & Evaluation in Visualizing Biological Data Chris North, Virginia Tech VizBi

2. Usomics & Evaluation in Visualizing Biological Data Chris North, Virginia Tech VizBi

3. Myths about Usability Usability = Voodoo

4. Science of Usability Measurement Modeling Engineering Science Phenomenon …analogy to biology

5. Usability Engineering User-centric Iterative Engineering = process to ensure usability goals are met 1. Analyze Requirements 2. Design 3. Develop 4. Evaluate

6. Myths about Usability Usability = Voodoo Usability = Learnability

8. Myths about Usability Usability = Voodoo Usability = Learnability Usability = Simple task performance

9. Impact on Cognition SpotfireGeneSpring Insight gained:

10. Myths about Usability Usability = Voodoo Usability = Learnability Usability = Simple task performance Usability = Expensive http://www.upassoc.org/usability_resources/usability_in_the_real_world/roi_of_usability.html

11. Usability Engineering 1. Analyze Requirements 2. Design 3. Develop 4. Evaluate

12. Requirements Analysis Goal = understand the user & tasks Methods: Ethnographic observation, interviews, cognitive task analysis Challenge: Find the hidden problem behind the apparent problem

14. Analysts’ Process Pirolli & Card, PARC

15. Systems Biology Analysis Beyond read-offs -> Model-based reasoning Mirel, U. Michigan

16. Usability Engineering 1. Analyze Requirements 2. Design 3. Develop 4. Evaluate

17. Why Emphasize Evaluation? Many useful guidelines, but… Quantity of evidence Exploit domain knowledge Hunter, Tipney, UC-Denver

18. Science of Usability Measurement Modeling Phenomenon

19. Measuring Usability in Visualization system, algorithm Measurements frame-rate capacity … realism data/ink … market ? ? 2 kinds of holes visual perception, interaction inference, insight goal, problem solving Phenomena task time accuracy …

20. Time & Accuracy Controlled Experiments Benchmark tasks

21. Results

22. + Consistent overall + Fast for single node analysis - Slow and inaccurate for expression across graph + Accurate for comparing timepoints p<0.05

23. Cerebral Munzner, UBC

24. Insight-based Evaluation Problem: Current measurements focus on low-level task performance and accuracy What about Insight? Idea: Treat tasks as dependent variable What do users learn from this Visualization? Realistic scenario, open-ended, think aloud Insight coding Information-rich results

25. Insight? Spotfire GeneSpring Cluster/Treeview TimeSearcher HCE Gene expression visualizations

26. Cluster- Time- Gene- ViewSearcher HCE Spotfire Spring Count of insights Total value of insights Average time to first insight (minutes) Results

27. Overall Visualization Tool Unexpected Insights Hypotheses Generated Incorrect Insights Clusterview320 TimeSearcher310 HCE512 Spotfire230 GeneSpring000

28. Insight Summary Time series Viral conditions Lupus screening Clusterview TimeSearcher HCE Spotfire GeneSpring

29. Users’ Estimation Total value of insights Users’ estimated insight percentage Cluster- Time- Gene- ViewSearcher HCE Spotfire Spring

30. Insight Methodology Difficulties: Labor intensive Requires domain expert Requires motivated subjects Short training and trial time Opportunities: Self reporting data capture Insight trails over long-term usage – Insight Provenance

31. Trend towards Longitudinal Evaluation Multidimensional in-depth long-term case studies (MILCS) Qualitative, ethnographic GRID: Study graphics, find features, ranking guides insight, statistics confirm But: Not replicable, Not comparative Shneiderman, U. Maryland

32. Onward… VAST Challenge Analytic dataset with ground truth E.g. Goerg, Stasko – JigSaw study BELIV Workshop – BEyond time and errors: novel evaLuation methods for Information Visualization

33. Visual Analytics VisualizationVisual Analytics Perception, Interaction Cognition, Sensemaking Visualization tasksWhole analytic process Visual representations, interaction techniques Connection to data mining, statistics, … Datatype scenariosReal usage scenarios, Analysts

34. Embodied Interaction GigaPixel Display Lab, Virginia TechCarpendale, U. Calgary 1) Cognition is situated. 2) Cognition is time-pressured. 3) We off-load cognitive work onto the environment. 4) The environment is part of the cognitive system. 5) Cognition is for action. 6) Off-line cognition is body-based. -- Margaret Wilson, UCSC