What are your interactions doing for your visualization? Remco Chang UNC Charlotte Charlotte Visualization Center

Outline: Three Areas of Proposed Research… What is the role of interaction in visual analytics? – Is there a science to designing interactions and applying them to visualizations? How do we know if an interaction is “good”? – Can we evaluate (quantify?) the benefits (or costs) of interactions? If analysts use interactions to perform analysis, can we store the knowledge in the interactions? – Is it possible to create a knowledge-base by extracting knowledge from interaction logs?

Introduction: Role of Interaction Most people in the visual analytics community believe that interactivity is essential for analysis: – “A [visual] analysis session is more of a dialog between the analyst and the data… the manifestation of this dialog is the analyst’s interactions with the data representation” [Thomas & Cook 2005] – “Without interaction, [a visualization] technique or system becomes a static image or autonomously animated images” [Yi et al. 2007]

Motivation: Role of Interaction More explicitly: [Pike et al. 2009] – “A central precept of visual analytics is that it is through the interactive manipulation of a visual interface – the analytic discourse – that knowledge is constructed, tested, refined, and shared.” – “These visual displays must be embedded in an interactive framework that scaffolds the human knowledge construction process with the right tools and methods to support the accumulation of evidence and observations into theories and beliefs.”

VISUAL analytics or visual ANALYTICS? Observation: Current designs of visual analytical systems start with visual representation and add in appropriate interactions afterwards. Visual analytics = visual representation + analytics – assume that (interaction == analytic discourse), then: – Visual analytics = visual representation + interaction – VISUAL analytics = VISUAL REPRESENTATION + interaction – visual ANALYTICS = visual representation + INTERACTION Proposal: If we start the design of visual analytical systems with interactions (i.e., how a user would perform a series of tasks, or to generate hypotheses), we could focus on the ANALYTICS aspect in the design. This seems pretty hard to do… Arguably because we don’t really understand the nature of interactions for the purpose of analytics.

Case Study: Brushing and Linking The linchpin in most visualizations that utilize multiple coordinated views. – Spotfire, GeoVISTA, JIGSAW, etc. However, when used in a collaborative environment, it’s purpose becomes slightly different even though the implementation is (mostly) the same. [Isenberg et al. 2009] Hypothesis: the nature of Brushing and Linking is to coordinate between different perspectives of the same data elements, especially for data of high dimensionality. It is now easier to consider a system design around this… VisualAnalytics

Evaluation: The Benefits of Interactions Scientifically, how is interaction useful? With interaction, – Does an analyst perform tasks faster? – Does an analyst perform tasks more accurately? Short answer: no [Lim et al. 1996] [Jeong et al. 2009a] [Jeong et al. 2009b] [Lipford et al. 2009]

Evidence: Interaction Is Useful in Visualizations… Empirical evidence that interactivity is useful… (1) Users don’t “give up” as easily [Jeong et al. 2009] Green bar denotes the number of participants who “gave up” during an analysis. (iPCA is an interactive visualization, and SAS/INSIGHT is a traditional text-based interface with limited interactivity)

Evidence: Interactivity Is Useful in Visualizations… Empirical evidence that interactivity is useful… (2) Users become more proficient faster The longer a user uses an interactive visualization, the better (faster) they become. Whereas when the same user uses a non-interactive visualization, the amount of time spent remains (roughly) the same. Time Spent using System User’s Task Completion Time Non-Interactive System Interactive System Slow Fast

Evidence: Interactivity Is Useful in Visualizations… Empirical evidence that interactivity is useful… (3) Users prefer interactivity [Jeong et al. 2009] Users giving letter grades to the two tools after using them during an experiment. (iPCA is an interactive visualization, and SAS/INSIGHT is a traditional text-based interface with limited interactivity)

Future Work: How is Interactivity Useful? We propose that: (1)Interactivity is indeed useful (2)We’ve been measuring the wrong things Hypothesis: – Interactivity is useful to keep a user “in a cognitive zone” which is why they don’t give up – Interactivity allows the user to gather more “contextual information” users spend more time to understand the problem before attempting to solve it – We need new metrics and methods to measure the “benefits of interactivity”

Provenance: Capturing User Interactions What is in a user’s interactions? If (interactions == analytic discourse), what can we learn from the user’s interactions? Is it possible to extract “analysis” from “interactions”?

Study: What is in a User’s Interactions? Goal: determine if there really is “analysis” in a user’s interactions. Analysts Grad Students (Coders) Logged (semantic) Interactions Compare! (manually) Strategies Methods Findings Guesses of Analysts’ thinking WireVis Interaction-Log Vis

Results: What’s in a User’s Interactions From this experiment, we find that interactions contains at least: – 60% of the (high level) strategies – 60% of the (mid level) methods – 79% of the (low level) findings R. Chang et al., Recovering Reasoning Process From User Interactions. IEEE Computer Graphics and Applications, R. Chang et al., Evaluating the Relationship Between User Interaction and Financial Visual Analysis. IEEE Symposium on VAST, 2009.

Provenance: Future Work Using semantic interaction capturing, we might be able to collect all the analysis processes of expert analysts and create a knowledge-base that is useful for – Training: many domain specific analytics tasks are difficult to teach – Guidance: use existing knowledge to guide future analyses – Verification and validation: check for accuracy and correctness But our study was crude and made lots of assumptions… – How do we extract analysis from interaction logs semi- automatically? – Can these methods be generalized to all visualizations? – What does a knowledge-base of interactions look like and how to use it? – A model of how and what to capture in a visualization for extracting an analytical process is necessary.

Conclusion We do not yet have a perfect foundation on the “science of interaction”, but we are getting there. The three areas that I propose that would have the highest impact in interaction research are: – Fundamental (Functional) understanding of interaction and interaction techniques – Evaluation methods and metrics for measuring the benefits (and costs) of interactions – Capturing and re-using interactions to create a knowledge-base of analyst’s strategies and methods

Thank you!

Backup Slides

Results: What’s in a User’s Interactions Why are these so much lower than others? – (recovering “methods” at about 15%) Only capturing a user’s interaction in this case is insufficient.

Understanding Interaction as a Science With deeper understanding of

Scenario Revisited: Task: design a visual analytical system to analyze IP logs, starting the design with interaction elements. Solution: A system that displays different aspects of the IP data (e.g., dest IP, orig IP, time, port number, etc.) that are coordinated through Brushing and Linking. – Visual displays (integrated or coordinated multi-views) is a secondary consideration. – The visual representation of what IP logs could be can also be considered independently. – (Number of users and the applied environment)?

Taxonomy of Interaction Techniques: [Yi et al. 2007] – Select: mark something as interesting – Explore: show me something else – Reconfigure: show me a different arrangement – Encode: show me a different representation – Abstract/Elaborate: show me more or less detail – Filter: show me something conditionally – Connect: show me related items Seems all 7 elements are necessary, but that itself doesn’t lead to a design of a system…

Is Visual ANALYTICS possible? We propose that indeed it is…