LECTURE 13: ONGOING RESEARCH: THE ROLE OF INDIVIDUAL DIFFERENCES April 25, 2016 SDS136: Communicating with Data

Announcements Last week of classes! Final project reception: Wednesday 6-8pm in Ford Atrium A note on digital displays: - please bring a laptop (with an HDMI connection or a dongle) - plan to arrive a little early to test / set up

Outline Highlight “Bayesian Reasoning” paper from IEEE VIS 2015 Discussion: what does this mean for visualization? Final project workshop (time permitting)

Improving Bayesian Reasoning: the Effects of Phrasing, Visualization, and Spatial Ability Alvitta Ottley, Evan M. Peck, Lane T. Harrison, Daniel Afergan, Caroline Ziemkiewicz, Holly A. Taylor, Paul K. J. Han, and Remco Chang

Communicating Bayesian reasoning is important for medical decision-making.

The probability of breast cancer is 1% for women at age forty who participate in routine screening. If a woman has breast cancer, the probability is 80% that she will get a positive mammography. If a woman does not have breast cancer, the probability is 9.6% that she will also get a positive mammography. If a woman at age 40 is tested positive, what are her chances of actually having breast cancer?

The chance of actually having breast cancer given a positive mammogram: 7.8%

95 out of 100 doctors 1 estimate this probability to be: 80% 1 Eddy, David M. "Probabilistic reasoning in clinical medicine: Problems and opportunities." (1982).

Why is this so hard? People are not very good at reasoning about probabilities Especially when they have no statistical training And when the consequences of their decisions are serious 10

VIS Community

The Problem? They disagree. “ ” “ ” ” “...” “

Why? No consistent wording No consistent visual metaphor No consistent problem

So how do we fix it? 1.Need to understand how the wording of the problem impacts accuracy. 2.Need to understand how different reasoning aides impact accuracy.

Experiment 1 1.Need to understand how the wording of the problem impacts accuracy.

Back to the Mammography Problem

Three Conditions OriginalProbeDisease X

Three Conditions ProbeDisease X

Three Conditions OriginalDisease X

Three Conditions OriginalProbe

Experiment 1: Design 3 conditions 100 participants Between subjects experiment

Experiment 1: Findings Disease X

Experiment 1: Findings OriginalProbe <

Experiment 1: Findings OriginalProbeDisease X <≤

Experiment 2 1.Need to understand how the wording of the problem impacts accuracy. 2.Need to understand how different reasoning aides impact accuracy.

Experiment 2 1.Need to understand how the wording of the problem impacts accuracy. 2.Need to understand how different reasoning aides impact accuracy. Specifically: does adding visualization to the text help?

Three Initial Conditions Text Only (Control) Text + Visualization Only

Three Initial Conditions Text + Visualization Only +

Three Initial Conditions Text Only (Control) Text + Visualization Only +

Three Initial Conditions Text Only (Control) Visualization Only +

Why this visual representation?

Visualization Advantage 32

Bridging the gaps Text Only (Control) Text + Visualization Only Complete Text Structured Text Story- boarding

Bridging the gaps Text Only (Control) Text + Visualization Only Complete Text Structured Text Story- boarding There is a total of 100 people in the population. Out of the 100 people in the population, 6 people actually have the disease. Out of these 6 people, 4 will receive a positive test result and 2 will receive a negative test result. On the other hand, 94 people do not have the disease (i.e., they are perfectly healthy). Out of these 94 people, 16 will receive a positive test result and 78 will receive a negative test result. Another way to think about this is... Out of the 100 people in the population, 20 people will test positive. Out of these 20 people, 4 will actually have the disease and 16 will not have the disease (i.e., they are perfectly healthy). On the other hand, 80 people will test negative. Out of these 80 people, 2 will actually have the disease and 78 will not have the disease (i.e., they are perfectly healthy).

Bridging the gaps Text Only (Control) Text + Visualization Only Complete Text Structured Text Story- boarding There is a total of 100 people in the population. Out of the 100 people in the population, 6 people actually have the disease. Out of these: 4 will receive a positive test result and 2 will receive a negative test result. On the other hand, 94 people do not have the disease (i.e., they are perfectly healthy). Out of these: 16 will receive a positive test result and 78 will receive a negative test result. Another way to think about this is... Out of the 100 people in the population, 20 people will test positive. Out of these…

Bridging the gaps Text Only (Control) Text + Visualization Only Complete Text Structured Text Story- boarding

Experiment 2: Design 6 conditions 377 participants Between subjects experiment Also measured spatial ability

Experiment 2: Results

So what’s going on? Visualizations should make difficult concepts clear. So what now?

Experiment 2: Design 6 conditions 377 participants Between subjects experiment Also measured spatial ability

Experiment 2: Design 6 conditions 377 participants Between subjects experiment Also measured spatial ability Can spatial ability help us understand this phenomenon?

Separated by Spatial Ability 42 Low spatial-ability High spatial-ability

Low Spatial Ability Users 43 Low spatial-ability High spatial-ability

High Spatial Ability Users 44 Low spatial-ability High spatial-ability

Effective representations 45 Low spatial-ability High spatial-ability

Text + Visualization 46 Low spatial-ability High spatial-ability

Spatial Ability Matters 47 Low spatial-ability High spatial-ability

Take away We need to look beyond task and data. How the problem is phrased matters. The interaction between text and visualization matters. Who we are matters.

Improving Bayesian Reasoning: the Effects of Phrasing, Visualization, and Spatial Ability Alvitta Ottley, Evan M. Peck, Lane T. Harrison, Daniel Afergan, Caroline Ziemkiewicz, Holly A. Taylor, Paul K. J. Han, and Remco Chang

Discussion Do what does this mean for visualization design? What does it mean for us in this course?