Exploiting the Cognitive and Social Benefits of Physically Large Displays Desney S. Tan Thesis Proposal Thesis Committee: Randy Pausch (Chair) Jessica Hodgins Scott Hudson Mary Czerwinski, Microsoft Research

Desney S. Tan, Thesis Proposal 2 Towards Human-Centered Computing Human-Computer Interaction Software Mechanical Issues Cognitive Issues Social Issues Hardware Software Mechanical Issues Physically Large Displays

Desney S. Tan, Thesis Proposal 3 Ignore Science Fiction at Our Peril Captain’s workplace on USS Enterprise

Desney S. Tan, Thesis Proposal 4 Large Displays are Here Gary Bishop’s workplace at UNC

Desney S. Tan, Thesis Proposal 5 Small vs. Large Displays

Desney S. Tan, Thesis Proposal 6 Thesis Statement “Information elicits fundamentally different cognitive and social reactions when presented on large wall- sized displays as compared to smaller displays, even at identical visual angles. These reactions can be quantified and understood in controlled experiments and can be exploited to design display systems that make users more productive than they were on traditional systems.”

Desney S. Tan, Thesis Proposal 7 Outline Introduction General approach Spatial cognition Social environments More proposed work Expected contributions Proposed schedule Conclude

Desney S. Tan, Thesis Proposal 8 General Approach Controlled Experiments/ Evaluation Formulate Design Principles Prototype Systems Identify Promising Areas Anecdotal Evidence Prior Theoretical Research HCI Iterative Design

Desney S. Tan, Thesis Proposal 9 Outline Introduction General approach Spatial cognition Social environments More proposed work Expected contributions Proposed schedule Conclude

Desney S. Tan, Thesis Proposal 10 Experimental Setup 19" Desktop Monitor Large Projection Display 14 " 76 " 136 " 25 "

Desney S. Tan, Thesis Proposal 11 Experimental Setup (actual) Small DisplayLarge Display

Desney S. Tan, Thesis Proposal 12 Initial Exploratory Experiments Reading task: No performance difference Spatial task: Large display 26% improvement

Desney S. Tan, Thesis Proposal 13 Guilford-Zimmerman Spatial Task

Desney S. Tan, Thesis Proposal 14 Spatial Task Procedure Paper-based instructions 3 practice questions –Feedback provided 30 questions in each display condition –5 minutes for each condition –No feedback

Desney S. Tan, Thesis Proposal 15 Spatial Task Performance Large Display Proportion Correct Small Display N =24

Desney S. Tan, Thesis Proposal 16 Follow-up Experiments: Hypotheses 1.Egocentric strategy more efficient than exocentric one for spatial orientation task –Telling users is sufficient to bias them 2.With no explicit strategy, display size biases users differently: –Small displays → exocentric strategy –Large displays → egocentric strategy

Desney S. Tan, Thesis Proposal 17 Three Different Instruction Sets Unbiased (original Guilford-Zimmerman): –“The boat is moving.” Egocentric bias: –“Imagine standing on a boat that is moving.” Exocentric bias: –“Imagine standing on a boat that is firmly attached to the ground. A painted image is moving in front of you.”

Desney S. Tan, Thesis Proposal 18 Experimental Procedure ExocentricUnbiased (Experiment 1) Egocentric Small Large Display (within Ss) Instruction Set (between Ss) Design: 42 participants Same setup and procedure as before

Desney S. Tan, Thesis Proposal 19 Spatial Results Exocentric Instructions Unbiased Instructions Egocentric Instructions Proportion Correct Different No Difference Small Display Large Display Small Display Large Display Small Display Large Display

Desney S. Tan, Thesis Proposal 20 Spatial Cognition: Proposed Work Examine interaction: –Will interaction create egocentric biases that negate the effects of screen size? Generalize: –For what tasks does this hold? Specify concrete principles for designing large display interfaces and systems

Desney S. Tan, Thesis Proposal 21 Outline Introduction General approach Spatial cognition Social environments More proposed work Expected contributions Proposed schedule Conclude

Desney S. Tan, Thesis Proposal 22 Social Environments Exploit social cues induced by physical size: –Help people communicate –Increase productivity on individual tasks Must quantify in order to exploit Information on large displays more public –Ask user? Cannot guarantee accuracy –Video? Cannot disambiguate glance from reading

Desney S. Tan, Thesis Proposal 23 Measuring ‘Peeking’ Implicit memory priming paradigm –Expose user to stimulus –Test user implicitly on how much they’ve processed stimulus Word Stem Completion Eg. Mon_____ –Priming measured by faster response or higher frequency of stimulus Monkey, money, monster, monday, monopoly, …

Desney S. Tan, Thesis Proposal 24 Experiment Materials Stimulus: 30 carefully chosen words –7 subject lines –2 messages Place where it can be seen by user Priming test to see if they’ve read it –Word stem completion

Desney S. Tan, Thesis Proposal 25 Experimental Setup 156″ 38″ 16″ 27.5″ 66″114″ Experimenter Participant Large Projection Screen Small Desktop Monitor

Desney S. Tan, Thesis Proposal 26 Implicit Memory Results Small DisplayLarge Display Average # of Target Words N =12

Desney S. Tan, Thesis Proposal 27 Other Converging Measures Users admitted reading text on: –Large Screen (7/12) vs. Small Screen (3/12) Comments indicate reading someone else’s more acceptable on large screen Video shows users glanced at: –Large Screen (M=19 seconds) vs. Small Screen (M=14 seconds)

Desney S. Tan, Thesis Proposal 28 Social Environments: Summary Implicit memory priming paradigms effective for measuring if someone has peeked at text People are more willing to read someone else’s on large wall-sized displays than on smaller displays –Even with identical visual angles and legibility

Desney S. Tan, Thesis Proposal 29 Social Environments: Proposed Work Understand why large displays more public –Physical proximity of information to owner? –Wall-mounted nature of large display? Protect private information from prying eyes –Private information never placed on public screens –Interface conventions that convey level of privacy Facilitate ad hoc collaboration –Display systems that make people interact more

Desney S. Tan, Thesis Proposal 30 Outline Introduction General approach Spatial cognition Social environments More proposed work Expected contributions Proposed schedule Conclude

Desney S. Tan, Thesis Proposal 31 More Proposed Work Defining large displays: –Do users rely more on environmental or optical cues in perceiving physical size? –What is the optimal physical size? Subjective responses: –Are there specific tasks that users prefer performing on large displays? –Are users more motivated by large displays?

Desney S. Tan, Thesis Proposal 32 Outline Introduction General approach Spatial cognition Social environments More proposed work Expected contributions Proposed schedule Conclude

Desney S. Tan, Thesis Proposal 33 Expected Contributions Fundamental understanding of physical display size on information perception and task performance New paradigms to measure display benefits Applications that large displays benefit Principles for designing large display systems

Desney S. Tan, Thesis Proposal 34 Proposed Schedule Completion: May FebruaryOctoberJuly Present: March Microsoft Research Internship Social Environments Thesis writing Subjective Response Spatial Cognition Defining Large Displays

Desney S. Tan, Thesis Proposal 35 Conclusions New Approach –Physical size of displays, even viewed at identical visual angles, affects performance –Formulate design principles for building large display systems Initial Results –Spatial cognition: Large displays bias users into more efficient egocentric strategies –Social environments: Large displays are more public

Desney S. Tan, Thesis Proposal 36 Conclusions Proposed Work –Extend results and find concrete applications in: Spatial cognition Social environments –Define what makes large displays different –Understand subjective responses to large displays

Desney S. Tan, Thesis Proposal 37 Acknowledgments

Desney S. Tan, Thesis Proposal 38 The End… Or has it just Begun?

Exploiting the Cognitive and Social Benefits of Physically Large Displays Desney S. Tan Thesis Proposal Thesis Committee: Randy Pausch (Chair) Jessica Hodgins Scott Hudson Mary Czerwinski, Microsoft Research

Desney S. Tan, Thesis Proposal 40 Visual Perception Retinal image Physical object Perceived image

Desney S. Tan, Thesis Proposal 41 Display Garden

Desney S. Tan, Thesis Proposal 42 Small vs. Large Displays (schematic) 76" 10.5" 136" 14" 57" 48" 25" Large Projection Display Desktop Monitor

Desney S. Tan, Thesis Proposal 43 Reading Task 1 practice + 6 test GRE passages 24 participants No significant differences in reading speed or comprehension 10 pt14 pt18 pt Small Large Display (within Ss) Font Size (within Ss)

Desney S. Tan, Thesis Proposal 44 Harder Guilford-Zimmerman Question

Desney S. Tan, Thesis Proposal 45 Spatial Updating: New Paradigm Navigate through virtual ‘maze’ –Spatially update mental map At the end, point at direction of objects Start End

Desney S. Tan, Thesis Proposal 46 Spatial Updating: Design Hypothesis: Physically large displays provide cues that allow for more efficient egocentric navigation even when the user has active control of a first person viewpoint PassiveActive Small Large Display (within Ss) Control (within Ss)

Desney S. Tan, Thesis Proposal 47 Spatial Updating: Prior Findings Egocentric strategy faster and more accurate than exocentric strategy The more cues present, the better –Proprioceptive cues better than visual cues or imagination –Active control better than passive viewing

Desney S. Tan, Thesis Proposal 48 Notification Study: Setup 103″ 25″ Large Projection Display 66″ Small Desktop Monitors 16″

Desney S. Tan, Thesis Proposal 49 Notification Study: Conditions (a) Monitor: Near-within(b) Monitor: Near-across(c) Monitor: Far- across (d) Projector: Near- within (e) Projector: Near-across(f) Projector: Far- across

Desney S. Tan, Thesis Proposal 50 Notification Study: Tasks Primary task – Proofreading –Identify grammatical errors within text article Secondary task – Notification Detection –Detect visual changes outside the focal region Tertiary task – Text Comparison –Determine whether two sets of text were identical

Desney S. Tan, Thesis Proposal 51 Notification Study: Results No effect of physical discontinuities (monitor bezel or depth difference) Significant, though small (~10%) performance difference when tasks further apart (55 vs. 27 degrees), but only when further separated in depth

Desney S. Tan, Thesis Proposal 52 Ch 3: Defining Large Displays Hypothesis 3a –Users rely on environmental cues more than optical cues in perceiving physical size Hypothesis 3b –The physical size of a display is optimal when it presents life-sized images

Desney S. Tan, Thesis Proposal 53 Ch 4: Spatial Cognition Hypothesis 4a –Large displays better bias users into egocentric representations than small displays, even when users are interacting with content Hypothesis 4b –Large displays improve performance on 3D navigation tasks Hypothesis 4c –Egocentric representations afforded by large displays transfer more readily to the real-world and are more efficient for training tasks

Desney S. Tan, Thesis Proposal 54 Ch 5: Social Environments Hypothesis 5a –People treat information on large displays as being public and are more likely to interact with it Hypothesis 5b –People working on displays that are publicly visible are more likely to stay on task and be more productive Hypothesis 5c –Systems that are aware of their physical displays as well as their social impact can automatically lay information out based on social context

Desney S. Tan, Thesis Proposal 55 Attention and Human Memory Hypothesis 5d –Large displays arouse users more, better capturing their attention as well as allowing them to remember more information presented on the displays

Desney S. Tan, Thesis Proposal 56 Ch 6: Subjective Responses Hypothesis 6a –Users prefer performing tasks requiring overviews of large areas on large displays; Users prefer performing tasks requiring detailed work on small displays Hypothesis 6b –Even with user interaction, there exists a (possibly non-linear) function that describes viewing distance preferences as they relate to the size of displays Hypothesis 6c –Users find large displays more engaging and are more motivated to perform tasks on them

Desney S. Tan, Thesis Proposal 57 Size-Distance Invariance Hypothesis Given a retinal image, the ratio of perceived size (s) to perceived distance (d) is constant S1 S2 d1 d2 α tan α = s/d

Desney S. Tan, Thesis Proposal 58 Ames Room Illusion Actual Person A Apparent Person A Actual + Apparent Person B Viewing Peephole Apparent Shape of Room

Desney S. Tan, Thesis Proposal 59 Risk Analysis Few significantly interesting effects –But: Prior work and anecdotal evidence indicates there are –But: Already shown non-intuitive benefits –But: Work with Stage 3 (CMU), Large Display User Experience (MSR), Proffitt Psychology Lab (UVA) Unable to reproduce and measure effects –But: Already shown this is possible –But: Work with Mary Czerwinski, Dennis Proffitt Create overly contrived studies, no real application –But: Work with Large Display User Experience (MSR)

Desney S. Tan, Thesis Proposal 60 Interesting Work Beyond Scope Interaction of physical size with: –Field of view –Fidelity/Resolution –Information content System with fluid notion of displays Interaction and visualization techniques Brain imaging to support behavioral results –fMRI, OTIS, …

Women Go With the (Optical) Flow Desney S. Tan, Mary Czerwinski, George Robertson 9:30am, Wednesday April 9 th, 2003 Information Voyeurism: Social Impact of Physically Large Displays on Information Privacy Desney S. Tan, Mary Czerwinski 5:30pm, Tuesday April 8 th, 2003 With Similar Visual Angles, Larger Displays Improve Spatial Performance Desney S. Tan, Darren Gergle, Peter Scupelli, Randy Pausch 10:00am, Wednesday April 9 th, 2003

Desney S. Tan, Thesis Proposal 62 That’s brilliant! I’m glad you’re on my committee…