CS 522: Human-Computer Interaction Research Methods in HCI Dr. Debaleena Chattopadhyay Department of Computer Science debchatt@uic.edu debaleena.com

Agenda Research methods in HCI Paper discussion

HCI HCI (roughly) = Computer Science + Human Factors + Design

What are the goals of human factors research? To discover principles about human beings that can improve the design of interfaces and workflows and how people engage with them, to enhance performance, health, and safety while reducing errors, accidents, distractions, inattention, and fatigue.

Basic vs. Applied research How do you distinguish basic research and applied research in human factors? The degree of application independence. Why do we need basic research? It is useful to identify principles that can be applied regardless of technological change.

Basic vs. Applied research (cont..)

Basic vs. Applied research (cont..) What is the relation between basic and applied research? Basic research provides the concepts and methodologies to solve specific issues. Applied research identifies specific issues that basic research needs to address.

What is Science? “The intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment.” “Science is the pursuit and application of knowledge and understanding of the natural and social world following a systematic methodology based on evidence.”

Foundations of Science What is the main principle on which science is based? Empiricism

What is empiricism? Verification by observation Scientific statements “are restricted to observable events and the inferences that can be drawn from them.” Science is self-correcting.

Scientific Method Observe Phenomenon Statement of Problem Development of Hypotheses Conduct Experiment Disseminate Results Evaluate Hypothesis

Why do empirical HCI research? How do we know what computer systems are easy, quick and productive for humans to use? We can theorize about human behavior based on existing studies in other domains, such as human factors, psychology, or physiology. But to ensure that our design guidelines are valid, reliable, and generalizable, we need systematic empirical studies that can be replicated.

Type of Empirical Studies in HCI Quantitative, such as A/B testing, between-subject studies, within-subject studies, or randomized clinical trials (RCT). Qualitative, such as grounded theory, ethnography, phenomenology, contextual inquiry, or content analysis. Quantitative—more prevalent in US, while Qualitative studies are more prominently used in Scandinavian countries and Europe.

Four HCI studies Touchless Circular Menus Explaining the Uncanny Valley Understanding advice sharing among physicians Office Social

Touchless Circular Menus Touchless circular menus relieve users from both recalling a precise vocabulary of hand-postures and strictly complying with them.

Linear menus using a grab gesture Evaluation 2 Linear menus using a grab gesture Touchless circular menus Vs. How effectiveness and efficiency of TCM is affected by their triggering locations on the visual interface? Participants (N = 15) were sitting away from a large display. 1 Experimental Control

Evaluation (cont.) 1 2 Sample Demographics Experiment Experiment 9 trials X 7 blocks X 15 participants = 945 trials Experiment 2 6 trials X 7 blocks X 15 participants = 630 trials Sample Demographics N = 15 All right-handed 4/15 females 8/15 had prior familiarity with touchless gestures 11/15 were below 30 years of age, Time/participant: 1 hour Efficiency: Time on task | Successful Trigger Rate Effectiveness: Error Rate User Satisfaction: SUS | NASA TLX Chattopadhyay, D., & Bolchini, D. (2014, May). Touchless circular menus: toward an intuitive UI for touchless interactions with large displays. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces (pp. 33-40). ACM.

Understanding Advice Sharing How to improve the design of Drug-Drug Interaction Alerts (DDI Alerts) to improve physician’s trust and adherence? DDI is a type of computerized clinical decision support.

Study Before looking at how to improve alerts, we looked at how to improve the trust between physicians and computerized advice. A starting point to address this issue is looking at whom physicians do trust: their medical colleagues and mentors. In 3 contextual inquires, we examined why clinical advice is trusted among physicians. 255 minutes – 22 health care professionals during three inpatient team meetings

Example of a work model (flow model)

The Emerging Themes Informed by our consolidated work models, we identified eight themes driving trusted advice among physicians in clinical settings Specialization Role in the Medical Hierarchy Demonstrated Experience Evidence of Understanding the Patient’s Situation Empathy Demonstrated Knowledge of Evidence from the Literature Collaborative and Inclusive Language Timeliness of the Advice

Validating Themes with Survey To validate with a larger sample of physicians the crucial themes emerging from our formative study, we designed and administered an online survey. Demographics: Of the 87 questionnaires sent, 37 were returned (22 females). Respondents were mostly less than 30 (17) or less than 40 (11) years old, and were mostly either resident (19) or attending physicians (17).

Connecting themes to survey

Mixed-method approach: Types of data analysis Chattopadhyay, D., Ghahari, R. R., Duke, J., & Bolchini, D. (2015). Understanding Advice Sharing among Physicians: Towards Trust-Based Clinical Alerts. Interacting with Computers, iwv030.

Explaining the Uncanny Valley Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley

Theory  Hypothesis  Data collection and analysis H1. Realism inconsistency makes anthropomorphic entities look unfamiliar. Operationalization: What is realism inconsistency? How do we measure it? What is familiarity? How do we measure it? Independent and dependent variables Research design

The experiment followed a within-group design The experiment followed a within-group design. The experiment consists of a two-alternative forced choice categorization task and a survey in which the task stimuli were rated. Category – Real or Computer-animated image IV: Images and different variations of each image DV: Reaction time, Ratings of familiarity.

Quantitative study Modeling where F[S] represents affinity for stimulus S, p(S) the probability of its occurrence, k the viewer's sensitivity to perceptual tension, and V[S] the perceptual tension.  Chattopadhyay, D., & MacDorman, K. F. (2016). Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley. Journal of Vision, 16(11), 7-7.

Office Social Designing Presentation Interactivity for Nearby Devices https://www.youtube.com/watch?v=JUyRDf9EMhE

Evaluation Longitudinal study with the same group of individuals. 4 observation sessions (video recording) average 9 users/session; longitudinal over 4 weeks 1 focus group 4 interviews (2 semi-structured + 2 stimulated recall) Chattopadhyay, D., O'Hara, K., Rintel, S., & Rädle, R. (2016, May). Office Social: Presentation Interactivity for Nearby Devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 2487-2491). ACM.

Paper discussion Oulasvirta, A., & Hornbæk, K. (2016). HCI Research as Problem-Solving. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 4956—4967, ACM. 

The Science in HCI Kostakos, V. (2015). The big hole in HCI research. Interactions, 22(2), 48-51. http://interactions.acm.org/archive/view/march-april-2015/the-big-hole-in-hci-research-insights

To do: Project proposal – DUE Today Readings: Shneiderman et al., 2017. Chapter 7 HW 2 – DUE next week HW 1 grades will be out by the end of this week (hopefully)