1. CS 522: Human-Computer Interaction Research Methods in HCI
Dr. Debaleena Chattopadhyay
Department of Computer Science
debaleena.com

2. Agenda
Research methods in HCI
Paper discussion

3. HCI
HCI (roughly) =
Computer Science
+ Human Factors
+ Design

4. What are the goals of human factors research?
To discover principles about human beings that can improve the design of interfaces and workflows and how people engage with them, to enhance performance, health, and safety while reducing errors, accidents, distractions, inattention, and fatigue.

5. Basic vs. Applied research
How do you distinguish basic research and applied research in human factors?
The degree of application independence.
Why do we need basic research?
It is useful to identify principles that can be applied regardless of technological change.

6. Basic vs. Applied research (cont..)

7. Basic vs. Applied research (cont..)
What is the relation between basic and applied research?
Basic research provides the concepts and methodologies to solve specific issues.
Applied research identifies specific issues that basic research needs to address.

8. What is Science?
“The intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment.” “Science is the pursuit and application of knowledge and understanding of the natural and social world following a systematic methodology based on evidence.”

9. Foundations of Science
What is the main principle on which science is based?
Empiricism

10. What is empiricism? Verification by observation
Scientific statements “are restricted to observable events and the inferences that can be drawn from them.”
Science is self-correcting.

11. Scientific Method Observe Phenomenon Statement of Problem Development
of Hypotheses
Conduct
Experiment
Disseminate
Results
Evaluate
Hypothesis

12. Why do empirical HCI research?
How do we know what computer systems are easy, quick and productive for humans to use? We can theorize about human behavior based on existing studies in other domains, such as human factors, psychology, or physiology. But to ensure that our design guidelines are valid, reliable, and generalizable, we need systematic empirical studies that can be replicated.

13. Type of Empirical Studies in HCI
Quantitative, such as A/B testing, between-subject studies, within-subject studies, or randomized clinical trials (RCT).
Qualitative, such as grounded theory, ethnography, phenomenology, contextual inquiry, or content analysis.
Quantitative—more prevalent in US, while Qualitative studies are more prominently used in Scandinavian countries and Europe.

14. Four HCI studies Touchless Circular Menus
Explaining the Uncanny Valley
Understanding advice sharing among physicians
Office Social

15. Touchless Circular Menus
Touchless circular menus relieve users from both recalling a precise vocabulary of hand-postures and strictly complying with them.

16. Linear menus using a grab gesture
Evaluation 2
Linear menus using a grab gesture
Touchless circular menus
Vs.
How effectiveness and efficiency of TCM is affected by their triggering locations on the visual interface?
Participants (N = 15) were sitting away from a large display. 1
Experimental
Control

17. Evaluation (cont.) 1 2 Sample Demographics Experiment Experiment
9 trials
X 7 blocks
X 15 participants
= 945 trials
Experiment 2
6 trials
X 7 blocks
X 15 participants
= 630 trials
Sample Demographics
N = 15
All right-handed
4/15 females
8/15 had prior familiarity with touchless gestures
11/15 were below 30 years of age, Time/participant: 1 hour
Efficiency:
Time on task | Successful Trigger Rate
Effectiveness:
Error Rate
User Satisfaction:
SUS | NASA TLX
Chattopadhyay, D., & Bolchini, D. (2014, May). Touchless circular menus: toward an intuitive UI for touchless interactions with large displays. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces (pp ). ACM.

18. Understanding Advice Sharing
How to improve the design of Drug-Drug Interaction Alerts (DDI Alerts) to improve physician’s trust and adherence?
DDI is a type of computerized clinical decision support.

19. Study
Before looking at how to improve alerts, we looked at how to improve the trust between physicians and computerized advice.
A starting point to address this issue is looking at whom physicians do trust: their medical colleagues and mentors.
In 3 contextual inquires, we examined why clinical advice is trusted among physicians.
255 minutes – 22 health care professionals during three inpatient team meetings

20. Example of a
work model (flow model)

21. The Emerging Themes
Informed by our consolidated work models, we identified eight themes driving trusted advice among physicians in clinical settings
Specialization
Role in the Medical Hierarchy
Demonstrated Experience
Evidence of Understanding the Patient’s Situation
Empathy
Demonstrated Knowledge of Evidence from the Literature
Collaborative and Inclusive Language
Timeliness of the Advice

22. Validating Themes with Survey
To validate with a larger sample of physicians the crucial themes emerging from our formative study, we designed and administered an online survey.
Demographics: Of the 87 questionnaires sent, 37 were returned (22 females). Respondents were mostly less than 30 (17) or less than 40 (11) years old, and were mostly either resident (19) or attending physicians (17).

23. Connecting themes
to survey

24. Mixed-method approach: Types of data analysis
Chattopadhyay, D., Ghahari, R. R., Duke, J., & Bolchini, D. (2015). Understanding Advice Sharing among Physicians: Towards Trust-Based Clinical Alerts. Interacting with Computers, iwv030.

25. Explaining the Uncanny Valley
Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley

26. Theory  Hypothesis  Data collection and analysis
H1. Realism inconsistency makes anthropomorphic entities look unfamiliar.
Operationalization: What is realism inconsistency? How do we measure it? What is familiarity? How do we measure it?
Independent and dependent variables
Research design

27. The experiment followed a within-group design
The experiment followed a within-group design. The experiment consists of a two-alternative forced choice categorization task and a survey in which the task stimuli were rated.
Category – Real or Computer-animated image
IV: Images and different variations of each image
DV: Reaction time, Ratings of familiarity.

29. Quantitative study Modeling
where F[S] represents affinity for stimulus S, p(S) the probability of its occurrence, k the viewer's sensitivity to perceptual tension, and V[S] the perceptual tension. 
Chattopadhyay, D., & MacDorman, K. F. (2016). Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley. Journal of Vision, 16(11), 7-7.

30. Office Social Designing Presentation Interactivity for Nearby Devices

31. Evaluation Longitudinal study with the same group of individuals.
4 observation sessions (video recording)
average 9 users/session;
longitudinal over 4 weeks
1 focus group
4 interviews
(2 semi-structured + 2 stimulated recall)
Chattopadhyay, D., O'Hara, K., Rintel, S., & Rädle, R. (2016, May). Office Social: Presentation Interactivity for Nearby Devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp ). ACM.

32. Paper discussion
Oulasvirta, A., & Hornbæk, K. (2016). HCI Research as Problem-Solving. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 4956—4967, ACM. 

33. The Science in HCI
Kostakos, V. (2015). The big hole in HCI research. Interactions, 22(2),

34. To do: Project proposal – DUE Today
Readings: Shneiderman et al., Chapter 7
HW 2 – DUE next week
HW 1 grades will be out by the end of this week (hopefully)