Human-Computer Interaction: User Study Examples CSCE 315 – Programming Studio, Fall 2017 Project 3, Lecture 3

Review: The User Study Questions to Address How to get participants? Where to gather data? What data to gather? How to gather that data? How to analyze that data?

How to Gather Data? Data Gathering Processes Interview Pre-questionnaire Interactive System Use Observation! Recording: photos, audio, video Logging Answers found/products produced Think-aloud Post-questionnaire Some material from Kerne’s slides

First Click Testing Determines what a user would click first on the interface Use to determine how well an interface lets someone navigate through it Why important? Clicking the first right link leads to success 87% of the time (compared to only 46% on a wrong first click) Give the user the task (e.g., set up auto bill pay from chase.com) Track each click (could be manual or automated) Record time taken to make click Can gather subjective information on how easy the user was able to do the task Usability.gov

System Usability Scale 5-point Likert scale (strongly disagree to strongly agree) 10 questions 5 are “positive”,5 are “negative” Given to users after they’ve used the system, before any debriefing From 1986, has become an industry standard Each question gives score 0 to 4. For the negative questions, reverse order. Sum them up, multiply by 2.5 and you have a scale from 0 to 100, with 100 being best A “quick-and-dirty” approach to usability testing Not ideal, but widely used/accepted, so easier to compare Just gives a score on usability – no diagnosis for how to improve That’s why, we wont use it in out Project 3 to improve UX

System Usability Scale Questions I think that I would like to use this system frequently. I found the system unnecessarily complex. I thought the system was easy to use. I think that I would need the support of a technical person to be able to use this system. I found the various functions in this system were well integrated. I thought there was too much inconsistency in this system. I would imagine that most people would learn to use this system very quickly. I found the system very cumbersome to use. I felt very confident using the system. I needed to learn a lot of things before I could get going with this system.

Test by Experts Heuristic Evaluation Expert Reviews Usability Experts review your UI and compare it against accepted usability principles Nielsen’s Heuristics are most commonly used set of heuristics Pros: Relatively inexpensive, can be applied early in the process Cons: Requires knowledge and experience that are hard to get Expert Reviews A less formal version of Heuristic Evaluation Requires the same level of expertise, but a different view

Interview Contextual Interview User interview Observer watches/listens as user works in user’s environment instead of the lab Collects observations, typically does not include measurements Observation is followed/interspersed with interview You find out such things as: Task duration, Internet speed at site, preference between keyboard and mouse usage etc. User interview Interview is more in-depth discussion (30-60 minutes) Often record interview for later review Not directly in the context of using the product

Interviews Unstructured - are not directed by a script. Rich but not replicable. Structured - are tightly scripted, often like a questionnaire. Replicable but may lack richness. Semi-structured - guided by a script but interesting issues can be explored in more depth. Can provide a good balance between richness and replicability. Focus groups – a group interview www.id-book.com

Interview questions Two types: Closed questions are easier to analyze ‘closed questions’ have a predetermined answer format, e.g.. ‘yes’ or ‘no’ ‘open questions’ do not have a predetermined format Closed questions are easier to analyze Avoid: Long questions Compound sentences - split them into two Jargon and language that the interviewee may not understand Leading questions that make assumptions e.g.. why do you like …? Unconscious biases e.g.. gender stereotypes www.id-book.com

Focus Group More efficient than one-on-one interview in terms of getting qualitative data from many people Less efficient, but allows deeper exploration than a survey Capture information (recording or notes) Moderator has to be able to progress through questions and handle group dynamics Poor moderation can lead to people not being forthright, getting into group-think, getting drowned out

Online surveys Low cost Broad audience Identify early on: The purpose of the survey Where you will get respondents Software to use Who will analyze data, and now Questions should be created to measure or get feedback on what you intend Can mix open-ended and closed questions Allows mix of quantitative and qualitative evaluation

Task Analysis Early stage user study Find how users go about some task Observe users in action to determine goals/scope of a project Find how users go about some task What their goals are, what they try to achieve What they actually do to achieve the goals What experiences (personal, social, cultural) they bring to the tasks How users are influenced by physical environment How users’ previous knowledge and experience influence: Their thought process Their workflow

Eye Tracking Use an eye tracker to determine where people are looking on a screen during a task Can measure several items: Where they are looking How long they are looking Focus/order of examination Where they don’t look Can infer where people are reading/studying vs. glancing Focus does not mean comprehension

Eye Tracking (2) Heat Maps Saccade Pathways

References Some slide material taken from information at: https://www.usability.gov/how-to-and- tools/methods/index.html Some qualitative slides taken from Andruid Kerne’s slides: Noted in footers Material is copyrighted Can be used and cited freely for non-commercial education and scholarship without profit provided that credit is given to the author and this notice is included Some topics taken from: Interaction Design, 4th edition, by Rogers, Sharp, and Preece, Wiley, 2014. www.id-book.com See chapters 7-8 in particular www.id-book.com