1. Asking Users and Experts Bobby Kotzev Adrian Sugandhi

2. Outline Asking Users Interviews Questioners Asking Experts Inspections Walkthroughs

3. Interviews, Survey, or Focus Group? Interview Take significant time and need more resources Survey Large data set Focus group Seeking multiple point of views in a shorter period of time

4. Preparing for Interviews: Things to be aware Be alert to unconscious biases Honesty Social desirability Prestige response bias Sensitive or highly personal topics – survey? Consider the interviewee as he/she is “Doing you a favor”

5. Preparing Interviews: Outcome Analyses Outcome Analyses Plan outcome-based interviews Associate questions with the goals Capture desired outcomes Differentiate between outcome and solution Organize the outcomes Rate the outcomes for importance and satisfactions Importance + (Importance-Satisfaction) = Opportunity Use the outcomes to jump-start innovation

6. Preparing Interviews: Pre-interview Identify the objective of the study Select type of interviews Medium of interview (in person/phone) Decide how you will analyze the data Write the questions Avoid long questions (Brevity) Avoid compound sentences Avoid using jargon Avoid leading questions Avoid Biases Clarity Avoid future prediction questions Avoid inaccessible topics Test your questions

7. Preparing Interviews: Players in the Activity Participants 6-10 of each user type (diversity) Interviewers Ensure participants understand the questions Must be skilled The note-taker Interviewers can focus more on body language The videographer Whenever possible, record the interview session

8. Conducting Interviews: Sections Introduction 5-10 Minutes Warm-up 5-10 Minutes Main section 85-100 Minutes Cool off period 5 Minutes Closing

9. Preparing for Interviews: Additional Preparations Run pilot study Be professional Dress similarly and appropriately Prepare informed consent Check and familiarize yourself with recording equipment Record answers exactly

10. Interviews Types Interview Types Open-ended Unstructured Structured Semi-structured

11. Unstructured Interviews A conversation that focus on a particular topic Open questions Can generate rich data which the interviewer haven’t thought about Generates a lot of unstructured data Impossible to replicate Hard to analyze

12. Unstructured Interviews: Prepare Have an agenda Be prepared to follow new lines Pay attention to ethical details Respond with sympathy but make sure to avoid putting ideas in the user’s head Analyze data as soon as possible after the interview

13. Structured Interviews Predetermined questions, similar to a questionnaire.

14. Semi-Structured Interviews Starts as a structured interview but can change direction and inject open-ended questions depending on responses Avoid preempting answers Probes – “Do you want to tell me anything else about…”

15. Interviewer’s Role Do not interrupt Keep on track Unstructured interview easy goes off-track Silence is golden “permission” to provide more detail Remain attentive Asking the tough questions Wait until you develop rapport Using examples Watch for generalities Do not force choices

16. Interviewer’s Role (cont’d) Watch for markers Key events to probe for more rich information Select the right type of probes Know when to move on Reflecting Summarize, reword, or reflect responses Empathy and antagonism Transitions Transition smoothly from one topic to another

17. Monitoring the Relationships with the Interviewee/Participant Watch participant’s body language Nervous, tenses – go to easier question or restate purpose/motivation of the study Fighting for control Ask yourself why one refusing your questions Hold your opinions Dos and Don’ts

18. Interview: Data Analysis and Interpretation Must be analyzed shortly after each interview Categorizing Affinity diagram Qualitative analysis tools Look for patterns

19. Interview: Communicate the Findings Over time By topic By participant Vehicles for communicating the results Summarized poster Identify follow-up activities based on the results Table of recommendations (uncovered issues and next steps)

20. Questionnaires Good, established technique for collecting demographic data and users opinion Can be Closed and Open. Can be distributed to large number of people

21. Questionnaires: Designing Start with basic demographic data Make questions clear and specific When possible ask closed questions and offer a range of answers Include no-opinion option Ordering matters Avoid complex and compound questions Use intuitive, consistent scaling Avoid jargons Provide clear instructions how to complete the questioner

22. Questionnaires: Question Types Checkboxes Likert Scales Semantic differential scales Gender:Male  Female 

23. Questionnaires: Administering Well designed Include stamped self addressed envelope Provide a short version Contact and respondents trough a follow-up, mail, e-mail phone Offer incentives like payments

24. Questionnaires: Online Forms Pros: Reach large audience Response is fast No postage costs Data is already in a electronic format and can be imported easily in DB Errors can be corrected easily and fast Cons: Obtaining random sample is difficult – web users represent only a certain demographic. Participants are self selecting – proclaimed nonscientific

25. Questionnaires: Online Forms (cont’) Construct Online questioners based on their paper forms Run pilot studies

26. Asking Experts Experts will be power users but also could be involved with the development of similar, successful products so they have an deeper understanding.

27. Asking Experts: Inspections Heuristic evaluations evaluate user-interface against a predetermined guidelines and principles Visibility of system status Are users kept informed about what is going on? Is appropriate feedback provided within reasonable time about a user action? User Control and freedom Are there ways of allowing users to easily escape from places they unexpectedly find themselves in? Consistency and standards Are the ways of performing similar actions consistent?

28. Asking Experts: Inspections (cont’) Different heuristics are needed for different products, devices and software Using heuristics and experts can reveal issues fast and inexpensively – 5 experts can reveal 75% of issues - graph Performing heuristic evaluation Spend 1-2 hours with the product. At least two passes – one to get the feel, second to walk trough the interface and address all aspects of usability. Have a specific task in mind when evaluating functional products

29. Asking Experts: Inspections (cont’) Heuristic evaluation of websites Internal consistency Simple dialog Shortcuts Minimizing the user memory load Preventing errors Feedback Internal locus of control Layout Internal consistency

30. Asking Experts: Problems Reporting issues which are not there – (Bill Bailey 2001)

31. Asking Experts: Walkthroughs Walk through a task and to notice problems “Cognitive walkthroughs” – simulating the user problem solving process for a task Figure if the users will know what to do, how to do it and weather the action was correct or not. Record assumptions, side issues and summaries of results Pluralistic walkthroughs Users developers and usability experts get together to step through a scenario

32. Summary Interviews Fresh input Questionnaires Reach the masses Experts Easy way to discover most problems