User Studies Basic principles, methods, and examples 30.10.2001 Sari Kujala Qure Project Software Business and Engineering Institute (SoberIT)

Contents User studies: goal and principles User study process: methods and templates Analysing stakeholders Planning Studying users Analysing and reporting Presenting results Effort estimations, lessons learned

User studies The goal is to gather USER NEEDS for product development User need is manifested by the context of use, problems, and possibilities Context of use means the characteristics of the intended users, users' present tasks, tools and environment Understanding users’ goals is essential

Principles of user studies Plan the focus and objectives for the visit Watch, listen to, and talk with users in their own environment Treat users as experts Take the role of the apprentice. Your task is to confess your ignorance and ask stupid questions.

User studies: What to look for? Background information User’s goals and preferences Current processes and problems User’s knowledge, skills, & experiences Context of using Pros and cons of the current tools and processes

User study process Analysing stakeholders Planning Seeing users Analysing and reporting Presenting results

Analysing stakeholders Utilising existing documents Identifying stakeholders Identifying user groups Preliminary user description

Planning (Hackos and Redish, 1998) Decide issues and objectives for the study Select user study methods Plan how data will be analysed and reported Choose participating users Choose locations where to go Form a user study team and decide roles Plan how the user visits will be recorded Prepare materials for user visits Plan a pleminary schedule

User study methods Observing and video-observing Interviewing Contextual Inquiry Artifact walkthrough Thinking-aloud method Simulations, role-play Interactive feature conceptualization

Selecting user study methods

Observing Users are observed in their own environment performing their tasks Focus on gathering non-verbal information: routinised actions, detailed behaviour, environment Can be time consuming, if not used together with other methods

Interviewing Users are interviewed in their own environment using their language Focus on verbal information: user goals, tasks and sequences in general level, domain specific information Predefined interview themes are used flexibly Efficient method, but detailed and nonverbal information may be lost

Contextual Inquiry (Holtzblatt and Byer, 1998) Interviewing and observing is combined; users are watched, listen to, and talked with while they work in their own environment Focus on uncovering detailed work practice and context of use

Artifact walkthrough Users are asked for examples of artifacts: forms, reports, hand-written manuals, notes The artifacts can help to understand how tasks are done what is difficult for users to do or remember what kind of information flows there are from user to user

Thinking aloud-method Focus on gathering non-verbal information: skills, selection rules and problem solving, beliefs Users are asked to tell and show how they use a tool

Simulations If it is not possible to disturb users in real situations, the use situation can be simulated Or you can videorecord the situation and discuss with users afterwards

Interactive feature conceptualization During an interview, tools, places, persons are written in sticky notes. In the end, user is asked to rearrange the notes in categories, the over all picture is drawn together with the interviewer using arrows etc. Focus on visualisation of user information, the overall picture, checking the validity of interviews

Analysing and reporting User profiles Task sequences and hierarchies Scenarios User needs tables -> use cases Affinity diagrams Videoshots and photographs

Example: A user need table

Example: Use Case

Report contents Case summary Customer descriptions User descriptions Context of use descriptions User needs tables Findings-tables

Presenting Presentations better way in sharing than a report alone Videoshots and photographs were prefered Presentations inspired salesmen to give more examples and information

Number of person hours spent on the user studies

Lessons learned from the cases A small-scale user study pilot has been a good way of introducing user needs gathering to companies Results visible: increased innovations and customer satisfaction Motivating for people and organisation Learning by doing A multi-disciplinary team work was efficient