1. Day 8
Usability testing

2. Objectives
Examine usability testing in more depth

3. Usability testing basics
Don’t depend on designers to judge their own designs
Test often using small groups
Select the appropriate type of usability test
exploratory tests
assessment tests
evaluation tests
comparison tests

4. Types of Usability Tests

5. Conducting a usability test
Six main steps
create the test plan
prepare the test materials
select the participants
conduct the test
post-test discussion and questionnaire
convert the test results to recommendations

6. 1. Create the test plan - purpose
Purpose of the usability test
examples
to obtain feedback on the initial design as shown on wireframes, specifically with regard to the whether the functions and features conform to what potential users would expect
to obtain feedback on two alternative designs, to identify any strengths or weaknesses of each design
to determine whether tasks can be performed in accordance with the set benchmark

7. 1. Create the test plan – specific goals
Goals of the usability test
examples
Do users complete a task successfully?
If so, how fast do they complete each task?
Is that fast enough to satisfy them?
What paths do they take when trying?
Do those paths seem efficient enough to them?
Where do they seem to stumble or get confused?
What words or paths are they looking for that are not currently on the site?

8. 1. Create the test plan – participant profiles
This section lists
the characteristics of the users who will participate in the usability test
the desired number of each type of user

9. 1. Create the test plan – environment
test environment
the room – quiet, free of other happenings
a computer – make sure your software is on it
a mounted video camera – (maybe, if possible)
roles of people involved in the test
the facilitator
the participant
the recorder (maybe)

10. A possible environment (overkill)

11. 1. Create the test plan – data to collect
examples
the actual steps the participant took to attempt to complete each task
the time it takes to complete each task
the participant’s rating of the ease or difficulty with which each task was completed
participant’s comments and recorder’s observations that indicate the participant’s satisfaction with the product being tested
the number of “errors” or diversions from the ideal steps outlined for each task

12. 2. Prepare the test materials
screening questionnaire
consent form
task sheets
questionnaire/interview

13. An example

15. Tasks (important … this is the crucial bit!)
For some examples, see the scenarios in section 7 of this paper:

17. 3. Select the participants
They should reflect the target audience for the product you are testing
Keep a database of possible participants
Use a screening questionnaire to find candidates that meet the criteria you set
You should pay or reward them for their time

18. 4. Conduct the test Put the participants at ease
Do not lead the participants.
Be flexible
Never lose patience with a participant
Be a good listener
Keep your body language and facial expressions neutral
Don’t jump to conclusions

19. 5. Post –test questionnaires/interviews
The post-test questionnaire/interview
It provides an opportunity for the participant to provide conclusions and recommendations about the product or design being tested

20. 6. Results to recommendations
Keep it brief
Include positive findings (not just problem areas)
Organise comments logically
Include an executive summary
Summarise the most important findings of the report in an executive summary that is placed at the front of the report

21. Usability Remember the definition
[ISO 13407] "The usability of an interface is a measure of the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in a particular environment with that interface.“
You would need to measure
Effectiveness
efficiency
Satisfaction
You need to measure these objectively

22. Some usability techniques with users
During the task
Think aloud protocol
Co-discovery learning
Question-asking protocol
Performance measurement
Logging actual use
Post test
questionnaire
interviews
focus groups

23. Think aloud protocol
Users are asked to vocalise their thoughts, feelings, and opinions whilst interacting with a system as they perform a task
Verbalizations are quite useful in understanding mistakes that are made and getting ideas about what the causes might be and how the interface could be improved to avoid those problems

24. Co-discovery learning
Two participants are asked to use the system together and are encouraged to think aloud and discuss the product together

25. Question-asking Protocol
While users are carrying out a task, a tester asks direct questions about the product, in order to understand their mental model of the system and the tasks, and any trouble they have in understanding and using the system. Like co-discovery learning, this is a more natural way of encouraging users to verbalise their thoughts than the thinking-aloud method.

26. Performance Measurement
Quantitative data are obtained about a test participant’s performance when performing tasks

27. Logging Actual Use
The computer is used to automatically collect statistics about the detailed use of the system. It shows exactly how users perform their work. Because data is automatically collected, data from a large number of users working under different circumstances can be collected.

29. Questionnaires
After using the product, users fill out a questionnaire. They are given a series of questions that have been formulated about the system.

30. Interviews
Users are interviewed after using the product. They are given a series of questions that have been formulated about the system.

31. Focus Groups
About 6 to 9 users are brought together to discuss issues relating to the system. A list of issues to be discussed beforehand are formulated and the discussion is moderated by a usability expert. This method can capture spontaneous user reactions and ideas that evolve in the dynamic group process.

32. Another paper