1. Usability testing The Think-aloud test Presented by: Connor Stroomberg

2. Two papers Jakob Nielsen, 1994: –Estimating the number of subjects needed for thinking aloud test Van Den Haak et al., 2003: –Retrospective vs. concurrent think-aloud protocols: testing the usability of a online library catalogue

3. Definition The think aloud method: A simplified thinking aloud test involves having a test user operate an interface to perform a set of pre-defined tasks while being asked to “think out loud”. By Listening in on the user’s thoughts, the experimenter can pinpoint misconceptions and other usability problems as they occur in the interaction (Nielsen, 1994).

4. Nielsen paper A lot of developers know about the think-aloud method (21%) but few use it (6%). Why ? –To complex and expensive –The need for highly trained specialists –Special equipment and laboratory Nielsen however says: –There is no need for these expensive and complex tools and specialists if we are only concerned with qualitative results. –Because the method can be used at various steps in de design process qualitative results will have a higher impact on the end product.

5. The Nielsen study two set of experiments using the same think- aloud method. The people doing the user testing were two groups of students, three hours of training. People doing the tests; 24 and 30 different students. Two applications being tested, a commercial word processor and a piece of shareware software.

6. Results

7. Results 2

8. Results 3

9. Nielson Conclusion The models says that only four to six participants are needed. This will give 75% of the usability problems. Major problems are more likely to occur during testing than minor ones.

10. van den Haak et al. Is think-aloud really a good measure for usability testing ? In what way should we conduct a think- aloud test ?

11. van den Haak et al. Retrospective vs. concurrent think-aloud That are the benefits of doing retrospective think-aloud ? That are the drawbacks ?

12. benefits of retrospective think-aloud 1.Decrease in reactivity 2.Recording of working times per task 3.Possibility for reflection by test subjects 4.Cross culture (?)

13. Drawbacks of retrospective think- aloud 1.Duration 2.Biased accounts Maybe 3.Equipment ? 4.Expertise ?

14. Van der Haak et al. Methode Online library catalogue. 40 participants. 2 groups ? Concurrent vs. retrospective Questionnaires

15. Results

18. Disscusion Limited contribution of verbalization in CTA (Nielsen does not mention this) Enhance and impede task performance Interaction between task difficulty, degree of verbalization and task performance CTA

19. Conclusion van der Haak CTA and RTA are equivalent, but clearly different evaluation methods. Task difficulties need to be controlled in CTA, however this may not always be possible.