What Do Usability Evaluators Do in Practice? An Explorative Study of Think-Aloud Testing

14 tænke højt test  7 virksomheder 2

3

Dataindsamling Hypotese 24 timer og 54 minutter 4

Dataanalyse 641 segmenter 35 nøgleord 6 grupper 5

Analysis of results from a test session Analyse er ustruktureret  9 Analyse er ufuldstændige  9 Analyse som et sammendrag med brugeren  3 6

Analysis of results from a test session ”Let’s sum up...” 7

Confirmation of known issues as a test focus På udkig efter kendte problemer  8 Evaluatorerne havde forudset problemer  5 8

Confirmation of known issues as a test focus Person 1: ” I think we agree on many of the issues” Person 2: ”Yes – I have already written the chapter, I just need the ammunition” 9

Practical realities influencing tests Tekniske problemer  8 Ikke færdige prototyper  6 10

Practical realities influencing tests ”Now...Let us see... [searches in prototype paper sheets]...these are brand new, so I have not looked at them before” 11

Questions asked during a test Problemer bliver forklaret, ikke oplevet  13 Ledende spørgsmål  13 Unødvendige eller indlysende spørgsmål  10 12

Questions asked during a test ”Did you notice this column?” ”What did you expect to see?” 13

Trying to meet labratory-style scientific standards Evaluatorerne vil have lignende forhold for brugere  5 Stive eller kunstige procedurer  3 14

Trying to meet labratory-style scientific standards ”We have to make sure that all the users get the same questions” 15

Uncovering usability problems or utility concerns Brugere peger på anvendelighed eller mangel på  10 Evaluatorerne peger på anvendelighed problemer  7 16

Uncovering usability problems or utility concerns ”This is just to tell you that I would not do it like this!” 17

18

System failure... 19

31 af 404 = 7,6 % 20