1. Usability Testing – Part II Teppo Räisänen http://www.oamk.fi/~teraisan/ Teppo.raisanen@oamk.fi

2. Practical Advice When a test person arrives, a written query of his/her background information is often useful It is not uncommon, that because of misunderstandings the test person does not belong to targeted user group

3. Practical Advice Often also written agreements are signed NDA Agreement of participation Agreement about recording (video/audio) the test session

4. Interpretation of the Results Vast amounts of information are collected during testing For this information to be useful, it must be correctly transformed and interpreted Just gathering technical data (response times etc.) is usually not enough

5. Interpretation of the Results Often the most critical findings are reported before in-depth analysis Product development cycles have become increasingly faster The results of quick analysis can be used for important corrections and updates

6. Interpretation of the Results The test data should be organized in an uniform way E.g. handwritten comments and audio tapes are transformed into digital formats Numerical data is collected and processed Backup copies are made

7. Interpretation of the Results Several statistical methods are used for intepretation In case of usability problems it should be investigated if just an individual test person had a spesific problem problem is common to a spesific sector of target group problem is common to all users

8. Interpretation of the Results After an usability problem has been recognized the origin of the problem should be tracked After origin has been tracked a solution to the problem is suggested Problems are usually prioritized Not all problems in all cases can be solved within resources available

9. Usability Lab Facilities that are needed vary a lot according to the case Just simple room might be enough Basic equipment consists of computer system for testing video camcorder video monitor (backup equipment)

10. Usability Lab Often two rooms divided by a see- through mirror are used The test person is not able to see to the observers’ room In many cases more than one camrecorder are used Information shown on computer screen is usually not taped but saved internally

11. Usability Lab If two rooms are used, the rooms are often soundproof Test person’s voice is transmitted to the control room via speaker system Instead of see-through mirror, the visual information about test session can be displayed using control room’s monitors

12. Usability Lab A good lab should be easily modified according to the needs Often some amount of staging is used to make test sessions more realistic Background audio tapes Lights Anything within the limits of imagination

13. Field Tests Modern usability research stresses the importance of use context Therefore usability testing should also take place within actual use context Lab conditions are often quite different from real world of target users It has even been predicted, that traditional lab tests will become extinct

14. Field Tests A cost-effective way is to use customer’s facilities for field testing Some modern mobile test labs use advanced methods for recording user’s actions pulse measurement eye movement tracking

15. Automatic Collecting of Log Data In case of software products it is usually quite easy to implement a tool for collecting log data during tests Actions will be recorded automatically Analyzing the data still requires human resources Google Analytics

16. Automatic Collecting of Log Data Many kinds of sources can be recorded Data about the performance of the system Response times Network usage User’s actions, e.g. choices of commands in spesific situations Use of menu commands/kb shortcuts

17. Example: Subjective Enjoyability Enjoyability is one aspect of usability Opinions about product’s enjoyability are often quite individualistic Enjoyability can be investigated using interviews after test sessions Test person can for example be asked to fill a questionnaire

18. Example: Subjective Enjoyability Simple12345Complicated Technical12345Untechnical Reliable12345Unreliable Easy to use12345Difficult to use Friendly12345Hostile Safe12345Unsafe Robust12345Fragile High quality12345Low quality Pleasant12345Unpleasant Well-defined12345Ill-defined

19. Example: Subjective Enjoyability Results of questionnaires can be statistically analyzed E.g. simple average values In many cases test persons have preconceptions about products and brands Therefore it can be useful to commit anonymous testing

20. Anonymous Testing Test person is not told the company behind the product It is important not to give even a hint Often outside consultants are used