1. Cultural Heritage in REGional NETworks REGNET Project Meeting Content Group 2002-06-24 - 2002-06-26 Part 1: Usability Testing

2. June 2002REGNET Project Team Meeting Content Group 2 Goals: To assure that the REGNET system is usable (for the different target groups) Usability Testing - An introduction The usability of an interface is a measure of the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in a particular environment with that interface are consistent support the users workflow avoid errors provide good feedback give users control Usable systems...

3. June 2002REGNET Project Team Meeting Content Group 3 Why usability testing? - To facilitate ease of learning in using the system - To increase the speed of task completion - To minimize the error rate whilst using the system- - To increase the subjective user satisfaction Results: detailed catalogue of recommendations to enhance the user interface Heuristic Tests: Tests by experts, using generally accepted usability criteria. User Tests (Scenario based tests): Get/collect users view of system features. Methods of usability testing proposed for the REGNET project: Card Sorting: Determine the optimal information architecture of the system Usability Testing - An introduction

4. June 2002REGNET Project Team Meeting Content Group 4 Usability Testing I - Heuristic tests Test areas: Navigation und Orientation Interaction and Exchange of Information Up-to-Dateness and Quality of Information Design of Information and Text Location and access to the system REGNET Usability Index Basis for the heuristic tests in order to define usability requirements for the portal site. REGNET UI, see working material

5. June 2002REGNET Project Team Meeting Content Group 5 Suggestion for testing Suggestion: 1. 3 partners should carry out a heuristic test (end-user-view of the portal). 2. Report the results (to IMAC) (excel spreadsheet together with comments). 3. IMAC will collect and analyse the results and formulate requirements for the improvement of the portal. Usability Testing I - Heuristic tests Usability experts / persons with experience in computers + internet (overview about similar web sites projects)! 3-5 experts will find about 90% of usability problems

6. June 2002REGNET Project Team Meeting Content Group 6 Evaluate Web Usability Index (As provided by IMAC) Which test criteria are needed for the REGNET Project? Which additional test criteria should be included for the REGNET Project? Agreement on task assignement How many tests? Responsible partners Agreement of test schedule To Dos (Working group and discussion) Usability Testing I - Heuristic tests Working group

7. June 2002REGNET Project Team Meeting Content Group 7 Usability Testing II - Scenario based tests Users perform different tasks on a system Activities are monitored by expert staff Users are debriefed after the experiment regarding their general impression Results of the experiment are evaluated by experts Scenario based tests - Asking user to work with the system Why scenario based tests? 1. Status Quo Investigation - Data on User Experience 2. Optimization - Navigation, Wording, Information Architecture, Look and Feel

8. June 2002REGNET Project Team Meeting Content Group 8 Experiments are performed by content partners (monitoring user) Each content partner identifies 2 potential system users Each content partner translates the questionnaires !? Each content partner carries out the tests according to the test manual Each content partner uses a standard sheet for the de-briefing of the test persons Results are gathered and evaluated by IMAC Usability Testing II - Scenario based tests Suggestion for testing Estimation: 30 tests, per partner 1 day.

9. June 2002REGNET Project Team Meeting Content Group 9 Agreement on test effort (No. of Test-Persons) per Content Partner (Proposal: 2 test persons from the specialists user group per partner. Each user group is represented with at least three test persons) Define user groups from which the test persons are selected Discuss and finalise questionnaire (Proposal for questionnaire provided by IMAC) Agree on the schedule for user testing To Dos (Working group and discussion) Usability Testing II - Scenario based tests Working group Questionnaire, see working material User groups

10. June 2002REGNET Project Team Meeting Content Group 10 List of professional user groups (internal) to be considered in the first phase (according to your feedback). All roles covered? Other groups to be added? (all groups should equally be covered...) Usability Testing II - Scenario based tests Test user groups

11. June 2002REGNET Project Team Meeting Content Group 11 Usability Testing II - Scenario based tests List of end user groups (external) to be considered in the second phase (according to your feedback). Test user groups

12. June 2002REGNET Project Team Meeting Content Group 12 Create test guidelines Translation of templates Set-up test environement Identify test persons Carry out tests Collect reports Evaluate Reports + Recommendations Usability Testing II - Scenario based tests Next steps (after Sofia) Working in groups

13. June 2002REGNET Project Team Meeting Content Group 13 Experimental approach for the validation of the information architecture (portal). Goal: To define the final structure of the web site (Administrativ view, end user view) IBMs Usort / EZCalc http://www-3.ibm.com/ibm/ easy/eou_ext.nsf/Publish/410 Usability Testing III - Card Sorting Card Sorting Why Card Sorting? - to validate the existing Regnet information architecture - easy and cheap to conduct - enables to understand how 'real people' are likely to group items - to identify terminology that is likely to be misunderstood - to identify items that are likely to be difficult to categorize and find Suggestion for testing: 3 tests per partner (professional test user groups, administrative view), 1 h per test (0,5 days per partner).

14. June 2002REGNET Project Team Meeting Content Group 14 Items of information (menu entries) are translated by the content partners and loaded into the system During tests the various itemsof information are dragged from a source window to the target windows Related Items are grouped together in the target window Not-related items are dragged into distinct groups Similar groups are put in higher-level groups Groups are given meaningful names by test-persons Result file are created and send to IMAC Usability Testing III - Card Sorting

15. June 2002REGNET Project Team Meeting Content Group 15 IMAC loads results files into evaluation software Diagrams are generated to guide the navigational design of the site to meet users expectations (cluster analysis) IMAC proposes renaming of menu entries / information architecture for end-users / specialist users portal view Usability Testing III - Card Sorting Life-Test - Administrative view - End-user view

16. June 2002REGNET Project Team Meeting Content Group 16 Agreement on carrying out such tests Agreement on Test Effort (No. of Test-Persons per Content Partner) Proposal: 3 test person from each of the specialists user group per content partner Agreement on the schedule for card sorting To Dos (Working group and discussion) Usability Testing III - Card Sorting Card Sorting exercise

17. June 2002REGNET Project Team Meeting Content Group 17 Create test guidelines Translate cards Set-up test environement Identify test persons Carry out tests Collect reports Evaluate Reports + Recommend Scenario based tests QA Usability Testing III - Card Sorting Next steps (after Sofia)