1 Technical & Business Writing (ENG-315) Muhammad Bilal Bashir UIIT, Rawalpindi

Outline 1. Guidelines for testing documentation 4. Set performance and learning objectives 5. Select testers and evaluators 6. Prepare the test materials 7. Set up the test environment 8. Record information accurately 9. Interpret the data 10. Incorporate the feedback 2. Summary 2

Conducting Usability Tests Lecture: 20 3

4. Set Performance and Learning Objectives Often called operational definitions  Performance objectives state, in clear terms, how long a procedure should take or with what frequency of correctness we can expect users to perform software tasks Performance does not always mean performing a task in the shorter period of time  Effectiveness of software use results from users mindfully following steps, applying their unspoken knowledge, and considering consequences of their actions in workplace  Effective performance may take longer to accommodate the complexity of doing work with software 4

Set Performance and Learning Objectives (Continue…) 5 Kind of Objectives DescriptionExample Time-relatedTime taken to perform a procedure, Time taken to find a topic The user can install the program under 5 minutes The user can locate the import function in under one minute Error-relatedNumber of errors made during a procedure Number of times a passage gets re-read before comprehending Number of tries in the index The user can perform the procedure with a 20% error rate The user can paraphrase the meaning of field variable after one reading The user can find the remove function in under three tries

5. Select Testers and Evaluators The tester is the person who administers the test: arrange the meetings with users, set up the test situations, record the test activities etc. Often you will administer you usability test yourself The evaluator is the person taking the usability test Selecting evaluator poses some interesting problems 6

7 EvaluatorCharacteristicsSuggestions Actual usersGiven User type (novice, experienced, expert) Given degree of subject-matter knowledge No compromise necessary Similar to your Actual Users Same user type but lacking in subject matter knowledge Acquaint the evaluators with the major use of the software Need to read the user scenarios for background Not Similar to your Actual Users Different user type and lacking in subject matter knowledge Acquaint the evaluators with the basics of software if they need to test advance tasks Remind them to assume the role of novice if testing basic tasks Acquaint the evaluators with the major use of the software Need to read the user scenarios for background

6. Prepare the Test Materials Depending on the complexity of your usability test, the written and other materials you supply for testers and evaluators can get very complicated You will probably not require all these materials for your test, unless you plan to set up a testing labs Along with the written test materials you need also to pay attention to the location of the test and kind of hardware and software equipment you require There are various kinds of written test materials  Evaluator selection survey, Instructions for evaluators, Test schedule, Instructions for testers, Test lab procedures etc. 8

Prepare the Test Materials (Continue…) Pilot Testing  It means you test the test, it’s a way of reviewing your test, trying it out in a kind of dry run  It tells about how well your questions work and what kind of data you can expect  Pilot testing doesn’t require a lot of extra effort and it provides a wealth of information to help your final version work better  It can help you in following areas Instructions: Do the guidelines you give to evaluators allow them to perform the test correctly? Terminologies: Do you use any terms the user can’t understand? Timing: Can the user perform the test in the time you have allotted? 9

7. Set Up the Environment The environment for your test may range from the user’s work environment (the field) to a controlled laboratory Your best chance to learn about actual use in the context of user’s work and information environment comes from filed testing But if you have one available, the laboratory offers you a great degree of control and you may find it more convenient, assuming you have a choice 10

Set Up the Environment (Continue…) Advantages and Disadvantages of various test environment 11 EnvironmentAdvantagesDisadvantages Field (User’s workplace) Irreplaceable similarity to actual work demands Intrusive in the user’s workplace Lab (Documentation, software, hardware usability test Controlled and consistent testing Better recording equipment Trained testing personal Lacks similarity to actual user environment More expensive CombinationPilot testing done in lab can lead to a better field results Require setting up two tests events (expensive)

8. Record Information Accurately As you conduct the test, you need accurate methods of recording what you see and hear Your observation serve to flesh out what the cameras and other pieces of equipment record For this reason you should observe some guidelines in recording information Synchronize your timing with the recorders so that you know when an event happened  If the voice recorder, say, indicates a pause in the tape, you can go back to the same place in your notes and see what you recorded the user as doing at that time 12

9. Interpret the Data Interpretation requires you to take into account all the elements that can go wrong with testing so that you get clear results A number of variables can have the effect of clouding the data Interpretation requires of course more than just calculating the data and making the changes that you justify by the numbers It requires that you make common sense decision about your manual design so that the changes make sense to you and to others involved in development 13

10. Incorporate the Feedback Testing does you no good unless you incorporate the information into the design of the documentation product Ideally, your testing produced such useful results that you could make the suggested change, then retest a few times with consistently positive results However, you may only obtain partial results, in which case you analyze them reasonably, make the changes you think they imply, and get on with the project 14

15 Summary Any Questions?