1. 1 SEG3120 Analysis and Design for User Interfaces LAB1: Video tape evaluation

2. 2 TAs TAs’ contacts: –ssamarah@site.uottawa.ca –xma093@site.uottawa.ca –hibrahim@mcrlab.uottawa.ca (French course) Please always start your email subject with “SEG3120”

3. 3 Lab 1: Videotaped Evaluation Definition: A recorded (videotaped) study of users actively engaging a user interface, performing tasks that might be predefined, in a series of steps that might also be predefined, usually following the “thinking-out-loud” protocol.

4. 4 Lab 1: Videotaped Evaluation Motivation: Videotaping is an evaluation method that addresses: –“Self-fulfilling prophecies” –“Heat-of-the-moment” effect –Attention to minor details –Advantages of repetition Purpose: –Minimizing Malfunctions.

5. 5 Lab 1: Videotaped Evaluation Procedure: 1.Select 6 to 8 representative users per user class E.g. client, salesperson, manager, accounts receivable 2.Invite them to individual sessions Sessions should last 30-90 minutes Schedule 4-6 per day 3.If system involves user's clients in the interaction: 1.Have users bring important clients 2.or have staff pretend to be clients

6. 6 Lab 1: Videotaped Evaluation Procedure: 4.Select facilitators/observers and note takers 5.Prepare tasks: Select the most commonly used tasks plus a few less important tasks Write task instructions for users Estimate the time it will take to complete each task plus extra time for discussion 6.Prepare notebook or form for organizing notes

7. 7 Lab 1: Videotaped Evaluation 7.Set up and test equipment 1.Hardware on which to run system 2.Audio or video recorder (one or more?) 3.Software logs 8.Do a dry run (pilot study)!

8. 8 Lab 1: Videotaped Evaluation 9.At the Start of an Observation Session explain : –nature of project –anticipated user contributions –why user's views are important –focus is on evaluating the user interface, not evaluating the user –all notes, logs, etc., are confidential –user can withdraw at any time –usage of devices –relax! Sign informed consent form: –very important

9. 9 Lab 1: Videotaped Evaluation 10.Start user verbalizing as they perform each task (thinking aloud) For co-operative evaluation, software engineer also verbalizes Appropriate questions to be posed by the observing software engineer: QuestionMalfunction if What do you want to do?They do not know; the system cannot do what they want What do you think would happen if...?They do not know; they give wrong answer. What do you think the system has done? They do not know; they give wrong answer. What do you think is this information telling you? They do not know; they give wrong answer. Why did the system do that?They do not know; they give wrong answer. What were you expecting to happen?They had no expectation; they were expecting something else.

10. 10 Lab 1: Videotaped Evaluation 11.Hold a wrap-up interview 1.What were the most significant problems? 2.What was most difficult to learn? 3.Etc. 12. Analyze the videotape to find malfunctions

11. 11 Malfunction Analysis A disciplined approach to analyzing malfunctions –Provides feedback into the redesign process. 1.Identify Malfunction 2.Answer four distinct questions –Q1: How is the malfunction manifested? What do you notice and who noticed it? –Q2. At what stage in the interaction is it occurring? Goal forming, action decision, action execution, interpretation of results

12. 12 Malfunction Analysis –Q3. At what level of the user interface is it occurring? Physical element level to task level –Q4. Why is it occurring? What is its root cause 3. List and prioritize possible cures

13. 13 Demos http://www.betterdesktop.org/wiki/index.php?title=Mai n

14. 14 Assignment 1 ( Assignment 1 (PowerPoint) 1.Create a PowerPoint presentation that consist of: A.A title slide with title" SEG 3120" (font size 40 pt and style Bold). B.A slide contains group names – (Numerical list, centered, and font type Verdana). C.A slide with a centered picture of computer.

15. 15 Assignment 1 2.Add a custom animation of type "Box" upon entrench once you click the mouse on the computer picture in the 3ed slide. 3.Choose "Cover Up" option for slide transaction to the second slide. 4.Choose slides' design to be "Maple". 5.Save the file.

16. 16 Assignment 1 Your report will contain the following: A summary of the procedures you used to do the evaluation (5 Marks) –When, where and how did you do the evaluation process? –What did the subject do to achieve the task, Pseudo code like description for the steps were taken to accomplish the required task. –What happened as the evaluation proceeded? Here you should provide sufficient detail so the marker can see that you followed good procedures and handled procedural problems well.

17. 17 Assignment 1 A complete list of malfunctions that you found (1 per line) (5 marks). A discussion of the four most significant malfunctions (5 marks). For each provide the following, –An excerpt of the protocol. i.e. a verbatim transcript of 5-15 lines describing what the user did and said, what you said and what happened (around the time the malfunction occurred) You can embellish this with a picture illustrating the malfunction if this makes it clearer –The result of malfunction analysis. –Brief recommendations for the changes.

18. 18 Assignment 1 General Notes: Do not forget to sign the Informed Consent Form, available on the course web site. For malfunction analysis follow (in detail) the procedures outlined in module B –Remember to do a short dry run (pilot study) so you become comfortable with the procedures and A-V equipment. The dry run must use a different task from the main session. –Remember that co-operative evaluation requires both you and the subject user to verbalize. –Your subject(s) should not be someone intimately familiar with the software (i.e. not a designer); however the subject should know or be taught the basics of the system.

19. 19 Assignment 1 Total videotaping time should be 20-30 minutes Videotape the session (the TA will help with this if needed) –You can arrange to borrow cameras from A-V services, although if you do your study with the TA, she can take care of this for several groups at once. –Do not hand in the tape with your report, but keep it in case the professor wants to see it. Erase the tape once you get your mark.

20. 20 Thank You