1. CS305, HW1, Spring 2008 Evaluation Assignment
Part 1: Proposal ( to by 5pm, Friday, Feb. 8)
A short description of what you plan to evaluate (2-3 lines is enough)
Names of group members (3-4 people)
Possible participants for your evaluation
Choose any software / system to evaluate as long as:
You are at least somewhat familiar with it
You can find two or more participant users who are not experts
You can run the software / system in an experimental manner
It must involve some input/interaction as well as output. (Don’t want it to be passive for participant. GUI not required)
Web site? Only if it has some interesting complexity
Device? Probably OK. Game? Probably OK. Ask me.

2. Evaluation Assignment
Focus on some aspect of the software
Just part of a system if it’s complex
Focus on several usability goals
The user interactions that implement one to three tasks that all together can be done by an expert in, say, 20 minutes
Ideally a highly used or complex part of the software
You choose what goals matter to you. E.g. maybe you do or don’t want to include learnability or efficiency

3. Evaluation Assignment
Perform a co-operative evaluation and analysis of usability defects
Do a dry run with someone in your team
Remember that co-operative evaluation requires both you and the participant to verbalize
Your participants should not be experts but should be able to use or learn the software fairly quickly
If asked, you must volunteer to be someone else’s participant!
Also, create a table like that presented in class to evaluate success in tasks
Keeps good notes and records!

4. Evaluation Assignment: Your Report
Turn in a report as follows on Mon., Feb, 18, 11 am
A description of the specific software or system and its aspects that are being evaluated.
Be sure to what the specific goals are for your evaluation! Just a subset of the system? Specific goals for this evaluation?
A summary of your procedures
When, where, how did you do your evaluation? (Only enough detail so I can see that you followed good procedures.)
Also, what tasks did you ask the participant to do?
Should include copies of the task descriptions you gave the user
(see next page)

5. Your report (cont’d)
List of all usability defects or problems by the user that you observe
What happened? What usability principle is violated?
OK if this has only a little detail, but feel free to add more detail if you wish
A more detailed analysis of the most serious defects (at least four defects)
Choose defects in the system’s design
How it supports tasks, or its conceptual model, or its interaction style, or problems with specific UI elements
State when error occurs in the goal/decide/execute cycle
Excerpt from your notes (quotes if possible) on what happened with the user(s) in the evaluation
Cause of the problem
More details than given in step 3 above. Make appropriate use of concepts of usability principles etc.
Any other appropriate detail from slides on usability defects
Recommendations for changes in the system

6. Your report (cont’d)
The table like Figure 12.9 (p. 610) for each task evaluated
But, list high-level tasks based on a user-goal here!
If something obvious, or easy, why bother?
Use this to evaluate how well the system serves user needs
Examples of good choices:
Add new entry to phonebook
Find and play specific MP3 file by name
Compose and send an
Examples of bad choices:
Type in phone number
Pause the player (assuming there’s a pause-button)
Enter the body of the message
A summary of any subjective comments made by your participants in the “wrap-up” discussion you have at the end of your evaluation.