1. CS 522: Human-Computer Interaction Lab: Formative Evaluation
Dr. Debaleena Chattopadhyay
Department of Computer Science
debaleena.com

2. Formative Evaluation
Assessing a product during its development or early implementation.
Formative evaluation results should inform your design revisions before high-fidelity prototyping or full-fledged implementation.

3. Getting the design right
Sketches using the elaboration-reduction method. —Getting the right design (to meet the functional requirements)
Testing with users/ user representatives.
—Getting the design right (to meet usability goals and provide a user-friendly design solution.) still focusing on ideas, not system’s performance or visual design

4. Formative evaluation methods
Heuristic Inspection
Think aloud protocol
Cognitive Walkthrough
Pluralistic Walkthrough
GOMS (Goals, Operators, Methods, Selection)
Interviews
Observational task analysis
..... Many more

5. Before Evaluation
You need prototypes

6. Prototypes Paper prototypes Wizard-Of-Oz prototypes
General overview: 
Paper prototype
A person simulates the computer’s operation in putting down and picking up paper mock ups. Tips: Make it larger than life. Keep pieces organized.
Wizard –of – Oz
Interaction simulation using human in the loop.
Wizard of Oz = Man behind the curtain
Wizard has to be mechanical and fake the computer’s interaction
The user should think that they are interacting with a computer, but a developer will respond to the input. Figure out how to fake the functionality you need to give the user an authentic experience from their viewpoint

7. Methods we will use in class
Cognitive walkthrough with paper prototypes
Think aloud with Wizard-of-Oz prototypes

8. What we need for the evaluation?
Assign roles to your design team: Computer, Facilitator, and Observer.
Revisit your user scenarios and define 3 to 5 tasks. Tasks should be real, complete, independent, and representative.
(For CW) A complete, written list of the actions needed to complete each task.
Computer:
Simulates prototype
Doesn’t give any feedback that the computer wouldn’t
Facilitator:
Presents interface and tasks to the user
Encourages user to think aloud by asking questions
Keeps user test from getting off track
Observer
Keeps mouth shut, sits on hands if necessary
Takes copious notes

9. Lab Create tasks Plan your evaluation
If you have prototypes, test with your peers as evaluators*
*recommended, NOT required

10. Formative Evaluation
2 members from each project group should participate in 2 formal evaluations of another group (mention in your report who participated in which group’s evaluation)
Find one evaluator outside class.
Deliverables:
Images of the prototype/setup you used for formative evaluation
Changes that you will make to the prototype based on testing
Observations that will inform your application development

11. To do: Due 11/2— Formative evaluation
Nov 2: Shneiderman et al., Chapter 15 (Information Search), pp. 478—502.