1. Design & Evaluation Methods
Wizard Of OZ
Design & Evaluation Methods

2. Wizard Of OZ In Design

3. Designer’s Role
Use a small part of the design to interact with the user.
Interact with perfect or near perfect speech recognition and responses.
Record conversation.
Base VUI Design on Wizard of OZ experiment trials.

4. Wizard Of OZ In Evaluation

5. Evaluator’s Role Never say “say” as part of the protocol.
Telling the user what to say is a violation of the experiment.
Only recognize things that are in your grammar.
All responses have to be verbatim from your VUI.
Record the session and make design adjustments as necessary.

6. Wizard of OZ
Pros
You get an idea of the real interactions that a person may expect under near perfect conditions.
Cons
A human Wizard doesn’t sound synthetic, therefore, the user or pilot tester may have different interactions based upon the believability of the Wizard.