Chapter 24 Final preparations for the evaluation

UIDE Chapter 24 Roles for Evaluators –Facilitator –Note-Taker –Equipment Operator –Observer –Meeter and Greeter –Recruiter –The Lone Evaluator

UIDE Chapter 24 Roles for Evaluators –Facilitator Key in relaxing participant and making them feel comfortable Secondary role to keep communication flowing Insure that objective of evaluation is met

UIDE Chapter 24 Roles for Evaluators –Note-Taker Record events in a non-distracted manner

UIDE Chapter 24 Roles for Evaluators –Equipment Operator Video, or other recording equipment? Equipment necessary but not germane to the participants experiences with the system Equipment necessary that is germane to the experience - printers

UIDE Chapter 24 Roles for Evaluators –Observer Does nothing but Aspects to remember –Be quiet –Do not include participant names in notes –This is not a “smoking & joking” session –Wait for results – don’t interject

UIDE Chapter 24 Roles for Evaluators –Meet & Greet Meets participants as they arrive Responsible for domestic arrangements –Rest rooms –Refreshments –Keeping participant from losing interest before their turn to participate

UIDE Chapter 24 Roles for Evaluators –Recruiter –Finds participants before the evaluation sessions –Evaluates proper candidates –Ensure compensation is made if appropriate

UIDE Chapter 24 Roles for Evaluators –The Lone Evaluator –Combines tasks –May have a Tonto to split multiple tasks with –If tasks are combined leave extra time to accomplish

UIDE Chapter 24 Preparation Steps for an Evaluation –Book/Arrange Location –Assign roles to evaluators –Brief Evaluators –Create Evaluation Script –Create Consent Form –Create Pre-test interview plan (to brief participants) –Recruit Participants –Create a non-disclosure form –Stage System for Prototype –Run-through (Cardinal rule – if it can go wrong it will)

UIDE Chapter 24 Creating an Evaluation Script –An Example of an Evaluation Script (pp ) Welcome Participant Brief the Participant Obtain Consent (after briefing) Pre-test the questionnaire ( hey dude, do you understand what I’m going to ask you? ) Explain the task, see if participant understands, train if necessary Observe and record during session Perform post-session questionnaire (retrospective protocol) Thank the participant, collect any materials, close session

UIDE Chapter 24 Forms to Use When Asking for Permission to Record –Informed Consent Minimal risk Information –inform them –get them to agree they have been informed Comprehension Voluntariness Participants rights (i.e. you can get up and leave) Nondisclosure Confidentiality Waivers (i.e. future use of recorded materials) Expectations (how we will use your participation)

UIDE Chapter 24 Forms to Use When Asking for Permission to Record –Nondisclosure AgreementsNondisclosure Agreements Business University

UIDE Chapter 24 The Pilot Test –Participants for Your Pilot Test –Design and Assemble the Test Environment –Run the Pilot Test