Chapter 24 Final preparations for evaluation

Completing preparations Geez, I thought we were all ready… This chapter dots the i’s and crosses the t’s I guess… Geez, I thought we were all ready… This chapter dots the i’s and crosses the t’s I guess…

Roles for evaluators If you’re the only person, you have to do everything…or, assign, I mean, delegate responsibilities Facilitator:  “flight attendant”, looks after well being of participant  “sportscaster”: data collector, keeps the participant talking (if talk-aloud, presumably)  “scientist”: ensuers data collected in scientific manner (objective, unbiased, etc.) If you’re the only person, you have to do everything…or, assign, I mean, delegate responsibilities Facilitator:  “flight attendant”, looks after well being of participant  “sportscaster”: data collector, keeps the participant talking (if talk-aloud, presumably)  “scientist”: ensuers data collected in scientific manner (objective, unbiased, etc.)

Other roles How many roles are there? Lots…  Note-taker: self-explanatory  Equipment operator: self-explanatory  Observer: good for “outsiders” (e.g., stakeholders)  Meeter and greeter  Recruiter Lone Evaluator: does all of the above How many roles are there? Lots…  Note-taker: self-explanatory  Equipment operator: self-explanatory  Observer: good for “outsiders” (e.g., stakeholders)  Meeter and greeter  Recruiter Lone Evaluator: does all of the above

Use a script (Seems like I said to use a script several times already…) Scripts are important for several reasons:  so you don’t forget to say or do something (e.g., get consent form signed)  so you don’t influence the participant, or if you do, at least the influence is consistent (e.g., between groups) (Seems like I said to use a script several times already…) Scripts are important for several reasons:  so you don’t forget to say or do something (e.g., get consent form signed)  so you don’t influence the participant, or if you do, at least the influence is consistent (e.g., between groups)

Script example Welcome the participant “Thanks for coming, please have a seat here.” Brief the participant “You have been invited to participate in an eye tracking study. We want to find out what you look at when navigating a web page. You don’t have to think about what you’re looking at, just act naturally. This is an eye tracker. No, it doesn’t have any lasers.” Obtain consentPlease read and sign both copies of the consent form -- the top copy is yours to keep for your record.

Informed Consent What’s on the Informed Consent Form?  Rights of the participant minimal risk (no harm) procedural info (what is going to happen) comprehension (does user understand) voluntariness (no coercion or pressure) human rights (participants must be informed)  Rights of the company/school nondisclosure (is an NDA needed?) confidentiality (thought this was subject’s identity kept secret) waivers (for pictures, video recordings, etc.) legalese (use plain English) expectations (not clear on this…) What’s on the Informed Consent Form?  Rights of the participant minimal risk (no harm) procedural info (what is going to happen) comprehension (does user understand) voluntariness (no coercion or pressure) human rights (participants must be informed)  Rights of the company/school nondisclosure (is an NDA needed?) confidentiality (thought this was subject’s identity kept secret) waivers (for pictures, video recordings, etc.) legalese (use plain English) expectations (not clear on this…)

Example of IC form See Clemson’s IRB:  RB.htm RB.htm See Clemson’s IRB:  RB.htm RB.htm

Pilot study Before running the full experiment, run a pilot study:  this will expose “minor annoyances” that you may have missed, that are easy/quick to fix, but that may hinder collection of the “real data”  you can generally expect to throw away all data collected in the pilot Before running the full experiment, run a pilot study:  this will expose “minor annoyances” that you may have missed, that are easy/quick to fix, but that may hinder collection of the “real data”  you can generally expect to throw away all data collected in the pilot