1. Ensuring Turkers enjoy your studies and perform them well (and seeing what they think of your HITs) reddit.com/r/HitsWorthTurkingFor Turkopticon

2. IRB statements Your HIT should contain a statement like the following (modified per your IRB-needs). I put mine on the instructions screen, usually: By answering the following questions, you are participating in a study being performed by psychologists in the Harvard Department of Psychology. If you have questions about this research, please contact Timothy Brady at tbrady@wjh.harvard.edu. Your participation in this research is voluntary. You may decline to answer any or all of the following questions. You may decline further participation, at any time, without adverse consequences. Your anonymity is assured; the researchers who have requested your participation will not receive any personal information about you.

3. Today Take our HIT from last time and: – Break each trial into a separate page, using Javascript (DO NOT SAY JAVA) – Don’t let you advance to the next page without answering the current question – Randomizing trials – Making it prettier using CSS (cascading style sheets), a language within HTML – Hiding the Submit button until they complete the HIT – Blocking them from completing the HIT if they are previewing it – Block people who have done similar HITs before – Use Javascript to make things appear/disappear/control appearance after delays Note: all of our Javascript will use “JQuery”, a ubiquitous toolkit that modifies Javascript to make it more convenient. This is, sadly, different than what TestMyBrain relies on. But the basic concepts are the same.

4. Not today (sorry) Using keyboard to collect responses Storing RTs Trials that hide and show things with timing (mostly) Command-line tools Ideas for catch trials Generating HITs using Matlab or Python code Using Flash/Actionscript to run experiments