Detecting and fixing bad clocks in computerized reaction time (RT) experiments J. P. Salmon1, M. A. Lawrence1, G. A. Eskes1 & R. M. Klein2 Dalhousie University, Department of Psychology1 & Department of Psychiatry Why should you care, 1) clinical, 2) small effects, 3) small sample… Also – make first figure bigger (doesn’t quite fit …) … Get rid of ABSTRACT??? Introduction In many experiments reaction time (RT) is a critical indicator of how quickly a person can perform a specific mental operation. Precise measuring of RT can be important particularly when exploring subtle differences of cognitive functioning, for example, some research publishes significant cognitive effects as small as 20 ms! Computers appear to be one of the best and easiest ways to collect reaction-times for cognitive experiments. Most people know that the average computer CPU can process data in microseconds, or millionths of a second! You may be surprised, however, to learn that computer hardware typically only operates in milliseconds (ms), or thousandths of second, slowing down your ability to calculate precise reaction times. When should you care about milliseconds? Millisecond precision can be particularly important if any of the following are true: You are studying a clinical sample You have a small sample You expect small effect sizes Reaction time (RT) is your critical measure (i.e. not accuracy). Problems to look out for HARDWARE problems Polling rates for mouse and keyboards – typically only 8 to 10 ms. Refresh rates for monitors– typically 60 Hz, or 16.67 ms SOFTWARE problems Synchronizing response-timer with screen refresh rate Measuring the response, on time! Simulating Reaction Times A response simulator (Figure 1) can be used to measure the temporal precision of your computerized task. It connects to a USB port on your computer and simulates a keyboard response with a known pattern of responses. Figure 1: The response simulator with USB plug-in and light detector. Light detector on the simulator The light detector on the simulator detects a change from white to black and uses the onset to start a precise timer to simulate a key press (response) with a known response function. Figure 2: The response simulator being used to simulate keyboard responses. The light detector detects the screen transition from white to dark. For most tasks this just requires the addition of a black square that is time-locked to stimulus presentation (or appears instead of the regular stimuli). Timing metrics of simulator The first simulated response occurs after 100 ms, then second at 102 ms, the third at 104 ms, etc., increasing by 2 ms up to 200 ms. Afterwards, the simulator resets to 100 ms, and begins again. In this way, the simulator will create a known response function, that can be compared to the actual function recorded by the software. For example, Figure 3 shows the function that response simulator is generating (in black), compared to the actual, temporally imprecise reaction times (in red) that were measured. Figure 3: Bad temporal precision (staircase – or binned RT) based on problems in the way the software (code) was written. The problem In order to collect a reaction time, the computer needs to start a timer (when the stimulus starts) and stop a timer (when the response is registered). In this particular case (Figure 3) the code was telling the computer to start the timer and the draw of stimulus at the same time. However, the command to draw was not being executed until the next natural refresh rate of the screen (60 Hz, or 16.67 ms). This meant the RT-timer could be started 0-17 ms before anything actually appeared on the display. This result was binning of RTs as shown in Figure 3. The fix Synchronizing the reaction timer with the screen refresh-rate so that the timer started when the stimulus was drawn (and not when the command to draw was sent) fixed this reaction-time function (see Figure 4). Figure 4: Simulated responses (red), after response- timer was properly synchronized with screen refresh rate. Other problems Failure to time-lock the reaction-timer to the onset of the stimulus or inadvertently delaying accurate precision collection of reaction-time collection, can also lead to noisy reaction-time functions such as the one below (Figure 5). Figure 5: Reaction-time function created by the response simulator in response to non-optimized software. Hardware solutions In addition to fixing & checking your software or writing good code, you can buy special devices, or response boxes that get polled by the computer more often. For mouse & keyboards, you can buy the “gaming” version, which boast faster polling times, up to 1 ms precision (Figure 6). Figure 6: Gaming keyboard (and box) boasting a 1000 Hz polling rate. This means responses on keyboard will be polled once every 1 ms, instead of after 10 ms delay (like a regular keyboard). Conclusions Don’t assume that your computerized task is giving microsecond or even millisecond precision. If precise reaction times are important to you, you should carefully check your software and hardware. Consider buying hardware that poll at 1000 Hz or 1 ms (like “gaming” mice & keyboards). Consider testing the precision of your software with something like the response- simulator discussed in this poster. Acknowledgements This research supported by the Atlantic Innovation Fund (ACOA). Thanks to Jesse Rusak for developing the software, and the software fix, that was used for this presentation. Thanks, also, to Chris Wright for designing the response simulator. Thanks to Christopher Dean for his advice on poster design. Finally, thanks to Carey Isenor for his help and support printing this (and many other) academic posters. Reaction Time (ms) Trials Trials Reaction Time (ms) Trials Reaction Time (ms)