1. Sequential Hypothesis Testing under Stochastic Deadlines Peter Frazier, Angela Yu Princeton University TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAAAAA

2. Sequential Hypothesis Testing

3. under Stochastic Deadlines

4. Peter Frazier & Angela Yu Princeton University

5. We consider the sequential hypothesis testing problem and generalize the sequential probability ratio test (SPRT) to the case with stochastic deadlines. This causes reaction times for correct responses to be faster than for errors, as seen in behavioral studies. Summary

6. Both decreasing the deadline’s mean and increasing its variance causes more response urgency. Results extend to the general case with convex continuation cost.

7. 1. Sequential Probability Ratio Test

8. Sequential Hypothesis Testing ABABAB wait At each time, the subject decides whether to act (A or B), or collect more information. This requires balancing speed vs. accuracy.

9. We observe a sequence of i.i.d. samples x 1,x 2,... from some density. The underlying density is unknown, but is known to equal either f 0 or f 1. We begin with a prior belief about whether f 0 or f 1 is the true density, which we update through time based on the samples. We want to maximize accuracy

10. Let  be the index of the true distribution. Let p 0 be the initial belief, P{  =1}. Let p t := P{  =1 | x 1,...,x t }. Let c be a cost paid per-sample. Let d be a cost paid to violate the deadline (used later) Let  be time-index of the last sample collected. Let  be the guessed hypothesis.

11. Posterior probabilities may be calculated via Bayes Rule: Time (t) Probability (p t )

12. Probability of Error Time Delay Penalty The objective function is: where we require that the decisions  and  are “non-anticipative”, that is, whether  <= t is entirely determined by the samples x 1,...,x t, and  is entirely determined by the samples x 1,...,x . Objective Function

13. Time (t) Probability (p t ) A B  Optimal Policy (SPRT) Wald & Wolfowitz (1948) showed that the optimal policy is to stop as soon as p exits an interval [A,B], and to choose the hypothesis that appears more likely at this time. This policy is called the Sequential Probability Ratio Test or SPRT.

14. 2. Models for Behavior

15. A classic sequential hypothesis testing task is detecting coherent motion in random dots. One hypothesis is that monkeys and people behave optimally and according to the SPRT.

16. (Roitman & Shadlen, 2002) Broadly speaking, the model based on the classic SPRT fits experimental behavior well. Accuracy vs. CoherenceReaction Time vs. Coherence There is one caveat, however…

17. Accuracy Mean RT RT Distributions (Data from Roitman & Shadlen, 2002; analysis from Ditterich, 2007) SPRT fails to predict the difference in response time distributions between correct and error responses. Correct responses are more rapid in experiments. SPRT predicts they should be identically distributed.

18. 3. Generalizing to Stochastic Deadlines

19. (Data from Roitman & Shadlen, 2002) (Analysis from Ditterich, 2006) Monkeys occasionally abort trials without responding, but it is always better to guess than to abort under the assumed objective function. To explain the discrepancy, we hypothesize a limit on the length of time that monkeys can fixate the target.

20. Error Penalty Deadline Penalty Hypothesizing a decision deadline D leads to a new objective function: Time Penalty We will assume that D has a non-decreasing failure rate, i.e. P{D=t+1 | D>t} is non-decreasing in t. This assumption is met by deterministic, normal, gamma, and exponential deadlines, and others. Objective Function

21. The resulting optimal policy is to stop as soon as p t exits a region that narrows with time. Probability (p t ) Generalized SPRT Classic SPRT Deadline Time (t) Optimal Policy

22. Reaction Time Frequency of Occurrence Correct Responses Error Responses Under this policy, correct responses are generally faster than error responses. Response Times

23. Influence of the Parameters Deadline Uncertainty Deadline Mean Time Penalty Deadline Penalty Plots of the continuation region C t (blue), and the probability of a correct response P{  =  |  =t} (red). D was gamma distributed, and the default settings were c=.001, d=2, mean(D)=40, std(D)=1. In each plot we varied one while keeping the others fixed.

24. Theorem : The continuation region at time t for the optimal policy, C t, is either empty or a closed interval, and it shrinks with time (C t+1 µ C t ). Proposition : If P{D<1} = 1 then there exists a T < 1 such that C T = ;. That is, the optimal reaction time is bounded above by T.

25. Proof Sketch Lemma 1: The continuation cost of the optimal policy, Q(t,p), is concave as a function of p. Lemmas 2 and 3: Wasting a time period incurs an opportunity cost in addition to its immediate cost c. Lemma 4: If we are certain which hypothesis is correct (p=0 or p=1), then the optimal policy is to stop as soon as possible. Its value is: Define Q(t,p t ) to be the conditional loss given p t of continuing once from time t and then behaving optimally.

26. C t+1 Proof Sketch CtCt p0 1 Expected Loss Q(t+1,p)-c Q(t,p) min(p,1-p)

27. References 1.Anderson, T W (1960). Ann. Math. Statist. 31: 165-97. 2.Bogacz, R et al. (2006). Pyschol. Rev. 113: 700-65. 3.Ditterich, J (2006). Neural Netw. 19(8):981-1012. 4.Luce, R D (1986). Response Times: Their Role in Inferring Elementary Mental Org. Oxford Univ. Press. 5.Mozer et al (2004). Proc. Twenty Sixth Annual Conference of the Cognitive Science Society. 981-86. 6.Poor, H V (1994). An Introduction to Signal Detection and Estimation. Springer-Verlag. 7.Ratcliff, R & Rouder, J N (1998). Psychol. Sci. 9: 347-56. 8.Roitman J D, & Shadlen M N (2002). J. Neurosci. 22: 9475- 9489. 9.Siegmund, D (1985). Sequential Analysis. Springer. 10.Wald, A & Wolfowitz, J (1948). Ann. Math. Statisti. 19:326-39.