1. Inference in Dynamic Environments Mark Steyvers Scott Brown UC Irvine This work is supported by a grant from the US Air Force Office of Scientific Research (AFOSR grant number FA9550-04-1-0317)

2. Overview Experiments with dynamically changing environments –How do observers adapt their decision-making strategies? –How quickly can observers detect changes? Theory development –Bayesian ideal observers –Process models –Measure individual differences in adaptation

3. Part I Decision Criteria Adaption ( a quick overview from last year’s presentation)

4. Difficult Block Easy Block Decision Environment parameter Subject’s decision criterion Decision lag Easy Block Difficult Block Time Environments alternating between easy and hard blocks Time

5. Realistic 3D gaming tasks Decide which of two types of missiles is approaching How quickly can a participant adapt to changes in the similarity between the two missiles? A=“not puffy” B=“very puffy”

6. Modeling Results Developed a lagged signal detection model Model estimates that participants take an average of 9 trials to switch to a new decision criterion Individual differences

7. Part II Prediction and Change Detection

8. Change detection and prediction Prediction of future observations in time- series data Changepoint models –E.g. changing to different coins at random times Accurate prediction requires detection of change –stock market –“hot hand” players

9. Basic Task Given a sequence of random numbers, predict the next one

10. = observed data = prediction Two-dimensional prediction task Touch screen monitor 1500 trials Self-paced Same sequence for all subjects

11. Sequence Generation (x,y) locations are drawn from a binomial distribution of size 10, and parameter θ At every time step, probability 0.1 of changing θ to a new random value in [0,1] Example sequence: Time θ=.12θ=.95θ=.46θ=.42θ=.92θ=.36

12. = observed sequence Optimal Bayesian Solution = prediction Subject 4 – change detection too slow Subject 12 – change detection too fast (sequence from block 5)

13. Tradeoffs Detecting the change too slowly will result in lower accuracy and less variability in predictions than an optimal observer. Detecting the change too quickly will result in false detections, leading to lower accuracy and higher variability in predictions.

14. Average Error vs. Movement = subject Relatively many changes Relatively few changes

15. A simple model 1.Make new prediction some fraction α of the way between old prediction and recent outcome. α = change proportion 2.Fraction α is a linear function of the error made on last trial 3.Two free parameters: A, B A<B bigger jumps with higher error A=B constant smoothing α 0 1 A B B A

16. Average Error vs. Movement = subject = model

17. One-dimensional Prediction Task 1 2 3 4 5 6 7 8 9 10 11 12 12 Possible Locations Where will next blue square arrive on right side?

18. Average Error vs. Movement = subject = model

19. Inference and Prediction Judgments Many errors in prediction –too much movement What is causing this? –Faulty change detection or faulty prediction judgments? –Gambler’s fallacy: people predict too many alternations New experiments: –ask directly about the generating process –inference judgment: what currently is the state of the system?

20. Tomato Cans Experiment Cans roll out of pipes A, B, C, or D Machine perturbs position of cans (normal noise) At every trial, with probability 0.1, change to a new pipe (uniformly chosen) (real experiment has response buttons and is subject paced) ABCDABCD

21. Tomato Cans Experiment (real experiment has response buttons and is subject paced) ABCDABCD Cans roll out of pipes A, B, C, or D Machine perturbs position of cans (normal noise) At every trial, with probability 0.1, change to a new pipe (uniformly chosen) Curtain obscures sequence of pipes

22. Tasks ABCDABCD Inference: what pipe produced the last can? A, B, C, or D? Prediction: in what region will the next can arrive? 1, 2, 3, or 4? 12341234

23. Experiment 1 63 subjects 12 blocks –6 blocks of 50 trials for inference task –6 blocks of 50 trials for prediction task –Identical trials for inference and prediction

24. INFERENCE PREDICTION Sequence ABCDABCD Trial

25. INFERENCE PREDICTION Sequence Ideal Observer ABCDABCD Trial

26. INFERENCE PREDICTION Sequence Ideal Observer Individual subjects Trial ABCDABCD

27. INFERENCE PREDICTION Sequence Ideal Observer Trial ABCDABCD Individual subjects

28. INFERENCE PREDICTION Sequence Ideal Observer Trial ABCDABCD Individual subjects

29. = Subject ideal INFERENCEPREDICTION

30. ideal INFERENCEPREDICTION = Process model = Subject

31. Varying Change Probability 136 subjects Inference judgments only  Subjects track changes in alpha ideal Prob. =.08 Prob. =.16 Prob. =.32

32. Number of Perceived Changes per Subject Low medium high Change Probability (Red line shows ideal number of changes) Subject #1

33. Number of Perceived Changes per Subject 55% of subjects show increasing pattern 45% of subjects show non- increasing pattern Low, medium, high change probability Red line shows ideal number of changes

34. Conclusion Adaptation in non-stationary decision environments Subjects are able to track changes in dynamic decision environments Individual differences –Over-reaction: perceiving too much change –Under-reaction: perceiving too little change

35. Potential deliverable outcomes Simple measurement models of decision making in dynamic environments. Classify different observers’ decision making abilities under different kinds of environments –E.g., does this observer respond too slowly to changes in their environment? –Is another observer close to the theoretically optimal decision mechanism? –Conversely, classify different observers as optimally suited to different tasks, depending on dynamic properties of decision making environments in those tasks.

36. Publications Brown, S.D., & Steyvers, M. (2005). The Dynamics of Experimentally Induced Criterion Shifts. Journal of Experimental Psychology: Learning, Memory & Cognition, 31(4), 587-599. Steyvers, M., & Brown, S. (2005). Prediction and Change Detection. In: Advances in Neural Information Processing Systems, 19 Navarro, D.J., Griffiths, T.L., Steyvers, M., & Lee, M.D. (2006). Modeling individual differences using dirichlet processes. Journal of Mathematical Psychology, 50, 101-122. Wagenmakers, E.J., Grunwald, P., & Steyvers, M. (2006). Accumulative prediction error and the selection of time series models. Journal of Mathematical Psychology, 50, 149-166.