Inference in Dynamic Environments Mark Steyvers Scott Brown UC Irvine This work is supported by a grant from the US Air Force Office of Scientific Research (AFOSR grant number FA )

Overview Experiments with dynamically changing environments –How do observers adapt their decision-making strategies? –How quickly can observers detect changes? Theory development –Bayesian ideal observers –Process models –Measure individual differences in adaptation

Part I Decision Criteria Adaption ( a quick overview from last year’s presentation)

Difficult Block Easy Block Decision Environment parameter Subject’s decision criterion Decision lag Easy Block Difficult Block Time Environments alternating between easy and hard blocks Time

Realistic 3D gaming tasks Decide which of two types of missiles is approaching How quickly can a participant adapt to changes in the similarity between the two missiles? A=“not puffy” B=“very puffy”

Modeling Results Developed a lagged signal detection model Model estimates that participants take an average of 9 trials to switch to a new decision criterion Individual differences

Part II Prediction and Change Detection

Change detection and prediction Prediction of future observations in time- series data Changepoint models –E.g. changing to different coins at random times Accurate prediction requires detection of change –stock market –“hot hand” players

Basic Task Given a sequence of random numbers, predict the next one

= observed data = prediction Two-dimensional prediction task Touch screen monitor 1500 trials Self-paced Same sequence for all subjects

Sequence Generation (x,y) locations are drawn from a binomial distribution of size 10, and parameter θ At every time step, probability 0.1 of changing θ to a new random value in [0,1] Example sequence: Time θ=.12θ=.95θ=.46θ=.42θ=.92θ=.36

= observed sequence Optimal Bayesian Solution = prediction Subject 4 – change detection too slow Subject 12 – change detection too fast (sequence from block 5)

Tradeoffs Detecting the change too slowly will result in lower accuracy and less variability in predictions than an optimal observer. Detecting the change too quickly will result in false detections, leading to lower accuracy and higher variability in predictions.

Average Error vs. Movement = subject Relatively many changes Relatively few changes

A simple model 1.Make new prediction some fraction α of the way between old prediction and recent outcome. α = change proportion 2.Fraction α is a linear function of the error made on last trial 3.Two free parameters: A, B A<B bigger jumps with higher error A=B constant smoothing α 0 1 A B B A

Average Error vs. Movement = subject = model

One-dimensional Prediction Task Possible Locations Where will next blue square arrive on right side?

Average Error vs. Movement = subject = model

Inference and Prediction Judgments Many errors in prediction –too much movement What is causing this? –Faulty change detection or faulty prediction judgments? –Gambler’s fallacy: people predict too many alternations New experiments: –ask directly about the generating process –inference judgment: what currently is the state of the system?

Tomato Cans Experiment Cans roll out of pipes A, B, C, or D Machine perturbs position of cans (normal noise) At every trial, with probability 0.1, change to a new pipe (uniformly chosen) (real experiment has response buttons and is subject paced) ABCDABCD

Tomato Cans Experiment (real experiment has response buttons and is subject paced) ABCDABCD Cans roll out of pipes A, B, C, or D Machine perturbs position of cans (normal noise) At every trial, with probability 0.1, change to a new pipe (uniformly chosen) Curtain obscures sequence of pipes

Tasks ABCDABCD Inference: what pipe produced the last can? A, B, C, or D? Prediction: in what region will the next can arrive? 1, 2, 3, or 4?

Experiment 1 63 subjects 12 blocks –6 blocks of 50 trials for inference task –6 blocks of 50 trials for prediction task –Identical trials for inference and prediction

INFERENCE PREDICTION Sequence ABCDABCD Trial

INFERENCE PREDICTION Sequence Ideal Observer ABCDABCD Trial

INFERENCE PREDICTION Sequence Ideal Observer Individual subjects Trial ABCDABCD

INFERENCE PREDICTION Sequence Ideal Observer Trial ABCDABCD Individual subjects

INFERENCE PREDICTION Sequence Ideal Observer Trial ABCDABCD Individual subjects

= Subject ideal INFERENCEPREDICTION

ideal INFERENCEPREDICTION = Process model = Subject

Varying Change Probability 136 subjects Inference judgments only  Subjects track changes in alpha ideal Prob. =.08 Prob. =.16 Prob. =.32

Number of Perceived Changes per Subject Low medium high Change Probability (Red line shows ideal number of changes) Subject #1

Number of Perceived Changes per Subject 55% of subjects show increasing pattern 45% of subjects show non- increasing pattern Low, medium, high change probability Red line shows ideal number of changes

Conclusion Adaptation in non-stationary decision environments Subjects are able to track changes in dynamic decision environments Individual differences –Over-reaction: perceiving too much change –Under-reaction: perceiving too little change

Potential deliverable outcomes Simple measurement models of decision making in dynamic environments. Classify different observers’ decision making abilities under different kinds of environments –E.g., does this observer respond too slowly to changes in their environment? –Is another observer close to the theoretically optimal decision mechanism? –Conversely, classify different observers as optimally suited to different tasks, depending on dynamic properties of decision making environments in those tasks.

Publications Brown, S.D., & Steyvers, M. (2005). The Dynamics of Experimentally Induced Criterion Shifts. Journal of Experimental Psychology: Learning, Memory & Cognition, 31(4), Steyvers, M., & Brown, S. (2005). Prediction and Change Detection. In: Advances in Neural Information Processing Systems, 19 Navarro, D.J., Griffiths, T.L., Steyvers, M., & Lee, M.D. (2006). Modeling individual differences using dirichlet processes. Journal of Mathematical Psychology, 50, Wagenmakers, E.J., Grunwald, P., & Steyvers, M. (2006). Accumulative prediction error and the selection of time series models. Journal of Mathematical Psychology, 50,