SUNY at Albany System Dynamics Colloquium, Spring 2008 Navid Ghaffarzadegan Effect of Conditional Feedback on Learning Navid Ghaffarzadegan PhD Student, the State University of New York at Albany MIT-Albany-WPI System Dynamics Colloquium, Spring 2008

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Introduction barriers to learning from feedback in a dynamic decision making environment: Complexity of the environment (Gonzalez 2005) Misperception of delays (Rahmandad et al. 2007, Rahmandad 2008) Feedback asymmetry (Denrell and March 2001), The existence of noise in feedback (Bereby-Meyer and Roth 2006), Problems of mental models (Senge 1996) … People ignore and misperceive feedback (Sterman 1989a, Sterman 1989b).

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Introduction A common theme in formal studies on learning a decision maker makes a decision and receives a payoff the question is whether or not the decision maker is capable of learning from the information. Full Feedback Decision Payoff Perceiving payoff LEARNING

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Introduction Little attention has been paid to the relevance of such an assumption. eg: Police Officer, Admission Office, Human Resources Manager Conditional Feedback For positive decisions we perceive feedback much easier than for negative decisions Decision Payoff Perceiving payoff LEARNING

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Research Problem What is the effect of conditional feedback on learning Or How relevant was the assumption of Full Feedback? Method: 1- Simulation. 1- Build a differential equation model in Signal Detection Framework 2- Experiment with the model 2- Test with data Second hand data: a published laboratory experiment

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Framework Signal Detection Theory Signal vs. Noise e.g. Guilty vs. Innocent, e.g. capable vs. incapable candidates Decision makers try to differentiate signals from noise Judgment and Decision Making Evidence is often ambiguous, and there is uncertainty in the environment (Hammond 1996, Stewart 2000)

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Framework Important concepts: Base rate – selection rate – d – threshold Payoff Threshold Learning

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Framework Conditional Feedback Threshold Learning (Cue Learning)

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback Set threshold make decision Receive Payoff Perceive Payoff correct threshold One stock: Threshold (experiment)

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback Learning Algorithm: 1. Learning from payoff shortfall: Payoff shortfall= maximum possible payoff (Q) – payoff (Q, d) maximum possible payoff (Q)= Vtn+ Q*(Vtp-Vtn) 2. Anchoring and adjustment assumption in correcting the threshold (Tversky and Kahneman 1974, Epley and Gilovich 2001, Sterman 1989.b)

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback Inputs: noise ~ N(0,1) and signal ~ N(d,1) Base rate = 0.5, values are symmetric, (To make Base rate traceable) correct decisions are more valued

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback (learning from Payoff Shortfall) Payoff shortfall=0 Payoff shortfall > 0

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback noise ~ N(0,1) and signal ~ N(d,1) Base rate = 0.5, values are symmetric, correct decisions are more valued

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback - Results In full feedback; the model is able to learn from feedback

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback - Results In full feedback; the model is able to learn from feedback Looking at payoff shortfall in enough to learn threshold The speed of approaching depends on the time to change threshold Dynamics of selection rate for base rates of 0.3 and 0.7

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model II: Conditional Feedback Our decision influence our payoff perception. How do we judge our negative decision's payoff. People can be different in interpreting their negative decisions. (Personality, second loop learning,..)

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model II: Conditional Feedback Constructivist strategy For negative decisions: perceived payoff= payoff (p,0) p = ratio of signals to total decisions for negative decisions p=0 means assuming all of our negative decisions are right. p=1 means assuming all of our negative decisions are wrong. P

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback For negative decisions: perceived payoff= payoff (p,0) p=ratio of signals to total decisions for negative decisions p=0 means assuming all of our negative decisions are right. p=1 means assuming all of our negative decisions are wrong.

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback For negative decisions: perceived payoff= payoff (p,0) p=ratio of signals to total decisions for negative decisions p=0 means assuming all of our negative decisions are right. p=1 means assuming all of our negative decisions are wrong.

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model II: Conditional Feedback - Results In conditional feedback; learning depends on how people code their negative decisions Elwin et. al (2007): p=0 Stewart et. al (2007): people underestimate selection rate and overestimate threshold

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback - Results Comparison of full feedback and conditional feedback in confident constructivist strategy Dynamics of selection rate in last 50 trials (a) and threshold (b) for base rate of 0.5

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Model I: Full Feedback - Results In conditional feedback and in confident constructivist strategy; the model is not able to learn from feedback What is the real p? How do really people code their negative decisions? Dynamics of selection rate for base rates of 0.3 and 0.7

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Replications of an empirical investigation Data from Elwin et. al (2007) Comparison of Full Feedback and Conditional Feedback Sixty four subjects performed a computerized task of predicting economic outcomes for companies The experiment had two major phases: training trials test phase In the training part, a group of subjects performed 120 trials of full feedback decision making, while the other group performed 240 trials of conditional feedback.

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Replications of an empirical investigation We use their published report in our model and test parameters that can replicate their findings. Main parameters: d and p. (and time to adjust threshold)

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Conclusion A new explanation for imperfectness of decision making in a series of tasks. (learning from clear shortfalls) Conditional feedback can result in bias and underestimation of the base rate. In respect to second loop learning: People do not find the optimal threshold even if, in the real world, second loop learning exists, it works for a limited number of people Warning about overestimation of relevance of full feedback assumption in formal studies.

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Conclusion Future works: Effects of personality traits Effect of Personality on Learning, e.g. using Big Five

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Conclusion Future works: Making confidence endogenous. Effect of Personality on Learning, e.g. using Big Five Dynamics of confidence building

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Conclusion Future works: Two individuals communicating

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Conclusion Future works: Two individuals influencing each others performance

SUNY at Albany Navid Ghaffarzadegan System Dynamics Colloquium, Spring 2008 Thanks FEEDBACK?