Cognitive feedback Public Administration and Policy PAD634 Judgment and Decision Making Behavior Cognitive feedback Thomas R. Stewart, Ph.D. Center for Policy Research Rockefeller College of Public Affairs and Policy University at Albany State University of New York T.STEWART@ALBANY.EDU

Alternative to outcome FB (knowledge of results) CFB is a way of learning to improve accuracy of judgment under uncertainty Alternative to outcome FB (knowledge of results) A. Difficult to learn with OFB because of causal ambiguity B. OFB may actually deter learning PAD634

Types of cognitive feedback A. Task information (TI) B. Cognitive information (CI) C. Functional validity information (FVI) PAD634

A. Task information (TI) Relations between cues and distal variable 1. Task uncertainty (Re) 2. Relations between each cue and distal variable (weights and function forms) 3. Cue intercorrelations 4. Frequency distribution of distal variable PAD634

B. Cognitive information (CI) 1. Judgmental consistency (Rs) 2. Relations between each cue and judgment (weights and function forms) 3. Cue intercorrelations (subjective) 4. Frequency distribution of judgments PAD634

C. Functional validity information (FVI) 1. ra 2. G 3. C PAD634

Review papers Kluger, A. N., & DeNisi, A. (1998). Feedback interventions: Toward the understanding of a double-edged sword. Current Directions in Psychological Science, 7(3), 67-72. Balzer, W. K., Doherty, M. E., & O’Connor, J. (1989). Effect of Cognitive Feedback on Performance. Psychological Bulletin, 106(3), 410-433. PAD634

Feedback Interventions (FI) Kluger & DeNisi (1998) Feedback Interventions (FI) Includes, but not limited to, CFB Widely used (performance appraisals, grades, teaching evaluations) Has variable effects on performance (positive, none, negative) Reasons not understood PAD634

Feedback Intervention Theory Kluger & DeNisi (1998) Feedback Intervention Theory (a) Behavior is regulated by comparisons of feedback with goals or standards (and the identification of gaps between the two). (b) Attention is limited, and only those feedback-standard gaps that receive attention actively participate in behavior regulation. (c) FIs change the locus of attention and therefore affect behavior. PAD634

Sometimes responses to feedback are symmetric, sometimes they are not Kluger & DeNisi (1998) (a) Behavior is regulated by comparisons of feedback with goals or standards (and the identification of gaps between the two). Symmetry? p. 69. …most cognitive treatments of the process of discrepancy reduction are indifferent to the valence (positive vs. negative) of the discrepancy. That is, these views suggest that effects are symmetrical, and that both a positive discrepancy and a negative discrepancy yield a self-regulatory action that is a function of the absolute magnitude of the discrepancy. e.g., positive and negative gaps (overshooting and undershooting) or reward and punishment Sometimes responses to feedback are symmetric, sometimes they are not Two response systems (analogous to system 1 and system 2)? One responds symmetrically One responds asymmetrically p. 69 However, most cognitive treatments of the process of discrepancy reduction are indifferent to the valence (positive vs. negative) of the discrepancy. That is, these views suggest that effects are symmetrical, and that both a positive discrepancy and a negative discrepancy yield a self-regulatory action that is a function of the absolute magnitude of the discrepancy. Similarly, behaviorism (Thorndike, 1927) has symmetrical predictions, in that rewards and punishment can produce learning equally. Other theorists have argued, however, that the reaction to positive and negative events is vastly different (cf. Taylor, 1991). That is, they contend that the direction of the feedback-standard discrepancy has major consequences, that reinforcement and punishment have different and asymmetric effects on behavior (Taylor, 1991). PAD634

Attention can be directed to self. Kluger & DeNisi (1998) (b) Attention is limited, and only those feedback-standard gaps that receive attention actively participate in behavior regulation. (c) FIs change the locus of attention and therefore affect behavior. Attention can be directed to self. This depletes the cognitive resources necessary for task performance. e.g., praise or destructive criticism can direct attention to self and has a negative effect on performance Attention should be directed to the task. PAD634

Task Properties Task mastery (subjective difficulty) Kluger & DeNisi (1998) Task Properties Task mastery (subjective difficulty) Task complexity (objective difficulty) FI’s that direct attention to self on complex tasks are bad. FI’s that direct attention to self on simple tasks may be good (similar to social facilitation--presence of other people). Effects of FI’s are more positive as the task becomes more subjectively familiar or more objectively simple. “Ironically, people who need feedback the most benefit the least from it.” PAD634

Task Properties: Hypothesis Kluger & DeNisi (1998) Task Properties: Hypothesis Two cognitive systems: rational or rule-based association-based or experiential-based “The rational system may be more susceptible to resource depletion, and hence tasks that are largely dependent on this system may be more susceptible to negative effects on performance” (p. 71) PAD634

Practical Implications Kluger & DeNisi (1998) Practical Implications Use FI’s only in combination with a goal setting intervention. FI’s then direct attention to the task. Goal-setting intervention augments FI’s effect on performance. Employee’s who want more feedback than they are getting often suffer from the absence of clear goals. Providing FI without clear goals increases the risk that the goals will not be the ones intended. PAD634

Balzer, Doherty, & O’Connor (1989) Cognitive Feedback Research shows that it improves performance in judgment task. They don’t discuss why it works, but one reason could be that it directs attention toward the task and away from the self. Another could be that it provides information about the task that is useful in making judgments (TI-baseball task example). Another could be that it provides useful insight into the judgment policy (CI) that is not available to the person. PAD634

Is Cognitive Information Necessary? Balzer, Doherty, & O’Connor (1989) Is Cognitive Information Necessary? No It may be that self insight is better than reported in many studies. PAD634

Are human judges necessary? Balzer, Doherty, & O’Connor (1989) Are human judges necessary? Linear model more reliable and therefore more accurate, but... People have ability to select and code information. Exclusion of people might lead decision makers to distrust the process. PAD634

Factors influencing effectiveness of CFB Balzer, Doherty, & O’Connor (1989) Factors influencing effectiveness of CFB Task complexity. One study found CFB more effective for more complex tasks, another did not. Task predictability? Not much evidence, but it should help more in low predictability tasks. Experience. One study found CFB more effective for more experienced judges. PAD634

Potential Applications Balzer, Doherty, & O’Connor (1989) Potential Applications Medical education Assessment of clinical judgment (NBME study) Personnel interviewers Stockbroker trainees Clinical psychologists Promising, but few, if any applications have been made. PAD634

Why doesn’t CFB work better in teaching medical judgment tasks Why doesn’t CFB work better in teaching medical judgment tasks? (Wigton, 1996, p. 187) Unlike lab studies, some task info already known. Even students have a good idea about direction and importance of cues. Precise cue weights may not make a difference. CFB might be more valuable for rare diseases, but the data needed would be difficult to obtain. Improved calibration from probability feedback is important. PAD634