CONFIDENCE – ACCURACY RELATIONS IN STUDENT PERFORMANCES We attempted to determine students’ ability to assess comprehension of course material. Students took a “practice exam” before their actual exam; each had questions about confidence in performance rating. Students accurately predicted practice exam performance and some in-class exam performance. An extra lab experiment was used to see if there was a correlation between the lab results and real exam scores. The lab results were found inconclusive and uncorrelated with the exam scores. Students rated future recognition performance unreliably. After recognition tests they made accurate estimates. ALLEN A. NEWTON & JOHN M. ACKROFF, Ph.D., RUTGERS UNIVERSITY – NEW BRUNSWICK Method: Research Participants: Students from two sections of an undergraduate course in Cognition at Rutgers University participated in these experiments for extra credit. Procedure: I. Lab Setting: A set of 49 words from the University of South Florida norms (Nelson et al., 2004), was chosen as a study list for students. The first day, students were asked to remember a 4 to 6 letter word and then asked how memorable each word was from a scale of 1 through 5. On day two (two days later), students again saw the word list and re-rated each word. After the word list was studied, students were given a distracter task rating the similarity of various random shapes. A forced choice test consisting of each target along with a high, medium, or low associate was then given. (For example, for the target DRAFT, the high, medium, and low associates were BEER, ARMY, and WIND, respectively.) Students returned 5 days later for a second recognition test. At the end of each recognition test, the students were asked “How many words do you think you recognized correctly?”, with choices of 90%, 80%, 70%, 60% and less than 60%. They were also asked to indicate how accurate they thought that prediction was. II. Practice and in-course exams: Practice exams were available to students before the the actual exam. Each practice exam had 20 questions which were followed by a confidence of the question and a “remember-know” question. The remember-know question asked the students where they remembered learning the answer. The actual exam consisted of 20 questions along with two follow- up questions at the end of the test. One question asked ths student to estimate the number of questions answered correctly. The other question asked how confident they were of that prediction. Results: There was a strong relationship between practice exam scores and prediction confidence (r = 0.360, df = 139, p <.001). Students accurately predicted actual exam scores when they were “moderately confident” (r = 0.334, df = 66, p <.01) or “fairly confident” (r = 0.377, df = 30, p <.05). Students who were “guessing” or thought they “may be right” were unable to make accurate predictions of their test scores on the actual exam. Students who were “certain” of their predictions tended to be accurate, although the results did not achieve significance. Students were more accurate on their first recognition test (Day 2) than on the second (Day 3) (t = 6.227, df=37, p <.001). The partial correlation between accuracy and retrospective predictions was significant (r = 0.443, df = 35, p <.05) on the first test when confidence was used as a control factor, but not on the second. Accuracy was better for the first test than for the second (t = 7.106, df = 37, p <.001). Discussion: When studying for exams students can sometimes find it hard to assess their knowledge of material before taking an actual exam. However, research has shown that students are able to recognize the difference between their accuracy and confidence in test taking. Once students can make accurate assessments of their confidence and accuracy they are able to determine which course materials to study. We have found that students are able to distinguish material they do not know and the material they do know based on their confidence rankings. Using the “retrospective confidence ratings” of Busey et al. (2000), (confidence rankings collected at the end of the recognition test), we found accurate assessments for each test. When confidence in the assessment was controlled for, however, the predictions were accurate for the first test only. Also, a rereading effect occurred from the first day to second day. Given that students were able to “re-read” the study list and re-rate their confidence, they predicted and performed better on the first lab exam than on the second lab exam. A 10% decrease in their prediction rate also suggests that the re-reading improves accuracy, suggested by Rawson and Dunlosky (2000). The practice exam was given to students to test their knowledge of the course material before an actual exam. The practice exam was a form of “prospective confidence rating”, and students’ performance on the practice exam was a good predictor of performance on the in-class exam. However, students’ “remember-know” judgments were unrelated to any other measure. References: Busey, T. A., Tunnicliff, J., Loftus, G. R. & Loftus, E. F.. (2000). Accounts of the confidence-accuracy relation in recognition memory. Psychonomic Bulletin and Review, 7, Nelson, D. L., McEvoy, C. L. & Schreiber, T. A. (2004). The University of South Florida word association, rhyme, and word fragment norms. Behavior and Research Methods, Instruments, & Computers,36, Rawson, K. & Dunlosky, J. (2000). The rereading effect: Metacomprehension accuracy improves across reading trials. Memory and Cognition, 28(6), Figure 3: Accuracy and confidence on the lab recognition tests Figure 2: Accuracy and Prediction on the lab recognition tests Figure 1: Accuracy and prediction on the in-class exam People are frequently asked to make decisions about how confident they are in judgments they have made. Studies in eyewitness identification have shown that confidence is not always related to accuracy. Most people have had the experience of being sure they were right about something, only to be proven wrong. Students are called upon to make decisions about the degree to which they know course material on a regular basis. Decisions about how and what to study, and for how long, are based on a perception of how well one understands the material for upcoming exams. Students who feel they are well-prepared sometimes receive feedback to the contrary – they perform poorly on exams. We decided to get a measure of how well students could predict how well they performed on classroom exams and laboratory recognition tests, and whether there was any relationship between their abilities to predict performance on these two seemingly unrelated tasks.