1. Lack of Learning or Lack of Studying? An Inquiry into Low Exam Scores Katherine M. Sauer Metropolitan State College of Denver ksauer5@mscd.edu February 19, 2010 21 st Annual Teaching Economics Conference Robert Morris University/McGraw Hill - Irwin

2. Outline I. background II. test assessment questionnaire III. meeting with students IV. participant discussion 2

3. 3

4. I. Background - “Why did I miss that test question?” - Revised it - Handed out every semester - This semester I gave incentive to complete it and meet with me to discuss it. - students responded very favorably - I gained insight I. background II. questionnaire III. meeting IV. discussion 4

5. A. My two reasons for trying this 1. Is it me or is it them? More and more time is devoted to assessment and assurance of learning. - design, revise rubrics - collecting data - analyzing This assumes that students are doing their part in the learning process. To what extent are they? I. background II. questionnaire III. meeting IV. discussion 5

6. 2. Are we on the same page? Student perceptions of “studying hard” differ from our perspectives as faculty. “I’ve never studied so hard in my life and I still didn’t do well!” It isn’t usually very effective to tell them to study harder. What are their study habits like? I. background II. questionnaire III. meeting IV. discussion 6

7. B. Benefits of this Method - Examining “does not meet expectations” - Way of demonstrating / communicating to students that I do care about student learning - Proactively promotes self-awareness and taking responsibility for one’s learning I. background II. questionnaire III. meeting IV. discussion 7

8. II. Test Assessment Questionnaire On the day I hand back the exam, I also hand out a copy of the Test Assessment. - I staple a copy to exams with grades of D or F. I explain that it is a tool for them to use if they would like to improve their grade and their learning. I explain that the answer to doing better is rarely as simple as “study harder”. If they take the time to fill out the test assessment, then I can give them individualized advice on how to improve. I. background II. questionnaire III. meeting IV. discussion 8

9. Test Assessment 1. Go through the questions you answered incorrectly on your exam. For each question, choose the reason you feel you got the answer wrong. Write the number of the question in the blank next to the reason. ~ Didn’t know a definition ~ Couldn’t apply a definition I knew ~ Didn’t read the question carefully / answered a different question than was asked I. background II. questionnaire III. meeting IV. discussion 9

10. ~ Knew the answer but couldn’t come up with it during the exam ~ Didn’t know how to set the problem up ~ Used the wrong formula ~ Just didn’t know the material ~ Other I. background II. questionnaire III. meeting IV. discussion 10

11. 2. Do you notice any patterns with the type of question you missed? Explain. 3. How do your exam scores compare with your assignment scores? If your exam scores are lower than your assignment scores, why do you think that is? 4. About how many times have you been to office hours to ask questions/clarify material? I. background II. questionnaire III. meeting IV. discussion 11

12. 5. How did you prepare for this exam? Did you… read the text? review your notes? review your in-class assignments? review your Aplia homework? complete the optional problem sets? other (make flash cards, friend quiz you, etc)? I. background II. questionnaire III. meeting IV. discussion 12

13. 6. How much time did you spend preparing for the exam? When did you seriously start studying for it and how much time did you spend? 7. In a regular week, how much time do you spend - reviewing your notes? - reading the text? - doing the Aplia? - doing optional problem sets? - reviewing previous weeks’ in-class assignments? I. background II. questionnaire III. meeting IV. discussion 13

14. 8. What letter grade do you hope to earn in this course? 9. To achieve your goal, what is your strategy for studying between now and the next exam? I. background II. questionnaire III. meeting IV. discussion 14

15. III. Meeting with Students The meetings usually take 5 to 10 minutes. 1. Small talk / thank them for coming by - students are often nervous 2. Ask them to tell me what they found out about their mistakes on the midterm - some confess right away to not studying at all for a particular reason (F students) - test anxiety / stress (D and C students) I. background II. questionnaire III. meeting IV. discussion 15

16. - patterns - definitions - applications - math problems - all the questions on the day they missed - second guessing For the A and B students, we usually end our discussion here. I tell them their current study methods seem to be working well and now they know what kind of question they’ve been missing. I. background II. questionnaire III. meeting IV. discussion 16

17. 3. I ask the students to tell me how they prepared for the exam Not adequate time/preparation - a few hours the night/week before - make flashcards the night/week before I explain how most brains don’t like to learn economics in huge doses and how studying for small periods of time more frequently seems to be a good way to make the material “stick”. - frequent 20 to 30 minute chunks are more manageable for students I. background II. questionnaire III. meeting IV. discussion 17

18. I also explain that students have a tendency to like to study what they already know. - skip over the confusing stuff to “come back to it later” - never come back to it I advise that in non-exam weeks they try the frequent, small chunks of time study method and then in an exam week increase the time. If they want to do flash cards, do them as we complete each chapter – not the week before the exam. I. background II. questionnaire III. meeting IV. discussion 18

19. 4. The discussion on exam preparation leads into how much time they are spending on the class in general. - not do homework - adult education - not do the reading - encounter the material in different ways - not spending any time other than homework I. background II. questionnaire III. meeting IV. discussion 19

20. 5. I ask what they plan to do differently between now and the next exam - optimistic about the small chunk studying - been meaning to work on their stress I advise they make a note whenever something doesn’t make sense and ask me. (much higher incidence of office hour visits after this meeting) 6. Thank them for stopping by / giving me insight into their learning experience. I. background II. questionnaire III. meeting IV. discussion 20

21. What students have to say… On the course evaluations: Please comment on the experience of completing the Test Assessment and/or meeting with me to discuss it. - It made me improve my studying skills. -Good, it gave me one on one time with you to figure out how I could improve. -Helpful in learning how I make mistakes and how to fix them better. I. background II. questionnaire III. meeting IV. discussion 21

22. - It helped me to personally analyze my test taking and study habits. Very helpful! I have never had a teacher who cared enough to take the time to help me improve in this aspect. -Your tips helped me in other classes and got me running again and I don’t feel as stressed. My mom thanks you. There was no negative feedback on this question. I. background II. questionnaire III. meeting IV. discussion 22

23. IV. Discussion A. Future Work - Collect data - Broaden scope? (final exam?) - Other faculty I. background II. questionnaire III. meeting IV. discussion 23

24. B. Questions for You How have you tackled the issue of using exam scores for assessment? What do you assume when a student gets a D or F on an exam? - not study? - not understand? Are the other questions that would be informative to add to the Test Assessment? (one page front and back) I. background II. questionnaire III. meeting IV. discussion 24

25. Comments? Questions? Katherine M. Sauer Metropolitan State College of Denver ksauer5@mscd.edu February 19, 2010 21 st Annual Teaching Economics Conference Robert Morris University/McGraw Hill - Irwin