1. Checking the ABC’S – Systematic Error Analysis in Student Assessments Scott Clark Molly Barrett CAIS 2007

2. Agenda History How we use this system Helpful hints for implementation Justification Practice What are others doing with this type of work?

3. History…Evolution of Error Coding Nov. 2005 article by Grant Wiggins and Stan Izen in Independent Teacher magazine on testing and assessment: –“…we need to make sure that students get feedback that they can use and profit from, [and] opportunities to use that feedback…” Four main categories: ABC’S

4. The Grid!

5. A - Assessment Day Issue Running out of time on test Misreading the directions or problem Making careless “OOPS!” errors Blanking out on a problem Not showing work! Not communicating ideas clearly Leaving problems blank by mistake

6. B = Background Topics/Skills Issue Forget math from previous years Forget math from previous chapters Weak in algebra skills

7. C = Current Topic/Skills Issue Missing basic facts - losing points on true/false, matching, or fill-in-blank Missing routine problems - losing points on problems that were just like homework and classwork only with different numbers Missing calculator skills - losing points for not knowing how to use the calculator (usually applies to graphing calculator skills)

8. S = Synthesis You are losing points on problems that present new challenges or situations. You are struggling to bring to bear your knowledge and talent on a problem that is slightly or even significantly different from those you saw in class.

9. How We Use This System Purpose explained to students in Sept. When grading test, teacher writes letter of type of mistake next to problem, and might circle where the mistake was made. Using coded test, the student completes test corrections. Part of the correction process - identify the specific type of error (e.g. A 2 ). When test corrections are turned in, 2 minutes are spent filling in the error coding grid. Teacher keeps grids until the next quiz/test.

10. Test Correction Guidelines  Complete all corrections on separate paper.  Mark error coding on each corrected problem (ex. A 2, B 1, etc..)  Correct each problem that had a mistake. Show all work.  Except for word problems, recopy all problems that are to be revised.  For word problems, recopy the question and the essential information (you do not need to copy the entire problem).  Revise the solutions so that they are correct.  Place a circle around revised answers.  At the end of each problem, describe in clear, written detail why the mistake occurred, and what has been learned in the revision process.

11. Sample Test Corrections:

12. This problem, I made a C2 mistake. I made a mistake on the original test because, instead of subtracting first, I divided. In the future, I will remember to use SADMEP on my problems.

13. Implementation Hints Copy grid and key on colored paper to make them easier to find later. Teacher should keep the grids so a consistent record can be kept. 3-4 times a year (or before midterm/final) have students make observations about any patterns in their errors. Refer students back to the advice part of the key – how to improve!

14. Justification

15. THINKING SKILLS CONTENT PROCESSES HABITS OF MIND FOUR LEVELS OF OUTCOMES: http://www.habits-of-mind.net/ppt/1BuildaThoughtfulLrnCommHOM.ppt. Costa & Kallick (2000). Discovering and Exploring Habits of Mind, Alexandria, VA: ASCD, p55. ACTIVITIES

16. Habits of Mind Persisting Managing impulsivity Listening with understanding and empathy Thinking flexibly Thinking about thinking (metacognition) Striving for accuracy Questioning & posing problems Applying past knowledge to new situations Thinking & communicating with clarity and precision Gathering data through all senses Creating, imagining & innovating Responding with wonderment & awe Taking responsible risks Finding humor Thinking interdependently Remaining open to continuous change

17. Habits of Mind PersistingPersisting Managing impulsivityManaging impulsivity Listening with understanding and empathy Thinking flexibly Thinking about thinking (metacognition)Thinking about thinking (metacognition) Striving for accuracy Questioning & posing problems Applying past knowledge to new situations Thinking & communicating with clarity and precision Gathering data through all senses Creating, imagining & innovating Responding with wonderment & awe Taking responsible risks Finding humor Thinking interdependently Remaining open to continuous change

18. Metacognition Thinking about thinking –Is this an A error or a C error? Students make the final call (but often need teacher oversight) –What kind of A error is this? –Are there patterns that I observe in the kinds of errors that I am making? –What can I do differently to improve my performance?

19. Student Perspectives Error coding helps me to see what I have done wrong on previous tests. This helps because I make a checklist in my mind of what I need to do, to make sure that I don’t make them again. - Kyle Error Coding: very useful because it tells me what mistakes I make most often. I had no idea that I made so many careless mistakes. Now I check my work more accurately. - Amara

20. Student Perspectives Error coding helps me realize what kinds of mistakes I make on test, and how I should fix them, instead of just getting points off with no explanation of why. - Victoria Error coding gives students a chance to see their accumulated mistakes without having to keep track of each test. - Kate

21. The Head-Royce Experiment in Grading Consistency (our own practice of metacognition)

22. Practice “Fake error coding” today: –Look at the problem and the sample response –Using the error coding key, write the letter of the type of mistake you think the student made –Talking is encouraged!

23. What are other schools doing with this type of work? Habits of Mind in Curriculum Maps (JRPO schools) Common scoring key (ex. HRS Algebra midterm) Checklists for self-monitoring (across curriculum, work habits) - check w/school learning specialists Apply similar system to homework Model hwk answers - what you want it to look like Oops errors - taking off 0 points Don’t tell students whether or not answer is correct - student must convince class of solution. Color coding grading - (green - correct; blue - can corrected for full credit; red - can be corrected for half credit) Bring note card to next test based on previous error coding. Include test-anxiety related section on grid

24. Evaluation Please give us feedback! Feel free to contact us: –Scott Clark: sclark@headroyce.orgsclark@headroyce.org –Molly Barrett: mbarrett@headroyce.orgmbarrett@headroyce.org