Seminar on Grading (1/24/2002): Pg 1www.physics.uiuc.edu The Problem with Exams The Problem with Exams M. Selen, UIUC Physics l Why exams are a sensitive issue… l Hand graded versus multiple choice. l How we do it in the intro Physics courses. l What is the cost?

Seminar on Grading (1/24/2002): Pg 2www.physics.uiuc.edu èExams typically represent > 50% of the final course grade. èStudents really focus on exams (perhaps too much). »Any perceived problems (real or imagined) will generate enormous resentment. èFaculty often don’t focus on exams. »Doing it right is time consuming, in particular of you are working alone. è“Fairness” is not a trivial issue. »Exam content. »Grading.

Seminar on Grading (1/24/2002): Pg 3www.physics.uiuc.edu l Hand Grading (what we used to do): l PROS: èStudents feel like everything they write is taken into account (i.e. partial credit). èAny mistakes in the exam can be adapted to (exam is more forgiving to the professor…less quality control is needed). l CONS: èHard to make grading fair & consistent. »TA, Handwriting, time, before/after pizza etc.. »Whiners are rewarded (i.e. re-grades). Hand Graded vs. Multiple Choice

Seminar on Grading (1/24/2002): Pg 4www.physics.uiuc.edu l Multiple Choice (what we do now): l PROS: èUniform & Fair. Everyone is treated exactly the same. »Fixing a bug in a problem help everyone. èLends itself to electronic publishing. »WEB interface possible for practice (before exam night) and help/explanations (after exam). èVery useful for analysis. »tracking changes, education research, exam problem quality control, problem “bank” etc. l CONS: èHarder to give partial credit (but not impossible…). èMore care is needed when preparing exam. èConsidered “inferior” by some (mostly faculty).

Seminar on Grading (1/24/2002): Pg 5www.physics.uiuc.edu About 1/3 of exam score is conceptual (2 & 3 choice) Quantitative problems (5-choice) allow students to select up to 3 answers. Partial credit ! Conceptual and quantitative problems are often paired.

Seminar on Grading (1/24/2002): Pg 6www.physics.uiuc.edu Analysis of exam “data” is very interesting (and useful for education research). Physics 101 Midterm Exam 1, Spring 2000

Seminar on Grading (1/24/2002): Pg 7www.physics.uiuc.edu More sophisticated analyses can be used to rate the effectiveness of each exam questions: not so good good

Seminar on Grading (1/24/2002): Pg 8www.physics.uiuc.edu We can look at the discriminating power of each problem: Not so good (weak correlation with the rest of the exam) Good (strong correlation with the rest of the exam) We can learn, quantitatively, how to build better exams.

Seminar on Grading (1/24/2002): Pg 9www.physics.uiuc.edu Instant exam feedback is possible: l The minute they leave the exam, students can go on the web, enter their answers into a web version of the exam they just took, and see what their raw score is: l After the exam has been graded (next day) students can find detailed statistics on each problem on the web. Students LOVE this !

Seminar on Grading (1/24/2002): Pg 10www.physics.uiuc.edu The Cost (is it more work?) l These courses are team-taught. èTypically 3 faculty »Lecturer, Discussion director, Lab director l The team course works together to produce exam. èThe subject material is divided up and each faculty submits a set of problems. »Old problems can be used for guidance. »Format is fixed (MS word in our case). èOne of the faculty is in charge of assembling exam (secretarial staff can help). èThe team meets several times to discuss & iterate the problems until the final draft is ready. »A senior TA works the exam out and provides comments on difficulty, length etc. èThree version of exam are produced by scrambling the order of the problems. »Secretaries do this. èAfter the exam, OIR “machine reads” the scantron forms and gives us the “raw data”. We have scripts that do the final analysis to yield grades etc.

Seminar on Grading (1/24/2002): Pg 11www.physics.uiuc.edu l We are getting good at this. Exam averages are consistently 70-75%. èCurving is unusual. èWe never curve down. èWe can honestly tell students that they are not competing against each-other. »Everyone could, in principle, get an “A” in Physics 11x. »This is a great motivator. èWe can tell students on the first day of the semester what final semester score they need to get the various letter grades: From Physics 101 web page