1. Exam Analysis Camp Teach & Learn May 2015 Stacy Lutter, D. Ed., RN Nursing Graduate Students: Mary Jane Iosue, RN Courtney Nissley, RN Jennifer Wierworka, RN

2. Objectives Understand the interpretation of exam reliability. Understand how to analyze test questions using point-biserial measures. Understand the relationship between student performance and exam analysis.

3. Nursing and NCLEX Uses a series of exam items that can include interpretation of charts, tables, or figures. Questions are multiple choice, multiple- response, order response items, hot-spot questions, or fill-in-the-blank. Examples of NCLEX questions should be used during nursing programs to assist students understanding of how to answer each question.

4. What was the overall performance? Mean – average of test scores of all students – Influenced by extreme scores Median – score at which point half the class scored higher and half the class scored lower – not influenced by extreme scores Discussion Question: What is a typical mean or median on your exams? What do you hope it will be?

5. What is the Reliability? Evaluates the consistency and accuracy of test scores – What is the probability that the same person would earn a similar score on the same exam at a different time? KR20 – Statistical measure of reliability – Appropriate when item difficulty levels are not the same – Can range from -1 to +1 (-1=totally unreliable, +1=totally reliable) – Acceptable reliability for teacher made tests is 0.65 or higher

7. Analyzing Test Items Item-analysis statistic terminology: – Difficulty Index (P-value) – percentage of students who choose the correct answer to a question.

9. Item Analysis Procedure looks at upper/lower 27% Test items answered correctly by <30% are too difficult – 25% of students could guess answer correctly on 4 choice question Test items answered correctly by >90% are usually too easy

11. Mastery test items Purpose is to reinforce content not make up more than 10% of an exam All items answered by more than 90% are not mastery items Examples in Nursing: drug calculation, skills questions, signs/symptoms of disease Discussion Question: What do you consider mastery test items in your discipline?

12. Point Biserial Analysis Can be + or -, ranges from 0-1 Point Biserial Interpretation: >0.3=excellent item 0.20-0.29=Good item 0.15-0.19=acceptable item 0.10-0.14=marginal item <0.09=poor item

14. Example of a Poor Multiple Choice Question The primary colors are: a.Red, yellow, blue b.Orange, green, turquoise, pink c.Green, orange d.None of the above Correct answer: A

15. Multiple Choice Question Revision What are the primary colors? a.Red, yellow, green b.Green, blue, orange c.Orange, blue, red d.Red, blue, yellow Correct answer: D

16. Distractor Analysis Were there nondistractors? – Work to edit nondistractors that weren’t selected – If <30 students, may be acceptable to have some nondistractors