1. Using Item Response Theory to Track Longitudinal Course Changes
Charles H. Atwood, Braden Ohlsen, and Brock Casselman
University of Utah
Department of Chemistry
Salt Lake City, UT

2. Quick Description of Item Response Theory (IRT)
Classical Test Theory (CTT)
Mean, median, mode
Discrimination Index
comparison of top and bottom quartiles
CTT is good for small samples
CTT is more dependent upon the subject group
Same assessment given to 2 groups will give different results

3. Quick Description of Item Response Theory (IRT)
More modern psychometric analysis tool than CTT
Works well for large sample sizes (> 200)
Iterative process which fits the students responses to a mathematical formula

4. Quick Description of Item Response Theory (IRT)
We use the program Bilog-MG3 to fit our data.
Bilog-MG3 assigns to every test item values for a, b, and c and displays them on an item characteristic curve
Plot of the probability of a student with a given IRT ability vs. probability that the student answered the question

5. Quick Description of Item Response Theory (IRT)
Item Characteristic Curve

6. Description of Item Response Theory (IRT)
Item Information Curve for every test question

7. Description of Item Response Theory (IRT)
Test information curve
Sum of Item Information Curves for an assessment

8. IRT Equating

9. CHEM 1210 Equated Average Scores 2014 - 2016
Fall 2014
2015 Equated
to 2014
2016 Equated
Midterm Exam 1
70.3%
74.2%
74.0%
Midterm Exam 2
62.3%
68.1%
68.5%
Midterm Exam 3
66.9%
79.0%
78.5%

10. Equated Test 1

11. Equated Test 2

12. Equated Test 3

13. ACS FINAL EXAM CHANGES FROM 2013 TO 2016

14. CHEM 1220 Equated Average Scores 2015 - 2016
Spring 2014
Spring 2015 Equated to 2014
Spring 2016 Equated to 2014
Midterm Exam 1
66.7%
75.9%
83.7%
Midterm Exam 2
62.9%
73.4%
78.5%

15. Equated Test 1

16. Equated Test 2