INTRODUCTION TO ITEM RESPONSE THEORY Malcolm Rosier Survey Design and Analysis Services Pty Ltd web: Copyright © 2000

IDEAS OF MEASUREMENT A trait is a concept not an entity: for example, temperature and weight A trait is not measured directly but by using an instrument

IDEAS OF MEASUREMENT: 2 The underlying trait should have a linear relationship to the measuring instrument : for example temperature has a linear relationship to the expansion of mercury inside a tube of glass weight has a linear relationship to the stretching of a spring (Hookes Law) Interval measurement a given change in the measuring instrument produces the same change in the trait at any point in the measurement range

Problems with traditional measurement Item difficulties depend on the ability of the group used for calibration. Since item facilities (proportions) are bounded by zero and one, they cannot form a scale at the interval level of measurement. Since items are not at the interval level of measurement, parametric statistics should not be used: means, standard deviations and statistics that depend on them. A total test score is obtained by simple summation: for example, Likert summation. This may not be justified

Advantages of item response theory Produces scales at the interval level of measurement. Assesses the dimensionality of scales. Measures the error associated with each case. Measures the consistency of the pattern of responses for each case. Enables persons to be measured using different sets of items. Handles cases with missing data.

A CONCEPTUAL VIEW OF ITEM RESPONSE THEORY The following picture offers a conceptual approach to item response theory. The dimension being measured is represented by a vertical scale at the interval level of measurement. The right hand side shows the items used for calibration. The left hand side shows the cases (persons) being measured.

ITEM RESPONSE THEORY: TEMPERATURE ANALOGY Dimension: temperature (central vertical line)

ITEM RESPONSE THEORY: TEMPERATURE ANALOGY Measurement is at the interval level The units used for calibration are arbitrary. They may be transformed to any convenient metric. The same units are used for calibrating and measuring.

PERSON - ITEM MATRIX The data needed for carrying out the item response theory procedure are arranged in a person - item matrix Persons are arranged by their scores on the trait (such as ability) Items are arranged by difficulty The entries in the matrix show the response of an encounter between a person of given ability and an item of given difficulty. For a dichotomous response (as in a mathematic test) the responses are: 1 = correct response 0 = incorrect response

Person - item matrix

THE ITEM RESPONSE THEORY MODEL A probability is associated with each encounter in the person - item matrix b is the person’s ability d is the difficulty of the item p(1) is the probability of a correct response If b > d, p(1) tends to 1.0 If b < d, p(1) tends to 0.0 If b = d, p(1) = 0.5 These probabilities can be met with the Rasch model: p(1) = exp (b-d) / [ 1 + exp (b-d) ]

ITEM RESPONSE THEORY: PARTIAL CREDIT MODEL Central line = scale at interval vel of measurement

PARTIAL CREDIT MODEL: ITEMS Item 1.3 refers to Item 1 Category 3 Item 1.2 refers to Item 1 Category 2 and so on. The difficulty associated with Category 3 of Item 1 is greater than the difficulty associated with Category 2 of Item 1, and so on (ordered categories) The location of Item 1.3 on the scale indicates the ability associated with a 50% probability of passing Category 3 of Item 1 (or of any of the lower categories).

PARTIAL CREDIT MODEL: PERSONS The location of person A on the scale indicates his ability. The probability of Person A passing categories at a lower level of difficulty is more than 50%. The probability of Person A passing categories at a higher level of difficulty is less than 50%. The probability of Person A passing categories at a level of difficulty that is the same as his ability is 50%.