Introduction to Item Analysis Objectives: To begin to understand how to identify items that should be improved or eliminated.

Item Analysis No item is perfect. No item is perfect. An item might be ambiguous, too simple, too difficult, or non-discriminating. An item might be ambiguous, too simple, too difficult, or non-discriminating. Non-discriminating means the item can not be used to measure individual differences on the trait that is measured by the test. Non-discriminating means the item can not be used to measure individual differences on the trait that is measured by the test.

Achievement Tests Item analysis can help diagnose student errors. Item analysis can help diagnose student errors. It can help improve the quality of tests. It can help improve the quality of tests. It can lead to instructional improvements. It can lead to instructional improvements.

Achievement Tests Item analysis can identify problems with the answer key on a teacher-made test, or problems with the machine scoring on a standardized test. Item analysis can identify problems with the answer key on a teacher-made test, or problems with the machine scoring on a standardized test. It can help isolate items where the students “guessed” a lot. It can help isolate items where the students “guessed” a lot.

The Basic Indexes of Item Analysis Difficulty – What percentage of respondents got the item “right” or indicated that they possess the trait being measured? Difficulty – What percentage of respondents got the item “right” or indicated that they possess the trait being measured? Discrimination – The extent to which the item differentiates between persons with high and low scores on the test. Discrimination – The extent to which the item differentiates between persons with high and low scores on the test.

The Basic Indexes of Item Analysis Difficulty – Measured by a simple percentage. Difficulty – Measured by a simple percentage. Discrimination – Measured by the difference between high and low scoring groups on the proportion answering the “right” answer. Discrimination – Measured by the difference between high and low scoring groups on the proportion answering the “right” answer.

Discrimination Items that are poor discriminators should be eliminated or modified. Items that are poor discriminators should be eliminated or modified. Balance content validity with construct validity. Balance content validity with construct validity. High discrimination tends to increase reliability. High discrimination tends to increase reliability.

Discrimination and Difficulty Discrimination and difficulty are related. Discrimination and difficulty are related. With very difficult items it is harder to show high discrimination. With very difficult items it is harder to show high discrimination. Balance purpose of assessment with the range of difficulty of items. Balance purpose of assessment with the range of difficulty of items. Generally difficulty is desired. Generally difficulty is desired.

Discrimination For educational achievment tests, you want to look at the discrimination for the “distractors” or wrong options on a multiple choice item. For educational achievment tests, you want to look at the discrimination for the “distractors” or wrong options on a multiple choice item. Ideally, you want them to be selected mostly by the low scoring respondents. Ideally, you want them to be selected mostly by the low scoring respondents.

Discrimination For Educational tests – For Educational tests – Form a 2 x 2 matrix that crosses “Right” vs. “Wrong” on the item by “High” vs. “Low” on the total score of the test. Form a 2 x 2 matrix that crosses “Right” vs. “Wrong” on the item by “High” vs. “Low” on the total score of the test. High and Low can be determined by a median split, or by quartiles, taking the highest and lowest quartile. High and Low can be determined by a median split, or by quartiles, taking the highest and lowest quartile.

Discrimination For Psychological tests – For Psychological tests – Form a 2 x 2 matrix that crosses “High” vs. “Low” on the item by “High” vs. “Low” on the total score of the test. Form a 2 x 2 matrix that crosses “High” vs. “Low” on the item by “High” vs. “Low” on the total score of the test. High and Low can be determined by a median split, or by quartiles, taking the highest and lowest quartile. High and Low can be determined by a median split, or by quartiles, taking the highest and lowest quartile.

An Example from the PRI “I am able to ask for emotional support.” “I am able to ask for emotional support.” Part of the Social Resourcefulness Factor Part of the Social Resourcefulness Factor Part of the Assistance in Relationships subscale. Part of the Assistance in Relationships subscale.

An Example from the PRI