ARDHIAN SUSENO CHOIRUL RISA PRADANA P. ITEM ANALYSIS ARDHIAN SUSENO CHOIRUL RISA PRADANA P.

ITEM ANALYSIS is a process which examines student responses to individual test items (questions) in order to assess the quality of those items and of the test as a whole SCOREPAK®

PURPOSE To improve items which will be used again in later tests. To eliminate ambiguous items in a single test administration. To increase instructors' skills in test construction. To identify specific areas of course content which need greater emphasis or clarity.

Method 3. Item validity (point biserial method) ITEM DIFFICULTY ITEM DISCRIMINATIO ITEM CHARACTERISTIC CURVE 3. Item validity (point biserial method) 4. Effectiveness of distractors

I. ITEM DIFFICULTY item difficulty is determined by the number of people who answer a particular test item correctly (p).  For example, if the first question on a test was answered correctly by 76% of the class, then the difficulty level (p or percentage passing) for that question is p = .76. If the second question on a test was answered correctly by only 48% of the class, then the difficulty level for that question is p = .48. The higher the percentage of people who answer correctly, the easier the item, so that a difficulty level of .48 indicates that question two was more difficult than question one, which had a difficulty level of .76.

Method of Item Difficulty Method for Dichotomously Scored Item Method for Polytomously Scored Item Grouping Method

a). Method for Dichotomously Scored Item Method for Dichotomously Scored Items p is the difficulty of a certain item. R is the number of examinees who get that item correct. N is the total number of examinees.

Example 1 There are 80 high school students attending a science achievement test, and 61 students pass item 1. Please calculate the difficulty for item 1. Answer : 0.7

b). Method for Polytomously Scored Items , the mean of total examinees’ scores on one item , the perfect scores of that item

Example 2 The perfect scores of one open- ended item is 20 points, the average score of total examinees on this item is 11 points. What is the item difficulty? Answer : 0.55

c). Grouping Method (Use of Extreme Groups) Upper (U) and Lower (L) Criterion groups are selected from the extremes of distribution of test scores or job ratings is the proportion for examinees of upper group who get the item correct. is the proportion for examinees of lower group who get the item correct.

Example 3 There are 370 examinees attending a language test. Known that 64 examinees of 27% upper extreme group pass item 5, and 33 examinees of 27% lower extreme group pass the same item. Please compute the difficulty of item 5. Answer : 0.49

II. ITEM DISCRIMINATION Item discrimination refers to the degree to which an item differentiates correctly among test takers in the behavior that the test is designed to measure. item discrimination determines whether those who did well on the entire test did well on a particular item.

Method of Item Discrimination a). Index of Discrimination D = pH - pL (7.5) We need to set one or two cutting scores to divide the examinees into upper scoring group and lower scoring group. pH is the proportion in the upper group who answer the item correctly and pL is the proportion in the lower group who answer the item correctly. Values of D may range from -1.00 to 1.00.

ILLUSTRATION BY GGS INFORMATION SERVICES. CENGAGE LEARNING, GALE.

Another example 50 Examinees’ Test Data on 8-Item Scale About Job Stress.

Question: There are 140 students attending a world history test. (1) If we use the ratio 27% to determine the upper and lower group, then how many examinees are there in the upper and lower group separately? (2)If 18 examinees in upper group answer item 5 correctly, and 6 examinees in lower group answer it correctly, then calculate the discrimination index for item 5.

ITEM VALIDITY :POINT-BISERIAL METHOD ILLUSTRATION BY GGS INFORMATION SERVICES. CENGAGE LEARNING, GALE.

Another way to determine the discriminability of an item is to determine the correlation coefficient between performance on an item and performance on a test, or the tendency of students selecting the correct answer to have high overall scores. POINT-BISERIAL METHOD

Distractor (Incorrect Alternatives) Analyzing the distractors (e.i., incorrect alternatives) is useful in determining the relative usefulness of the decoys in each item. Items should be modified if students consistently fail to select certain multiple choice alternatives. The alternatives are probably totally implausible and therefore of little use as decoys in multiple choice items. A discrimination index or discrimination coefficient should be obtained for each option in order to determine each distractor's usefulness .