Download presentation
Presentation is loading. Please wait.
Published byJoan Palmer Modified over 9 years ago
1
Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology
2
2 Outline Review of existing methods Globally Optimal Weight: a set of weights that maximizes the Expected Test Information Intrinsic Category Weights Examples Conclusions
3
3 Background Estimation of IRT ability on the basis of simple and weighted summed score X. Conditional distribution of X given as the distribution of the weighted sum of the Scored Multinomial Distribution. Posterior Distribution of given X. h( x) f(x| ) h( ) Posterior Mean(EAP) of given X. Posterior Standard Deiation(PSD)
4
4 Item Score We must choose w to calculate X. IRF
5
5 Item Score We must choose w and v to calculate X. ICRF
6
6 Conditional distribution of X given Binary items Conditional distribution of summed score X. Simple sum: Walsh(1955), Lord(1969) Weighted sum: Mayekawa(2003) Polytomous items Conditional distribution of summed score X. Simple sum: Hanson(1994), Thissen et.al.(1995) With Item weight and Category weight: Mayekawa & Arai(2007)
7
7 Example Eight Graded Response Model items 3 categories for each item.
8
8 Example (choosing weight) Example: Mayekawa and Arai (2008) small posterior variance good weight. Large Test Information (TI) good weight
9
9 Test Information Function Test Information Function is proportional to the slope of the conditional expectation of X given (TCC), and inversely proportional the squared width of the confidence interval (CI) of given X. Width of CI Inversely proportional to the conditional standard deviation of X given .
10
10 Confidence interval (CI) of given X
11
11 Test Information Function for Polytomous Items ICRF
12
12 Maximization of the Test Information when the category weights are known. Category weighted Item Score and the Item Response Function
13
13 Maximization of the Test Information when the category weights are known.
14
14 Maximization of the Test Information when the category weights are known. Test Information
15
15 Maximization of the Test Information when the category weights are known. First Derivative
16
16 Maximization of the Test Information when the category weights are known.
17
17 Globally Optimal Weight A set of weights that maximize the Expected Test Information with some reference distribution of . It does NOT depend on .
18
18 Example NABCT A B1 B2 GO GOINT A AINT Q1 1.0 -2.0 -1.0 7.144 7 8.333 8 Q2 1.0 -1.0 0.0 7.102 7 8.333 8 Q3 1.0 0.0 1.0 7.166 7 8.333 8 Q4 1.0 1.0 2.0 7.316 7 8.333 8 Q5 2.0 -2.0 -1.0 17.720 18 16.667 17 Q6 2.0 -1.0 0.0 17.619 18 16.667 17 Q7 2.0 0.0 1.0 17.773 18 16.667 17 Q8 2.0 1.0 2.0 18.160 18 16.667 17 LOx LO GO GOINT A AINT CONST 7.4743 7.2993 7.2928 7.2905 7.2210 7.2564 5.9795
19
19 Maximization of the Test Information with respect to the category weights. Absorb the item weight in category weights.
20
20 Maximization of the Test Information with respect to the category weights. Test Information Linear transformation of the category weights does NOT affect the information.
21
21 Maximization of the Test Information with respect to the category weights. First Derivative
22
22 Maximization of the Test Information with respect to the category weights. Locally Optimal Weight
23
23 Globally Optimal Weight Weights that maximize the Expected Test Information with some reference distribution of .
24
24 Intrinsic category weight A set of weights which maximizes: Since the category weights can be linearly transformed, we set v0=0, ….. vmax=maximum item score.
25
25 Example of Intrinsic Weights
26
26 Example of Intrinsic Weights h( )=N(-0.5, 1): v0=0, v1=*, v2=2
27
27 Example of Intrinsic Weights h( )=N(0.5, 1): v0=0, v1=*, v2=2
28
28 Example of Intrinsic Weights h( )=N(1, 1 ): v0=0, v1=*, v2=2
29
29 Summary of Intrinsic Weight It does NOT depend on , but depends on the reference distribution of : h( ) as follows. For the 3 category GRM, we found that For those items with high discrimination parameter, the intrinsic weights tend to become equally spaced: v0=0, v1=1, v2=2 The Globally Optimal Weight is not identical to the Intrinsic Weights.
30
30 Summary of Intrinsic Weight For the 3 category GRM, we found that The mid-category weight v1 increases according to the location of the peak of ICRF. That is: The more easy the category is, the higher the weight. v1 is affected by the relative location of other two category ICRFs.
31
31 Summary of Intrinsic Weight For the 3 category GRM, we found that The mid-category weight v1 decreases according to the location of the reference distribution of h( ) If the location of h( ) is high, the most difficult category gets relatively high weight, and vice versa. When the peak of the 2nd category matches the mean of h( ), we have eqaully spaced category weights: v0=0, v1=1, v2=2
32
32 Globally Optimal w given v
33
33 Test Information LOx LO GO GOINT CONST 30.5320 30.1109 30.0948 29.5385 24.8868
34
34 Test Information
35
35 Bayesian Estimation of from X
36
36 Bayesian Estimation of from X
37
37 Bayesian Estimation of from X (1/0.18)^2 = 30.864
38
38 Conclusions Polytomous item has the Intrinsic Weight. By maximizing the Expected Test Information with respect to either Item or Category weights, we can calculate the Globally Optimal Weights which do not depend on . Use of the Globally Optimal Weights when evaluating the EAP of given X reduces the posterior variance.
39
39 References
40
40 ご静聴 ありがとう ございました 。 Thank you.
41
41
42
42
43
43
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.