Download presentation
Presentation is loading. Please wait.
Published byAugustine Jordan Modified over 8 years ago
1
Nonequivalent Groups: Linear Methods Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and practices (2 nd ed.). New York, NY: Springer. EPSY 82251
2
Equating Common Population Observed Score True Score: CTT/IRT Anchor Test Observed Score True Score: CTT/IRT EPSY 82252
3
Nonequivalent Groups Only one test form is administered at a time. The different groups cannot be assumed to be randomly equivalent. Each group comes from a different population. To allow the linking of the two forms, a set of common items is included on each form. Common items could be internal (scored) or external (not part of the score on the form). EPSY 82253
4
Notation Y = score on the old form X = score on the new form V = score on the common item set EPSY 82254
5
The Synthetic Population The design involves two populations. But the equating function is defined for a single population. We define a synthetic population, where each of the independent populations is weighted by w 1 and w 2 where w 1 + w 2 = 1 and w 1, w 2 0 EPSY 82255
6
Linear Equating The linear conversion is defined by setting standardized deviation scores equal EPSY 82256
7
Linear Equating Function Where s indicates the synthetic population EPSY 82257
8
Synthetic Population Parameters s (X) = w 1 1 (X) + w 2 2 (X) s (Y) = w 1 1 (Y) + w 2 2 (Y) EPSY 82258
9
Synthetic Population Parameters In population 1, Y is not administered In population 2, X is not administered The following parameters cannot be estimated directly: EPSY 82259
10
Parameters Estimated from Data Form X administered to population 1 Form Y administered to population 2 EPSY 822510
11
Parameters Estimated from Assumptions Form X moments in Population 2 Form Y moments in Population 1 EPSY 822511
12
Nonequivalent Group Equating What sets apart these equating methods is the set of statistical assumptions used. Assumptions must be introduced to estimate the parameters not observed. Different methods employ different assumptions. EPSY 822512
13
Tucker Method Assumption 1: regression of total scores on common-item scores – The regression of X on V is the same linear function for both populations 1 and 2 – The same assumption is made for the regression of Y on V The slope and regression intercept are assumed to be the same for the observed data with each population and the unobserved parameters in the other population EPSY 822513
14
Tucker Method Assumption 2: conditional variances of total scores given common-item scores – Conditional variance of X given V is the same for population 1 and 2 – The same assumption is made for the conditional variance of Y given V EPSY 822514
15
Tucker Method The conditional mean score on the new form increases linearly with scores on the anchor – Use a simple formula to estimate the conditional mean in the synthetic population The conditional standard error is the same at all levels of the anchor score – Estimate a single value for the conditional SE Need: Mean and SD of anchor scores in synthetic population EPSY 822515
16
Tucker Method Result: the synthetic population means and variances for X and Y are adjusted to directly observable quantities. The adjustment is a function of the differences in means and variances for the common items across the two populations. If 1 (V) = 2 (V) and the synthetic parameters would equal observable means and variances. EPSY 822516
17
Estimating Synthetic Pop Values s (X) = 1 (X) - w 2 γ 1 [ 1 (V) - 2 (V)] s (Y) = 2 (Y) - w 1 γ 2 [ 1 (V) - 2 (V)] Where γ 1 and γ 2 are the regression slopes from the regression of X on V for the two populations. Similarly, we can estimate EPSY 822517
18
Weighting Options w 1 + w 2 = 1 and w 1, w 2 0 One could conceive of the synthetic population to be the new population: w 1 = 1 and w 2 = 0 Weights can be proportional to population sizes: w 1 = N 1 /(N 1 + N 2 ) and w 1 = N 2 /(N 1 + N 2 ) Weights can be equal, combining both populations equally: w 1 + w 2 =.5 EPSY 822518
19
Tucker Issues Problems in the equating can occur if – the ability distributions between those who take the different forms differ a great deal – When the anchor is not strongly correlated with the test scores – The test scores and the anchor scores do not yield near perfect reliabilities EPSY 822519
20
Levine Method Assumes that X, Y, and V are all measuring the same thing so that T X and T V as well as T Y and T V are perfectly correlated in both populations. Assumptions about true scores are made in terms of the linear regression of X on V and Y on V Assumptions about the error variances (measurement error) are made similarly EPSY 822520
21
Levine The method is one that relates observed scores from form X to the scale of observed scores on form Y. However, the assumptions underlying Levine method are about the true scores, T X, T Y, and T V. These are related to observed scores as in CTT, where the error has E[ε] = 0 and ρ εT = 0. EPSY 822521
22
Levine The assumptions are that X, Y, and V all measure the same thing, which implies True scores of X and V as well as True scores of Y and V are perfectly correlated in both populations 1 and 2. ρ 1 = (T X, T V ) = ρ 2 (T X, T V ) = 1 ρ 1 = (T Y, T V ) = ρ 2 (T Y, T V ) = 1 EPSY 822522
23
Levine True Score Method Similar assumptions about true scores are made in this method Instead of equating observed scores, true scores are equated Although the derivations employ true scores, the equating is actually done on observed scores. EPSY 822523
24
Levine True Score Method In CTT, observed score means = true score means And based on previous derivation results: Observed scores are used in place of true scores. The results do not rely on the synthetic population. EPSY 822524
25
Levine True-Score Property Turns out, using observed scores in Levine’s true score equating function for the common- item nonequivalent groups design results in first-order equity, under the congeneric model. For the population with a given true score on Y, the expected value for the linearly transformed scores on X equals the expected value of the scores on Y, for all true scores on Y. EPSY 822525
26
Nonequivalent Groups: Equipercentile Methods EPSY 822526
27
Frequency Estimation f (x, v ) is the joint distribution; the probability that X = x and V = v. f (x) is the marginal distribution of scores on X; the probability of obtaining a score of x on X. h( v ) is the marginal distribution of scores on V. f (x| v ) is the conditional distribution of scores on Form X for examinees with a particular score on V. EPSY 822527
28
The conditional expectation is It follows that EPSY 822528
29
Synthetic Populations f s (x) = w 1 f 1 (x) + w 2 f 2 (x) g s (y) = w 1 g 1 (y) + w 2 g 2 (y) Because form X is not administered to population 2, f 2 (x) is not directly estimable Because form Y is not administered to population 1, g 1 (y) is not directly estimable EPSY 822529
30
Assumptions For both Form X and Form Y, the conditional distribution of total score given each score, V = v, is the same in both populations. EPSY 822530
31
Estimation These assumptions can be used to find expressions for f 2 (x) and g 1 (y) using quantities for which direct estimates are available. EPSY 822531
32
Chained Equipercentile Equating Angoff (1971) Form X scores are converted to scores on the common items using examinees from population 1 Scores on the common items are equated to form Y scores using examinees from Population 2. These conversions are chained together to produce a conversion of Form X to Form Y EPSY 822532
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.