Chapter 2 Norms and Reliability

The essential objective of test standardization is to determine the distribution of raw scores in the norm group so that the test developer can publish derived scores known as norms.

Raw scores In the case of criterion-referenced tests, norms are not essential. There are many different kinds of norms, but they share one characteristic: Each incorporate a statistical summary of a large body of scores.

Essential Statistical Concepts Frequency Distributions Measures of Central Tendency Measures of Variability The Normal Distribution Skewness

Raw Score Transformations Percentiles and Percentile Ranks Standard Scores T scores and Other Standardized Scores Normalizing Standard Scores Stanines, Stens, and C scale

Selecting A Norm Group Age and Grade Norms Local and Subgroup Norms Expectancy Tables

Criterion-Referenced Tests Criterion-referenced tests are used to compare examinees’ accomplishments to a predefined performance standard. The focus is on what the test taker can do rather than on comparisons to the performance levels of others. Table 3.7

Concepts of Reliability Classical test theory and the sources of measurement error Sources of measurement error Measurement error and reliability The reliability coefficient The correlation coefficient The correlation coefficient as a reliability coefficient

Concepts of Reliability Reliability as temporal stability Reliability as internal consistency Item response theory The new rules of measurement Special circumstances in the estimation of reliability The interpretation of reliability coefficients Reliability and the standard error of measurement

Classical test theory and the sources of measurement error Test scores result from the influence of two factors: 1.Factors that contribute to consistency. 2.Factors that contribute to inconsistency

Sources of measurement error Item selecting Test administration Test scoring Systematic measurement error

Measurement error and reliability 1.unsystematic measurement error act as random influences 2.mean error of measurement is zero. 3.measurement errors are not correlated with true scores 4.measurement errors are not correlated with errors on other tests.

The reliability coefficient The ratio of true score variance to the total variance of test scores.

The correlation coefficient as a reliability coefficient If test results are highly consistent, then scores of persons taking the test on two occasions will be strongly correlated. Reliability as temporal stability Reliability as internal consistency

Reliability as temporal stability Test-retest reliability Alternate-forms reliability

Reliability as internal consistency Split-Half reliability The Spearman-Brown Formula Coefficient Alpha The Kuder-Richardson Estimate of reliability Interscorer reliability

Item response theory Figure 3.13 Item response Function or item characteristic curve Information functions Invariance in IRT

Special circumstances in the estimation of reliability Unstable characteristics Speed and power tests Restriction of range Reliability of criterion-referenced tests

Reliability and the standard error of measurement Computing the standard error of measurement The SEM and individual test scores The SEM and differences between scores