1. The first test of validity
Reliability
The first test of validity

2. Reliability
Degree to which an assessment tool yields consistent evaluations across time, situations, and raters
Is the instrument trustworthy to measure what it says it measures?

3. Types of Reliability
Interrater
Stability
Internal Consistency

4. Model of Reliability Obtained score = True score + error
Determines how much error is present in an obtained score

5. Standard Error of Measurement
Standard deviation of error around a true score
SEM = SD
1- rxx

6. True scores Mean score obtained if the entire domain were tested
Because an entire domain cannot be tested, the true score can only be estimated

7. True Score Estimate
X’ = X + rt1t2(X - X)

8. Confidence Intervals for Estimated True Scores
C.I. = X’ ± (z) (SEM)

9. Common % Confidence z-score Intervals

10. Test-retest reliability
Measures consistency of administration
Test the same group with the same test over a short period of time
one-two weeks is appropriate

11. Equivalent Forms
Measures consistency across tests of same content but varying questions.
Tests given to same group at the same time and correlated

12. Split-half reliability
Measures consistency across items, i.E, internal consistency
Divide a single test in half and correlate
Different approaches to splitting: even/odd, 1st/2nd section, random selection
Other possible split half measures include
Coefficient alpha (Cronbach’s )
Kr-20

13. Interrater reliability
Useful in measuring reliability across observers or scorers.
Number of agreements
Number of agreements + number of disagreements
100
% Agreement =

14. Desirable Standards for Reliability
Test authors need to report rxx and validation data.
test validation should be also be present for subtest or subscales.
Group data for administrative purposes
is desirable
.6 is a minimum

15. Desirable Standards for Reliability
Individual Data
For placement decisions
.9 is the minimum.
For screening decisions
.8 is recommended.