1. PSYCHOMETRICS RELIABILITY VALIDITY

2. RELIABILITY X obtained = X true – X error IDEAL DOES NOT EXIST USEFUL CONCEPTION

3. ERROR SCORE AFFECT TRUE SCORE NOT RELATED TO TRAIT MEASURED ERRORS COME FROM INDIVIDUAL ADMINISTRATOR SCORER ENVIRONMENT

4. RELIABILITY TYPES TEST-RETEST PARALLEL FORMS INTERNAL CONSISTENCY

5. INTERRATER RELIABILITY #1 SUBJECTIVE SCORES DEPEND ON ATTRIBUTE RATER BIASES/OPINIONS JA RATINGS PA RATINGS INTERVIEWER JUDGMENTS THREE CATEGORIES

6. INTERRATER RELIABILITY #2 INTERRATER AGREEMENT DID SOMETHING OCCUR INTERCLASS CORRELATIONS TWO RATER JUDGMENTS INTRACLASS CORRELATIONS THREE OR MORE RATERS

7. INTRACLASS CORRELATION INTERVIEWER Mr. Brown Ms. Black Ms. Green INTERVIEWEES Pat 988 John 565 Hal 432

8. RELIABILITY INTERPRETATION r =.90 90% OF DIFFERENCES DUE TO TRUE VARIANCE 10% DUE TO ERROR

9. FACTORS AFFECTING RELIABILITY METHOD INDIVIDUAL DIFFERENCES LENGTH DIFFICULTY HOMOGENEITY RESPONSE FORMAT ADMINISTRATION IDIOSYNCRASIES

10. STANDARD ERROR OF MEASUREMENT ESTIMATE ________ s meas = s x √1 - r xx r =.90, sd = 10 ________ s meas = 10 √1 -.90 ________ s meas = 10 √.10 = 10(.316) = 3.16 50 ± 3.16

11. VALIDITY EVIDENCE SUPPORTS INFERENCES DEPENDS ON RELIABILITY _______ r xy ≤ √r xx r yy VALIDITY ≤.70 IF RELIABILITY.81 AND.70

12. VALIDITY STRATEGIES CONCURRENT CRITERION-RELATED PREDICTIVE CRITERION-RELATED CONTENT

13. CONTENT EVIDENCE CONDUCT JA DESCRIPTION OF TASKS ASSESS CRITICALITY/IMPORTANCE KSA SPECIFICATION CRITICALITY OF KSA SELECT SME’s SPECIFY MEASURE CONTENT ASSESS RELEVANCE

14. ASSESSING RELEVANCE CONTENT VALIDITY RATIO n e – N/2 CVR = ----------- N/2 n e = number rating essential N = total number of judges -1 TO +1

15. VALIDITY STRATEGIES CONCURRENT CRITERION-RELATED PREDICTIVE CRITERION-RELATED CONTENT CONSTRUCT

16. VALIDITY COEFFICIENT INTERPRETATION r xy 2 % OF VARIANCE IN CRITERION.30 2 =.09 9% FORECASTING EFFICIENCY % IMPROVEMENT OVER NONVALID _______ 1 - √1 - r xy 2 r xy 2 =.30 16.3% IMPROVEMENT

17. FACTORS AFFECTING VALIDITY #1 PREDICTOR/CRITERION RELIABILITY ^ ____ r xy = r xy / √r yy VALIDITY =.35 CRITERION RELIABILITY =.70 CORRECTED =. 42

18. FACTORS AFFECTING VALIDITY #2 PREDICTOR/CRITERION RELIABILITY RESTRICTION OF RANGE ^ r xyr *(SDu/SDr) rxy = ------------------------------- __________________________________ √ 1–(r xyr ) 2 + (r xyr ) 2 [(SDu) 2 /(SDr) 2

19. RANGE RESTRICTION IV VALIDITY =.20 UNRESTRICTED GROUP sd = 10 RESTRICTED GROUP sd = 2 ^.20*(10/2) rxy = ------------------------------- ________________________________ √ 1–(.20) 2 + (.20) 2 [(10) 2 /(2) 2 = 1.00/1.40 =.71

20. FACTORS AFFECTING VALIDITY #3 PREDICTOR/CRITERION RELIABILITY RESTRICTION OF RANGE CRITERION CONTAMINATION VIOLATION STATISTICAL ASSUMPTION