1. Validity and Reliability EAF 410 July 9, 2001

2. Validity b Degree to which evidence supports inferences made b Appropriate b Meaningful b Useful

3. Appropriate Inference b Relevant to the study

4. Meaningful Inference b Says something about the meaning of the information obtained through an instrument

5. Useful Inference b Helps researchers make decisions related to what they studied

6. Do we get useful information about the topic/variables?

7. Validity depends on: b The amount and type of evidence supporting the interpretations

8. Types of Evidence b Content-related b Criterion-related b Construct-related

9. Content-related b Appropriate? b Comprehensive? b Logical? b Adequate?

10. b Adequacy of the sampling representative of content?representative of content? b Format of the instrument language, directions, etc.language, directions, etc. b Who judges the validity of the content? InterpretationInterpretation judgementjudgement

11. Criterion-related b Relationship between scores from different instruments b Strong relationship? b Accurate?

12. b Instrument results compared with independent criterion a second test or measurementa second test or measurement b Predictive validity time between measurementstime between measurements b Concurrent validity instrument and criterion measurements at same timeinstrument and criterion measurements at same time

13. b Validity Coefficient r = correlation coefficientr = correlation coefficient degree of relationship between scores from instrument and criteriondegree of relationship between scores from instrument and criterion Between -1.00 and +1.00Between -1.00 and +1.00.00 = no relationship.00 = no relationship

14. Construct-related b How well does the theoretical construct account for or explain differences? b Is the construct sufficient?

15. b Three steps: variable measured clearly definedvariable measured clearly defined hypotheses (based on theory)hypotheses (based on theory) hypotheses tested logically and empiricallyhypotheses tested logically and empirically b Collect variety of different types of evidence

16. Reliability b Consistency of the scores among individualsamong individuals among itemsamong items

17. b Reliable -- valid or not valid b Not reliable -- not valid b Validity and reliability dependent on the context in which the instrument is used.

18. Measurement Error b Same test, different scores b Caused by variety of factors: motivationmotivation anxietyanxiety environmentenvironment etc.etc.

19. Reliability Coefficient b Reliability estimates b Relationship between scores at two different times, or b Relationship between two parts of the same instrument

20. Test-Retest b Same test, same group, two times b Reliability coefficient indicates relationship between the two scores

21. Equivalent-Forms b Two different, equivalent tests, same group b Reliability coefficient calculated

22. Internal-Consistency b Single administration of the instrument b Split-half procedure b Kuder-Richardson approach b Alpha coefficient

23. Split-Half Procedure b Score two halves of test separately b Degree to which the two halves provide same results b Reliability increases with length of the test/instrument

24. Standard Error of Measurement b Index showing how much a measurement would vary under changed circumstances