Validity and Reliability EAF 410 July 9, 2001

Validity b Degree to which evidence supports inferences made b Appropriate b Meaningful b Useful

Appropriate Inference b Relevant to the study

Meaningful Inference b Says something about the meaning of the information obtained through an instrument

Useful Inference b Helps researchers make decisions related to what they studied

Do we get useful information about the topic/variables?

Validity depends on: b The amount and type of evidence supporting the interpretations

Types of Evidence b Content-related b Criterion-related b Construct-related

Content-related b Appropriate? b Comprehensive? b Logical? b Adequate?

b Adequacy of the sampling representative of content?representative of content? b Format of the instrument language, directions, etc.language, directions, etc. b Who judges the validity of the content? InterpretationInterpretation judgementjudgement

Criterion-related b Relationship between scores from different instruments b Strong relationship? b Accurate?

b Instrument results compared with independent criterion a second test or measurementa second test or measurement b Predictive validity time between measurementstime between measurements b Concurrent validity instrument and criterion measurements at same timeinstrument and criterion measurements at same time

b Validity Coefficient r = correlation coefficientr = correlation coefficient degree of relationship between scores from instrument and criteriondegree of relationship between scores from instrument and criterion Between and +1.00Between and = no relationship.00 = no relationship

Construct-related b How well does the theoretical construct account for or explain differences? b Is the construct sufficient?

b Three steps: variable measured clearly definedvariable measured clearly defined hypotheses (based on theory)hypotheses (based on theory) hypotheses tested logically and empiricallyhypotheses tested logically and empirically b Collect variety of different types of evidence

Reliability b Consistency of the scores among individualsamong individuals among itemsamong items

b Reliable -- valid or not valid b Not reliable -- not valid b Validity and reliability dependent on the context in which the instrument is used.

Measurement Error b Same test, different scores b Caused by variety of factors: motivationmotivation anxietyanxiety environmentenvironment etc.etc.

Reliability Coefficient b Reliability estimates b Relationship between scores at two different times, or b Relationship between two parts of the same instrument

Test-Retest b Same test, same group, two times b Reliability coefficient indicates relationship between the two scores

Equivalent-Forms b Two different, equivalent tests, same group b Reliability coefficient calculated

Internal-Consistency b Single administration of the instrument b Split-half procedure b Kuder-Richardson approach b Alpha coefficient

Split-Half Procedure b Score two halves of test separately b Degree to which the two halves provide same results b Reliability increases with length of the test/instrument

Standard Error of Measurement b Index showing how much a measurement would vary under changed circumstances