Lecture 5 Validity and Reliability Research Methods and Statistics

Validity and Reliability of Measurements Validity (accuracy) - The degree to which a data collection instrument “accurately” measures what it is supposed to measure for a particular study. Reliability (consistency) - The degree to which a data collection instrument “consistently” measures what it is supposed to measure for a particular study.

Validity Validity includes the appropriateness and meaningfulness of the specific inferences a researcher makes on the basis of the data the researcher collects. Methods of collecting evidence of validity - Content Validity - Criterion Validity

Content Validity Content validity is the degree to which items on a data collection instrument measure an intended content area. How to collect evidence of content validity - Review theory and the literature: how content has been measured in the past, an extensive review of the literature is warranted - Assemble a “panel of experts” to judge your instrument

Criterion Validity Criterion validity is the degree to which data from your instrument are related to data from another instrument (the criterion). Two types of criterion validity - Concurrent validity: degree to which data from your instrument are related to data from another instrument (the criterion) that is collected at the same time. - Predictive validity: degree to which data from your instrument predict future outcomes based on data from another instrument

Criterion Validity How to collecting evidence of criterion validity - Determine appropriate criterion (using theory and literature) against which you will correlate data from your instrument. - Measure the relationship between (correlate) your data and the criterion data and examine the validity coefficient.

Reliability Reliability is the degree to which an instrument consistently measures what it is supposed to measure. Methods for obtaining evidence of reliability - Stability: Test-Retest Reliability - Equivalence: Equivalent Forms Reliability - Internal Consistency: Internal Consistency Reliability Reliability is expressed numerically as a “reliability coefficient” (0<cronbach’s alpha<1)

Test-Retest Reliability Test-Retest Reliability is the degree to which scores from the same instrument are consistent or stable over time. The biggest challenge - Determining how long the time interval between the first and second administration should be.

Equivalent Forms Reliability Equivalent Forms Reliability is the degree to which scores are consistent across two different forms of the same instrument. The biggest challenge with equivalent forms reliability is constructing two forms of the same instrument that are essentially equivalent.

Internal Consistency Reliability Internal Consistency Reliability is the degree to which items on an instrument are consistent among themselves and with the instrument as a whole. Two types of internal consistency reliability - Split-Half : dividing an instrument into 2 equivalent halves and correlating the scores of each half. - Coefficient Alpha (Cronbach’s Alpha): determining how all items of an instrument relate to all other items and to the overall instrument.