1. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Ch. 6: Reliability and Validity in Measurement and Research

2. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Validity and Reliability Validity: How well does the measure or design do what it purports to do? Reliability: How consistent or stable is the instrument? Is the instrument dependable?

3. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Random and Systematic Error Random Error: Chance fluctuations Tend to cancel out over repeated measurements. Systematic Error: Fluctuations that are slanted in a particular direction. Also known as “bias”

4. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Types of Reliability Test-Retest Reliability: Degree of temporal stability of the instrument. Assessed by having instrument completed by same people during two different time periods. Alternate-Forms Reliability: Degree of relatedness of different forms of test. Used to minimize inflated reliability correlations due to familiarity with test items.

5. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Types of Reliability (cont.) Internal-Consistency Reliability: Overall degree of relatedness of all test items or raters. Also called reliability of components. Item-to-Item Reliability: The reliability of any single item on average. Judge-to-Judge Reliability: The reliability of any single judge on average.

6. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Spearman-Brown Formula Used to estimate the internal consistency of a test or judges overall. The formula is: Where: R SB = the overall internal consistency reliability n = the total number of items in the test r ii = the average intercorrelation among the items or among the judges.

7. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Reliability, Replication, and External Validity External Validity: The generalizability of an inferred causal relationship. The dependability of causal generalizations is based on replicable findings. But how know if have replicated a finding? Role of the effect size.

8. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Validity in Test and Instrument Construction Content Validity: Is the relevant material adequately sampled? Criterion Validity: How well does the test correlated with outcome criteria? Concurrent Validity Predictive Validity

9. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Construct Validity in Test Development Construct Validity: What does the test really assess? Does the test have the “ability to discriminate”? Establishing construct validity Convergent validity Discriminant validity

10. Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall Validity and Causal Inference in Experimental Design External Validity Construct Validity: The validity of the hypothetical idea linking the IV and DV. Statistical Conclusion Validity: Are the statistical conclusions well-grounded? Internal Validity: Ability to rule out plausible rival hypotheses.