Multiple-Group Threats to Internal Validity

The Central Issue l When you move from single to multiple group research the big concern is whether the groups are comparable. l Usually this has to do with how you assign units (for example, persons) to the groups (or select them into groups). l We call this issue selection or selection bias.

The Multiple Group Case AdministerprogramMeasureoutcomesMeasurebaseline Alternativeexplanations AlternativeexplanationsXOOOO Do not administer program MeasureoutcomesMeasurebaseline

ExampleExample l Compensatory education in math for 1st graders l Pre-post program-comparison group design l Measures (O) are standardized achievement tests (at start of grade 1 and start of grade 2; forms A & B)

Selection Threats l Any factor other than the program that leads to posttest differences between groups. l For example, because of group differences, kids in one group watch Sesame Street more frequently and pick up more math concepts. XOOOO

Selection-History Threat l Any other event that occurs between pretest and posttest that the groups experience differently. l For example, kids in one group pick up more math concepts because they watch more Sesame Street. XOOOO

Selection-Maturation Threat l Differential rates of normal growth between pretest and posttest for the groups. l They are learning at different rates, even without program. XOOOO

Selection-Testing Threat l Differential effect on the posttest of taking the pretest. l The test may have “primed” the kids differently in each group or they may have learned differentially from the test, not the program. XOOOO

Selection-Instrumentation Threat l Any differential change in the test used for each group from pretest and posttest l For example, change due to different forms of test being given differentially to each group, not due to program XOOOO

Selection-Mortality Threat l Differential nonrandom dropout between pretest and posttest. l For example, kids drop out of the study at different rates for each group. XOOOO

Selection-Regression Threat l Different rates of regression to the mean because groups differ in extremity. l For example, program kids are disproportionately lower math scorers and consequently have greater regression to the mean. XOOOO