Experimental and Ex Post Facto Designs Chapter Ten Experimental and Ex Post Facto Designs

Independent and Dependent Variables Variable: any quality or characteristic in a research investigation that has two or more values. Cause-and-Effect Relationship: the extent to which one variable (the cause) influences another variable (the effect). Independent Variable: a variable that the researcher studies as a possible cause of something else; the variable that the researcher directly manipulates. Dependent Variable: a variable that is potentially influenced by the independent variable; a variable that is influenced by and to some extent depends on the independent variable.

What is a Variable? Simply, something that varies. Specifically, variables represent persons or objects that can be manipulated, controlled, or merely measured for the sake of research. Variation: How much a variable varies. Those with little variation are called constants.

Independent Variables These variables are ones that are more or less controlled. Scientists manipulate these variables as they see fit. They still vary, but the variation is relatively known or taken into account. Often there are many in a given study.

Dependent Variables Dependent variables are not controlled or manipulated in any way, but instead are simply measured or registered. These vary in relation to the independent variables, and while results can be predicted, the data is always measured. There can be any number of dependent variables, but usually there is one to isolate reason for variation.

Independent V. Dependent Intentionally manipulated Controlled Vary at known rate Cause Intentionally left alone Measured Vary at unknown rate Effect

The Importance of Control Internal Validity: the extent to which the design of a research study and the data it yields allows the researcher to draw accurate conclusions about cause-and-effect and other relationships. Without internal validity in experimental designs, the results are not interpretable. Confounding Variables: account for differences in two or more groups that are not attributable to the particular treatment or intervention being studied. The greatest threat to internal validity is confounding. Confounding occurs whenever an extraneous variable changes systematically along with the independent variable. Confounding prevents us from inferring a causal relationship between the independent and dependent variables

Strategies for Controlling Confounding Variables Keep some things constant. Include a control group. Randomly assign people to groups. Assess equivalence before the treatment with one or more pretests. 5. Expose participants to all experimental conditions. 6. Statistically control for confounding variables.

Categories of Experimental Designs Pre-Experimental Designs True Experimental Designs Quasi-Experimental Designs Ex Post Facto Designs Factorial Designs

Pre-Experimental Designs Not possible to show cause-and-effect relationships because (a) the independent variable doesn’t vary or (b) experimental and control groups are not comprised of equivalent or randomly selected individuals. Design 1: One-Shot Experimental Case Study (low internal validity) Group 1 Tx Obs Design 2: One-Group Pretest-Posttest Design Group 1 Obs Tx Obs Design 3: Static Group Comparison Group 1 Tx Obs Group 2 --- Obs In a pre experimental design something is missing from the true experimental design What is usually missing is the control group is missing. Pre-experimental designs are used when we have time series or longitudinal data.

True Experimental Designs Compared to pre-experimental designs, experimental designs offer a great degree of control and greater internal validity. Design 4: Pretest-Posttest Control Group Design (random assignment) Group 1 Obs Tx Obs Group 2 Obs --- Obs Design 5: Solomon Four-Group Design (enhances external validity) Group 3 --- Tx Obs Group 4 --- --- Obs Design 6: Within-Subjects Design (repeated measures) Group 1 Txa Obsa Txb Obsb R A N D O M R A N D O M Basic research Pure research Cross section of time Experimental designs Greatest strength: strong internal validity able to control for threats—alternate explanations for treatment effects) Attributed to random assignment Relies heavily on random assignment to yield equivalent groups Can be more certain (say with confidence) than any other design about attributing the cause to the independent variable Greatest weakness: external validity—may inappropriate to attribute generalize beyond study populations May be difficult to implement with integrity where individuals responsible for random assignment procedures are not compliant with research protocol May be difficult to implement due to ethical considerations and inability to deny groups treatment Must consider the effects of attrition

Quasi Experimental Designs A type of research design for conducting studies in the field or real-life situations where the researcher may be able to manipulate the some independent variable but CANNOT randomly assign subjects to the control and experimental groups

When do we use a Quasi Experimental Design When randomization is not possible Independent variable precludes the use of random assignment Retrospective studies Studies that focus on economic or social conditions Randomization is too expensive, not feasible, or impossible to monitor Ethical issues as they relate to treatment Quick timeline

Quasi-Experimental Designs if one of the following conditions cannot be met is necessary to use quasi experimental designs 1. manipulation of at least one independent variable 2. random assignment of subjects to groups 3. random assignment of independent variable (treatment) to groups 4. Exposure of experimental group to treatment in isolation from other factors.

Strengths and Limitations supports causal inferences explain the uncertainty surrounding the existence of the causal relationship nonequivalent groups can help strengthen the design Limitations Comparison base may be biased—doesn’t give Strengths Provides an approximation to the experimental design and supports causal inferences Provides a means for explaining the uncertainty surrounding the existence of the causal relationship Using nonequivalent groups can help strengthen the design Limitations Comparison base may be biased

Types of Quasi Experimental Designs Comparison Group Pre/post design (experimental and comparison group) Post test only Interrupted Time series

Quasi-Experimental Designs Randomness is not possible or practical; can’t control for all confounding variables. Design 8: Nonrandomized Control Group Pretest-Posttest Design Group 1 Obs Tx Obs Group 2 Obs --- Obs Design 9: Simple Time-Series Design Group 1 Obs Obs Obs Obs Tx Obs Obs Obs Obs Design 10: Control Group, Time-Series Design Group 2 Obs Obs Obs Obs --- Obs Obs Obs Obs A type of research design for conducting studies in the field or real-life situations where the researcher may be able to manipulate the some independent variable but CANNOT randomly assign subjects to the control and experimental groups Quasi –experimental designs partial control over independent variables Lacking random assignment Time series design Interrupted time series Interrupted Time Series Design • Effects of “treatment” (IV) inferred from comparison of outcome measures (DV) • Outcome measures obtained at different time intervals – Before and after treatment is introduced 1. Define period of observation broadly – Observe DV before, during, and after intervention 2. Same units used throughout analysis – Observations and time points equally spaced 3. Time points have to be sensitive to the particular effects of interest 4. Measurements can’t fluctuate much – No “instrument” changes multiple time series

Quasi-Experimental Designs (con’t) Design 11: Reversal Time-Series Design Group 1 Tx Obs --- Obs Tx Obs --- Obs Design 12: Alternating Treatment Design Group 1 Txa Obs --- Obs Txb Obs --- Obs Txa Obs --- Obs Txb Obs Design 13: Multiple Baseline Design Group 1 Baseline ------ Obs Treatment Tx Obs Tx Group 2 -------- ---- Obs

Ex-post-facto designs Studies that investigate the relationships between independent and dependent variables in situations where it is impossible or unethical to manipulate the independent variable Example - what is the effect of pre-kindergarten (Pre-K) attendance on first grade achievement Cannot mandate Pre-K attendance for children Characteristics and resources of families who do and do not send their children to Pre-K may influence first grade achievement Similarities with correlational and experimental research designs Ex-post-facto designs Issues of concern Selecting participants who are as similar as possible on all characteristics except the independent variable Generalizing beyond the participants studied    

Ex Post Facto Designs The researcher identifies events that have already occurred or conditions that are already present and then collects data to investigate a possible relationship between these factors and subsequent characteristics or behaviors. - like correlational research, ex post facto research involves looking at existing circumstances; - like experimental research, ex post facto research has clearly identifiable independent and dependent variables; - unlike experimental research, ex post facto research involves no direct manipulation of the independent variable – the presumed “cause” has already occurred. Design 14: Simple Ex Post Facto Design Prior event Investigation period Group 1 exp Obs Group 2 ---- Obs

Factorial Designs - 1 Intervention studies with 2 or more categorical explanatory variables leading to a numerical outcome variable are called Factorial Designs. A factor is simply a categorical variable with two or more values, referred to as levels.

Factorial Designs Examination of the effects of two or more independent variables in a single study. Design 15: Randomized Two-Factor Design Treatment (var.1) Treatment (var. 2) Group 1 Tx1 Tx2 Obs Group 2 Tx1 ----- Obs Group 3 ---- Tx2 Obs Group 4 ---- ---- Obs Design 16: Combined Experimental and Ex Post Facto Design Group 1a Txa Obs Group 1 Expa Random assign Group 1b Txb Obs Group 2 Expb Random assign Group 2a Txa Obs Group 2b Txb Obs

Factorial Designs Advantages of factorial Designs are: A greater precision can be obtained in estimating the overall main factor effects. Interaction between different factors can be explored. Additional factors can help to extend validity of conclusions derived.