Ron Sterr Kim Sims Heather Cruz aka “The Carpool” Experimental Studies Ron Sterr Kim Sims Heather Cruz aka “The Carpool”

These imply cause and effect relationships Independent variable = the cause or the action Dependent variable = the outcome Also referred to as the: criterion, effect or consequence

For this causation to be established, three conditions must be met: A statistical relationship between the two variable must be demonstrated The presumed cause must occur BEFORE the presumed effect All other causes of the presumed effect must be ruled out

Statistical Relationship To do this you must run some form of statistics to prove this relationship T-test Chi square

Cause BEFORE Effect World War II cannot be considered a cause of the Great Depression because one physically happened before the other

Rule out other causes Make sure there are no other variables that might have an “effect” on your action. Rival hypotheses - alternative causes - you want to proves these false in order for the third condition to be met. These suggests that an extraneous variable was the entire or part of the cause for your statistical relationship.

Internal Validity The extent to which one can claim that the independent variable was the cause of the dependent variable Threats to Internal Validity refers to rival hypotheses Specific threats: history, maturation, testing, instrumentation, nonequivalence, regression, and mortality

A Closer look at threats History - specific events that are not part of the independent variable but occur during the time period in which changes in the dependent variable are observed Maturation - occurs when internal events occur that may cause changes to to dependent variable such as age

A Closer look at threats Testing - in a pretest/treatment /posttest design there is sometimes an effect on the dependent variable from the pretest, itself - this is called the practice effect Instrumentation - occurs when the method of measuring the dependent variable changes from group to group

A Closer look at threats Nonequivalence - when a subject characteristic that makes the groups compared unequal in any respect Regression - occurs when the subjects in a study are chosen because of their extreme position on some variable - I.e. mentally retarded students, gifted students, etc.

A Closer look at threats Mortality - attrition is another word for this - when people that started the study do not finish the study

How to Control Threats to Internal Validity Logical argument Use of control groups - a group that is not exposed to the “treatment Random assignment is used to control nonequivalence Matching is another way to control for nonequivalence Random is considered the best way to control internal validity

More threats - how can there be more? Non-significant differences on the dependent variable To control these, consider alternative hypotheses; Sample size too small Dependent variable was poorly measured Treatment was not administered long enough

3 Types of Research That Address Cause and Effect True Experiments The independent variable is a treatment that the researcher deliberately introduces and manipulates, and the researcher’s control extends the ability to assign subjects at random to the levels of the independent variable

3 Types of Research That Address Cause and Effect Quasi Experiments When the researcher has only partial control over the independent variables employed as treatments

3 Types of Research That Address Cause and Effect Causal-Comparative Studies - aka ex post facto studies The researcher does not have control over independent variables (IV) The IV has occurred in the past, prior to the study Subjects have assigned themselves to the various “treatment conditions” - public or private schools IV is some fixed characteristic of the subjects - birth order

True Experiments in greater detail •Randomized, pretest-posttest, control group design Step 1 - select subjects - either randomly (R-S) or nonrandom (No R-S) Step 2 - Assign subjects to groups - either by random (R-A) or nonrandom (No R-A) Step 3 - Pretest (O1) - this is optional Step 4 - Treatment period - make sure to hold the conditions constant between groups and with how the treatment is delivered Step 5 - Posttest (O2) - All differences are subject to inferential tests of statistical significance - you also have to rule out threats to internal validity in order to attribute your differences to the treatment

Variety of True Experiments Randomized, pretest/postest, control group design Randomized, posttest only, control group design Soloman four-group design - random assignment - two groups tested, two are not Factorial experiment with than one independent variable - all subjects are pretested - treatment is administered individually - all subjects are posttested One more independent variable plus one or more moderator variable (a subject, task, or method characteristic that can be used as a factor in an experiment along with the variable of interest.

External vs. Internal External Control - how far can you generalize your results? Internal Control - to what extent is the effect due to the treatment?

Population External Validity - how far can this be generalized? 1st consideration - random selection is the best way to be able to generalize your results 2nd consideration - description and judgment - If the researcher provides description and judgment for the sample along with clues about how the sample might have been atypical, then the reader can make an informed judgment about similarities of the sample and the target population.

Noncomparability Threats Lack of comparability between samples Treatment effect is limited to individuals with certain subject characteristics Ecological Threats - Lack of control of the environment of the treatment - setting Demand and reactive characteristics - alerting participants to the purpose of the study can skew your effect Hawthorne Effect - Western Electric Company

Noncomparability Threats Novelty and disruption effects Operations Effects Experimenter Effects Task Effects

More threats to external validity Poorly defined IV Placebo effect Poorly defined DV Interaction of treatment and time of measurement Pretest sensitization

Criteria for Judging Experimental Studies 1 - Were the methods for selecting the subjects clearly defined? 2 - Have the methods for assigning subjects to treatment conditions been described? 3 - Could another researcher replicate the study? 4 - Does the treatment produce statistical significance?

Criteria for Judging Experimental Studies 5 - Is there any mortality effect? 6 - Has the IV had an effect on the control group? 7 - Has their been implementation fidelity? 8 - Has the DV been reliably used and measured for validity? 9 - Have you controlled for threats to external validity?