Experimental Designs and An Introduction to the t Test Analysis Procedure

Review of Research Types Three general forms of quantitative research studies Observational Descriptive: Describes a situation Relational : Relationship between variables Causal (experimental designs) Can also be classified “non-experimental” or “experimental”. An “experimental design” is needed to show a cause and effect relationship. Internal Validity: Particularly relevant to experimental designs. Approximate truth of inferences regarding cause-effect or causal relationships.

Key Considerations External validity is established through “random selection” from the identified population (identify an accessible population and then sample). “Random Assignment” is different from “random selection”. Randomized experimental designs (a.k.a., “true experimental designs”) use “random assignment” to assign participants to comparison groups. Quasi-experimental designs do not apply “random assignment”, and instead use “intact groups”.

Cause and Effect: Three Key Considerations Covariation Temporal Precedence No plausible alternative explanations

Pre-experimental Design (single group experimental design) Posttest only: X O or Pretest-posttest O1 X 02 What are the validity threats? Why would one conduct a pre-experimental study?

R X O1 R O2 What is this design called? Experimental Design R X O1 R O2 What is this design called? What are some strengths/advantages of this design?

Factorial Experimental Design Factorial Experimental Designs allow the researcher to examine multiple independent variables (factors) that have multiple levels. Allows for the examination of “main effects” and “interaction effects”. See Figure 9.9 & 9.11 from RMK FACTORS: Factor 1 = Setting, Factor 2 = Time LEVELS: Factor 1 = In-class or Pull-out Factor 2 = 1 hr/wk or 4 hr/wk

Interaction Effects 6 6 6 6 Time The 1-hour amount works 1 hr 4 hrs The 1-hour amount works well with pull-outs while the 4 hour works as well with in class. 7 5 6 Out Setting 5 7 6 In 6 6

Factorial Experimental Design Notation R X11 O1 R X12 O2 R X21 O3 R X22 O4 Time 1 hr 4 hrs 7 5 6 Out Setting 5 7 6 In 6 6

Quasi-Experimental Designs (1 of 3) Lacks the key element of “random assignment”. The notation changes from our previous posttest-only experimental design, to a posttest only quasi- experimental design. N X O1 N O2 What is the problem with this design, and how might you overcome it?

Quasi-Experimental Designs (2 of 3) N X O1 N O2 What is the problem with this design, and how might it be overcome? N O1 X O2 N O3 O4 What is a potential problem with this pretest posttest quasi-experimental design and how might it be overcome?

Experimental Designs ( 3 of 3) Solomon Four Group Design Q.E.: N X O1 R.E: R X O1 N O2 R O2 What is the problem(s) with the above designs, and how might they be overcome? Could the provided solution below be changed to a quasi- experimental design (change the R to N)? What are the validity concerns? Solution: The design below is called the “Solomon Four Group Design”. R O1 X O2 R O3 O4 R X O5 R O6

Experimental Designs Ethical Considerations (1 of 2) Ethical decisions can arise with experimental designs. Treatment that can have a positive effect is given to some (treatment group), but withheld from others (control or comparison group). How might this ethical dilemma be addressed through research design?

Experimental Designs Ethical Considerations (2 of 2) Treatment that can have a positive effect is given to some (treatment group), but withheld from others (control or comparison group). How might this ethical dilemma be addressed through research design? Switching-Replications Design N O1 X O2 O3 N O4 O5 X O6

Experimental Designs Conclusion Designs can be customized based on the objectives, constraints and needs of the study.

Statistical Analysis of the Difference Between Groups (t Tests) See provided Excel data file for example data. Review example file and conduct basic analysis. Comparison of group means to determine if there is a “statistically significant difference” between groups. Do you know what it means when we say “statistically significant difference”? Hint – consider issues of sampling error.