E XPERIMENTAL D ESIGNS AND A N I NTRODUCTION TO THE T T EST A NALYSIS P ROCEDURE

R EVIEW OF R ESEARCH T YPES Three general forms of quantitative research studies Observational Descriptive: Describes a situation Relational : Relationship between variables Causal (experimental designs) Can also be classified “non-experimental” or “experimental”. An “experimental design” is needed to show a cause and effect relationship. Internal Validity : Particularly relevant to experimental designs. Approximate truth of inferences regarding cause-effec t or causal relationships.

K EY C ONSIDERATIONS External validity is established through “random selection” from the identified population (identify an accessible population and then sample). “Random Assignment” is different from “random selection”. Randomized experimental designs (a.k.a., “true experimental designs”) use “random assignment” to assign participants to comparison groups. Quasi-experimental designs do not apply “random assignment”, and instead use “intact groups”.

C AUSE AND E FFECT : T HREE K EY C ONSIDERATIONS Covariation Temporal Precedence No plausible alternative explanations

P RE - EXPERIMENTAL D ESIGN ( SINGLE GROUP EXPERIMENTAL DESIGN ) XO or O 1 X0 2 What are the validity threats? Why would one conduct a pre-experimental study?

E XPERIMENTAL D ESIGN RXO 1 RO 2 What is this design called? What are some strengths/advantages of this design?

F ACTORIAL E XPERIMENTAL D ESIGN Factorial Experimental Designs allow the researcher to examine multiple independent variables (factors) that have multiple levels. Allows for the examination of “main effects” and “interaction effects”. See Figure 9-6 from RMK FACTORS: Factor 1 = Setting, Factor 2 = Time LEVELS: Factor 1 = In-class or Pull-out Factor 2 = 1 hr/wk or 4 hr/wk

I NTERACTION E FFECTS hr4 hrs Out In Time Setting The 1-hour amount works well with pull-outs while the 4 hour works as well with in class.

F ACTORIAL E XPERIMENTAL D ESIGN N OTATION RX 11 O 1 RX 12 O 2 RX 21 O 3 RX 22 O hr4 hrs Out In Time Setting 5 7

Q UASI -E XPERIMENTAL D ESIGNS (1 OF 3) Lacks the key element of “random assignment”. The notation changes from our previous posttest-only experimental design, to a posttest only quasi- experimental design. NXO 1 NO 2 What is the problem with this design, and how might you overcome it?

Q UASI -E XPERIMENTAL D ESIGNS (2 OF 3) NXO 1 NO 2 What is the problem with this design, and how might it be overcome? NO 1 XO 1 NO 3 XO 2 What is a potential problem with this pretest posttest quasi-experimental design and how might it be overcome?

Q UASI -E XPERIMENTAL D ESIGNS ( 3 OF 3) S OLOMON F OUR G ROUP D ESIGN NXO 1 NO 2 What is the problem with the above design, and how might it be overcome? Could the provided solution below be changed to a randomized experiment? (change the N to R). NO 1 XO 1 NO 3 XO 2 NXO 2 NO 2

E XPERIMENTAL D ESIGNS E THICAL C ONSIDERATIONS (1 OF 2) Ethical decisions can arise with experimental designs. Treatment that can have a positive effect is given to some (treatment group), but withheld from others (control or comparison group). How might this ethical dilemma be addressed through research design?

E XPERIMENTAL D ESIGNS E THICAL C ONSIDERATIONS (2 OF 2) Treatment that can have a positive effect is given to some (treatment group), but withheld from others (control or comparison group). How might this ethical dilemma be addressed through research design? Switching-Replications Design NO 1 XO 2 O 3 NO 4 O 5 XO 6

E XPERIMENTAL D ESIGNS C ONCLUSION Designs can be customized based on the objectives, constraints and needs of the study. Note variations presented at the end of RMK Ch. 10 and Ch. 11.

S TATISTICAL A NALYSIS OF THE D IFFERENCE B ETWEEN G ROUPS ( T T ESTS ) See provided Excel data file for example data. Review example file and conduct basic analysis. Comparison of group means to determine if there is a “statistically significant difference” between groups. Do you know what it means when we say “statistically significant difference”? Hint – consider issues of sampling error.