Conducting Experimental and Quasi-Experimental Research R. Eric Heidel, PhD University of Tennessee Graduate School of Medicine © 2014 Pearson Education, Inc.

Characteristics of Experimental Research Experimental research is designed to establish cause-and-effect relationships. Researchers intend to establish the relationship between an "independent variable" and a "dependent variable." Independent variable—a variable that stands alone and is not changed by the other variables being measured Dependent variable—a variable that is dependent upon some other factor © 2014 Pearson Education, Inc.

Level of Evidence: Level of Control in Experiments To establish a cause-and-effect relationship, it is important to control for other variables that may influence an outcome. Extraneous variables are variables that may affect the dependent variable and are not related to the major purpose of the experiment. Experimental designs can control for extraneous variables. Without controlling variables, the experimental method cannot exist. © 2014 Pearson Education, Inc.

Level of Evidence: Level of Control in Experiments (cont'd) Control allows researchers to arrange the experiment so that the effect of the variables can be studied. Researchers cannot control for everything, but it is acceptable to attempt to control for variables that might have a significant impact on the experiment. Because experiments require rigorous control over the variables, they are generally considered to be the highest level of evidence. Not all experimental designs are equal in regard to their level of evidence. © 2014 Pearson Education, Inc.

Level of Evidence: Level of Control in Experiments (cont'd) Levels of evidence: Level I—controlled and randomized Level II—1- controlled but not randomized Level II—2- well-designed cohort or case-controlled analytic study Level II—3- multiple time series Level III—expert opinions based on clinical experience, expert committees, or descriptive studies © 2014 Pearson Education, Inc.

Level of Evidence: Level of Control in Experiments (cont'd) When an experiment is carried out, the investigators must take precautions to be sure that there is as much equivalence as possible among the groups in the study. Several procedures are used to ensure equal groups: Random assignment: the assignment of experimental subjects to groups such that every member of the population has an equal chance of being assigned to any of the groups © 2014 Pearson Education, Inc.

Level of Evidence: Level of Control in Experiments (cont'd) Randomized matching: subjects are matched on as many extraneous variables as could possibly affect the dependent variable Statistical control: used to control for differences among the groups in the experiment Analysis of covariance or ANCOVA: analyzes differences of the experimental groups on the dependent variable only after initial differences on the pretest measures are taken into account © 2014 Pearson Education, Inc.

Level of Evidence: Level of Control in Experiments (cont'd) Controlling the variables can be done by holding the variables constant, manipulating the variables systematically, and randomizing the situations. Holding the variables constant is achieved by treating all subjects alike except for exposure to the treatment. Manipulating the variables systematically is done by controlling the order in which the experiment is given to the subjects. Randomizing the situation variables is done by randomly assigning half of the participants to the control group and the other half to the experimental group. © 2014 Pearson Education, Inc.

Internal Validity Controlling the extraneous variables increases the level of trust or level of evidence about the relationship between independent and dependent variables. The greater the control, the greater the internal validity. When designs allow extraneous variables to influence the outcome variable, internal validity is threatened, thereby decreasing the level of evidence. Internal validity is extremely difficult to achieve outside of the laboratory because there are too many extraneous variables to control. © 2014 Pearson Education, Inc.

Threats to Internal Validity Extraneous variables that threaten internal validity and level of evidence: maturation effect history effect testing effect instrumentation effects regression to the mean differential selection experimental mortality selection–maturation interaction diffusion of treatment sequence effects contamination © 2014 Pearson Education, Inc.

External Validity External validity is the researcher's ability to generalize the findings of an experiment. Researchers want to be able to generalize their results to groups, environments, or circumstance beyond the actual groups in their current study. Attempts to control threats to internal validity may jeopardize external validity. Some actions taken by the investigator to control all the external variables may be so stringent that the results cannot be generalized to other populations or settings. © 2014 Pearson Education, Inc.

External Validity (cont'd) Population Validity Refers to the extent to which the results of an experiment can be generalized from the sample used in the study to a larger group of similar people. Two types of population validity: First type: the extent to which results can be generalized from the experimental sample to a defined population Second type: the extent to which personological variables interact with treatment effects © 2014 Pearson Education, Inc.

External Validity (cont'd) Ecological Validity It is the extent to which the results of an experiment can be generalized from the set of environmental conditions in the experiment to other environmental conditions. If the results can be obtained only under a very limited set of conditions, those results have low ecological validity. There are 10 factors that may contribute to the ecological validity of an experiment. If investigators find that a discrepancy exists between the experimental condition and real-world setting, then they should note this in the report as a limitation to the generalizability of the study. © 2014 Pearson Education, Inc.

Constructing Experimental Designs to Control Variables The major purpose in constructing an experimental design is to control as many extraneous variables as possible. In attempting to control for so many variables, real-world research can product artificial results. This occurs because the environment and/or the subjects are sometimes put into unnatural situations. © 2014 Pearson Education, Inc.

Constructing Experimental Designs to Control Variables (cont'd) There are several experimental designs that can be divided into the following categories: preexperimental quasi-experimental true experimental © 2014 Pearson Education, Inc.

Constructing Experimental Designs to Control Variables (cont'd) Preexperimental Designs The One-Shot Case Study Design The One-Group Pretest-Posttest Design The Static-Group Comparison Design Quasi-Experimental Designs The Nonequivalent Control Group Design The Counterbalanced Design The Interrupted Time Series Design The Equivalent Time Sample Design © 2014 Pearson Education, Inc.

Constructing Experimental Designs to Control Variables (cont'd) True Experimental Designs The Posttest-Only Control Group Design The Pretest-Posttest Control Group Design The Solomon Four-Group Design © 2014 Pearson Education, Inc.

Constructing Experimental Designs to Control Variables (cont'd) Factorial or Multivariable Designs One in which two or more variables are manipulated simultaneously to allow study of the independent effect of each variable on the dependent variable. The effects caused by the interaction among the several variables are assessed. The effect of each independent variable on the dependent variable is called a main effect. © 2014 Pearson Education, Inc.

Constructing Experimental Designs to Control Variables (cont'd) The effect of the interaction of two or more independent variables on the dependent variable is termed an interaction effect. Factorial designs allow the researcher to investigate several factors or variables in a single experiment. Only factorial designs allow the investigator to test for interaction among variables. © 2014 Pearson Education, Inc.

Clinical Trials: The Randomized, Controlled Trial Generally thought of as an experimental design for medical and pharmaceutical investigation but has been broadened to incorporate prospective clinical designs to test the efficacy of an intervention against a controlled condition. As clinical entities embrace evidence-based practice, the RCT is viewed as the best type of evidence on which to base decisions and establish guidelines. © 2014 Pearson Education, Inc.

Clinical Trials: The Randomized, Controlled Trial (cont'd) A clinical trial: is a study of human volunteers to address specific health questions is the most expedient and safe way to find treatments to improve health has inclusion and exclusion criteria © 2014 Pearson Education, Inc.

Clinical Trials: The Randomized, Controlled Trial (cont'd) Phase I introduction of the new drug or device into a human being is most concerned about the safety of the drug or device in humans Phase II still has safety and efficacy as high priorities and only occurs if safety has been demonstrated in Phase I trials participants usually have the condition under study trials provide much needed evidence of clinical significance trials are often placebo-controlled and double-blinded © 2014 Pearson Education, Inc.

Clinical Trials: The Randomized, Controlled Trial (cont'd) Phase III trials are usually a full-blow RCT trials prove the safety and efficacy of the intervention (drug/device) over a longer period of time randomization, placebo-control, and double-blinding are customary Phase IV trials are conducted after approval by the U.S. Food and Drug Administration (FDA) and are used to monitor adverse effects, check for new usage indications, gain more information about the product, or educate clinical researchers © 2014 Pearson Education, Inc.

Clinical Trials: The Randomized, Controlled Trial (cont'd) Parallel Group Trial pretest-posttest control group design only one set of patients receives the new drug comparison is between the two groups © 2014 Pearson Education, Inc.

Clinical Trials: The Randomized, Controlled Trial (cont'd) Crossover Trial All participants receive the new drug. Patients are randomized into an intervention group or a control group, with the intervention group taking the medication for a selected period of time. After that period of time, the original intervention group becomes the control group taking the placebo while the original control group serves as the intervention group receiving the new drug. © 2014 Pearson Education, Inc.

Clinical Trials: The Randomized, Controlled Trial (cont'd) The crossover design allows for evaluation after intervention with one group and then again following intervention with the second group participants serve as their own controls, and within-group comparisons are made © 2014 Pearson Education, Inc.

Comparative Effectiveness Research (CER) Studies are conducted to generate new evidence about the effectiveness or comparative effectiveness of health care service, a test, a procedure, or treatment. There are seven steps in conducing this research. Identify emerging clinical interventions. Synthesize current medical research. Look for gaps between clinical practice needs and existing medical research. © 2014 Pearson Education, Inc.

Comparative Effectiveness Research (CER) (cont'd) Generate new scientific evidence. Educate clinical researchers. Distribute findings to all stakeholders including practicing health care workers. Reach out to all stakeholders through several means such as a citizens forum. Although CER is clinically driven, those in other fields, such as education, can apply this format to their respective fields. © 2014 Pearson Education, Inc.

Advantages and Disadvantages of the Experimental Method convenience replication adjustment of variables establishment of cause-and-effect relationships Disadvantages cost inability to generalize securing cooperation © 2014 Pearson Education, Inc.