Quantitative Research Designs

Questions for Thought What is the difference between experimental, quasi-experimental, and nonexperimental?

Research Design Research Design It is the researcher’s overall plan for Answering the research question Testing the research hypotheses

Research Design Research Design Will involve decisions regarding Will there be an intervention What types of comparison will be made What procedures will be used to control extraneous variables Extraneous are variables that may affect the independent or dependent variables, and need to be controlled When and how many times will data be collected from study participants In what setting will the study occur

Types of Quantitative Research Designs 1. Experimental 2. Quasi-experimental 3. Non-experimental

Experimental Research Designs Experimental Designs

Experimental Research Designs Greatest amount of control over the independent variable Researcher is an active agent rather than a passive observer

Experimental Research Designs Characteristics of Experiments Manipulation Control Randomization Need to have these three factors to be considered an experimental design

Characteristics of Experiments Manipulation The researcher does something to study participants Independent variable is manipulated by administering an experimental treatment or intervention to some participants and withholding it from others or administering another treatment (control group) The researcher controls and varies the independent variable and observes its effect on dependent variable

Characteristics of Experiments Control The researcher introduces controls over the experimental situation through use of control groups Control group is the participants in an experiment that do not receive the experimental treatment and whose performance provides a baseline against which the effects of the treatment can be measured The researcher compares the control group’s performance on a dependent variable to the experimental group

Characteristics of Experiments Randomization (random assignment) The researcher assigns study participants to control or experimental groups randomly Each participant has an equal chance of being included in any group Each group is considered to be comparable therefore any changes could be attributed to the independent variable (treatment)

Experimental Designs Types: 1. After-only (post-test-only) design 2. Before-after (pretest-posttest) design 3. Factorial Design 4. Repeated-Measures Design 5. Clinical Trials

Experimental Designs 1. After-only (post-test-only) design Two groups Collection of data after intervention

Experimental Designs 2. Before-after (pretest-posttest) design Collects baseline (pretest data) data before intervention Then collects data after the intervention (post-test data)

Experimental Designs 3. Factorial Design Manipulation of two or more variables simultaneously Allows evaluation of main effects (effects resulting from the intervention) and interaction effects (effects resulting from combining the treatment methods) Participants are assigned at random to a combination of treatments

Experimental Designs 4. Repeated Measures Design (crossover design) Within-subjects designs A research design in which a single group of subjects is compared under different conditions or at different points in time (i.e. before and after surgery) The same study participants are used to evaluate more than one treatment/intervention Participants are randomly assigned Participants serve as their own control group

Experimental Designs 5. Clinical Trials Involves the testing of a clinical treatment Random assignment of participants to experimental or control groups Large sample, can be across the world to increase generalizability Usually use a before-after or after-only design

Advantages/Disadvantages of Experimental Design Most powerful for testing cause and effect hypotheses Disadvantages Some variables can not be manipulated Not feasible Not ethical Potential of Hawthorne effect Being in a study causes people to change their behaviour

Advantages/Disadvantages of Experiments To reduce the potential for the Hawthorne effect Researchers use double-blind experiments Neither researcher nor participant know who is getting which treatment

Quasi-Experimental Research Designs Quasi-Experimental Designs

Quasi-experimental Research Designs Involves manipulation of an independent variable Lacks either randomization or control-group or both *** Weaker than experimental designs Uses the term comparison group instead of the term control group Also known as pre-experimental design

Quasi-experimental Research Designs Types: 1. Nonequivalent Control-Group Design 2. Time-Series Design

Quasi-experimental Research Designs 1. Nonequivalent Control-Group Before-After Design Most frequently used quasi-experimental design Involves a treatment and two or more groups of participants Collects data before and after intervention No randomization Therefore the groups can not be assumed to be equivalent

Quasi-experimental Research Designs 2. Time-Series Design Has neither a control group nor randomization Involves the collection of data over an extended period of time and the introduction of the intervention during that period Therefore data is collected before the intervention (multiple collection points) and again after implementation (multiple collection points)

Advantages/Disadvantages of Quasi-Experiments Practical Disadvantages Cause and effect can not be determined as easily as experimental designs Could be other reasons for the change in dependent variable (rival hypotheses)

Nonexperimental Research Designs Nonexperimental Designs

Nonexperimental Research Designs Used where the independent variable can not be manipulated Used when it is unethical to manipulate the independent variable Useful in descriptive studies

Nonexperimental Research Designs Types: 1. Ex post facto (correlational) 2. Descriptive

Nonexperimental Research Designs 1. Ex post facto (correlational) Research is conducted after the independent variable has been manipulated or intervention applied Studys relationships among variables No control of independent variable Can be retrospective or prospective studies

Nonexperimental Research Designs Retrospective Looks at dependent variable in the present and attempts to link this effect to cause in the past Looks at present outcomes and tries to determine what factors caused it Lung cancer currently, linked to smoking in the past Prospective Looks at the presumed cause and then goes forward in time to observe presumed effects Considered stronger than retrospective studies Smoking currently may cause lung cancer in the future

Nonexperimental Research Designs 2. Descriptive Design Purpose is to observe, describe and document aspects of a situation

Advantages/Disadvantages of Nonexperimental Research Designs Good for problems not able to be studied through experimentation Disadvantages Can’t determine causal relationships conclusively

Research Design and Time When to use multiple points of data collection Time-related processes Phenomena evolves over time i.e. healing, growth Time-sequenced phenomena Sequencing of phenomena Comparative purposes Compare phenomena over time Enhancement of research control

Research Design and Time Cross-sectional Studies Collection of data at one point in time Difficult to infer changes and trends over time Longitudinal Studies Collect data over an extended period of time Can show changes over time

Research Design and Time Longitudinal Studies Trend studies Different samples from a population are studied over time, always from same population Panel studies Same participants supply the data at two or more points in time Follow up studies Determine the outcomes of people with a specific condition or who have received a specific treatment

Techniques of Research Control A major purpose of research design is to maximize the researcher’s control over the research situation

Techniques of Research Control Concerned with eliminating possible extraneous influences on the dependent variable A variable that confounds (confuses) the relationship between the independent and dependent variables Concerned with understanding the true relationship between independent and dependent variables Research control attempts to exclude contaminating factors, to control the extraneous influences

Techniques of Research Control Controlling External Factors Want to achieve constancy of conditions Researcher is confident that situational contaminants or conditions are not affecting the data Want to control the environment As environment influences people’s emotions and behaviour Want to control the time Maintain constancy of time, control time of day, time of year Research protocols Specific procedures are outlined to maintain consistency

Techniques of Research Control Controlling Intrinsic Factors Intrinsic factors are participant's characteristics Age, gender, habits

Techniques of Research Control Methods of Controlling Intrinsic Factors Randomization Equalized groups in relation to extraneous variables The best, most effective method Homogeneity Only participants who are the same with respect to extraneous variables are included in the study Matching Using information about participants’ characterisitcs to form comparison groups Statistical Control Statistical analysis

Internal and External Validity Internal Validity The extent to which it is possible to make an inference that the independent variable is truly influencing the dependent variable Quasi-experimental, preexperimental and correlational studies are especially susceptible to threats to internal validity Experimental designs usually reduce threats to internal validity

Internal and External Validity Threats to Internal Validity History threat Selection threat Maturation threat Mortality threat

Internal and External Validity Threats to Internal Validity History threat Refers to events external to the intervention but occur at the same time which can affect the dependent variable Threat in quasi-experimental and non-experimental designs Not usually threat in experimental studies as all groups will be affected by history

Internal and External Validity Threats to Internal Validity Selection threat (self-selection) Results from pre-existing differences between groups which may affect the dependent variable Groups may not be equivalent Most problematic threat to studies not using experimental design

Internal and External Validity Threats to Internal Validity Maturation threat Occurs when changes to the dependent variable (outcome) results from the passage of time

Internal and External Validity Threats to Internal Validity Mortality threat Refers to the loss of participants (attrition) from the different groups in the study

Internal and External Validity Refers to the generalizability of the research findings to other settings or samples Does intervention work in another setting or with different people A good sampling design increases chance of generalizability

Reference Loiselle, C. G., Profetto-McGrath, J., Polit, D. F., & Beck, C. T. (2011). Canadian essentials of nursing research (3rd ed.). Philadelphia, PA: Lippincott, Williams & Wilkins.