Chapter 9 Scrutinizing Quantitative Research Design

Aspects of Quantitative Research Design Intervention versus no intervention Nature of any comparisons Methods of controlling extraneous variables Timing of data collection Research sites and settings Communication with subjects

Dimensions of Quantitative Research Design Control over independent variable (experimental, quasi-experimental, preexperimental, nonexperimental) Type of group comparisons (between- subjects vs. within-subjects designs) Timeframes (Cross-sectional vs. longitudinal designs)

Dimensions of Quantitative Research Design (cont’d) Observance of independent and dependent variables (retrospective, prospective) Setting (Naturalistic setting, laboratory)

Characteristics of True Experiments Manipulation—researcher does something to some subjects (introduces an intervention or treatment) Control—researcher introduces controls, including a control group

Characteristics of True Experiments (cont’d) Randomization (also called random assignment)—researcher assigns subjects to groups at random –Typical assignment is to an experimental group or a control group –May be done by computer or through a table of random numbers

Experimental Designs After-only (posttest-only) design Before-after (pretest-posttest) design Factorial design Crossover (repeated measures) design

Example of a Factorial Design—Infant Stimulation

The Control Condition or Counterfactual No intervention An alternative intervention A placebo or pseudo-intervention Standard methods of care Different doses or intensities of treatment Delayed treatment

Quasi-Experiments Lack either randomization or control group, but introduce other controls Types of quasi-experimental designs: –Nonequivalent control group pretest- posttest design –Time series designs

Preexperimental Designs Lack control group and/or randomization Lack controls of quasi-experiments Examples of preexperimental designs: –Nonequivalent control group, after-only design –One group before-after design

Evaluation of quasi-experimental and pre- experimental designs May be easier, more practical than true experiments BUT More difficult to infer causality Usually several alternative rival explanations for results

Nonexperimental research Correlational (ex post facto) research –Prospective designs –Retrospective designs Case-control designs Descriptive research –Purely descriptive –Descriptive correlational study

Research Design and the Time Dimension Cross-sectional design—Data are collected at a single point in time Longitudinal design—Data are collected two or more times over an extended period –Trend studies –Panel studies –Follow-up studies

Controlling Extraneous Variables Controlling external factors –Achieving constancy of conditions –Control over environment, setting, time Controlling intrinsic factors –Control over subject characteristics

Methods of Controlling Intrinsic Factors Randomization Subjects as own controls (crossover design) Homogeneity (restricting sample) Matching Statistical control (e.g., analysis of covariance)

Characteristics of Good Research Design in Quantitative Studies Statistical conclusion validity —the ability to detect true relationships statistically Internal validity –the extent to which it can be inferred that the independent variable caused or influenced the dependent variable

Characteristics of Good Research Design in Quantitative Studies (cont’d) External validity–the generalizability of the findings to other samples or settings Construct validity–the adequacy of measuring key constructs

Threats to Statistical Conclusion Validity Low statistical power Weak construction of independent variable and counterfactual –Large differences between groups needed Unreliable implementation of a treatment Inadequate participation in treatment conditions

Threats to Internal Validity History threat Selection threat Maturation threat Mortality threat –Often a result of differential attrition from groups

Threats to External Validity Inadequate sampling Novelty effect Expectancy effect (Hawthorne effect) Placebo effect Artificiality of research environment