Experimental Design Chapter 1 Research Strategies and the Control of Nuisance Variables

Introduction Experimental design – A plan for assigning subjects to experimental conditions and the statistical analysis associated with the plan – Identifies the independent, dependent, and nuisance variables and indicates the way in which the randomization and statistical aspects of an experiment are to be carried out

Formulation of Plans for Collection and Analysis of Data Acceptable research hypotheses Distinction between independent and dependent variables Selection of independent variable – Quantitative independent variables – Qualitative independent variables Selection of the dependent variable – Multivariate designs Nuisance variables – Bias

Research Strategies Experiments Quasi-experiments Surveys Case studies Naturalistic observation

Other Research Strategies Ex post facto studies – Retrospective and prospective studies Longitudinal and cross-sectional studies Longitudinal-overlapping and time-lag studies Time-series and single-subject studies

Threats to Valid Inference Making Four categories of threats to valid inference making 1. Statistical conclusion validity 2. Internal validity 3. Construct validity of causes and effects 4. External validity

Threats to Statistical Conclusion Validity 1. Low statistical power 2. Violated assumptions of statistical tests 3. Fishing for significant results and the error rate problem 4. Reliability of measures 5. Reliability of treatment implementation 6. Random irrelevancies in the experimental setting 7. Random heterogeneity of respondents

Threats to Internal Validity 1. History Events other than the treatment that occur between the time the treatment level is assigned and the DV is measured 2. Maturation  Processes not related to treatment that occur within the subject 3. Testing  Repeated testing may result in familiarity 4. Instrumentation  Changes in calibration of a measuring instrument

Threats to Internal Validity 5. Statistical regression  Occurs when the measurement of the DV is not perfectly reliable 6. Selection  Prior differences in subjects 7. Mortality  Loss of subjects 8. Interactions with selection  Selection-history effects or Selection-maturation effects

Threats to Internal Validity 9. Ambiguity about the direction of causal influence 10. Diffusion or imitation of treatments  If the subjects in different levels can communicate, differences may be compromised 11. Compensatory rivalry by respondents receiving less desirable treatments 12. Resentful demoralization of respondents receiving less desirable treatments

Threats to External Validity 1. Interaction of testing and treatment 2. Interaction of selection and treatment 3. Interaction of setting and treatment 4. Interaction of history and treatment 5. Reactive arrangements 6. Multiple-treatment interference

Other Threats to Valid Inference Making Experimenter-expectancy effect Demand characteristics Subject-predisposition effects – Cooperative-subject effect – Screw you effect – Evaluation apprehension – Faithful subjects Placebo effect

Controlling Nuisance Variables General Approaches to Control – Experimental control – Statistical control Some Specific Approaches to Control – Blind procedures – Deception – Unobtrusive experimentation – Multiple researchers

Controlling Nuisance Variables Some Specific Approaches to Control – Debriefing – Experimenter-expectancy control groups – Unrelated-experiment technique – Quasi-control group – Yoked control procedure

Ethical Treatment of Subjects 1. Responsibility of researcher 2. Informed consent 3. Protection from harm 4. Protection of rights 5. Research deception 6. Confidentiality 7. Debriefing 8. Accurate and reporting