2.4. Design in quantitative research Karl Popper’s notion of falsification and science – If a theory is testable and incompatible with possible empirical observations, it is falsifiable (scientific); if not, it is unscientific Positivism – Only authentic knowledge is scientific knowledge – Logical positivism - holds that philosophy should aspire to the rigor of science. Causally-oriented and descriptively-oriented research

Causally-oriented research

The notion of cause Causal and non-causal associations – David Hume’s criteria for demonstrating causal association: 1. Contiguity - nearness or contact between presumed cause and effect; 2. Temporal precedence - cause has to precede the effect in time; and 3. Constant conjunction - cause has to be present whenever the effect is obtained. The Hypothetico-deductive approach (Popper) Postulate theory based on observation Explicate implications and develop a set of falsifiable hypotheses Develop and implement methods to test these hypotheses

Experimental Research

Internal Validity Approximate validity with which a cause-and-effect relationship between two variables can be inferred Threats to Internal Validity – Selection – History – Maturation – Testing – Instrumentation – Statistical Regression – Attrition – Instability

External Validity Approximate validity of cause-effect generalisations to target populations of persons, settings, and times Threats to External Validity – Selection-Treatment and History-Treatment Interactions (population validity) – Setting-Treatment Interactions (ecological validity) – Confounding (e.g., hypothesis-guessing within experimental conditions; experimenter expectancies; confounding constructs and levels of constructs; interaction of testing and treatment)

Pseudo-Experimental Designs – No control groups established – Can include pre-post measures Quasi-Experimental Designs Compares groups that are not randomly assigned to conditions (i.e., intact groups) Example: The Non-Equivalent Control-Group Design Pretest Treatment Posttest O X O O O Between-Case Experimental Research

Randomised Designs Compares groups that are randomly assigned to conditions Example: The Pretest-Posttest Control Group Design Pretest Treatment Posttest R O X O R O O

Threats to Internal Validity in Pseudo- Experimental Designs No threats ruled out conclusively Alternative approaches taken when no other design is possible: – Pre-post measures – Use of archival data

Quasi-Experimental Designs Compares groups that are not randomly assigned to conditions (i.e., intact groups) Example: The Non-Equivalent Control-Group Design Pretest Treatment Posttest O X O O O Threats to Internal Validity – Selection – Selection-History – Selection-Maturation – Selection-Testing – Selection-Instrumentation – Selection-Statistical Regression – Differential Attrition Threats to Internal Validity: Quasi- Experimental Designs

Threats to Internal Validity Ruled Out – Selection/Selection Interactions – History – Maturation – Testing – Instrumentation – Statistical Regression Threats not ruled out: – Differential Attrition Threats to Internal Validity: Randomized Experimental Designs

Within-case experimental research Standard within-case designs Single-Case (Interrupted Time Series) Designs – Phase change (AB, ABA, and ABAB) Designs Establishing consistency in behaviour before the intervention First observation period establishes baseline levels Withdrawal components can be added to strengthen interpretation – Multiple Baseline Designs Multiple baseline across behaviours Multiple baseline across people

– Threats to internal validity – within-subject threats (e.g., all threats listed for pre- experimental designs for AB designs; practice effects) – failures to reverse to baseline (ABA and ABAB designs) – covariation (e.g., multiple baseline designs) – Threats to external validity

Mixed Experimental Design (Chambers, 2009) Rationale: Reported that individuals AD/HD may perform better on cognitive tasks when background music is used in the workplace. Aim : Evaluate whether this effect varies with the extent to which the rhythmic component of the music is dominant; determine whether the effect is due to enhanced self-regulation. Participants: Individuals diagnosed with AD/HD and age- and gender-matched, non-diagnosed community participants. Design: Participants completed five types of cognitive task under each of three conditions: (i) Rhythm dominant background music, (ii) non-rhythm dominant background music, and (iii) silence. Results: No evidence for superiority for background music. Implications: Use of background music to improve performance of adults with AD/HD is not supported.