KNR 295 Quasi-Exptl Design Slide 1 Quasi-Experimental Design Ch. 8 – Non-random assignment

KNR 295 Quasi-Exptl design Slide 2 Quasi-Experimental Design  Basic issue throughout is lack of random assignment…must look elsewhere for assurances of group equality (and therefore internal validity) The non-equivalent groups design (NEGD)

KNR 295 Quasi-Exptl design Slide 3 The NEGD  Examining the design more closely… These diagrams are more confusing than those that follow, but are really useful to bring home the concepts of central tendency, spread and consequently of statistical significance (see KNR 445)

KNR 295 Quasi-Exptl design Slide 4 The NEGD  Possible outcomes…1. (as slide 3)   ?

KNR 295 Quasi-Exptl design Slide 5 The NEGD  Possible outcomes…2.

KNR 295 Quasi-Exptl design Slide 6 The NEGD  Possible outcomes…3.

KNR 295 Quasi-Exptl design Slide 7 The NEGD  Possible outcomes…4.

KNR 295 Quasi-Exptl design Slide 8 The NEGD  Possible outcomes…5.

KNR 295 Quasi-Exptl design Slide 9 The RD design  Regression-discontinuity  Assigns by cut-off  Differences maximal on pre-test  (Median split is an example often used in KNR)  Can be strong on internal validity  You need to understand regression…so we stop at analysis (it’s rather complicated model verification stuff)  Can be good ethically, as one assigns the treatment to those who need it  Is costly in terms of statistical power – need nearly 3 times as many participants as randomized design

KNR 295 Quasi-Exptl design Slide 10 The RD design  RD with a null treatment

KNR 295 Quasi-Exptl design Slide 11 The RD design  RD with a 10 point treatment effect Discontinuity is observed here – the reliability of the discontinuity gives the size of the treatment effect

KNR 295 Quasi-Exptl design Slide 12 The RD design  RD with a 10 point treatment effect – showing only regression lines this time

KNR 295 Quasi-Exptl design Slide 13 The RD design  Interpretation… …depends on the meaning of the dependent variable and which side of the line is assigned to the treatment

KNR 295 Quasi-Exptl design Slide 14 The RD design  Internal validity…  It’s surprisingly good, although the justifications are complicated (to do with the fact that the treatment group is by all accounts an extension of the control group [described by the same regression line as the control])  Analysis  Well, that’s a problem ~ the stats are beyond our level of analytical expertise within this degree, so we’ll stop here  So why did I include this? Because it’s novel, distinct and interesting, of course

KNR 295 Quasi-Exptl design Slide 15 Other Quasi-Exptl. designs  A variety of other designs are available  These vary in their suitability depending on the situation in which they are used  No need to go over them all – see text for examples

KNR 295 Quasi-Exptl design Slide 16 Notes on Proposal  Preparation  Select one paper only (should be recent)  Analyze that paper in terms of its validity  See research paper evaluation, parts A & B  Identify one or more ways in which the study might be improved or modified  See research paper evaluation, parts C & D  Form a hypothesis for the modified study  See research paper evaluation, part E

KNR 295 Quasi-Exptl design Slide 17 Notes on Proposal  Completion  Introduction  Review & critique paper. Describe proposed modification. Give rationale for proposed modification. Provide support, if appropriate, using other research and/or reference to research methods text. End with statement of hypothesis  Overall, 2 goals of introduction  Introduce and justify the proposed modification  State and justify the experimental hypothesis

KNR 295 Quasi-Exptl design Slide 18 Notes on Proposal  Completion  Methods  Describe key elements of study (see proposal)  Results  Describe expected results (see proposal)  Critique  Given the modification you have developed, evaluate the validity of the experiment you have proposed  No need to repeat parts already covered in introduction, but you should refer to them where necessary.