Quasi-experimental Designs & Survey Research

Quasi-experiments Hedrick, Bickman & Rog (1993) “a quasi-experimental design is not the method of choice, but rather a fallback strategy” Cannot infer cause and effect, BUT well designed quasi-experiments enable you to demonstrate that rival interpretations are rendered unlikely

Non-equivalent Group Designs Posttest only nonequivalent control group design (aka static group comparison) X O (treatment grp) O (nonequivalent control grp) X=treatment O=measurement/observation Because there is no random assignment to groups, confounding variables may explain any difference observed

Pre-test/Post-test Non-equivalent Control Group O X O treatment group O O non-equivalent control group Pre Post

Example: Research Methods and Reasoning Ability Intervention: critical thinking seminar Research Methods students receive the intervention (i.e., participate in the critical thinking seminar) Developmental Psychology students are used as a nonequivalent control group (i.e., do not attend the seminar)

Interrupted Time Series Designs Extension of the simple one group pre and post design Participants are pre-tested a number of times and then post-tested a number of times after being exposed to the treatment intervention O1 O2 O3 O4 X O5 O6 O7 O8

Interrupted Time Series Designs Useful when: cannot randomize participants it is possible to obtain a series of assessments of the DV before and after treatment

Example: Interrupted Time Series Designs Intervention: Course to change students’ study habits, implemented during the summer (after semester 4). DV: semester GPA Post-tx Baseline X

How Many Measurements Are Needed for a Time Series Design? Depends on: amount of random fluctuation (noise) that may occur in the outcome being measured how much of an impact the intervention is expected to have

Multiple Time Series Designs Add a comparison group to the simple interrupted time series design: O1 O2 O3 O4 X O5 O6 O7 O8 ------------------------------------------------------------------ O1 O2 O3 O4 O5 O6 O7 O8 Campbell & Stanley consider this an excellent quasi-experimental design

Survey Research 11

Surveys Items are composed of 2 parts: stem & response options The university should not raise tuition under any circumstances (stem) Agree Disagree (response options) 12

Question Wording Important consideration when writing questions: Simplicity Avoid jargon and technical terms Avoid abbreviations Be specific Avoid double-barreled questions - 13

Responses to Questions Open-ended Closed-ended 14

Open vs. Closed Questions “…closed questions should be used where alternative replies are known, are limited in number, and are clear cut. Open ended questions are used where the issue is complex, where relevant dimensions are not known, and where process is being explored.” Stacey (1969) 15

Rating Scales Likert Scales Semantic Differential Graphic Rating Scale Non-verbal scale 16

Question Placement Very important Confusing questionnaire = demoralizing to respondents Can affect people’s responses to them 17

Layout of Questionnaire Use closed-ended questions Use a consistent item format Don’t crowd questions together Don’t split questions or response options between pages It is most likely to be completed if easy on the eye, relevant, logical and as short as possible. 18

Response Biases Social desirability Acquiescence response - tendency either to agree/disagree with statements Yea-Saying Nay-Saying 19

Checklist for Survey Use Do the items provide a representative sample of the behaviour under study? Do the items ask for irrelevant information? Does the question have the same meaning for researcher and respondent? Is the question worded clearly? Are the words too difficult? Does the question concern a sensitive issue? 20

Checklist for Survey Use Does the question suggest a socially desirable answer? Open vs closed questions? How many questions are necessary to obtain the required information? How might the questions be arranged to encourage the respondent’s participation? 21