PS300 Research Methods Unit 8: Quasi-Experimental Designs and Program Evaluation Research Prof. Abby Ngwako, M.Ed.

Objectives 0 True experiments vs. Quasi-experiments 0 Threats to internal validity 0 Combatting internal validity threats 0 Discussion: Autism and Measles/Mumps/Rubella (MMR) Vaccine 0 Activity: Autism and (MMR) Vaccine 0 Program evaluation 0 Reminders for Unit 8 assignments 0 Tips on Final Project

True Experiments vs. Quasi- experiments True ExperimentsQuasi-experiments 0 Intervention implemented 0 High degree of control (random assignment) 0 Includes a comparison group (control group) 0 In natural setting, can be difficult to obtain permission and access 0 Intervention implemented 0 Lack a degree of control (lack random assignment) 0 Includes comparison group (nonequivalent control group) 0 Alternative when “true experiments” not possible.

Threats to internal validity “Only by eliminating all possible alternative explanations can we arrive at a definite conclusion about cause and effect” (J. Zechmeister, E. Zechmeister & Shaughnessy, 2001, p. 241). 1. History 2. Maturation 3. Testing 4. Instrumentation 5. Regression 6. Subject mortality 7. Selection 8. Interactions with selection (Campbell & Stanley, 1966).

Combatting internal validity threats 0 In order to combat these threats to internal validity Quasi-Experiments ruled them out by adding the comparison group or nonequivalent control group. 0 Some other types of Quasi-experiments… 0 Simple Interrupted Time-series design 0 Time series with nonequivalent control group design

Discussion: Autism and Measles/Mumps/Rubella (MMR) Vaccine 0 A number of studies have suggested a link between the MMR vaccine and autism. The one that became infamous was Andrew Wakfield’s study conducted in 1998 which relied on reports of parents and families of the 12 children with autism (Wakefield et al, 1998). 0 Check out the following video…. And then come back so that we can discuss how this relates to the importance of internal validity in such research studies.

0 According to the National Institute of Child Health and Human Development (NICHD) based on this study many parents feared that this vaccine was the cause of their child getting autism. Since the symptoms of autism began around the same time as when the child received MMR vaccination (NICHD, 2001). 0 Which one of the threats to internal validity would this be? Activity: Autism and (MMR) Vaccine

0 Several studies have been conducted to prove that there is no evidence that MMR vaccine causes autism. Organizations such as CDC (Center for Disease Control) and Collaborative Programs of Excellence in Autism (CPEA) as well as many others have come together to study autism and MMR vaccine. They will examine people diagnosed with autism but develop normally and then start to show autistic symptoms. This will allow researchers to learn as much as possible about patients by comparing to those who do not have autism and individuals with classic autism (showed autistic symptoms since birth) (NICHD, 2001). 0 This type of research would be checking for what internal validity threat?

Activity: Autism and (MMR) Vaccine 0 Other studies are being conducted to follow 100,000 children from before birth to age 20 to track growth and development, genetic blueprints, as well as environmental factors they may encounter. Researchers hope to look at whether environmental pollutants cause abnormal development such as autism (NICHD, 2001). 0 This would be further research to check for what internal validity threat?

True or False: True experiments vs. Quasi-experiments 0 The difference between true experiments and quasi- experiments is that quasi-experiments have greater internal validity. True or False? 0 The most critical defining characteristic of a true experiment is often seen as the high degree of control over random assignment of participants. True or False? 0 When individuals in one group of a nonequivalent control group design are growing more experienced, more tired or more bored at a faster rate than individuals in the other group, there is a danger of a threat to internal validity called interaction of selection and maturation. True or False?

Program Evaluation 0 Used to assess effectiveness of human service 0 Program evaluators assess needs, process, outcome and efficiency of social services 0 Practical and not theoretical (can have internal validity issues) 0 True experiments or quasi-experiments best approaches for evaluating social and institutional reforms

Unit 8: Reminders 0 As we near end of session, make sure any late DB responses from units 5-8 are posted ASAP or before end of unit 9. Late penalties may apply according to individual situations and only primary response will be accepted. 0 Also, remember that any late unit 6 projects will not be accepted after unit 8 0 Please respond to both discussion boards and utilize at least 1 source to help support each of your responses 0 Make sure to respond to at least 2 of your peers and participate throughout the unit.

TIPS on Final Project 0 Experimental Research method is covered in chapter 6 of E-book. 0 I would briefly describe the different Experimental designs of repeated measures and independent group designs to help justify which one you chose based on the scenario about the appetite reducing drug. 0 Then, make sure to identify whether using repeated measures design or independent group designs that we discussed tonight in seminar. There are 3 independent group designs so make sure if you choose one that you do explain which one and why? 0 Please use knowledge you have gained on sampling, target populations, ethics, internal and external validity in previous units to help support your project. How would they all specifically apply to the experimental method you chose and the example scenario given (testing effectiveness of the new drug for morbidly obese individuals)? 0 Please cite the sources used whether e-Book or other researched sources. Should be cited at the end in Reference list and throughout your paper as in- text citations.

References Campbell, D.T., & Stanley, J.C. (1966). Experimental and quasi-experimental designs for research. Chicago: Rand McNally. National Institute of Child Health and Human Development (NICHD). (2001 June). Autism and the MMR Vaccine. Retrieved from df Posavac, E.J., & Carey, R. G. (1997). Program Evaluation (5 th ed.). Englewood Cliffs, NJ: Prentice-Hall Wakefield AJ, Murch S, Anthony A, et al. (1998). Ileal lymphoid nodular hyperplasia, non-specific colitis, and 349: regressive developmental disorders in children. Lancet, 351: Zechmeister, J.S., Zechmeister, E.B., & Shaughnessy, J.J. (2001). Essentials of Research Methods in Psychology. New York: McGraw Hill.