Experiments Uniquely suited to identify cause-effect relationships To study effect of one variable (treatment) on another (outcome/dependent variable)

Slides:



Advertisements
Similar presentations
Evaluation Procedures
Advertisements

Validity (cont.)/Control RMS – October 7. Validity Experimental validity – the soundness of the experimental design – Not the same as measurement validity.
Experimental and Quasi-Experimental Research
Defining Characteristics
Types of Experimental Designs Non-experimental –Post-test only –Pre-post test Experimental –Pre & post-test with control group –Pre & multiple post-test.
Experimental Research Designs
Research Design: The Experimental Model and Its Variations
Research Design and Validity Threats
Research Problems.
QUASI-EXPERIMENTS w Compare subjects in different conditions on a DV w Lacks one or more criteria for an experiment (cause, comparison, control) w Interpreted.
Lecture 10 Psyc 300A. Types of Experiments Between-Subjects (or Between- Participants) Design –Different subjects are assigned to each level of the IV.
Types of Group Designs ____________ group design. The experiment compares groups that receive or _______________ the IV (control group) e.g., behavior.
Rest of Course Proposals & Research Design Measurement Sampling Survey methods Basic Statistics for Survey Analysis Experiments Other Approaches- Observation,
Criteria for Establishing Causal Relationships Concomitant variation Temporal ordering of variables Control over other possible causal factors.
Measuring Environmental Benefits: Revealed Preference Approaches Travel Cost and Hedonic Methods.
Aaker, Kumar, Day Seventh Edition Instructor’s Presentation Slides
Chapter 9 Experimental Research Gay, Mills, and Airasian
Experimental Research
Problem identification Research questions Constructs & Variables Research design.
Experimental Design 264a Marketing Research. Criteria for Establishing a Causal Relationship Concomitant variation Temporal variation Control over other.
EVALUATING YOUR RESEARCH DESIGN EDRS 5305 EDUCATIONAL RESEARCH & STATISTICS.
Chapter 8 Experimental Research
Experimental Design The Gold Standard?.
I want to test a wound treatment or educational program in my clinical setting with patient groups that are convenient or that already exist, How do I.
Research Design for Quantitative Studies
Learning Objectives 1 Copyright © 2002 South-Western/Thomson Learning Primary Data Collection: Experimentation CHAPTER eight.
Power Point Slides by Ronald J. Shope in collaboration with John W. Creswell Chapter 11 Experimental Designs.
Chapter 9 Causal Research Designs and Test Markets
INTRO TO EXPERIMENTAL RESEARCH, continued Lawrence R. Gordon Psychology Research Methods I.
Chapter Four Experimental & Quasi-experimental Designs.
By: Valerie Passalacqua Alana Goldstein Gloren Brown.
1 Experimental Research Cause + Effect Manipulation Control.
Research methods and statistics.  Internal validity is concerned about the causal-effect relationship in a study ◦ Can observed changes be attributed.
Primary Data Collection: Experimentation Chapter 7.
Chapter 6 Research Validity. Research Validity: Truthfulness of inferences made from a research study.
Chapter 10 Experimental Research Gay, Mills, and Airasian 10th Edition
Research Design ED 592A Fall Research Concepts 1. Quantitative vs. Qualitative & Mixed Methods 2. Sampling 3. Instrumentation 4. Validity and Reliability.
Nonexperimental and Quasi- Experimental Designs Distinction is the degree of control over internal validity.
1 Module 3 Designs. 2 Family Health Project: Exercise Review Discuss the Family Health Case and these questions. Consider how gender issues influence.
Experimental Research Hanser and Wheeler. Principles Independent Variable Dependent Variable.
Experimental & Quasi-Experimental Designs Dr. Guerette.
Research Design. Time of Data Collection Longitudinal Longitudinal –Panel study –Trend study –Cohort study Cross-sectional Cross-sectional.
Chapter 11.  The general plan for carrying out a study where the independent variable is changed  Determines the internal validity  Should provide.
SOCW 671: #6 Research Designs Review for 1 st Quiz.
The Experiment Chapter 7. Doing Experiments In Everyday Life Experiments in psychology use the same logic that guides experiments in biology or engineering.
Types of Experimental Designs (Educational research) True Experimental Quasi-Experimental.
Handbook for Health Care Research, Second Edition Chapter 7 © 2010 Jones and Bartlett Publishers, LLC CHAPTER 7 Designing the Experiment.
IREL 561: Research Methods Fall 2013 Week 10 Based largely on Neuman’s Basics of Social Research, Chapter 8 Prepared by Craig Webster, Ph.D.
Chapter 7 Causal Designs. Issues Addressed l conditions of causality l what is an experiment? l differences between a lab vs field experiment l two types.
Experimental and Ex Post Facto Designs
Experiments.  Labs (update and questions)  STATA Introduction  Intro to Experiments and Experimental Design 2.
Rest of Course Proposals & Research Design Measurement Sampling
Table of Contents  Research Design  Threats to Internal Validity  Threats to External Validity  Proposed Data  Pre-Test Post-Test Data Results 
Can you hear me now? Keeping threats to validity from muffling assessment messages Maureen Donohue-Smith, Ph.D., RN Elmira College.
Research designs Research designs Quantitative Research Designs.
Educational Research Experimental Research Chapter 9 (8 th Edition) Chapter 13 (7 th Edition) Gay and Airasian.
Chapter 9 Scrutinizing Quantitative Research Design.
Experimental Research Designs. Experimental Design Advantages  Best establishes cause-and-effect relationships Disadvantages  Artificiality of experiments.
Experimental and Quasi-Experimental Research
Chapter 8 Experimental Design The nature of an experimental design
Experiments Uniquely suited to identify cause-effect relationships
Chapter 9 Experimental Research: An Overview
The Nonexperimental and Quasi-Experimental Strategies
Introduction to Experimental Design
Chapter 18: Experimental and Quasi-Experimental Research
Experimental Research
Group Experimental Design
Experimental Research
Types of Designs: R: Random Assignment of subjects to groups
Reminder for next week CUELT Conference.
Presentation transcript:

Experiments Uniquely suited to identify cause-effect relationships To study effect of one variable (treatment) on another (outcome/dependent variable) Use a control group to rule out other causes Program is the “treatment” in a program evaluation; desired outcomes are “effect” Measure change with vs without the program, not just before vs after

Uses of Experiments in PRTR Effects of information or promotion programs –on knowledge, attitudes, or behavior. Consumer response to marketing mix changes –price, product, promotion, place Effectiveness of various TR interventions Impacts of tourism on community/region – community attitudes, social, economic, and environmental impacts. Benefits/Effects of recreation and tourism activity – physical health, mental health, family bonding, economic impacts, learning, etc.. Studying preferences –for landscapes and more generally to measure the relative importance of different product attributes in consumer choices. e.g. conjoint analysis

Characteristics of a true Experiment 1. Sample equivalent experimental and control groups 2. Isolate and control the treatment 3. Measure the effect

Pre-test/Post-test with Control R MB1 X MA1 Experimental group R MB2 MA2 Control group R denotes random assignment to groups X denotes the treatment Measure of effect =  Expmt gp -  Control gp = (MA1-MB1) - (MA2-MB2) = with vs without

Example PrePost Expmt75%90% Control70%80% Effect= (90-75) - (80-70) = 15% - 10% = 5% With vs without the treatment = 5% Before vs After = 15%

Threats to Internal validity * Pre-measurement (Testing) : effect of pre- measurement on dependent variable (post-test) * Selection: nonequivalent experimental & control groups, (statistical regression a special case) * History: impact of any other events between pre- and post measures on dependent variable * Interaction: alteration of the “effect” due to interaction between treatment & pre-test. Maturation: aging of subjects or measurement procedures Instrumentation: changes in instruments between pre and post. Mortality: loss of some subjects

Threats to external validity Reactive error - Hawthorne effect - artificiality of experimental situation Measurement timing - measure dependent variable at wrong time, miss effect. Surrogate situation: using population, treatment or situation different from “real” one.

Quasi-experimental designs Ex post facto (after the fact) No control group Subjects self-select to be in expmt group 1. Travel Bureau compares travel inquiries in 1991 and 1994 to evaluate 1992 promotion efforts. 2. To assess effectiveness of an interpretive exhibit, visitors leaving park are asked if they saw exhibit or not, Two groups are compared relative to knowledge, attitudes etc.

Lab vs Field Experiments Internal vs External Validity Internal validity - are findings correct for the particular subjects & setting External validity - can we generalize results to other similar situations/populations? Lab Expmt: high internal validity, low external Field Expmt: high external validity, low internal

Ad Evaluation -Woodside Example Design: 30,000 magazine subscribers, randomly assign 10K to each of three groups A, B and C. Treatments: 2 expmt’l groups, 1 control –A- fun in sun message –B – relax with family message –C – no ad, control group Measures of effect: –Total inquiries received –Unaided ad recall via phone survey of 3,000 subscribers –Expenditures of predicted visitors from each group (phone survey)

Results Measure of effectABC Inquiries/1000 subscribers Unaided awareness of destin. 12%4%2% Party visits/1000 subscribers Spending per trip $400 $200 Total spending/1,000 subsc. $3,600$ 800$ 100 Net tax revenue (10%) per K $360$80$10 Ad costs / 1,000 subscribers $40 $0 Net tax/ ad $ (ROI) $9.0$2.0- Tax revenue/ad cost $320$40$10

Recommendations A-B-C Copy Split Large sample sizes – 1,000 plus Compare alternatives with each other and to no ad - A to B and A/B to C Track multiple measures of impact/effect Gather spending to estimate ROI

Pricing Expmt- Bamford/Manning Design: Vary campsite pricing for prime campsites at Vermont State Treatments: Price differentials of $1-$5 –Assign state parks to treatment groups Measures of effect: –Percent choosing prime sites –Campsite occupancy shift index (compare with previous year) –Revenue generated –Equity – acceptance of policy, income group differences

Pricing Expmt Results Occupancy shift of 5% for each $1 differential Pct choosing prime = * Pctage Price Increase –E.g. $ 0 differential – 54% choose prime – 10% differential - 49% ; 20% diff - 44% Revenue increase of 4 -22% Small differences in income groups –Pct choose prime 20% for L, 25% M 26% H –Fee Fair? 49% L, 51% M, 60% H