1. Experiments
Uniquely suited to identify cause-effect relationships
To study effect of one variable (treatment) on another (outcome/dependent variable)
Use a control group to rule out other causes
Program is the “treatment” in a program evaluation; desired outcomes are “effect”
Measure change with vs without the program, not just before vs after

2. Uses of Experiments in PRTR
Effects of information or promotion programs
on knowledge, attitudes, or behavior.
Consumer response to marketing mix changes
price, product, promotion, place
Effectiveness of various TR interventions
Impacts of tourism on community/region
community attitudes, social, economic, and environmental impacts.
Benefits/Effects of recreation and tourism activity
physical health, mental health, family bonding, economic impacts, learning, etc..
Studying preferences
for landscapes and more generally to measure the relative importance of different product attributes in consumer choices. e.g. conjoint analysis

3. Characteristics of a true Experiment
1. Sample equivalent experimental and control groups
2. Isolate and control the treatment
3. Measure the effect

4. Pre-test/Post-test with Control
R MB1 X MA1 Experimental group
R MB MA2 Control group
R denotes random assignment to groups
X denotes the treatment
Measure of effect =  Expmt gp -  Control gp
= (MA1-MB1) - (MA2-MB2)
= with vs without

5. With vs without the treatment = 5%
Example
Pre Post
Expmt 75% 90%
Control 70% 80%
Effect = (90-75) - (80-70)
= % % = 5%
With vs without the treatment = 5%
Before vs After = 15%

6. Threats to Internal validity
* Pre-measurement (Testing) : effect of pre-measurement on dependent variable (post-test)
* Selection: nonequivalent experimental & control groups, (statistical regression a special case)
* History: impact of any other events between pre- and post measures on dependent variable
* Interaction: alteration of the “effect” due to interaction between treatment & pre-test.
Maturation: aging of subjects or measurement procedures
Instrumentation: changes in instruments between pre and post.
Mortality: loss of some subjects

7. Threats to external validity
Reactive error - Hawthorne effect - artificiality of experimental situation
Measurement timing - measure dependent variable at wrong time, miss effect.
Surrogate situation: using population, treatment or situation different from “real” one.

8. Quasi-experimental designs
Ex post facto (after the fact)
No control group
Subjects self-select to be in expmt group
1. Travel Bureau compares travel inquiries in 1991 and 1994 to evaluate 1992 promotion efforts.
2. To assess effectiveness of an interpretive exhibit, visitors leaving park are asked if they saw exhibit or not, Two groups are compared relative to knowledge, attitudes etc.

9. Lab vs Field Experiments Internal vs External Validity
Internal validity - are findings correct for the particular subjects & setting
External validity - can we generalize results to other similar situations/populations?
Lab Expmt: high internal validity, low external
Field Expmt: high external validity, low internal

10. Ad Evaluation -Woodside Example
Design: 30,000 magazine subscribers, randomly assign 10K to each of three groups A, B and C.
Treatments: 2 expmt’l groups, 1 control
A- fun in sun message
B – relax with family message
C – no ad , control group
Measures of effect:
Total inquiries received
Unaided ad recall via phone survey of 3,000 subscribers
Expenditures of predicted visitors from each group (phone survey)

11. Results Measure of effect A B C 30 10 5 12% 4% 2% 9.0 2.0 .5 $400 $200
Inquiries/1000 subscribers 30 10 5
Unaided awareness of destin.
12% 4% 2%
Party visits/1000 subscribers
9.0
2.0 .5
Spending per trip
$400
$200
Total spending/1,000 subsc.
$3,600
$ 800
$ 100
Net tax revenue (10%) per K
$360
$80
$10
Ad costs / 1,000 subscribers
$40 $0
Net tax/ ad $ (ROI)
$9.0
$2.0 -
Tax revenue/ad cost
$320

12. Recommendations A-B-C Copy Split Large sample sizes – 1,000 plus
Compare alternatives with each other and to no ad - A to B and A/B to C
Track multiple measures of impact/effect
Gather spending to estimate ROI

13. Pricing Expmt- Bamford/Manning
Design: Vary campsite pricing for prime campsites at Vermont State
Treatments: Price differentials of $1-$5
Assign state parks to treatment groups
Measures of effect:
Percent choosing prime sites
Campsite occupancy shift index (compare with previous year)
Revenue generated
Equity – acceptance of policy, income group differences

14. Pricing Expmt Results Occupancy shift of 5% for each $1 differential
Pct choosing prime = * Pctage Price Increase
E.g. $ 0 differential – 54% choose prime
10% differential % ; 20% diff - 44%
Revenue increase of 4 -22%
Small differences in income groups
Pct choose prime 20% for L, 25% M 26% H
Fee Fair? 49% L, 51% M , 60% H