Improving experiment internal validity: application to measuring the effect of extrinsic information on the perceived quality of olive oil DEKHILI & d’HAUTEVILLE SupAgro Montpellier (France) Quantitative Methods in the Social Sciences (QMSS) Conference, Prague, June 2007

Outline Introduction Internal Validity and Satistical Power Method Results

Introduction Experiments: relation of cause and effect (Shadish et al. 2002) In most experiments examining the effect of information on hedonic and sensory perception of food products: the stability of subjects’evaluations is not controled Internal validity ?? Pre-test/post-test designs without control groups Control: one of the fundamental components of experiments (Fisher, 1925; in Shadish et al. 2002)

Objectif of this paper : Use a quasi-experimental design: pre-test/post-test with a control group in an olive oil preference study To detect, in as pertinent a manner as possible, the effect of product-specific regional information on the perception of sensory properties and on perceived quality

Internal Validity and Satistical Power Internal Validity : Relevance of control groups Control of external variables Equivalence/non-equivalence between the two groups ANCOVA Model Statistical Power : Ability to detect the effect of any treatment Can be determined before or after the study Acceptable level: > 0.6 (Verma and Goodale, 1995) Sizes of test and control samples…

Method Consumers (test and control groups) 2 countries (France and Tunisia): 251 consumers Tunisia: - a test group : 107 consumers - a control group : 21 consumers - 13 sessions (1h30mn) France: -a test group : 100 consumers -a control group : 23 consumers - 10 sessions (1h20mn) We were unable to obtain 2 samples of equal sizes. We needed a qualitative and tasting information from a large test sample size

Condition for recruiting participants : they consume + purchase olive oil Participant recruiment methods : different according to the context (directly, ads) Consumers’assignment to the test or control groups on the basis of their availability

Products and procedure  4 « extra virgin » olive oils from the same region of production (Vallée des Baux de Provence in France and Zaghouan in Tunisia)  Information on the dimensions of the image of the region of origin specific to olive oil: was varied  Variety of olive: single variety oil, oil made with a blend of varieties of olive  Natural conditions: soil, climate  Human factor: modern mill, traditional mill  Evaluations in blind condition Expectation test on the basis of labels Evaluations in labelled condition

Extra virgin olive oil Region of origin Zaghouan Variety of olive Beldi Zaghouan Particular soils Modern mill Extra virgin olive oil Region of origin Zaghouan Blend of varieties of olive Beldi Zaghouan, Chetoui, Sehli Particular soils Traditionnel mill Extra virgin olive oil Region of origin Zaghouan Variety of olive Beldi Zaghouan Local climate Traditionnel mill Extra virgin olive oil Region of origin Zaghouan Blend of varieties of olive Beldi Zaghouan, Chetoui, Sehli Local climate Modern mill

Results  Statistical Power of the experiment prtocol: SamplePower Software : 0.7 in the French case in the Tunisian case Acceptable level (Verma et Goodale, 1995)  Effect of the information: 1 st step: Equivalence of the evaluations of the 2 groups in the pre-test -T-test (overall and attribute appreciation, maximum price to pay) - Khi square test (purchase intention)

TunisiaFrance - KD2 price - BR5 overall appreciation - BR5 taste - HE7 overall appreciation - HE7 taste - HE7 purchase intention - CM9 overall appreciation - CM9 taste - CM9 price - CM9 purchase intention - HE7 price differences in the hedonic and monetary evaluations : - 2 nd step: ANCOVA Model Variables showing differences between the 2 groups in pre-test: adjust the non-equivalence Scores of the post-test = f (pre-test scores covariate, « group » variable)

TunisiaFrance - KD2 price - BR5 overall appreciation - BR5 taste - HE7 overall appreciation - HE7 taste - HE7 purchase intention - CM9 overall appreciation - CM9 taste - CM9 price - CM9 purchase intention - HE7 price 3rd step: variables that didn’t show significant differences between the 2 groups in the pre-test Effect of the information: T-test, Khi square test

We summarise the effect of the region of origin image on the assessment of the olive oils : Olive oilsSignificant effect of the information TunisiaFrance CM9Price KD2PriceOverall apreciation, price BR5Price HE7 --

Thank you for your attention