Analyzing the Results of an Experiment… -not straightforward.. Why not?

Variability and Random/chance outcomes

Inferential Statistics Statistical analysis appropriate for inferring causal relationships and effects. Many different formulas…which one do you use?

Inferential Stat selection -Determine that you are analyzing the results of an experimental manipulation, not a correlation Identify the IV and DV. The IV Will always be nominal on some level, even when it may seem to be continuous..low, medium and high doses of a drug

Inf. Stat Selection What is the scale of the DV? Scale of DV -Statistic to use Nominal Chi-squared Ordinal Mann-Whitney U-test Continuous T-test or ANOVA

t-test or ANOVA? How many levels of the IV are there? 2 levels more than 2 levels T-test or ANOVA ANOVA

There are different forms of T-tests and ANOVA’s: Did the Study Use a Within Group or Between group Experimental Design? Between Group Within Group Only 2 levels of the IV Unpaired t-tests (or “t for independent samples”). “Paired t-tests ( or “t for dependent samples”) Or…ANOVA ( the basic ANOVA is fitted for between group designs) Or…Within group ANOVA (often referred to as a “repeated measures ANOVA”) More than 2 levels of the IV ANOVA Repeated Measures ANOVA

In some ways all inferential Stats are similar. They calculate the probability that a result was due to the IV as opposed to random variability… Let’s focus on the Basic ANOVA since it is likely to be the statistic you may use most commonly.

ANOVA ANOVA produces an F-value. F values are the ratio of overall between group Variability to the Mean within group variability Between Var. (+ chance) /Mean within grp. Variability (+ chance) What does this mean?

Lets suppose: Experiment- IV marijuana Control Placebo control Low dose High dose

Dependent Variable is: Performance on a short term memory task measured number correct out of 10 test items. 9 subjects in each group

Possible out come 1

Possible Outcome 1 Control Placebo Low dose High dose 4 2 2 2 5 3 3 3 6 4 4 5 5 6 4 3 5 5 5 4 6 5 4 4 4 4 5 4 3 4 6 6 7 3 3 5

Distribution of scores for control sample

Placebo scores

Low dose scores

High dose scores

The population distribution of scores

F value relatively low High low placebo control w/in grp. var Between grp. Var

Now consider this: Possible Outcome 2 Control. Placebo. Low dose Now consider this: Possible Outcome 2 Control Placebo Low dose High dose 4 2 2 2 5 3 3 3 6 4 4 5 5 6 4 3 5 5 5 4 6 5 4 4 4 4 5 4 3 4 6 6 7 3 3 5

Distribution of scores for control sample

Placebo scores

Low dose scores

High dose scores

F value relatively High low placebo control w/in grp. var Between grp. Var

The high F value reflects Logic! Distribution of score are much more obviously separated, and in this case are completely non-overlapping Low F values indicate highly overlapping score distributions

So how do we decide if an F value is large enough to consider the result as causal? We consult a table of established probabilities of different F values, within the context of Degree of freedom terms:

ANOVA Significance table

Where is/are the difference (s)?

Inferential Statistics

The story of “Scratch”

Why not jus use repeated t-tests? Probability pyramiding 15 t-tests required for this data set Post-hocs include compensations for repeated testing of a large data set

After all this where so we stand? We can still be wrong.

Factors that affect “power.” Sample size

One vs two-tailed testing

Effect size