Hypothesis Testing I 2/8/12 More on bootstrapping Random chance Null and alternative hypotheses Randomization distribution p-value Section 4.1, 4.2 Professor Kari Lock Morgan Duke University

Announcements Homework 3 (due Monday) Research question and data for Project 1 (proposal due next Wednesday)

Bootstrap CI Option 1: Estimate the standard error of the statistic by computing the standard deviation of the bootstrap distribution, and then generate a 95% confidence interval by Option 2: Generate a P% confidence interval as the range for the middle P% of bootstrap statistics

Suppose we have a random sample of 6 people: Patti

Original Sample Patti Create a “sampling distribution” using this as our simulated population

Bootstrap Sample: Sample with replacement from the original sample, using the same sample size. Patti Original Sample Bootstrap Sample

Continuous versus Discrete A continuous distribution can take any value within some range. The distribution will look like a smooth curve, without any gaps A discrete distribution only takes certain values. The distribution will be spiky, with gaps. Continuous Discrete

Criteria for Bootstrap CI Using the percentile method for a confidence interval bootstrapping for a confidence interval works for any statistic, as long as the bootstrap distribution is Approximately symmetric Approximately continuous Using the standard error method also requires Approximately bell-shaped Always look at the bootstrap distribution to make sure these are true!

Criteria for Bootstrap CI

Number of Bootstrap Samples When using bootstrapping, you may get a slightly different confidence interval each time. This is fine! The more bootstrap samples you use, the more precise your answer will be. For the purposes of this class, 1000 bootstrap samples is fine. In real life, you probably want to take 10,000 or even 100,000 bootstrap samples

Paul the Octopus http://www.youtube.com/watch?v=3ESGpRUMj9E

Paul the Octopus Paul the Octopus predicted 8 World Cup games, and predicted them all correctly Is this evidence that Paul actually has psychic powers? How unusual would this be if he was just randomly guessing (with a 50% chance of guessing correctly)? How could we figure this out?

Coins and Paul Did you get all 8 heads? (a) Yes (b) No

10,000 Simulations www.lock5stat.com/statkey If Paul is just guessing, the chance of him getting all 8 correct is 4/1000.

Hypotheses Null Hypothesis, H0 : Claim that there is no effect or no difference Alternative Hypothesis, Ha : Claim that we seek evidence for, usually that there is some effect Hypotheses are always given in terms of population parameters Paul the Octopus: H0 : 50% chance of correctly predicting each game Ha : >50% chance of correctly predicting each game

Paul the Octopus In the case of Paul the Octopus, what type of parameter are we interested in? (Hint: Are there one or two variables? Are the variable(s) categorical or quantitative?) (a) Proportion (b) Mean (c) Difference in proportions (d) Difference in means (e) Correlation Paul the Octopus: H0 : 50% chance of correctly predicting each game Ha : >50% chance of correctly predicting each game

Paul the Octopus Ha : p > 1/2 Let p denote the proportion of games that Paul guesses correctly (of all games he may have predicted) H0 : p = 1/2 Ha : p > 1/2

Hypotheses The alternative hypothesis is supported by finding evidence (data) that contradicts the null hypothesis (and supports the alternative hypothesis) Data can only contradict or not contradict the null hypothesis, but can never confirm it

Alternative Hypothesis Hypotheses Null Hypothesis Alternative Hypothesis Usually the null is a very specific statement, and so straightforward to assess evidence against ALL POSSIBILITIES

Paul and Hypotheses H0 : p = 1/2 Ha : p > 1/2 What if Paul had gotten 4 out of 8 correct? What would you conclude? H0 is true Ha is true H0 is false Ha is false Nothing

Your Own Hypotheses Come up with a situation where you want to establish a claim based on data What parameter(s) are you interested in? What would the null and alternative hypotheses be? What type of data would lead you to believe the null hypothesis is probably not true?

Measuring Evidence against H0 To see if a statistic provides evidence against H0, we need to see what kind of sample statistics we would observe, just by random chance, if H0 were true

Paul the Octopus

Randomization Distribution A randomization distribution is the distribution of sample statistics we would observe, just by random chance, if the null hypothesis were true Simulate many randomizations, assuming H0 is true, calculate the sample statistic each time, and collect these together to form a distribution

Randomization Distribution

p-value We calculate this from a randomization distribution The p-value is the probability of getting a statistic as extreme (or more extreme) as that observed, just by random chance, if the null hypothesis is true We calculate this from a randomization distribution

Paul the Octopus p-value

p-value What kinds of statistics would we get, just by random chance, if the null hypothesis were true? (randomization distribution) What proportion of these statistics are as extreme as our original sample statistic? (p-value)

Exercise and Pulse Does just 5 seconds of exercise raise your pulse rate? Let’s find out! How can we answer this question?