Bootstraps and Scrambles: Letting a Dataset Speak for Itself Robin H. Lock Patti Frazer Lock ‘75 Burry Professor of Statistics Cummings Professor of MathematicsSt.

Slides:

Advertisements

Similar presentations

Panel at 2013 Joint Mathematics Meetings

Advertisements

StatKey Online Tools for Teaching a Modern Introductory Statistics Course Robin Lock St. Lawrence University USCOTS Breakout – May 2013 Patti Frazer Lock.

What Can We Do When Conditions Arent Met? Robin H. Lock, Burry Professor of Statistics St. Lawrence University BAPS at 2012 JSM San Diego, August 2012.

Statistical Inference Using Scrambles and Bootstraps Robin Lock Burry Professor of Statistics St. Lawrence University MAA Allegheny Mountain 2014 Section.

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse –

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock MAA Minicourse – Joint Mathematics.

StatKey Online Tools for Teaching a Modern Introductory Statistics Course Robin Lock Burry Professor of Statistics St. Lawrence University

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 10/23/12 Sections , Single Proportion, p Distribution (6.1)

Early Inference: Using Bootstraps to Introduce Confidence Intervals Robin H. Lock, Burry Professor of Statistics Patti Frazer Lock, Cummings Professor.

Hypothesis Testing I 2/8/12 More on bootstrapping Random chance

Intuitive Introduction to the Important Ideas of Inference Robin Lock – St. Lawrence University Patti Frazer Lock – St. Lawrence University Kari Lock Morgan.

Models and Modeling in Introductory Statistics Robin H. Lock Burry Professor of Statistics St. Lawrence University 2012 Joint Statistics Meetings San Diego,

A Fiddler on the Roof: Tradition vs. Modern Methods in Teaching Inference Patti Frazer Lock Robin H. Lock St. Lawrence University Joint Mathematics Meetings.

STAT 101 Dr. Kari Lock Morgan Exam 2 Review.

Connecting Simulation- Based Inference with Traditional Methods Kari Lock Morgan, Penn State Robin Lock, St. Lawrence University Patti Frazer Lock, St.

Dr. Kari Lock Morgan Department of Statistics Penn State University Teaching the Common Core: Making Inferences and Justifying Conclusions ASA Webinar.

Starting Inference with Bootstraps and Randomizations Robin H. Lock, Burry Professor of Statistics St. Lawrence University Stat Chat Macalester College,

Using Simulation Methods to Introduce Statistical Inference Patti Frazer Lock Kari Lock Morgan Cummings Professor of Mathematics Assistant Professor of.

Building Conceptual Understanding of Statistical Inference with Lock 5 Dr. Kari Lock Morgan Department of Statistical Science Duke University Wake Forest.

Bootstrapping: Let Your Data Be Your Guide Robin H. Lock Burry Professor of Statistics St. Lawrence University MAA Seaway Section Meeting Hamilton College,

Introducing Inference with Simulation Methods; Implementation at Duke University Kari Lock Morgan Department of Statistical Science, Duke University

StatKey Online Tools for Teaching a Modern Introductory Statistics Course Robin Lock Burry Professor of Statistics St. Lawrence University

Using Bootstrap Intervals and Randomization Tests to Enhance Conceptual Understanding in Introductory Statistics Kari Lock Morgan Department of Statistical.

Statistics: Unlocking the Power of Data Lock 5 Inference for Proportions STAT 250 Dr. Kari Lock Morgan Chapter 6.1, 6.2, 6.3, 6.7, 6.8, 6.9 Formulas for.

Introducing Inference with Bootstrap and Randomization Procedures Dennis Lock Statistics Education Meeting October 30,

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University

Understanding the P-value… Really! Kari Lock Morgan Department of Statistical Science, Duke University with Robin Lock, Patti Frazer.

Using Simulation Methods to Introduce Inference Kari Lock Morgan Duke University In collaboration with Robin Lock, Patti Frazer Lock, Eric Lock, Dennis.

Statistics: Unlocking the Power of Data Lock 5 Normal Distribution STAT 250 Dr. Kari Lock Morgan Chapter 5 Normal distribution Central limit theorem Normal.

Statistics: Unlocking the Power of Data Lock 5 Synthesis STAT 250 Dr. Kari Lock Morgan SECTIONS 4.4, 4.5 Connecting bootstrapping and randomization (4.4)

Using Lock5 Statistics: Unlocking the Power of Data

What Can We Do When Conditions Aren’t Met? Robin H. Lock, Burry Professor of Statistics St. Lawrence University BAPS at 2011 JSM Miami Beach, August 2011.

How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.

Statistics: Unlocking the Power of Data Lock 5 Afternoon Session Using Lock5 Statistics: Unlocking the Power of Data Patti Frazer Lock University of Kentucky.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University

Statistics: Unlocking the Power of Data Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University University of Kentucky.

6.1 - One Sample One Sample  Mean μ, Variance σ 2, Proportion π Two Samples Two Samples  Means, Variances, Proportions μ 1 vs. μ 2.

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 9/18/12 Confidence Intervals: Bootstrap Distribution SECTIONS 3.3, 3.4 Bootstrap.

Significance Tests: THE BASICS Could it happen by chance alone?

Introducing Inference with Simulation Methods; Implementation at Duke University Kari Lock Morgan Department of Statistical Science, Duke University

Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse- Joint.

Robin Lock St. Lawrence University USCOTS Opening Session.

CHAPTER 17: Tests of Significance: The Basics

1 Chapter 10: Introduction to Inference. 2 Inference Inference is the statistical process by which we use information collected from a sample to infer.

StatKey Online Tools for Teaching a Modern Introductory Statistics Course Robin Lock Burry Professor of Statistics St. Lawrence University

Introducing Inference with Bootstrapping and Randomization Kari Lock Morgan Department of Statistical Science, Duke University with.

Implementing a Randomization-Based Curriculum for Introductory Statistics Robin H. Lock, Burry Professor of Statistics St. Lawrence University Breakout.

Statistics: Unlocking the Power of Data Lock 5 Bootstrap Intervals Dr. Kari Lock Morgan PSU /12/14.

The Practice of Statistics, 5th Edition Starnes, Tabor, Yates, Moore Bedford Freeman Worth Publishers CHAPTER 10 Comparing Two Populations or Groups 10.1.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University Canton, New York.

Statistics: Unlocking the Power of Data Lock 5 Exam 2 Review STAT 101 Dr. Kari Lock Morgan 11/13/12 Review of Chapters 5-9.

Using Bootstrapping and Randomization to Introduce Statistical Inference Robin H. Lock, Burry Professor of Statistics Patti Frazer Lock, Cummings Professor.

Give your data the boot: What is bootstrapping? and Why does it matter? Patti Frazer Lock and Robin H. Lock St. Lawrence University MAA Seaway Section.

Early Inference: Using Randomization to Introduce Hypothesis Tests Kari Lock, Harvard University Eric Lock, UNC Chapel Hill Dennis Lock, Iowa State Joint.

Statistics: Unlocking the Power of Data Lock 5 Inference for Means STAT 250 Dr. Kari Lock Morgan Sections 6.4, 6.5, 6.6, 6.10, 6.11, 6.12, 6.13 t-distribution.

Statistics: Unlocking the Power of Data Lock 5 Normal Distribution STAT 250 Dr. Kari Lock Morgan Chapter 5 Normal distribution (5.1) Central limit theorem.

Synthesis and Review 2/20/12 Hypothesis Tests: the big picture Randomization distributions Connecting intervals and tests Review of major topics Open Q+A.

StatKey Online Tools for Teaching a Modern Introductory Statistics Course Robin Lock Burry Professor of Statistics St. Lawrence University

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock Minicourse – Joint Mathematics.

Making computing skills part of learning introductory stats

Patti Frazer Lock Cummings Professor of Mathematics

Introducing Statistical Inference with Resampling Methods (Part 1)

When we free ourselves of desire,

Connecting Intuitive Simulation-Based Inference to Traditional Methods

Using Simulation Methods to Introduce Inference

Improving Conceptual Understanding in Intro Stats

Using Simulation Methods to Introduce Inference

Improving Conceptual Understanding in Intro Stats

Teaching with Simulation-Based Inference, for Beginners

Randomization Tests - Beyond One/Two Sample Means & Proportions

Presentation transcript:

Bootstraps and Scrambles: Letting a Dataset Speak for Itself Robin H. Lock Patti Frazer Lock ‘75 Burry Professor of Statistics Cummings Professor of MathematicsSt. Lawrence University Colgate University October 11, 2012

The Lock 5 Team Robin & Patti St. Lawrence Dennis Iowa State Eric UNC/Duke Kari Harvard/Duke Statistics: Unlocking the Power of Data, Wiley, 2013

“Modern” Re-sampling Methods? "Actually, the statistician does not carry out this very simple and very tedious process, but his conclusions have no justification beyond the fact that they agree with those which could have been arrived at by this elementary method." -- Sir R. A. Fisher, 1936

Bootstrap Confidence Intervals and Randomization Hypothesis Tests

Example 1: What is the average price of a used Mustang car? Select a random sample of n=25 Mustangs from a website (autotrader.com) and record the price (in $1,000’s) for each car.

Sample of Mustangs: Our best estimate for the average price of used Mustangs is $15,980, but how accurate is that estimate?

Traditional Inference 1. Which formula? 2. Calculate summary stats 5. Plug and chug 3. Find t * 4. df? OR t * = Interpret in context CI for a mean 7. Check conditions

Bootstrapping Brad Efron Stanford University Assume the “population” is many, many copies of the original sample. Key idea: To see how a statistic behaves, we take many samples with replacement from the original sample using the same n. “Let your data be your guide.”

Suppose we have a random sample of 6 people:

Original Sample A simulated “population” to sample from Bootstrap Sample

Original Sample Bootstrap Sample

Creating a Bootstrap Distribution 1. Compute a statistic of interest (original sample). 2. Create a new sample with replacement (same n). 3. Compute the same statistic for the new sample. 4. Repeat 2 & 3 many times, storing the results. Important point: The basic process is the same for ANY parameter/statistic. Bootstrap sample Bootstrap statistic Bootstrap distribution

Original Sample Bootstrap Sample ●●●●●● Bootstrap Statistic Sample Statistic Bootstrap Statistic ●●●●●● Bootstrap Distribution

We need technology! StatKey

StatKey

Using the Bootstrap Distribution to Get a Confidence Interval – Method #1 The standard deviation of the bootstrap statistics estimates the standard error of the sample statistic. Quick interval estimate : For the mean Mustang prices:

Using the Bootstrap Distribution to Get a Confidence Interval – Method #2 Keep 95% in middle Chop 2.5% in each tail We are 95% sure that the mean price for Mustangs is between $11,930 and $20,238

Example #2 : According to a recent CNN poll of n=722 likely voters in Ohio: 368 choose Obama (51%) 339 choose Romney (47%) 15 choose otherwise (2%) Find a 95% confidence interval for the proportion of Obama supporters in Ohio.

StatKey

Why does the bootstrap work?

Sampling Distribution Population µ BUT, in practice we don’t see the “tree” or all of the “seeds” – we only have ONE seed

Bootstrap Distribution Bootstrap “Population” What can we do with just one seed? Grow a NEW tree! µ

Golden Rule of Bootstraps The bootstrap statistics are to the original statistic as the original statistic is to the population parameter.

What About Hypothesis Tests?

P-value: The probability of seeing results as extreme as, or more extreme than, the sample results, if the null hypothesis is true. Say what????

Example 1: Beer and Mosquitoes Does consuming beer attract mosquitoes? Experiment: 25 volunteers drank a liter of beer, 18 volunteers drank a liter of water Randomly assigned! Mosquitoes were caught in traps as they approached the volunteers. 1 1 Lefvre, T., et. al., “Beer Consumption Increases Human Attractiveness to Malaria Mosquitoes, ” PLoS ONE, 2010; 5(3): e9546.

Beer and Mosquitoes Beer mean = 23.6 Water mean = Does drinking beer actually attract mosquitoes, or is the difference just due to random chance? Beer mean – Water mean = 4.38 Number of Mosquitoes BeerWater

Traditional Inference 1. Which formula? 2. Calculate numbers and plug into formula 3. Plug into calculator 4. Which theoretical distribution? 5. df? 6. find p- value < p-value < 0.001

Simulation Approach Beer mean = 23.6 Water mean = Does drinking beer actually attract mosquitoes, or is the difference just due to random chance? Beer mean – Water mean = 4.38 Number of Mosquitoes BeerWater

Simulation Approach Number of Mosquitoes BeerWater Find out how extreme these results would be, if there were no difference between beer and water. What kinds of results would we see, just by random chance?

Simulation Approach Number of Mosquitoes BeerWater Find out how extreme these results would be, if there were no difference between beer and water. What kinds of results would we see, just by random chance? Number of Mosquitoes Beverage

Simulation Approach Beer Water Find out how extreme these results would be, if there were no difference between beer and water. What kinds of results would we see, just by random chance? Number of Mosquitoes Beverage

StatKey! P-value

Traditional Inference 1. Which formula? 2. Calculate numbers and plug into formula 3. Plug into calculator 4. Which theoretical distribution? 5. df? 6. find p- value < p-value < 0.001

Beer and Mosquitoes The Conclusion! The results seen in the experiment are very unlikely to happen just by random chance (just 1 out of 1000!) We have strong evidence that drinking beer does attract mosquitoes!

“Randomization” Samples Key idea: Generate samples that are (a)based on the original sample AND (a)consistent with some null hypothesis.

Example 2: Malevolent Uniforms Sample Correlation = 0.43 Do teams with more malevolent uniforms commit more penalties, or is the relationship just due to random chance?

Simulation Approach Find out how extreme this correlation would be, if there is no relationship between uniform malevolence and penalties. What kinds of results would we see, just by random chance? Sample Correlation = 0.43

Randomization by Scrambling

StatKey P-value

Malevolent Uniforms The Conclusion! The results seen in the study are unlikely to happen just by random chance (just about 1 out of 100!) We have some evidence that teams with more malevolent uniforms get more penalties!

P-value: The probability of seeing results as extreme as, or more extreme than, the sample results, if the null hypothesis is true. Yeah – that makes sense!

Summary These randomization-based methods tie directly to the key ideas of statistical inference. They are ideal for building conceptual understanding of the key ideas. Not only are these methods great for teaching statistics, but they are increasingly being used for doing statistics.

It is the way of the past… "Actually, the statistician does not carry out this very simple and very tedious process [the randomization test], but his conclusions have no justification beyond the fact that they agree with those which could have been arrived at by this elementary method." -- Sir R. A. Fisher, 1936

… and the way of the future “... the consensus curriculum is still an unwitting prisoner of history. What we teach is largely the technical machinery of numerical approximations based on the normal distribution and its many subsidiary cogs. This machinery was once necessary, because the conceptually simpler alternative based on permutations was computationally beyond our reach. Before computers statisticians had no choice. These days we have no excuse. Randomization-based inference makes a direct connection between data production and the logic of inference that deserves to be at the core of every introductory course.” -- Professor George Cobb, 2007

Thanks for joining us!