Using Simulation Methods to Introduce Inference Kari Lock Morgan Duke University In collaboration with Robin Lock, Patti Frazer Lock, Eric Lock, Dennis.

Slides:

Advertisements

Similar presentations

Panel at 2013 Joint Mathematics Meetings

Advertisements

Statistical Inference Using Scrambles and Bootstraps Robin Lock Burry Professor of Statistics St. Lawrence University MAA Allegheny Mountain 2014 Section.

Guidelines for Assessment and Instruction in Statistics Education (GAISE) Kari Lock Morgan STA 790: Teaching Statistics 9/19/12.

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse –

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock MAA Minicourse – Joint Mathematics.

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 10/23/12 Sections , Single Proportion, p Distribution (6.1)

Hypothesis Testing: Intervals and Tests

Bootstrap Distributions Or: How do we get a sense of a sampling distribution when we only have ONE sample?

Hypothesis Testing A hypothesis is a claim or statement about a property of a population (in our case, about the mean or a proportion of the population)

Hypothesis Testing I 2/8/12 More on bootstrapping Random chance

Intuitive Introduction to the Important Ideas of Inference Robin Lock – St. Lawrence University Patti Frazer Lock – St. Lawrence University Kari Lock Morgan.

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: p-value STAT 250 Dr. Kari Lock Morgan SECTION 4.2 Randomization distribution p-value.

A Fiddler on the Roof: Tradition vs. Modern Methods in Teaching Inference Patti Frazer Lock Robin H. Lock St. Lawrence University Joint Mathematics Meetings.

STAT 101 Dr. Kari Lock Morgan Exam 2 Review.

Connecting Simulation- Based Inference with Traditional Methods Kari Lock Morgan, Penn State Robin Lock, St. Lawrence University Patti Frazer Lock, St.

Section 4.4 Creating Randomization Distributions.

Dr. Kari Lock Morgan Department of Statistics Penn State University Teaching the Common Core: Making Inferences and Justifying Conclusions ASA Webinar.

Using Simulation Methods to Introduce Statistical Inference Patti Frazer Lock Kari Lock Morgan Cummings Professor of Mathematics Assistant Professor of.

Building Conceptual Understanding of Statistical Inference with Lock 5 Dr. Kari Lock Morgan Department of Statistical Science Duke University Wake Forest.

Inference for Categorical Variables 2/29/12 Single Proportion, p Distribution Intervals and tests Difference in proportions, p 1 – p 2 One proportion or.

Bootstrapping: Let Your Data Be Your Guide Robin H. Lock Burry Professor of Statistics St. Lawrence University MAA Seaway Section Meeting Hamilton College,

Introducing Inference with Simulation Methods; Implementation at Duke University Kari Lock Morgan Department of Statistical Science, Duke University

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: p-value STAT 101 Dr. Kari Lock Morgan 9/25/12 SECTION 4.2 Randomization distribution.

Using Bootstrap Intervals and Randomization Tests to Enhance Conceptual Understanding in Introductory Statistics Kari Lock Morgan Department of Statistical.

ANOVA 3/19/12 Mini Review of simulation versus formulas and theoretical distributions Analysis of Variance (ANOVA) to compare means: testing for a difference.

Introducing Inference with Bootstrap and Randomization Procedures Dennis Lock Statistics Education Meeting October 30,

Simulation and Resampling Methods in Introductory Statistics Michael Sullivan Joliet Junior College

Randomization Tests Dr. Kari Lock Morgan PSU /5/14.

Synthesis and Review 3/26/12 Multiple Comparisons Review of Concepts Review of Methods - Prezi Essential Synthesis 3 Professor Kari Lock Morgan Duke University.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University

More Randomization Distributions, Connections

Understanding the P-value… Really! Kari Lock Morgan Department of Statistical Science, Duke University with Robin Lock, Patti Frazer.

Let’s flip a coin. Making Data-Based Decisions We’re going to flip a coin 10 times. What results do you think we will get?

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: Significance STAT 101 Dr. Kari Lock Morgan 9/27/12 SECTION 4.3 Significance level Statistical.

+ Chapter 9 Summary. + Section 9.1 Significance Tests: The Basics After this section, you should be able to… STATE correct hypotheses for a significance.

Statistics: Unlocking the Power of Data Lock 5 Synthesis STAT 250 Dr. Kari Lock Morgan SECTIONS 4.4, 4.5 Connecting bootstrapping and randomization (4.4)

Using Lock5 Statistics: Unlocking the Power of Data

Hypothesis Testing: p-value

How to Handle Intervals in a Simulation-Based Curriculum? Robin Lock Burry Professor of Statistics St. Lawrence University 2015 Joint Statistics Meetings.

Statistics: Unlocking the Power of Data Lock 5 Afternoon Session Using Lock5 Statistics: Unlocking the Power of Data Patti Frazer Lock University of Kentucky.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University

Statistics: Unlocking the Power of Data Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University University of Kentucky.

Introducing Inference with Simulation Methods; Implementation at Duke University Kari Lock Morgan Department of Statistical Science, Duke University

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 11/1/12 ANOVA SECTION 8.1 Testing for a difference in means across multiple.

Statistics: Unlocking the Power of Data Lock 5 Normal Distribution STAT 101 Dr. Kari Lock Morgan 10/18/12 Chapter 5 Normal distribution Central limit theorem.

Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse- Joint.

Robin Lock St. Lawrence University USCOTS Opening Session.

1 Chapter 10: Introduction to Inference. 2 Inference Inference is the statistical process by which we use information collected from a sample to infer.

1 Chapter 8 Hypothesis Testing 8.2 Basics of Hypothesis Testing 8.3 Testing about a Proportion p 8.4 Testing about a Mean µ (σ known) 8.5 Testing about.

Introducing Inference with Bootstrapping and Randomization Kari Lock Morgan Department of Statistical Science, Duke University with.

Implementing a Randomization-Based Curriculum for Introductory Statistics Robin H. Lock, Burry Professor of Statistics St. Lawrence University Breakout.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University Canton, New York.

Statistics: Unlocking the Power of Data Lock 5 Exam 2 Review STAT 101 Dr. Kari Lock Morgan 11/13/12 Review of Chapters 5-9.

Give your data the boot: What is bootstrapping? and Why does it matter? Patti Frazer Lock and Robin H. Lock St. Lawrence University MAA Seaway Section.

AP Statistics Section 11.1 B More on Significance Tests.

+ Using StatCrunch to Teach Statistics Using Resampling Techniques Webster West Texas A&M University.

Early Inference: Using Randomization to Introduce Hypothesis Tests Kari Lock, Harvard University Eric Lock, UNC Chapel Hill Dennis Lock, Iowa State Joint.

Chapter 9 Day 2 Tests About a Population Proportion.

Statistics: Unlocking the Power of Data Lock 5 Normal Distribution STAT 250 Dr. Kari Lock Morgan Chapter 5 Normal distribution (5.1) Central limit theorem.

Synthesis and Review 2/20/12 Hypothesis Tests: the big picture Randomization distributions Connecting intervals and tests Review of major topics Open Q+A.

Bootstraps and Scrambles: Letting a Dataset Speak for Itself Robin H. Lock Patti Frazer Lock ‘75 Burry Professor of Statistics Cummings Professor of MathematicsSt.

Today: Hypothesis testing p-value Example: Paul the Octopus In 2008, Paul the Octopus predicted 8 World Cup games, and predicted them all correctly Is.

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: p-value STAT 250 Dr. Kari Lock Morgan SECTION 4.2 p-value.

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock Minicourse – Joint Mathematics.

Making computing skills part of learning introductory stats

Patti Frazer Lock Cummings Professor of Mathematics

What Is a Test of Significance?

When we free ourselves of desire,

Using Simulation Methods to Introduce Inference

Using Simulation Methods to Introduce Inference

Presentation transcript:

Using Simulation Methods to Introduce Inference Kari Lock Morgan Duke University In collaboration with Robin Lock, Patti Frazer Lock, Eric Lock, Dennis Lock CAUSE Webinar 12/13/11

Inference: confidence intervals and hypothesis tests Simulation methods: bootstrapping and randomization I’ll focus on randomization tests, because Chris Wild just gave a great CAUSE webinar on bootstrapping: Simulation Methods

To generate a distribution assuming H 0 is true: Traditional Approach: Calculate a test statistic which should follow a known distribution if the null hypothesis is true (under some conditions) Randomization Approach: Decide on a statistic of interest. Simulate many randomizations assuming the null hypothesis is true, and calculate this statistic for each randomization Hypothesis Testing

Why not? With a different formula for each test, students often get mired in the details and fail to see the big picture Plugging numbers into formulas does little to help reinforce conceptual understanding Traditional Hypothesis Testing

Paul the Octopus

Paul the Octopus predicted 8 World Cup games, and predicted them all correctly Is this evidence that Paul actually has psychic powers? How unusual would this be if he was just randomly guessing (with a 50% chance of guessing correctly)? How could we figure this out? Paul the Octopus

Students each flip a coin 8 times, and count the number of heads Count the number of students with all 8 heads by a show of hands (will probably be 0) If Paul was just guessing, it would be very unlikely for him to get all 8 correct! How unlikely? Simulate many times!!! Simulate with Students

Simulate with StatKey

In a randomized experiment on treating cocaine addiction, 48 people were randomly assigned to take either Desipramine (a new drug), or Lithium (an existing drug) The outcome variable is whether or not a patient relapsed Is Desipramine significantly better than Lithium at treating cocaine addiction? Cocaine Addiction

RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRR RRRRRR RRRRRR RRRRRR RRRR RRRRRR RRRRRR RRRRRR New Drug Old Drug 1. Randomly assign units to treatment groups

RRRR RRRRRR RRRRRR NNNNNN RRRRRR RRRRNN NNNNNN RR NNNNNN R = Relapse N = No Relapse RRRR RRRRRR RRRRRR NNNNNN RRRRRR RRRRRR RRNNNN RR NNNNNN 2. Conduct experiment 3. Observe relapse counts in each group Old Drug New Drug 10 relapse, 14 no relapse18 relapse, 6 no relapse 1. Randomly assign units to treatment groups

If the null hypothesis is true (if there is no difference in treatments), then the outcomes would not change under a different randomization Simulate a new randomization, keeping the outcomes fixed (as if the null were true!) For each simulated randomization, calculate the statistic of interest Find the proportion of these simulated statistics that are as extreme (or more extreme) than your observed statistic Randomization Test

RRRR RRRRRR RRRRRR NNNNNN RR RRRR RRRRNN NNNNNN RR NNNNNN 10 relapse, 14 no relapse18 relapse, 6 no relapse

RRRRRR RRRRNN NNNNNN NNNNNN RRRRRR RRRRRR RRRRRR NNNNNN RNRN RRRRRR RNRRRN RNNNRR NNNR NRRNNN NRNRRN RNRRRR Simulate another randomization New Drug Old Drug 16 relapse, 8 no relapse12 relapse, 12 no relapse

RRRR RRRRRR RRRRRR NNNNNN RR RRRR RNRRNN RRNRNR RR RNRNRR Simulate another randomization New Drug Old Drug 17 relapse, 7 no relapse11 relapse, 13 no relapse

Give students index cards labeled R (28 cards) and N (20 cards) Have them deal the cards into 2 groups Have them contribute to a class dotplot for the randomization distribution Simulate with Students

Simulate with StatKey The probability of getting results as extreme or more extreme than those observed if the null hypothesis is true, is about.02. p-value observed statistic

Why did you re-deal your cards? Why did you leave the outcomes (relapse or no relapse) unchanged on each card? Cocaine Addiction You want to know what would happen by random chance (the random allocation to treatment groups) if the null hypothesis is true (there is no difference between the drugs)

Bootstrapping Middle 95% of bootstrap statistics: (27.4, 30.9)

Simulation vs Traditional Simulation methods intrinsically connected to concepts same procedure applies to all statistics no conditions to check minimal background knowledge needed Traditional methods (normal and t based) Familiarity expected after intro stats Needed for future statistics classes Only summary statistics are needed Insight from standard error

Simulation AND Traditional? Currently, we introduce inference with simulation methods, then cover the traditional methods Students have seen the normal distribution appear repeatedly via simulation; use this common shape to motivate traditional inference “Shortcut” formulas give the standard error, avoiding the need for thousands of simulations Students already know the concepts, so can go relatively fast through the mechanics

Student Preferences Which way do you prefer to do inference (confidence intervals and hypothesis tests)? Bootstrapping and Randomization Formulas and Theoretical Distributions %28%

Student Preferences Which way did you prefer to learn inference? Bootstrapping and Randomization Formulas and Theoretical Distributions %33%

Student Preferences Which way of doing inference gave you a better conceptual understanding of confidence intervals and hypothesis tests? Bootstrapping and Randomization Formulas and Theoretical Distributions %27%

Student Preferences DO inferenceSimulationTraditional AP Stat1810 No AP Stat246 LEARN inferenceSimulationTraditional AP Stat1315 No AP Stat264 UNDERSTANDSimulationTraditional AP Stat1711 No AP Stat255

Student Preferences LEARN inference SimulationTraditional DO inference Simulation339 Traditional610 Student PreferencesUNDERSTAND inference SimulationTraditional DO inference Simulation348 Traditional88 Student PreferencesUNDERSTAND inference SimulationTraditional LEARN inference Simulation345 Traditional811

Simulation methods are useful for teaching statistics… The methods reinforce the concepts A randomization test is based on the definition of a p-value A bootstrap confidence interval is based on the idea that statistics vary over repeated samples Very little background is needed, so the core ideas of inference can be introduced early in the course, and remain central throughout the course

… and for doing statistics! Introductory statistics courses now (especially AP Statistics) place a lot of emphasis on checking the conditions for traditional hypothesis tests However, students aren’t given any tools to use if the conditions aren’t satisfied! Randomization-based inference has no conditions, and always applies (even with non-normal data and small samples!)

It is the way of the past… "Actually, the statistician does not carry out this very simple and very tedious process [the randomization test], but his conclusions have no justification beyond the fact that they agree with those which could have been arrived at by this elementary method." -- Sir R. A. Fisher, 1936

… and the way of the future “... the consensus curriculum is still an unwitting prisoner of history. What we teach is largely the technical machinery of numerical approximations based on the normal distribution and its many subsidiary cogs. This machinery was once necessary, because the conceptually simpler alternative based on permutations was computationally beyond our reach. Before computers statisticians had no choice. These days we have no excuse. Randomization-based inference makes a direct connection between data production and the logic of inference that deserves to be at the core of every introductory course.” -- Professor George Cobb, 2007

Thank you!