Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse- Joint.

Slides:

Advertisements

Similar presentations

Panel at 2013 Joint Mathematics Meetings

Advertisements

What Can We Do When Conditions Arent Met? Robin H. Lock, Burry Professor of Statistics St. Lawrence University BAPS at 2012 JSM San Diego, August 2012.

Objectives 10.1 Simple linear regression

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse –

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock MAA Minicourse – Joint Mathematics.

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 10/23/12 Sections , Single Proportion, p Distribution (6.1)

Hypothesis Testing: Intervals and Tests

Bootstrap Distributions Or: How do we get a sense of a sampling distribution when we only have ONE sample?

Early Inference: Using Bootstraps to Introduce Confidence Intervals Robin H. Lock, Burry Professor of Statistics Patti Frazer Lock, Cummings Professor.

Hypothesis Testing An introduction. Big picture Use a random sample to learn something about a larger population.

Hypothesis Testing I 2/8/12 More on bootstrapping Random chance

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: p-value STAT 250 Dr. Kari Lock Morgan SECTION 4.2 Randomization distribution p-value.

Introduction to Hypothesis Testing AP Statistics Chap 11-1.

A Fiddler on the Roof: Tradition vs. Modern Methods in Teaching Inference Patti Frazer Lock Robin H. Lock St. Lawrence University Joint Mathematics Meetings.

STAT 101 Dr. Kari Lock Morgan Exam 2 Review.

Connecting Simulation- Based Inference with Traditional Methods Kari Lock Morgan, Penn State Robin Lock, St. Lawrence University Patti Frazer Lock, St.

Section 4.4 Creating Randomization Distributions.

Starting Inference with Bootstraps and Randomizations Robin H. Lock, Burry Professor of Statistics St. Lawrence University Stat Chat Macalester College,

Using Simulation Methods to Introduce Statistical Inference Patti Frazer Lock Kari Lock Morgan Cummings Professor of Mathematics Assistant Professor of.

Building Conceptual Understanding of Statistical Inference with Lock 5 Dr. Kari Lock Morgan Department of Statistical Science Duke University Wake Forest.

Inference for Categorical Variables 2/29/12 Single Proportion, p Distribution Intervals and tests Difference in proportions, p 1 – p 2 One proportion or.

Bootstrapping: Let Your Data Be Your Guide Robin H. Lock Burry Professor of Statistics St. Lawrence University MAA Seaway Section Meeting Hamilton College,

Introducing Inference with Simulation Methods; Implementation at Duke University Kari Lock Morgan Department of Statistical Science, Duke University

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: p-value STAT 101 Dr. Kari Lock Morgan 9/25/12 SECTION 4.2 Randomization distribution.

Using Bootstrap Intervals and Randomization Tests to Enhance Conceptual Understanding in Introductory Statistics Kari Lock Morgan Department of Statistical.

Statistics: Unlocking the Power of Data Lock 5 Inference for Proportions STAT 250 Dr. Kari Lock Morgan Chapter 6.1, 6.2, 6.3, 6.7, 6.8, 6.9 Formulas for.

ANOVA 3/19/12 Mini Review of simulation versus formulas and theoretical distributions Analysis of Variance (ANOVA) to compare means: testing for a difference.

Introducing Inference with Bootstrap and Randomization Procedures Dennis Lock Statistics Education Meeting October 30,

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: Hypotheses STAT 101 Dr. Kari Lock Morgan SECTION 4.1 Statistical test Null and alternative.

Randomization Tests Dr. Kari Lock Morgan PSU /5/14.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University

More Randomization Distributions, Connections

Understanding the P-value… Really! Kari Lock Morgan Department of Statistical Science, Duke University with Robin Lock, Patti Frazer.

Using Simulation Methods to Introduce Inference Kari Lock Morgan Duke University In collaboration with Robin Lock, Patti Frazer Lock, Eric Lock, Dennis.

Confidence Intervals: Bootstrap Distribution

Section 5.2 Confidence Intervals and P-values using Normal Distributions.

Statistics: Unlocking the Power of Data Lock 5 Normal Distribution STAT 250 Dr. Kari Lock Morgan Chapter 5 Normal distribution Central limit theorem Normal.

Normal Distribution Chapter 5 Normal distribution

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: Significance STAT 101 Dr. Kari Lock Morgan 9/27/12 SECTION 4.3 Significance level Statistical.

Statistics: Unlocking the Power of Data Lock 5 Synthesis STAT 250 Dr. Kari Lock Morgan SECTIONS 4.4, 4.5 Connecting bootstrapping and randomization (4.4)

Using Lock5 Statistics: Unlocking the Power of Data

Hypothesis Testing: p-value

What Can We Do When Conditions Aren’t Met? Robin H. Lock, Burry Professor of Statistics St. Lawrence University BAPS at 2011 JSM Miami Beach, August 2011.

Statistics: Unlocking the Power of Data Lock 5 Afternoon Session Using Lock5 Statistics: Unlocking the Power of Data Patti Frazer Lock University of Kentucky.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University

Statistics: Unlocking the Power of Data Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University University of Kentucky.

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 9/18/12 Confidence Intervals: Bootstrap Distribution SECTIONS 3.3, 3.4 Bootstrap.

Introducing Inference with Simulation Methods; Implementation at Duke University Kari Lock Morgan Department of Statistical Science, Duke University

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 11/1/12 ANOVA SECTION 8.1 Testing for a difference in means across multiple.

A Broad Overview of Key Statistical Concepts. An Overview of Our Review Populations and samples Parameters and statistics Confidence intervals Hypothesis.

Statistics: Unlocking the Power of Data Lock 5 Normal Distribution STAT 101 Dr. Kari Lock Morgan 10/18/12 Chapter 5 Normal distribution Central limit theorem.

Introducing Inference with Bootstrapping and Randomization Kari Lock Morgan Department of Statistical Science, Duke University with.

Implementing a Randomization-Based Curriculum for Introductory Statistics Robin H. Lock, Burry Professor of Statistics St. Lawrence University Breakout.

Statistics: Unlocking the Power of Data Lock 5 Bootstrap Intervals Dr. Kari Lock Morgan PSU /12/14.

Building Conceptual Understanding of Statistical Inference Patti Frazer Lock Cummings Professor of Mathematics St. Lawrence University Canton, New York.

Bayesian Inference, Review 4/25/12 Frequentist inference Bayesian inference Review The Bayesian Heresy (pdf)pdf Professor Kari Lock Morgan Duke University.

Statistics: Unlocking the Power of Data Lock 5 Exam 2 Review STAT 101 Dr. Kari Lock Morgan 11/13/12 Review of Chapters 5-9.

Using Bootstrapping and Randomization to Introduce Statistical Inference Robin H. Lock, Burry Professor of Statistics Patti Frazer Lock, Cummings Professor.

Statistics: Unlocking the Power of Data Lock 5 STAT 101 Dr. Kari Lock Morgan 12/6/12 Synthesis Big Picture Essential Synthesis Bayesian Inference (continued)

Early Inference: Using Randomization to Introduce Hypothesis Tests Kari Lock, Harvard University Eric Lock, UNC Chapel Hill Dennis Lock, Iowa State Joint.

Statistics: Unlocking the Power of Data Lock 5 Inference for Means STAT 250 Dr. Kari Lock Morgan Sections 6.4, 6.5, 6.6, 6.10, 6.11, 6.12, 6.13 t-distribution.

Synthesis and Review 2/20/12 Hypothesis Tests: the big picture Randomization distributions Connecting intervals and tests Review of major topics Open Q+A.

Bootstraps and Scrambles: Letting a Dataset Speak for Itself Robin H. Lock Patti Frazer Lock ‘75 Burry Professor of Statistics Cummings Professor of MathematicsSt.

Statistics: Unlocking the Power of Data Lock 5 Hypothesis Testing: p-value STAT 250 Dr. Kari Lock Morgan SECTION 4.2 p-value.

Using Randomization Methods to Build Conceptual Understanding in Statistical Inference: Day 1 Lock, Lock, Lock, Lock, and Lock Minicourse – Joint Mathematics.

Connecting Intuitive Simulation-Based Inference to Traditional Methods

Stat 217 – Day 28 Review Stat 217.

Using Simulation Methods to Introduce Inference

Using Simulation Methods to Introduce Inference

Presentation transcript:

Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2 Lock, Lock, Lock Morgan, Lock, and Lock MAA Minicourse- Joint Mathematics Meetings Baltimore, MD January 2014

Schedule: Day 2 Saturday, 1/18, 1:00 – 3:00 pm 5. More on Randomization Tests How do we generate randomization distributions for various statistical tests? How do we assess student understanding when using this approach? 6. Connecting Intervals and Tests 7. Connecting Simulation Methods to Traditional 8. Technology Options Other software for simulatiions (Minitab, Fathom, R, Excel,...) Bascis summary stats/graphs with StatKey A more advanced randmization with StatKey 9. Wrap-up How has this worked in the classroom? Participant comments and questions 10. Evaluations

In a randomized experiment on treating cocaine addiction, 48 people were randomly assigned to take either Desipramine (a new drug), or Lithium (an existing drug) The outcome variable is whether or not a patient relapsed Is there convincing evidence to conclude that Desipramine is better than Lithium at treating cocaine addiction? Cocaine Addiction

RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRRRR RRRR RRRRRR RRRRRR RRRRRR RRRR RRRRRR RRRRRR RRRRRR Desipramine Lithium 1. Randomly assign units to treatment groups

RRRR RRRRRR RRRRRR NNNNNN RRRRRR RRRRNN NNNNNN RR NNNNNN R = Relapse N = No Relapse RRRR RRRRRR RRRRRR NNNNNN RRRRRR RRRRRR RRNNNN RR NNNNNN 2. Observe relapse counts in each group 3. Compare the proportions who relapse Lithium Desipramine 10 relapse, 14 no relapse18 relapse, 6 no relapse 1. Randomly assign units to treatment groups Is this convincing evidence?

Assume the null hypothesis is true Simulate new randomizations For each, calculate the statistic of interest Find the proportion of these simulated statistics that are as extreme as your observed statistic Randomization Test

RRRR RRRRRR RRRRRR NNNNNN RR RRRR RRRRNN NNNNNN RR NNNNNN 10 relapse, 14 no relapse18 relapse, 6 no relapse

RRRRRR RRRRNN NNNNNN NNNNNN RRRRRR RRRRRR RRRRRR NNNNNN RNRN RRRRRR RNRRRN RNNNRR NNNR NRRNNN NRNRRN RNRRRR Simulate another randomization Desipramine Lithium 16 relapse, 8 no relapse12 relapse, 12 no relapse

RRRR RRRRRR RRRRRR NNNNNN RR RRRR RNRRNN RRNRNR RR RNRNRR Simulate another randomization Desipramine Lithium 17 relapse, 7 no relapse11 relapse, 13 no relapse

Physical Simulation

The observed difference in proportions was How unlikely is this if there is no difference in the drugs? To begin to get a sense of this: How many of you had a randomly simulated difference in proportions this extreme? Was your simulated difference in proportions more extreme than the sample statistic (that is, was it less than or equal to -0.33)? A.Yes B.No

A randomization sample must: Use the data that we have (That’s why we didn’t change any of the results on the cards) AND Match the null hypothesis (That’s why we assumed the drug didn’t matter and combined the cards) Cocaine Addiction

StatKey The probability of getting results as extreme or more extreme than those observed if the null hypothesis is true, is about.02. p-value Proportion as extreme as observed statistic observed statistic Distribution of Statistic Assuming Null is True

How can we do a randomization test for a correlation?

Is the number of penalties given to an NFL team positively correlated with the “malevolence” of the team’s uniforms?

Ex: NFL uniform “malevolence” vs. Penalty yards r = n = 28 Is there evidence that the population correlation is positive?

Key idea: Generate samples that are (a) consistent with the null hypothesis (b) based on the sample data. H 0 :  = 0 r = 0.43, n = 28 How can we use the sample data, but ensure that the correlation is zero?

Randomization by Scrambling

Randomize one of the variables! Let’s look at StatKey.

Traditional Inference 1. Which formula? 2. Calculate numbers and plug into formula 3. Plug into calculator 4. Which theoretical distribution? 5. df? 6. find p- value 0.01 < p-value < 0.02

How can we do a randomization test for a mean?

Example: Mean Body Temperature Data: A random sample of n=50 body temperatures. Is the average body temperature really 98.6 o F? H 0 :μ=98.6 H a :μ≠98.6 Data from Allen Shoemaker, 1996 JSE data set article

Key idea: Generate samples that are (a) consistent with the null hypothesis (b) based on the sample data. How to simulate samples of body temperatures to be consistent with H 0 : μ=98.6?

Randomization Samples How to simulate samples of body temperatures to be consistent with H 0 : μ=98.6?

Let’s try it on StatKey.

Playing with StatKey! See the orange pages in the folder.

Choosing a Randomization Method A=Sleep mean=15.25 B=Caffeine mean=12.25 Example: Word recall Option 1: Randomly scramble the A and B labels and assign to the 24 word recalls. H 0 : μ A =μ B vs. H a : μ A ≠μ B Option 2: Combine the 24 values, then sample (with replacement) 12 values for Group A and 12 values for Group B. Reallocate Resample

Question In Intro Stat, how critical is it for the method of randomization to reflect the way data were collected? A. Essential B. Relatively important C. Desirable, but not imperative D. Minimal importance E. Ignore the issue completely

How do we assess student understanding of these methods (even on in-class exams without computers)? See the blue pages in the folder.

Connecting CI’s and Tests Randomization body temp means when μ=98.6 Bootstrap body temp means from the original sample What’s the difference?

Fathom Demo: Test & CI Sample mean is in the “rejection region” Null mean is outside the confidence interval

What about Traditional Methods?

Transitioning to Traditional Inference AFTER students have seen lots of bootstrap distributions and randomization distributions… Students should be able to Find, interpret, and understand a confidence interval Find, interpret, and understand a p-value

Slope :Restaurant tips Correlation: Malevolent uniforms Mean :Body Temperatures Diff means: Finger taps Mean : Atlanta commutes Proportion : Owners/dogs What do you notice? All bell-shaped distributions! Bootstrap and Randomization Distributions

The students are primed and ready to learn about the normal distribution!

Transitioning to Traditional Inference Introduce the normal distribution (and later t) Introduce “shortcuts” for estimating SE for proportions, means, differences, slope…

z*z* -z * 95% Confidence Intervals

Test statistic 95% Hypothesis Tests Area is p-value

Yes! Students see the general pattern and not just individual formulas!

Other Technology Options Your binder includes information for doing several simulation examples using Minitab (macros) R (defined functions) Excel (PopTools ad-in) Fathom (drag/drop/menus) Matlab (commands) SAS (commands)

More StatKey Basic Statistics/Graphics

Sampling Distribution Capture Rate

Example: Sandwich Ants Experiment: Place pieces of sandwich on the ground, count how many ants are attracted. Does it depend on filing? Favourite Experiments: An Addendum to What is the Use of Experiments Conducted by Statistics Students? Margaret Mackisack

Student Preferences Which way of doing inference gave you a better conceptual understanding of confidence intervals and hypothesis tests? Bootstrapping and Randomization Formulas and Theoretical Distributions %31%

Student Preferences Which way did you prefer to learn inference (confidence intervals and hypothesis tests)? Bootstrapping and Randomization Formulas and Theoretical Distributions %36% SimulationTraditional AP Stat3136 No AP Stat7424

Student Behavior Students were given data on the second midterm and asked to compute a confidence interval for the mean How they created the interval: Bootstrappingt.test in RFormula %8%

A Student Comment " I took AP Stat in high school and I got a 5. It was mainly all equations, and I had no idea of the theory behind any of what I was doing. Statkey and bootstrapping really made me understand the concepts I was learning, as opposed to just being able to just spit them out on an exam.” - one of Kari’s students

Thank you for joining us! More information is available on Feel free to contact any of us with any comments or questions.