P Values - part 4 The P value and ‘rules’ Robin Beaumont 10/03/2012 With much help from Professor Geoff Cumming.

Slides:



Advertisements
Similar presentations
Introduction to Hypothesis Testing
Advertisements

1 COMM 301: Empirical Research in Communication Lecture 15 – Hypothesis Testing Kwan M Lee.
Lecture (11,12) Parameter Estimation of PDF and Fitting a Distribution Function.
CHAPTER 21 Inferential Statistical Analysis. Understanding probability The idea of probability is central to inferential statistics. It means the chance.
Hypothesis testing Week 10 Lecture 2.
Statistical Significance What is Statistical Significance? What is Statistical Significance? How Do We Know Whether a Result is Statistically Significant?
HYPOTHESIS TESTING Four Steps Statistical Significance Outcomes Sampling Distributions.
EPIDEMIOLOGY AND BIOSTATISTICS DEPT Esimating Population Value with Hypothesis Testing.
Chapter 8 Hypothesis Testing I. Significant Differences  Hypothesis testing is designed to detect significant differences: differences that did not occur.
Behavioural Science II Week 1, Semester 2, 2002
Cal State Northridge  320 Ainsworth Sampling Distributions and Hypothesis Testing.
Statistical Significance What is Statistical Significance? How Do We Know Whether a Result is Statistically Significant? How Do We Know Whether a Result.
Chapter 6 Hypotheses texts. Central Limit Theorem Hypotheses and statistics are dependent upon this theorem.
Chapter Sampling Distributions and Hypothesis Testing.
Chapter 3 Hypothesis Testing. Curriculum Object Specified the problem based the form of hypothesis Student can arrange for hypothesis step Analyze a problem.
Chapter 9 Hypothesis Testing II. Chapter Outline  Introduction  Hypothesis Testing with Sample Means (Large Samples)  Hypothesis Testing with Sample.
Inference about Population Parameters: Hypothesis Testing
INFERENTIAL STATISTICS – Samples are only estimates of the population – Sample statistics will be slightly off from the true values of its population’s.
Chapter Ten Introduction to Hypothesis Testing. Copyright © Houghton Mifflin Company. All rights reserved.Chapter New Statistical Notation The.
AM Recitation 2/10/11.
Overview of Statistical Hypothesis Testing: The z-Test
Overview Definition Hypothesis
Hypothesis Testing – Introduction
1 © Lecture note 3 Hypothesis Testing MAKE HYPOTHESIS ©
Descriptive statistics Inferential statistics
Copyright © 2012 by Nelson Education Limited. Chapter 8 Hypothesis Testing II: The Two-Sample Case 8-1.
Chapter 8 Introduction to Hypothesis Testing
Copyright © Cengage Learning. All rights reserved. 13 Linear Correlation and Regression Analysis.
1 Today Null and alternative hypotheses 1- and 2-tailed tests Regions of rejection Sampling distributions The Central Limit Theorem Standard errors z-tests.
P Values Robin Beaumont 10/10/2011 With much help from Professor Chris Wilds material University of Auckland.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 17 Inferential Statistics.
Copyright © 2008 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 22 Using Inferential Statistics to Test Hypotheses.
Week 8 Chapter 8 - Hypothesis Testing I: The One-Sample Case.
Chapter 8 Hypothesis Testing I. Chapter Outline  An Overview of Hypothesis Testing  The Five-Step Model for Hypothesis Testing  One-Tailed and Two-Tailed.
Hypothesis Testing: One Sample Cases. Outline: – The logic of hypothesis testing – The Five-Step Model – Hypothesis testing for single sample means (z.
Chapter 9 Hypothesis Testing II: two samples Test of significance for sample means (large samples) The difference between “statistical significance” and.
Copyright © 2012 by Nelson Education Limited. Chapter 7 Hypothesis Testing I: The One-Sample Case 7-1.
Significance Tests: THE BASICS Could it happen by chance alone?
P Values - part 3 The P value as a ‘statistic’ Robin Beaumont 1/03/2012 With much help from Professor Geoff Cumming.
Chapter 9 Power. Decisions A null hypothesis significance test tells us the probability of obtaining our results when the null hypothesis is true p(Results|H.
Probability.  Provides a basis for thinking about the probability of possible outcomes  & can be used to determine how confident we can be in an effect.
P Values - part 2 Samples & Populations Robin Beaumont 11/02/2012 With much help from Professor Chris Wilds material University of Auckland.
1 Psych 5500/6500 The t Test for a Single Group Mean (Part 1): Two-tail Tests & Confidence Intervals Fall, 2008.
Confidence intervals and hypothesis testing Petter Mostad
Chapter 8 Introduction to Hypothesis Testing ©. Chapter 8 - Chapter Outcomes After studying the material in this chapter, you should be able to: 4 Formulate.
Economics 173 Business Statistics Lecture 4 Fall, 2001 Professor J. Petry
1 Chapter 8 Introduction to Hypothesis Testing. 2 Name of the game… Hypothesis testing Statistical method that uses sample data to evaluate a hypothesis.
1 When we free ourselves of desire, we will know serenity and freedom.
Copyright © 2011 Pearson Education, Inc. Putting Statistics to Work.
Chapter 8 Hypothesis Testing I. Significant Differences  Hypothesis testing is designed to detect significant differences: differences that did not occur.
Stats Lunch: Day 3 The Basis of Hypothesis Testing w/ Parametric Statistics.
26134 Business Statistics Tutorial 11: Hypothesis Testing Introduction: Key concepts in this tutorial are listed below 1. Difference.
P Values Robin Beaumont 8/2/2012 With much help from Professor Chris Wilds material University of Auckland.
© Copyright McGraw-Hill 2004
Introduction to Testing a Hypothesis Testing a treatment Descriptive statistics cannot determine if differences are due to chance. Sampling error means.
Copyright © 1998, Triola, Elementary Statistics Addison Wesley Longman 1 Assumptions 1) Sample is large (n > 30) a) Central limit theorem applies b) Can.
Education 793 Class Notes Inference and Hypothesis Testing Using the Normal Distribution 8 October 2003.
P Values - part 2 Samples & Populations Robin Beaumont 2011 With much help from Professor Chris Wilds material University of Auckland.
Course Overview Collecting Data Exploring Data Probability Intro. Inference Comparing Variables Relationships between Variables Means/Variances Proportions.
Copyright© 1998, Triola, Elementary Statistics by Addison Wesley Longman 1 Testing a Claim about a Mean: Large Samples Section 7-3 M A R I O F. T R I O.
Statistical Inference for the Mean Objectives: (Chapter 8&9, DeCoursey) -To understand the terms variance and standard error of a sample mean, Null Hypothesis,
ENM SIMULATION PROBABILITY AND STATISTICS REVIEW 1.
15 Inferential Statistics.
More on Inference.
Hypothesis Testing: One Sample Cases
Assumptions For testing a claim about the mean of a single population
Hypothesis Testing – Introduction
Hypothesis Testing: Hypotheses
More on Inference.
Statistical Inference
Presentation transcript:

P Values - part 4 The P value and ‘rules’ Robin Beaumont 10/03/2012 With much help from Professor Geoff Cumming

Putting it all together

Summary so far A P value is a conditional probability which considers a range of outcomes = ‘area’ in a PDF graph. The SEM formula allows us to: predict the accuracy of your estimated mean across a infinite number of samples! Taking another random sample will give us a different P value How different? - Does not follow a normal distribution Dance of the p values – Geoff Cumming Depends upon if the null hypothesis is actually true in reality! Remember we have assumed so far that the null hypothesis is true. Review

Rules t density:s x = n = t Shaded area = Original units: 0 Rule -> If our result has a P value more extreme than our level of acceptability = our critical value Reject the parameter value we based the p value on. = reject the null hypothesis Given that the sample was obtained from a population with a mean (i.e. parameter value) of 120 a sample with a T (n=15) statistic of or or one more extreme with occur 1.8% of the time. This is less than one in twenty (i.e. P value < 0.05). Mark the one in twenty value = the critical value for T (n=15) = = alpha (α) level =0.05) Therefore we dismiss the possibility that our sample came from a population with a mean of 120

Rules Say one in twenty 1/20 = Or 1/100 Or 1/1000 or.... What value do we now give the parameter value ? When p value is in the critical region Reject the parameter value we based the p value on is considered untenable = reject the null hypothesis What is the mean of the population if we have now ruled out one value? Set a level of acceptability = critical value (CV)

Allows decision making Moves thing forward If the sample did not come from the ‘null distribution’ indicates there is some effect Population mean of 120 for substance X indicates they have a propensity to a range of nasty diseases Given them miracle drug Y which is believed to reduce the level has this have any effect in our sample of 15? If the probability of obtaining our t value (i.e. + associated p value) is below a certain critical threshold we can say that our sample does not come from the null distribution and there is a effect. Why do we want to reject the null distribution?

Fisher – only know and only consider the model we have i.e. The parameter we have used in our model – when we reject it we accept that any value but that one can replace it. Neyman and Pearson + Gossling Bayesians H null = μ=120 versus alternative H alt = μ≠120 [μ = population mean] H null = T= 0 versus alternative H alt = T≠ 0 [T = t statistic] H null = μ=120 versus alternative H alt = μ = XXX [μ = population mean] H null = T= 0 versus alternative H alt = T = xxx [T = t statistic]

Fisher – infinitely many alternative ‘red’ distributions View 1 - The infinite variety of alternative distributions - Fisher Alternative distributions: Become flatter further away from null When coincides with null = same shape How do we define the alternative hypothesis? Distance measures: Effect size / Non centrality Parameter Become more asymmetrical as further away from null Null distribution t (df, ncp=0) Alternative distributions t (df, ncp)

View 2 - The single specified alternative – Neyman + Pearson Take the distribution around the sample value Then work our the difference SD’s: Delta = Δ Non centrality parameter = capital = d x √15 = x = Population mean of 120 for substance X, SD= 35 units/100 ml. 15 random subjects take miracle drug have a mean = 96

Gpower shows it all! Population mean of 120 for substance X, SD= 35 units/100 ml. 15 random subjects take miracle drug have a mean = 96 Red line = null distribution Red areas = critical regions = α alpha regions Blue line = specified alternative distribution Blue shaded area = β beta regions (far right– very small)

α = the reject region = 120= 96 Correct decisions incorrect decisions Two correct and 2 incorrect decisions

Correct decisions Power = 1 - Beta Power is good

Insufficient power – unlikely to get a p value in the critical region Too much power always p value in critical region but possibly trivial effect size More Power the better up unto a certain point!

Cumming Replication and P intervals Red line = null distribution Red areas = critical regions = α alpha regions Blue line = specified alternative distribution Blue shaded area = β beta regions (far right– very small)

Implications Power and Cohens effect size measure d reflect one another. When power high p value distribution does provides a measure of evidence for the specific alternative hypothesis. Computer simulations - reality different Replication is a vitally important research strategy – meta analysis. Power analysis during study design and Confidence intervals along with effect size measures when reporting results – alternative strategy? Always specify specific alternative hypothesis

Students bloomers The p value did not indicate much statistic significance Given that the population comes from one population The p value is thus rejecting the null hypothesis and there is a statistical significance Correlation = 0.25 (p<0.001) indicating that assuming that the data come from a bivariate normal distribution with a correlation of zero you would obtain a correlation of < There is 95% chance that the relationship among the variables is not due to chance