Power and Sample Size I HAVE THE POWER!!! Boulder 2006 Benjamin Neale.

Slides:



Advertisements
Similar presentations
Our goal is to assess the evidence provided by the data in favor of some claim about the population. Section 6.2Tests of Significance.
Advertisements

1 COMM 301: Empirical Research in Communication Lecture 15 – Hypothesis Testing Kwan M Lee.
Hypothesis testing Another judgment method of sampling data.
Lecture (11,12) Parameter Estimation of PDF and Fitting a Distribution Function.
Anthony Greene1 Simple Hypothesis Testing Detecting Statistical Differences In The Simplest Case:  and  are both known I The Logic of Hypothesis Testing:
CHAPTER 21 Inferential Statistical Analysis. Understanding probability The idea of probability is central to inferential statistics. It means the chance.
Statistical Significance What is Statistical Significance? What is Statistical Significance? How Do We Know Whether a Result is Statistically Significant?
HYPOTHESIS TESTING Four Steps Statistical Significance Outcomes Sampling Distributions.
 Once you know the correlation coefficient for your sample, you might want to determine whether this correlation occurred by chance.  Or does the relationship.
Hypothesis Testing Steps of a Statistical Significance Test. 1. Assumptions Type of data, form of population, method of sampling, sample size.
DATA ANALYSIS I MKT525. Plan of analysis What decision must be made? What are research objectives? What do you have to know to reach those objectives?
Statistical Significance What is Statistical Significance? How Do We Know Whether a Result is Statistically Significant? How Do We Know Whether a Result.
Lecture 11 Psyc 300A. Null Hypothesis Testing Null hypothesis: the statistical hypothesis that there is no relationship between the variables you are.
Chapter 9 Hypothesis Testing.
Power and Sample Size Adapted from: Boulder 2004 Benjamin Neale Shaun Purcell I HAVE THE POWER!!!
Inferential Statistics
Overview of Statistical Hypothesis Testing: The z-Test
Hypothesis Testing.
8 - 1 © 2003 Pearson Prentice Hall Chi-Square (  2 ) Test of Variance.
Chapter 8 Introduction to Hypothesis Testing
Means Tests Hypothesis Testing Assumptions Testing (Normality)
4-1 Statistical Inference The field of statistical inference consists of those methods used to make decisions or draw conclusions about a population.
+ Chapter 9 Summary. + Section 9.1 Significance Tests: The Basics After this section, you should be able to… STATE correct hypotheses for a significance.
The Probability of a Type II Error and the Power of the Test
1 Power and Sample Size in Testing One Mean. 2 Type I & Type II Error Type I Error: reject the null hypothesis when it is true. The probability of a Type.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 17 Inferential Statistics.
1 Lecture note 4 Hypothesis Testing Significant Difference ©
Inference and Inferential Statistics Methods of Educational Research EDU 660.
Inferential Statistics Body of statistical computations relevant to making inferences from findings based on sample observations to some larger population.
Power and Sample Size Boulder 2004 Benjamin Neale Shaun Purcell.
Academic Research Academic Research Dr Kishor Bhanushali M
Education 793 Class Notes Decisions, Error and Power Presentation 8.
Power and Sample Size Anquan Zhang presents For Measurement and Statistics Club.
© Copyright McGraw-Hill 2004
Education 793 Class Notes Inference and Hypothesis Testing Using the Normal Distribution 8 October 2003.
Major Steps. 1.State the hypotheses.  Be sure to state both the null hypothesis and the alternative hypothesis, and identify which is the claim. H0H0.
Chapter 13 Understanding research results: statistical inference.
Hypothesis Testing Steps for the Rejection Region Method State H 1 and State H 0 State the Test Statistic and its sampling distribution (normal or t) Determine.
If we fail to reject the null when the null is false what type of error was made? Type II.
Chapter 8 Introducing Inferential Statistics.
Logic of Hypothesis Testing
Chapter 9 Hypothesis Testing.
Power and p-values Benjamin Neale March 10th, 2016
Lecture Nine - Twelve Tests of Significance.
Hypothesis Testing: One Sample Cases
Unit 3 Hypothesis.
Hypothesis Testing.
CHAPTER 9 Testing a Claim
Chapter 8: Hypothesis Testing and Inferential Statistics
Central Limit Theorem, z-tests, & t-tests
Hypothesis Testing: Hypotheses
Inferential Statistics:
Reasoning in Psychology Using Statistics
CONCEPTS OF HYPOTHESIS TESTING
Chapter 9: Hypothesis Testing
Chapter 9: Hypothesis Tests Based on a Single Sample
Statistical Inference about Regression
INTRODUCTION TO HYPOTHESIS TESTING
Chapter 3 Probability Sampling Theory Hypothesis Testing.
Inferential Statistics
Inference Concepts Hypothesis Testing.
Psych 231: Research Methods in Psychology
Chapter 9: Testing a Claim
Reasoning in Psychology Using Statistics
The POWER of a hypothesis test
Testing Hypotheses I Lesson 9.
Reasoning in Psychology Using Statistics
Section 8.2 Day 2.
Type I and Type II Errors
Rest of lecture 4 (Chapter 5: pg ) Statistical Inferences
Presentation transcript:

Power and Sample Size I HAVE THE POWER!!! Boulder 2006 Benjamin Neale

To Be Accomplished Introduce concept of power via correlation coefficient (ρ) example Identify relevant factors contributing to power Practical: Empirical power analysis for univariate twin model (simulation) How to use mx for power

Simple example Investigate the linear relationship (r) between two random variables X and Y: r=0 vs. r0 (correlation coefficient). draw a sample, measure X,Y calculate the measure of association r (Pearson product moment corr. coeff.) test whether r  0.

How to Test r  0 assumed the data are normally distributed defined a null-hypothesis (r = 0) chosen a level (usually .05) utilized the (null) distribution of the test statistic associated with r=0 t=r  [(N-2)/(1-r2)]

How to Test r  0 Sample N=40 r=.303, t=1.867, df=38, p=.06 α=.05 As p > α, we fail to reject r = 0 have we drawn the correct conclusion?

= type I error rate probability of deciding r  0 (while in truth r=0) a is often chosen to equal .05...why? DOGMA

N=40, r=0, nrep=1000 – central t(38), a=0.05 (critical value 2.04)

Observed non-null distribution (r=.2) and null distribution

In 23% of tests of r=0, |t|>2. 024 (a=0 In 23% of tests of r=0, |t|>2.024 (a=0.05), and thus draw the correct conclusion that of rejecting r = 0. The probability of rejecting the null-hypothesis (r=0) correctly is 1-b, or the power, when a true effect exists

Hypothesis Testing Correlation Coefficient hypotheses: ho (null hypothesis) is ρ=0 ha (alternative hypothesis) is ρ ≠ 0 Two-sided test, where ρ > 0 or ρ < 0 are one-sided Null hypothesis usually assumes no effect Alternative hypothesis is the idea being tested

Summary of Possible Results H-0 true H-0 false accept H-0 1-a b reject H-0 a 1-b a=type 1 error rate b=type 2 error rate 1-b=statistical power

Nonsignificant result STATISTICS Rejection of H0 Non-rejection of H0 Type I error at rate  Nonsignificant result (1- ) H0 true R E A L I T Y Type II error at rate  Significant result (1-) HA true

Power The probability of rejection of a false null-hypothesis depends on: the significance criterion () the sample size (N) the effect size (Δ) “The probability of detecting a given effect size in a population from a sample of size N, using significance criterion ”

Standard Case  Sampling distribution if H0 were true Sampling distribution if HA were true alpha 0.05 POWER: 1 -    T Non-centrality parameter

Increased effect size  Sampling distribution if HA were true Sampling distribution if H0 were true alpha 0.05 POWER: 1 -  ↑   T Non-centrality parameter

Impact of more conservative Sampling distribution if H0 were true Sampling distribution if HA were true alpha 0.01 POWER: 1 -  ↓   T Non-centrality parameter

Impact of less conservative Sampling distribution if H0 were true Sampling distribution if HA were true alpha 0.10 POWER: 1 -  ↑   T Non-centrality parameter

Increased sample size  Sampling distribution if HA were true Sampling distribution if H0 were true alpha 0.05 POWER: 1 -  ↑   T Non-centrality parameter

Effects on Power Recap Larger Effect Size Larger Sample Size Alpha Level shifts <Beware the False Positive!!!> Type of Data: Binary, Ordinal, Continuous Multivariate analysis Empirical significance/permutation

When To Do Power Calculations? Generally study planning stages of study Occasionally with negative result No need if significance is achieved Computed to determine chances of success