Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly Significance and Sample Size Refresher Harrison W. Kelly III, Ph.D. Lecture # 3.

Slides:



Advertisements
Similar presentations
Hypothesis Testing An introduction. Big picture Use a random sample to learn something about a larger population.
Advertisements

Inference Sampling distributions Hypothesis testing.
Chapter 10 Section 2 Hypothesis Tests for a Population Mean
STATISTICAL INFERENCE PART V
Copyright ©2011 Brooks/Cole, Cengage Learning Testing Hypotheses about Means Chapter 13.
Comparing Two Population Means The Two-Sample T-Test and T-Interval.
Nemours Biomedical Research Statistics March 19, 2009 Tim Bunnell, Ph.D. & Jobayer Hossain, Ph.D. Nemours Bioinformatics Core Facility.
Hyp Test II: 1 Hypothesis Testing: Additional Applications In this lesson we consider a series of examples that parallel the situations we discussed for.
Fundamentals of Hypothesis Testing. Identify the Population Assume the population mean TV sets is 3. (Null Hypothesis) REJECT Compute the Sample Mean.
Hypothesis Testing Steps of a Statistical Significance Test. 1. Assumptions Type of data, form of population, method of sampling, sample size.
MARE 250 Dr. Jason Turner Hypothesis Testing II To ASSUME is to make an… Four assumptions for t-test hypothesis testing: 1. Random Samples 2. Independent.
MARE 250 Dr. Jason Turner Hypothesis Testing II. To ASSUME is to make an… Four assumptions for t-test hypothesis testing:
Cal State Northridge  320 Ainsworth Sampling Distributions and Hypothesis Testing.
6.4 One and Two-Sample Inference for Variances. Example - Problem 26 – Page 435  D. Kim did some crude tensile strength testing on pieces of some nominally.
Chapter 9 Hypothesis Testing.
Hypothesis Testing For a Single Population Mean. Example: Grade inflation? Population of 5 million college students Is the average GPA 2.7? Sample of.
AM Recitation 2/10/11.
Chapter 10 Hypothesis Testing
Introduction to Biostatistics and Bioinformatics
1/2555 สมศักดิ์ ศิวดำรงพงศ์
Inference in practice BPS chapter 16 © 2006 W.H. Freeman and Company.
14. Introduction to inference
Education 793 Class Notes T-tests 29 October 2003.
More About Significance Tests
Dependent Samples: Hypothesis Test For Hypothesis tests for dependent samples, we 1.list the pairs of data in 2 columns (or rows), 2.take the difference.
1 Design of Engineering Experiments Part 2 – Basic Statistical Concepts Simple comparative experiments –The hypothesis testing framework –The two-sample.
Statistical Power The ability to find a difference when one really exists.
Hypothesis Testing: One Sample Cases. Outline: – The logic of hypothesis testing – The Five-Step Model – Hypothesis testing for single sample means (z.
Copyright © 2012 by Nelson Education Limited. Chapter 7 Hypothesis Testing I: The One-Sample Case 7-1.
One Sample Inf-1 If sample came from a normal distribution, t has a t-distribution with n-1 degrees of freedom. 1)Symmetric about 0. 2)Looks like a standard.
A Broad Overview of Key Statistical Concepts. An Overview of Our Review Populations and samples Parameters and statistics Confidence intervals Hypothesis.
Chapter 10: Analyzing Experimental Data Inferential statistics are used to determine whether the independent variable had an effect on the dependent variance.
Statistical Hypotheses & Hypothesis Testing. Statistical Hypotheses There are two types of statistical hypotheses. Null Hypothesis The null hypothesis,
Lecture 9 Chap 9-1 Chapter 2b Fundamentals of Hypothesis Testing: One-Sample Tests.
Statistical Inference for the Mean Objectives: (Chapter 9, DeCoursey) -To understand the terms: Null Hypothesis, Rejection Region, and Type I and II errors.
Chapter 8 Parameter Estimates and Hypothesis Testing.
Ex St 801 Statistical Methods Inference about a Single Population Mean.
Welcome to MM570 Psychological Statistics
Basic Business Statistics, 11e © 2009 Prentice-Hall, Inc. Chap 9-1 Chapter 9 Fundamentals of Hypothesis Testing: One-Sample Tests Basic Business Statistics.
A review of key statistical concepts. An overview of the review Populations and parameters Samples and statistics Confidence intervals Hypothesis testing.
PHANTOMS: A Method of Testing Hypotheses
Statistical Inference Drawing conclusions (“to infer”) about a population based upon data from a sample. Drawing conclusions (“to infer”) about a population.
ISMT253a Tutorial 1 By Kris PAN Skewness:  a measure of the asymmetry of the probability distribution of a real-valued random variable 
Statistical Inference Statistical inference is concerned with the use of sample data to make inferences about unknown population parameters. For example,
STATISTICAL INFERENCE PART VI HYPOTHESIS TESTING 1.
T tests comparing two means t tests comparing two means.
1 Testing Statistical Hypothesis The One Sample t-Test Heibatollah Baghi, and Mastee Badii.
Statistical Inference for the Mean Objectives: (Chapter 8&9, DeCoursey) -To understand the terms variance and standard error of a sample mean, Null Hypothesis,
+ Homework 9.1:1-8, 21 & 22 Reading Guide 9.2 Section 9.1 Significance Tests: The Basics.
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Statistics for Business and Economics 7 th Edition Chapter 9 Hypothesis Testing: Single.
When the means of two groups are to be compared (where each group consists of subjects that are not related) then the excel two-sample t-test procedure.
Chapter 9 Fundamentals of Hypothesis Testing: One-Sample Tests
Lecture #8 Thursday, September 15, 2016 Textbook: Section 4.4
Hypothesis Tests l Chapter 7 l 7.1 Developing Null and Alternative
More on Inference.
Hypothesis Testing: One Sample Cases
INF397C Introduction to Research in Information Studies Spring, Day 12
Unit 5: Hypothesis Testing
This Week Review of estimation and hypothesis testing
Hypothesis Testing and Confidence Intervals (Part 1): Using the Standard Normal Lecture 8 Justin Kern October 10 and 12, 2017.
Examples of testing the mean and the proportion with single samples
Hypothesis Testing: Hypotheses
INTRODUCTORY STATISTICS FOR CRIMINAL JUSTICE Test Review: Ch. 7-9
More on Inference.
Hypothesis Testing.
Section 10.3 Making Sense of Statistical Significance
Significance Tests: The Basics
Psych 231: Research Methods in Psychology
Comparing Two Proportions
Comparing Two Proportions
Presentation transcript:

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly Significance and Sample Size Refresher Harrison W. Kelly III, Ph.D. Lecture # 3

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 2 Significance Statistical Significance An observed difference that is unlikely to have occurred by chance Practical Significance - The “ Real ” Question The point at which you decide that the difference that exists between two things is important enough to take action on

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 3 Hypothesis Testing The process of taking a practical problem translating it to a statistical problem Because we are using samples (and relatively small ones at that) to estimate population parameters, there is always a chance that we can select a “ weird ” sample for our experiment that may not represent a “ typical ” set of observations Because of this, inferential statistics with some assumptions allows us to estimate the probability of getting a “ weird ” sample

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 4 Hypothesis Testing Testing is performed on the Null Hypothesis, if the null hypothesis is not supported, we must default to the alternative hypothesis! We never actually test the hypothesis that we are biased in favor of, rather we show that the converse is not true, so therefore, our hypothesis should be supported The Null hypothesis (Ho) is assumed to be true, the data must provide sufficient evidence to support that the Null hypothesis is probably not true

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 5 Hypothesis Testing After data is collected, a test statistic is calculated A p-value is then calculated based on the test statistic The p-value tells you the the chance that the results would have occurred if Ho were true The probability that an event like the one observed would happen when sampling from a single population Correct Decision Type 2 Error (  Type 1 Error (  Ho Ho Ha HaTruthConclusion

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 6 Academic Example Say you study conditions conducive to potato rot by injecting potatoes with bacteria that cause rotting and subjecting them to two temperatures. You measure the percent of potatoes that have not rotted. We want to compare the average percent of unrotted potatoes at the two levels studied and determine if is different than the historical percentage of 13%. EXH_AOV.MTW

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 7 Academic Example Data RotTemp

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 8 Are the data Normally distributed?

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 9 Minitab Session Window Test for Equal Variances: Rot versus Temp 95% Bonferroni confidence intervals for standard deviations Temp N Lower StDev Upper F-Test (normal distribution) Test statistic = 0.68, p-value = Levene's Test (any continuous distribution) Test statistic = 0.05, p-value = Asymmetric 95% confidence intervals based on the Chi-square distribution Both p-values are greater than reasonable choices of , so you fail to reject the null hypothesis

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 10 Minitab Output

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 11 Where are we? You fail to reject the null hypothesis of the variances being equal and conclude that there is no evidence for unequal variances. What do we need to know now?

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 12 Minitab Output Two-Sample T-Test and CI: Rot, Temp Two-sample T for Rot Temp N Mean StDev SE Mean Difference = mu (10) - mu (16) Estimate for difference: % CI for difference: ( , ) T-Test of difference = 0 (vs not =): T-Value = P-Value = DF = 15

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 13 Minitab Output

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 14 Minitab Output

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 15 Real-Life Example Say you had a gel dispense process and you were asked to find out with high confidence (95%) if the average dispense weight was similar to the target weight of 4 grams How would you you go about it?

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 16 How should we collect the data? Post-weigh parts or containers subtract the mean part or container weight Pre-weigh parts or containers, dispense, post-weigh parts or containers, calculate delta - bulk population Pre-weigh parts or containers, dispense, post-weigh parts or containers, calculate delta - individuals

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 17 Things to consider How will you physically capture the gel? What will you use as a balance? What kinds of data do you need? How much data do you need?

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 18 Key Phrases Say you had a gel dispense process and you were asked to find out with high confidence (95%) if the average dispense weight was similar to the target weight of 4 grams

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 19 95% Confidence Means that there is a 95% chance that your sample estimate will not be different from the population parameter Would you want to know the probability of detecting a difference from target if one is present? What is this known as? Does the question contain this information? Do we need this information to answer the question in an efficient way?

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 20 Power 1-  = the probability of detecting a difference if a difference is present to detect Let ’ s look at a range of powers (0.7, 0.8, and 0.9)

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 21 Similar to How similar to the target do we mean? We know that the average will not equal the target, so how different will we consider to be important enough so that we will take action Let ’ s use 10% of the target value or 0.4 grams

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 22 Summary so far We know: how confident we want to be in our estimate of mean weight how likely it is that we will detect a difference if a difference exists how big a difference we are concerned with What are we missing? Why do we need statistics? How much variability is there in the process?

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 23 Variability Affects the total number of samples we need to collect The greater the variability, the more samples needed to overcome the uncertainty caused by the variability Say for our example that our process variability is 1.0 gram

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 24 Minitab Output MTB > Power; SUBC> TOne; SUBC> Difference 0.4; SUBC> Power ; SUBC> Sigma 1.0. Power and Sample Size 1-Sample t Test Testing mean = null (versus not = null) Calculating power for mean = null + difference Alpha = 0.05 Sigma = 1 Sample Target Actual Difference Size Power Power

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 25 Ok what do we do? We want to collect enough samples to answer the question We can ’ t (or don ’ t want to) afford the number of samples needed We can ’ t reduce the process variation arbitrarily We know that using 30 is not the right answer Let ’ s try one of the other Minitab options and see if they can help

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 26 Minitab Output Power and Sample Size 1-Sample t Test Testing mean = null (versus not = null) Calculating power for mean = null + difference Alpha = 0.05 Sigma = 1 Sample Difference Size Power

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 27 Something has to give! We check with the engineer and find out that our real concern is detecting a change and knowing (>60% chance) that it has really changed Ideally, we would like to catch it as soon as it happens (small difference) but we could tolerate up to a 1 standard deviation difference Oh, we also find out that the engineer doesn ’ t want to spend lots of time or money on this project

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 28 Minitab Output Power and Sample Size 1-Sample t Test Testing mean = null (versus not = null) Calculating power for mean = null + difference Alpha = 0.05 Sigma = 1 Sample Difference Size Power

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 29 Here ’ s what we decide Alpha = 0.05 Power = 62% Difference able to detect = 0.8 grams (about 20% of the target) Sample size = 10

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 30 Example So, if I go out and dispense 10 parts, I should be able to detect a difference from target of 0.8 grams with a 62% chance of detecting the change if the change exists! I generated 10 random samples using Minitab and the following parameters: MTB > random 10 c40; SUBC> normal I should be able to detect this change from target

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 31 Minitab Output MTB > TTest 4 'SampRot'; SUBC> Alternative 0. T-Test of the Mean Test of mu = vs mu not = Variable N Mean StDev SE Mean T P SampRot It worked and I detected the change!

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 32 Had we collected 30 samples We could have: Detected a smaller difference Had greater confidence Had more power We could have: Collected to many samples (wasted time and money) Detected too small a difference

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 33 Consider Why we collected the data the way we did Why did we use measures of the same vessels before and after the dispense? What did we try to remove from the data set? This method of treating samples is related to another type of t test called a paired t test

Lecture Notes and Electronic Presentations, © 2013 Dr. Kelly 34 Sample Size In hypothesis testing, sample size is everything Too small a sample - risk not being able to determine a difference even if there is one Too large a sample - risk wasting too much money, time and resources showing that there is a difference