Lesson Significance Tests: The Basics

Vocabulary Hypothesis – a statement or claim regarding a characteristic of one or more populations Hypothesis Testing – procedure, base on sample evidence and probability, used to test hypotheses Null Hypothesis – H 0, is a statement to be tested; assumed to be true until evidence indicates otherwise Alternative Hypothesis – H 1, is a claim to be tested(what we will test to see if evidence supports the possibility) Level of Significance – probability of making a Type I error, α

Steps in Hypothesis Testing A claim is made Evidence (sample data) is collected to test the claim The data are analyzed to assess the plausibility (not proof!!) of the claim Note: Hypothesis testing is also called Significance testing

Hypotheses: Null H 0 & Alternative H a Think of the null hypothesis as the status quo Think of the alternative hypothesis as something has changed or is different than expected We can not prove the null hypothesis! We only can find enough evidence to reject the null hypothesis or not.

Hypotheses Cont Our hypotheses will only involve population parameters (we know the sample statistics!) The alternative hypothesis can be –one-sided: μ > 0 or μ < 0 (which allows a statistician to detect movement in a specific direction) –two-sided: μ  0 (things have changed) Read the problem statement carefully to decide which is appropriate The null hypothesis is usually “=“, but if the alternative is one-sided, the null could be too

Three Ways – H o versus H a 1.Equal versus less than (left-tailed test) H 0 : the parameter = some value (or more) H 1 : the parameter < some value 2. Equal hypothesis versus not equal hypothesis (two-tailed test) H 0 : the parameter = some value H 1 : the parameter ≠ some value 3. Equal versus greater than (right-tailed test) H 0 : the parameter = some value (or less) H 1 : the parameter > some value b a b a Critical Regions 123

English Phrases Revisited Math SymbolEnglish Phrases ≥At leastNo less than Greater than or equal to >More thanGreater than <Fewer thanLess than ≤No more thanAt most Less than or equal to =ExactlyEqualsIs ≠Different from

Example 1 A manufacturer claims that there are at least two scoops of cranberries in each box of cereal Parameter to be tested: Test Type: H 0 : H a : left-tailed test  The “bad case” is when there are too few Scoops = 2 (or more) (s ≥ 2) Less than two scoops (s < 2) number of scoops of cranberries in each box of cereal  If the sample mean is too low, that is a problem  If the sample mean is too high, that is not a problem

Example 2 A manufacturer claims that there are exactly 500 mg of a medication in each tablet Parameter to be tested: Test Type: H 0 : H a : Two-tailed test  A “bad case” is when there are too few  A “bad case” is also where there are too many amount of a medication in each tablet  If the sample mean is too low, that is a problem  If the sample mean is too high, that is a problem too Amount = 500 mg Amount ≠ 500 mg

Example 3 A pollster claims that there are at most 56% of all Americans are in favor of an issue Parameter to be tested: Test Type: H 0 : H a : right-tailed test  The “bad case” is when sample proportion is too high population proportion in favor of the issue  If p-hat is too low, that is not a problem  If p-hat is too high, that is a problem P = 56% (or less) P > 56%

Conditions for Significance Tests SRS –simple random sample from population of interest Normality –For means: population normal or large enough sample size for CLT to apply or use t-procedures –t-procedures: boxplot or normality plot to check for shape and any outliers –For proportions: np ≥ 10 and n(1-p) ≥ 10 Independence –Population, N, such that N > 10n

Test Statistics Principles that apply to most tests: The test is based on a statistic that compares the value of the parameter as stated in H 0 with an estimate of the parameter from the sample data Values of the estimate far from the parameter value in the direction specified by H a give evidence against H 0 To assess how far the estimate is from the parameter, standardize the estimate. In many common situations, the test statistic has the form: estimate – hypothesized value test statistic = standard deviation of the estimate (ie SE)

Example 4 Several cities have begun to monitor paramedic response times. In one such city, the mean response time to all accidents involving life-threatening injuries last year was μ=6.7 minutes with σ=2 minutes. The city manager shares this info with the emergency personnel and encourages them to “do better” next year. At the end of the following year, the city manager selects a SRS of 400 calls involving life-threatening injuries and examines response times. For this sample the mean response time was x-bar = 6.48 minutes. Do these data provide good evidence that the response times have decreased since last year? List parameter, hypotheses and conditions check

Example 4 cont Parameter: response times of paramedics to life threatening events H 0 : H a : Conditions Check: 1) : 2) : 3) : μ = 6.7 minutes (unchanged) μ < 6.7 minutes (they got “better”) SRS stated in problem statement Normality n = 400 suggest CLT would apply to x-bar Independence n = 400 means we must assume over 4000 calls each year that involve life-threatening injuries

Homework Pg. 693 & , 4, 5, 7, 8, 9

Hypothesis Testing Approaches P-Value –Logic: Assuming H 0 is true, if the probability of getting a sample mean as extreme or more extreme than the one obtained is small, then we reject the null hypothesis (accept the alternative). Classical (Statistical Significance) –Logic: If the sample mean is too many standard deviations from the mean stated in the null hypothesis, then we reject the null hypothesis (accept the alternative) Confidence Intervals –Logic: If the sample mean lies in the confidence interval about the status quo, then we fail to reject the null hypothesis

-z* α/2 z* α/2 Reject Regions x – μ 0 Test Statistic: z 0 = z* = invnorm(1-α/2) σ/√n Reject null hypothesis, if Left-TailedTwo-TailedRight-Tailed Not usually done z 0 < - z* or z 0 > z* Not usually done Confidence Interval Approach μ0μ0 UBLB Reject Regions FTR Region

zαzα -z α/2 z α/2 -z α Reject Regions x – μ 0 Test Statistic: z 0 = σ/√n Reject null hypothesis, if Left-TailedTwo-TailedRight-Tailed z 0 < - z α z 0 < - z α/2 or z 0 > z α/2 z 0 > z α Classical Approach

P-value P-value is the probability of getting a more extreme value if H 0 is true (measures the tails) Small P-values are evidence against H 0 –observed value is unlikely to occur if H 0 is true Large P-values fail to give evidence against H 0

z0z0 -|z 0 | |z 0 | z0z0 P-Value is the area highlighted x – μ 0 Test Statistic: z 0 = σ/√n Reject null hypothesis, if P-Value < α P-Value Approach Probability(getting a result further away from the point estimate) = p-value P-value is the area in the tails!!

Example 5: P-Values For each α and observed significance level (p-value) pair, indicate whether the null hypothesis would be rejected. a)α =. 05, p =.10 b)α =.10, p =.05 c)α =.01, p =.001 d)α =.025, p =.05 e) α =.10, p =.45 α < P  fail to reject H o P < α  reject H o

Example 4 cont What is the P-value associated with the data in example 4? What if the sample mean was 6.61? x – μ – 6.7 Z 0 = = σ/√n 0.10 = -2.2 P(z < Z 0 ) = P(z < -2.2) = (unusual !) x – μ – 6.7 Z 0 = = = σ/√n 0.10 P(z < Z 0 ) = P(z < -0.9) = (not unusual !)

Two-sided Test P-value P-value is the sum of both tail areas in the two sided test case

Statistical Significance Statistically significant means simply that it is not likely to happen just by chance Significant in the statistical sense does not mean important Very large samples can make very small differences statistically significant, but not practically important

Statistical Significance – P-value When using a P-value, we compare it with a level of significance, α, decided at the start of the test. Not significant when α < P Significant when α ≥ P Fail to Reject H 0 Reject H 0

Statistical Significance Interpretation Remember the three C’s: Conclusion, connection, context Conclusion: Either we have evidence to reject H 0 in favor of H a or we fail to reject Connection: connect your calculated values to your conclusion Context: Always put it in terms of the problem (don’t use generalized statements)

Statistical Significance Warnings If you are going to draw a conclusion based on statistical significance, then the significance level α should be stated before the data are produced –Deceptive users of statistics might set an α level after the data have been analyzed to manipulate the conclusion –P-values give a better sense of how strong the evidence against H 0 is This is just as inappropriate as choosing an alternative hypothesis to be one-sided in a particular direction after looking at the data

Summary and Homework Summary –Significance test assesses evidence provided by data against H 0 in favor of H a –H a can be two-sided (different, ≠) or one-sided (specific direction, ) –Same three conditions as with confidence intervals –Test statistic is usually a standardized value –P-value, the probability of getting a more extreme value given that H 0 is true  is small we reject H 0 Homework –13, 14, 16