Hypothesis Testing

Introduction Always about a population parameter Attempt to prove (or disprove) some assumption Setup: alternate hypothesis: What you wish to prove Example: Person is guilty of crime null hypothesis: Assume the opposite of what is to be proven. The null is always stated as an equality. Example: Person is innocent

The test 1. Take a sample, compute statistic of interest. The evidence gathered against defendent 2. How likely is it that if the null were true, you would get such a statistic? (the p-value) How likely is it that an innocent person would be found at the scene of crime, with gun in hand, etc. 3. If very unlikely, then null must be false, hence alternate is proven beyond reasonable doubt. 4. If quite likely, then null may be true, so not enough evidence to discard it in favor of the alternate.

Types of Errors Null is really True Null is really False reject null, assume alternate is proven Type I Error (convict the innocent) Good Decision do not reject null, evidence for alternate not strong enough Good DecisionType II Error (let guilty go free)

Hypothesis Testing Roadmap Hypothesis Testing AttributeContinuous Normal, Interval Scaled Non-Normal, Ordinal Scaled  2 Contingency Tables Correlation Same tests as Non-Normal Medians Variance Medians Variance Means Levene’s Correlation Sign Test Wilcoxon Kruskal- Wallis Mood’s Friedman’s 22 F-test Bartlett’s Z-tests t-tests ANOVA Correlation Regression

Parametric Tests Use parametric tests when: 1. The data are normally distributed 2. The variances of populations (if more than one is sampled from) are equal 3. The data are at least interval scaled

One sample z - test Used when testing to see if sample comes from a known population. A sample of 25 measurements shows a mean of 17. Test whether this is significantly different from a the hypothesized mean of 15, assuming the population standard deviation is known to be 4. One-Sample Z Test of mu = 15 vs not = 15 The assumed standard deviation = 4 N Mean SE Mean 95% CI Z P ( , )

Z-test for proportions 70% of 200 customers surveyed say they prefer the taste of Brand X over competitors. Test the hypothesis that more than 66% of people in the population prefer Brand X. Test and CI for One Proportion Test of p = 0.66 vs p > % Lower Sample X N Sample p Bound Z-Value P-Value

One sample t-test BP Reduction % The data show reductions in Blood Pressure in a sample of 17 people after a certain treatment. We wish to test whether the average reduction in BP was at least 13%, a benchmark set by some other treatment that we wish to match or better.

One Sample t-test – Minitab results One-Sample T: BP Reduction Test of mu = 13 vs > 13 95% Lower Variable N Mean StDev SE Mean Bound T P BP Reduction The p-value of 0.20 indicates that the reduction in BP could not be proven to be greater than 13%. There is a 0.20 probability that it is not greater than 13%.

Two Sample t-test M F You realize that though the overall reduction is not proven to be more than 13%, there seems to be a difference between how men and women react to the treatment. You separate the 17 observations by gender, and wish to test whether there is in fact a significant difference between genders.

Two Sample t-test Two-sample T for BP Reduction M vs BP Reduction F N Mean StDev SE Mean BP Red M BP Red F Difference = mu (BP Red M) - mu (BP Red F) Estimate for difference: % CI for difference: ( , ) T-Test of difference = 0 (vs not =): T-Value = P-Value = DF = 15 Both use Pooled StDev = The test for equal variances shows that they are not different for the 2 samples. Thus a 2-sample t test may be conducted. The results are shown below. The p-value indicates there is a significant difference between the genders in their reaction to the treatment.

Basics of ANOVA Analysis of Variance, or ANOVA is a technique used to test the hypothesis that there is a difference between the means of two or more populations. It is used in Regression, as well as to analyze a factorial experiment design, and in Gauge R&R studies. The basic premise of ANOVA is that differences in the means of 2 or more groups can be seen by partitioning the Sum of Squares. Sum of Squares (SS) is simply the sum of the squared deviations of the observations from their means. Consider the following example with two groups. The measurements show the thumb lengths in centimeters of two types of primates. Total variation (SS) is 28, of which only 4 (2+2) is within the two groups. Thus 24 of the 28 is due to the differences between the groups. This partitioning of SS into ‘between’ and ‘within’ is used to test the hypothesis that the groups are in fact different from each other. See for more details. Obs.Type AType B Mean SS Overall Mean = 5 SS = 28

Results of ANOVA One-way ANOVA: Type A, Type B Source DF SS MS F P Factor Error Total ___________________________________ S = 1 R-Sq = 85.71% R-Sq(adj) = 82.14% The results of running an ANOVA on the sample data from the previous slide are shown here. The hypothesis test computes the F-value as the ratio of MS ‘Between’ to MS ‘Within’. The greater the value of F, the greater the likelihood that there is in fact a difference between the groups. looking it up in an F-distribution table shows a p-value of 0.008, indicating a 99.2% confidence that the difference is real (exists in the Population, not just in the sample). Minitab: Stat/ANOVA/One-Way (unstacked)

Two-Way ANOVA Two-way ANOVA: Strength versus Temp, Speed Source DF SS MS F P Temp Speed Interaction Error Total S = R-Sq = 94.08% R-Sq(adj) = 91.86% StrengthTempSpeed 20.0LowSlow 22.0LowSlow 21.5LowSlow 23.0LowFast 24.0LowFast 22.0LowFast 25.0HighSlow 24.0HighSlow 24.5HighSlow 17.0HighFast 18.0HighFast 17.5HighFast Is the strength of steel produced different for different temperatures to which it is heated and the speed with which it is cooled? Here 2 factors (speed and temp) are varied at 2 levels each, and strengths of 3 parts produced at each combination are measured as the response variable. The results show significant main effects as well as an interaction effect.

Two-Way ANOVA The box plots give an indication of the interaction effect. The effect of speed on the response is different for different levels of temperature. Thus, there is an interaction effect between temperature and speed.