Chapter 11: Multiple Comparisons & Analysis of Variance

One population, Two population,... Chapter 9: Inference (confidence intervals, hypothesis testing) for mean for one group/one populationChapter 9: Inference (confidence intervals, hypothesis testing) for mean for one group/one population Chapter 9: Inference (confidence intervals, hypothesis testing) to compare the means of two groups/two populationsChapter 9: Inference (confidence intervals, hypothesis testing) to compare the means of two groups/two populations To review... briefly look at a few of those one and two- mean inference procedures/situationsTo review... briefly look at a few of those one and two- mean inference procedures/situations

H o : μ = 1H a : μ > 1 where μ = mean heat conductivity transmitted per square meter of surface per degree Celsius difference on the two sides of the glass Is there evidence that the conductivity of this type of glass is greater than 1? Carry out an appropriate test.

Does logging significantly change the mean number of species in a plot after 8 years? Give appropriate statistical evidence to support your conclusion. Assume both populations are Normally distributed. We want to test H o : μ U = μ L OR μ U – μ L = 0 H a : μ U ≠ μ L OR μ U – μ L ≠ 0 H a : μ U ≠ μ L OR μ U – μ L ≠ 0 where μ U & μ L are the mean number of species in unlogged and logged plots, respectfully

Is there good evidence that red wine drinkers’ mean polyphenol levels were different from white wine drinkers’ mean polyphenol levels? Assume both populations are approximately Normal.Is there good evidence that red wine drinkers’ mean polyphenol levels were different from white wine drinkers’ mean polyphenol levels? Assume both populations are approximately Normal. We want to test:We want to test: H o : μ R = μ W or μ R – μ W = 0 H a : μ R ≠ μ W or μ R – μ W ≠ 0 where μ R & μ W are the mean percent change in polyphenols for men who drink red and white wine, respectfully.

Nothing magical about the numbers one or two... Sometimes there is a need to compare three, four, five, or more groups with each other.Sometimes there is a need to compare three, four, five, or more groups with each other. ANOVA (Analysis of Variance) is a method for doing that; tests whether there is an association between a categorical variable that identifies different groups, and a numerical variable.ANOVA (Analysis of Variance) is a method for doing that; tests whether there is an association between a categorical variable that identifies different groups, and a numerical variable. The phrase “Analysis of Variance” can be misleading; the procedure really looks at means/compares means.The phrase “Analysis of Variance” can be misleading; the procedure really looks at means/compares means.

ANOVA example... We may want to know which diet (Weight Watchers, Jenny Craig, Atkins, Slim-Fast, etc.), the categorical variable, is best for losing particular amounts of weight (3 pounds, 10 pounds, 25 pounds), the numerical variable.We may want to know which diet (Weight Watchers, Jenny Craig, Atkins, Slim-Fast, etc.), the categorical variable, is best for losing particular amounts of weight (3 pounds, 10 pounds, 25 pounds), the numerical variable. ANOVA accurately performs multiple comparisonsANOVA accurately performs multiple comparisons If we chose to conduct several individual comparisons (several two-sample/two-population) procedures (two- sample t procedure, confidence interval, hypothesis test), life gets very messy (more on this in a moment)If we chose to conduct several individual comparisons (several two-sample/two-population) procedures (two- sample t procedure, confidence interval, hypothesis test), life gets very messy (more on this in a moment)

How does anova work? Basic idea: Accurately calculates variation within a given group as well as between several groupsBasic idea: Accurately calculates variation within a given group as well as between several groups This leads to a test statistic (like a t score/value) that accurately compares several different groups without the problem of multiple comparisons.This leads to a test statistic (like a t score/value) that accurately compares several different groups without the problem of multiple comparisons. Like all procedures we have discussed, ANOVA works best if certain conditions are met; more on this later...Like all procedures we have discussed, ANOVA works best if certain conditions are met; more on this later...

...popcorn & kernels popped & problems with multiple comparison... In chapter 9, we would have... Ho: mean no oil = mean medium oil Ha: mean no oil ≠ mean medium oil Ho: mean no oil = mean maximum oil Ha: mean no oil ≠ mean maximum oil Ho: mean medium oil = mean maximum oil Ha: mean medium oil ≠ mean maximum oil Three different hypothesis tests... This is called multiple comparison... Comparing multiple pairs of means

Remember α... Rejection zone (when conducting an hypothesis test); significance level; usually 5% (0.05)Rejection zone (when conducting an hypothesis test); significance level; usually 5% (0.05) α is also the probability of committing a type I error (rejecting the null hypothesis when it really is true) α is also the probability of committing a type I error (rejecting the null hypothesis when it really is true) Basic problem with multiple comparisons is that even though the probability of something going wrong (making an incorrect decision; committing an error) on one occasion is small (5%), if we keep repeating the experiment, eventually something will go wrong.Basic problem with multiple comparisons is that even though the probability of something going wrong (making an incorrect decision; committing an error) on one occasion is small (5%), if we keep repeating the experiment, eventually something will go wrong.

Big chances to make big mistakes... Essentially, by doing multiple tests, we are creating more opportunities to mistakenly reject the null hypothesis.Essentially, by doing multiple tests, we are creating more opportunities to mistakenly reject the null hypothesis. The more tests we do, the greater the probability that we will mistakenly reject the null hypothesis at least one.The more tests we do, the greater the probability that we will mistakenly reject the null hypothesis at least one. For our three hypothesis tests, each with α = 0.05, the overall significance level (or probability that we will conclude that at least one amount of oil is more effective than another, when the truth is that all amounts are equally effective is 14%.For our three hypothesis tests, each with α = 0.05, the overall significance level (or probability that we will conclude that at least one amount of oil is more effective than another, when the truth is that all amounts are equally effective is 14%.

Big chances to make big mistakes... 14% doesn’t seem too high... But we were shooting for 5%... And this is just with 3 procedures.14% doesn’t seem too high... But we were shooting for 5%... And this is just with 3 procedures. DETOUR...by the way, how did I calculate that 14%?DETOUR...by the way, how did I calculate that 14%? Remember binomial distributions? Let’s take a moment and review how I calculated that 14%. Pull up Minitab...Remember binomial distributions? Let’s take a moment and review how I calculated that 14%. Pull up Minitab...

So, anova to the rescue... ANOVA tests whether a categorical variable is associated with a numerical variable. This is the same as testing whether the mean value of a numerical variable is different in different groupsANOVA tests whether a categorical variable is associated with a numerical variable. This is the same as testing whether the mean value of a numerical variable is different in different groups ANOVA looks at the variation within each group and between all groups; then creates a ratio comparing these numbers called the F-statisticANOVA looks at the variation within each group and between all groups; then creates a ratio comparing these numbers called the F-statistic F =F =

ANOVA looks at variation within & between Look at variation within each groupLook at variation within each group Look at variation between all groups groupLook at variation between all groups group

ANOVA looks at variation within & between Look at variation within each groupLook at variation within each group Look at variation between all groups groupLook at variation between all groups group

ANOVA looks at variation within & between Look at variation within each group Look at variation between all groups group

Like all other procedures... We have conditions that must be checked and metWe have conditions that must be checked and met Random Sample & Independent MeasurementsRandom Sample & Independent Measurements Independent GroupsIndependent Groups Same VarianceSame Variance

Let’s practice a few... Does the amount of oil used affect the number of kernels popped when one is making popcorn? Researchers randomly assigned bags with 50 unpopped kernels to be popped with no oil, a medium amount of oil (1/2 tsp), or the maximum amount of oil (1 tsp). Thirty-six bags were assigned to each group. After 75 seconds, the popped kernels were counted.Does the amount of oil used affect the number of kernels popped when one is making popcorn? Researchers randomly assigned bags with 50 unpopped kernels to be popped with no oil, a medium amount of oil (1/2 tsp), or the maximum amount of oil (1 tsp). Thirty-six bags were assigned to each group. After 75 seconds, the popped kernels were counted. Using significance level of 5%, fan ANOVA test was carried out.Using significance level of 5%, fan ANOVA test was carried out. Hypothesize:Hypothesize: H o : μ none = μ medium = μ maxH o : μ none = μ medium = μ max H a : The mean number of kernels popped differs by amount of oil used.H a : The mean number of kernels popped differs by amount of oil used.

H o : μ none = μ medium = μ max H a : The mean number of kernels popped differs by amount of oil used. Does the amount of oil used affect the number of kernels popped when one is making popcorn? Using significance level of 5%, carry out an ANOVA test.Does the amount of oil used affect the number of kernels popped when one is making popcorn? Using significance level of 5%, carry out an ANOVA test. Check Conditions:Check Conditions: Random Sample & Independent MeasurementsRandom Sample & Independent Measurements Independent GroupsIndependent Groups Same VarianceSame Variance

H o : μ none = μ medium = μ max H a : The mean number of kernels popped differs by amount of oil used. Does the amount of oil used affect the number of kernels popped when one is making popcorn? Using significance level of 5%, carry out an ANOVA test.Does the amount of oil used affect the number of kernels popped when one is making popcorn? Using significance level of 5%, carry out an ANOVA test.

H o : μ none = μ medium = μ max H a : The mean number of kernels popped differs by amount of oil used. Reject null hypothesis. With an α of 5% and a p-value of about 2%, we reject the null hypothesis that the mean number of kernels popped are all equal.Reject null hypothesis. With an α of 5% and a p-value of about 2%, we reject the null hypothesis that the mean number of kernels popped are all equal.

Now let’s use minitab...

Test weather the mean number of tv’s differs among these three schools. Use a 5% significance level. Assume conditions have been checked and met.

H o : All populations are = h a : the population means are ≠ Minitab; enter data into three columns with labels for each (OC, VA, MC)Minitab; enter data into three columns with labels for each (OC, VA, MC) Go to ANOVA, one-way ANOVA, choose ‘responses are in separate columns for each factor level’Go to ANOVA, one-way ANOVA, choose ‘responses are in separate columns for each factor level’ In responses box, insert your three columns of data (OC, VA, MC); then OKIn responses box, insert your three columns of data (OC, VA, MC); then OK F-Value = 0.48; p-value = F-Value = 0.48; p-value = Interpretation: Fail to reject. With an α level of 5% and a p-value over 62%, we do not have enough evidence to show that the population means are ≠Interpretation: Fail to reject. With an α level of 5% and a p-value over 62%, we do not have enough evidence to show that the population means are ≠

ANOVA HW & Test questions... No need to do the HW in this sectionNo need to do the HW in this section Just be familiar with the general PP conceptsJust be familiar with the general PP concepts Test questions on ANOVA? Minimal. Will be on final, but not a main focus (probably just 1-2 simple questions about ANOVA)Test questions on ANOVA? Minimal. Will be on final, but not a main focus (probably just 1-2 simple questions about ANOVA)