1 Power, Power Curves and Sample Size

2 Planning Reliable and Efficient Tests For any job, you need a tool that offers the right amount of power for the task at hand. You wouldn't use a telescope to examine a stamp collection, or a handheld magnifying glass to search for new galaxies, because neither would provide you with meaningful observations. To complicate matters, if detecting a galaxy really was your goal, the cost of gaining the necessary power might be more than you can afford.

3 Planning Reliable and Efficient Tests Anyone using statistical tests faces the same issues. You must consider the precision you need to meet your goals (should your test detect subtle effects or massive shifts?), and balance it against the cost of sampling your population (are you testing toothpicks or jet engines?).

4 Planning Reliable and Efficient Tests You also want the confidence in your results that's appropriate for your situation (testing seat belts demands a greater degree of certainty than testing shampoo). We measure this certainty with statistical power – the probability your test will detect an effect that truly exists.

5 Planning Reliable and Efficient Tests Minitab's Power and Sample Size tools, with Power Curves, help you balance these issues that may compete for your limited resources. Here are three examples of how a quick Power and Sample Size test can help you save time and money getting results you can trust.

6 Don't Leave Success to Chance A paper clip manufacturer wants to detect significant changes in clip length. They sample thousands of clips because it is cheap and quick to do. But this huge sample makes the test too sensitive: the broken line (next slide) shows it will sound the alarm if the average length differs by a trivial amount (0.05).

7 Don't Leave Success to Chance

8 This Power Curve shows they are wasting resources on excessive precision. A sample size of just 100 will detect meaningful differences (0.25) without “crying wolf” at every negligible blip.

9 Don't Leave Success to Chance

10 Don't Leave Success to Chance An aerospace company is designing an experiment to test a new rocket. Each rocket is very expensive, so it is critical to test no more than necessary.

11 Don't Leave Success to Chance

12 Don't Leave Success to Chance This Power Curve confirms an experiment with 6 replicates will give researchers the power they need without spending more than they must.

13 Don't Leave Success to Chance

14 Don't Leave Success to Chance “We've always done it this way.” That's why a lumber company would sample 10 beams to test whether their strength meets the target.

15 Don't Leave Success to Chance

16 Don't Leave Success to Chance According to the Power Curve, this small sample size made their test incapable of detecting important effects. They must sample 34 beams to detect meaningful differences (0.50).

17 Don't Leave Success to Chance

18 See the Big Picture A power analysis helps you weigh your resources against your demands, and quantifies a test's ability to answer your question. It can expose design problems, like the lumber company's insufficient sample size. It can also reveal design solutions you hadn't considered.

19 See the Big Picture Take for instance the packaging plant of a snack company. Customers complain that the company's pretzel bags are sealed with glue that's too strong, so researchers use One-Way ANOVA to compare their current glue with three potential replacements.

20 See the Big Picture Differences in seal strength less than 10 are undetectable to most people, so their test only needs to detect a difference of 10. A power value of 80% is acceptable, but 90% is ideal. What sample size meets their needs?

21 See the Big Picture 30 samples of each glue ensure the test detects a difference of 10 with 90% power.

22 See the Big Picture Or, they could detect the same difference with 23 samples and 80% power. If this represents considerable savings, the researchers may consider using the smaller sample.

23 See the Big Picture The Power Curve illustrates this information, but it also charts every other combination of power and difference for a given sample size.

24 See the Big Picture

25 See the Big Picture The solid line indicates researchers can attain 90% power with just 23 samples if they are willing to seek a difference of 12 instead of 10. This might just be the ideal choice.

26 See the Big Picture

27 How to Create Power Curves in Minitab Performing power analyses with Power Curves couldn't be simpler. You supply the factors you know, and Minitab calculates the one you omit.

28 How to Create Power Curves in Minitab Suppose a trainer wants to compare two training courses for forklift operators. She will use a two-sample t-test to compare the average scores that operators from each course earned on the final exam.

29 How to Create Power Curves in Minitab She knows she must detect a difference of 5 in either direction between the two courses with 80% power, and historical data suggest a standard deviation of 5. But how many participants must she sample from each course?

30 How to Create Power Curves in Minitab Choose Stat > Power and Sample Size > 2-Sample t In Differences, type -5, 5 In Power values, type 0.80 In Standard deviation, type 5 Click OK

31 How to Create Power Curves in Minitab

32 How to Create Power Curves in Minitab

33 How to Create Power Curves in Minitab According to this Power Curve, sampling 17 participants from each class enables her test to detect the difference she seeks with 80% power.

34 How to Create Power Curves in Minitab

35 Putting Power Curves to Use Without knowing the power of your test, it's hard to know if you can trust your results: your test could be too weak to answer your question, or too strong for your needs. Minitab's Power Curves (available for many common statistical procedures) help you balance your resources against your goals and design a test you can trust that costs no more than necessary.

36 Putting Power Curves to Use Power Curves graph the dynamic relationships that define power, revealing the big picture and ensuring no option escapes your consideration. And perhaps most importantly, they make power analysis an easier and more accessible part of every project. Empower your test. Trust your results.