1. chapter 7 Statistical Issues in Research Planning and Evaluation

2. Chapter Outline Probability Meaningfulness Power Using information in context of the study Reporting statistical data

3. Probability What are the odds that a certain thing(s) will happen? –Equally likely events –Relative frequency Alpha (α): –Level of chance occurrence set by the researcher prior to the study –Typical: p <.05 or p <.01 Odds that findings are due to chance are either 5 or 1 in 100 –Controls for Type I error

4. Truth Table for the Null Hypothesis H o trueH o false AcceptCorrect decisionType II error (beta) RejectType I error (alpha) Correct decision

5. Probability Beta (β): –Magnitude of a Type II error –Acceptance of null hypothesis when it is FALSE As alpha is set increasingly smaller, beta becomes larger

6. Sampling for Null Hypothesis From Experimental procedures for behavioral science, 3rd ed., by R.E. Kirk © 1995. Reprinted with permission of Brooks/Cole, an imprint of the Wadsworth Group, a division of Thomson Learning. Fax 800-730-2215.

7. Meaningfulness The practical significance of an effect or relationship Effect Size (ES) –Standardized value that is the difference between the means divided by the standard deviation Formula: –ES = (M 1 – M 2 )/s

8. Estimating Effect Size ES allows comparison between studies using different dependent variables because it puts data in standard deviation units. Scale: –ES of 0 is no difference, –ES of 0.2 or less is small, –ES of ~0.5 is medium, and –ES of 0.8 or more is large

9. Power The probability of rejecting the null hypothesis when it is false (detecting a real difference), or The probability of making a correct decision This concepts answers the following questions: –How large a difference is important in theory or practice? –How many participants are needed to declare an important difference as significant?

10. Effect Size Curve to Estimate Sample Size When p =.05

11. Effect Size Curve to Estimate Sample Size When p =.01

12. Using Information in the Context of the Study How do findings from the study fit within the context of –Theory –Practice Context is what matters with regard to “meaningfulness” –Estimates of significance are driven by sample size –Estimates of meaningfulness are driven by the size of the difference –Context is driven by how the findings will be used

13. Summary Information needed when planning research 1.Alpha Establishes the acceptable magnitude of Type I error Usually.05 or.01 2.Effect size The outcome of a study expressed in standard deviation units 3.Power The chance of rejecting a false null hypothesis 1 - beta 4.Sample size Number of participants in the study

14. Using the Power Calculator When Reading a Research Study When reading research, often sample size, means, and standard deviations are supplied. You can calculate the effect size by the formula in chapter 7. Using this data and the Power Calculator at the Web site below, you can estimate the power to detect a difference or relationship. http://calculators.stat.ucla.edu/powercalc/

15. Using the Power Calculator to Plan Research If you are planning your own research, you can often estimate the effect size from other studies. By setting your alpha (say.05) and power (say.8), you can use the Power Calculator at the website below to estimate the sample size you need. http://calculators.stat.ucla.edu/powercalc/