Stat 100 Chapter 23, Try prob. 5-6, 9 –12 Read Chapter 24

Example N=20 teens with high blood pressure take calcium supplements to reduce b.p. After two months, mean decrease for these 20 teens was 6 points

Hypotheses for Significance Test null : mean b.p. does not change with calcium supplements alternative: mean b.p. decreases with calcium supplements These statements are for larger population represented by the sample of 20 teens

How the decision is made Determine the probability that observed decrease would be as large as 6 if calcium really has no effect This is called the “p-value” We’ll skip the details of finding this Rule is: reject the null if the p-value is less than.05 (5%).

P-value and Conclusion Suppose p-value in our example is found to be.03 (3%) This is below 5% guideline for significance. We can reject the null hypothesis Conclude that calcium causes drop in b.p.

Example Suppose you are on a panel considering the case of a student accused of cheating in a class. If “convicted” the student will fail the class. What do you think would be appropriate null and alternative hypotheses in this case?

Hypotheses Null: Student did not cheat Alternative: Student did cheat

Possible errors What are the two decision errors that might happen over many cases like this? Might convict somebody who did not cheat Might fail to convict somebody who did cheat

Which error would be worse? Probably convicting an innocent person. So rules of evidence might protect against this. Downside would be we might often let off guilty people (because it’s hard to convict)

Two Possible errors in significance testing Type 1 Error: picking alternative when null is really true Type 2 Error : picking null when alternative is really true

Example Researchers compare effectiveness of placebo and Zoloft for treating depression What are the null and alternative hypotheses? Null: no difference in effectiveness Alternative: Zoloft is more effective

Possible errors Type 1 = picking the alternative when the null is true = saying Zoloft is more effective when it’s not Type 2 = picking the null when the alternative is true = saying there’s no difference when there really Zoloft is better

Most Common Cause of Type 2 Error Small sample size Small study may not be definitive about significance so there’s a risk of not be able to reject the null Similar to not having enough evidence to convict somebody who’s really guilty

The Effect of Sample Size The larger the study, the smaller the risk of Type 2 error Put another way - The larger the study, the better the chance of finding a true difference.

Power The term “power” defines the chance of not making a type 2 error That is, power = chance of finding a true difference As sample size is increased, power is increased

The Problem with A Huge Sample A small, unimportant difference may be called “statistically significant” Headline: “Spring Birth Provides Height Advantage” Data: N=400,000 Austrian military recruits Observed diff in heights: Spring versus rest=1/4 of inch. The sample was so big, it was possible to say there was a difference

Meta-Analysis Approach Combine results of different researchers' studies of the same problem Example - recent news item about zinc's affect on cold symptoms –combined five different studies of zinc's affect on cold duration Concluded taking zinc may have some benefit

Why meta-analysis ? Basically increases sample size and generates more power Good to combine all known information Smaller studies might all be inconclusive, but the combined effect could be significant

Difficulties Studies may not be comparable –different populations – different methods Nonsignificant studies may not have been published (called the file drawer problem) Biased analyst may give more weight to studies proving his or her point