Sociology 5811: Lecture 9: CI / Hypothesis Tests Copyright © 2005 by Evan Schofer Do not copy or distribute without permission

Announcements Problem Set #3 Due next week Problem set posted on course website We are a bit ahead of reading assignments in Knoke book Try to keep up; read ahead if necessary

Review: Confidence Intervals General formula for Confidence Interval: Where: Y-bar is the sample mean Sigma sub-Y-bar is the standard error of the mean Z (alpha/2) is the critical Z-value for a given level of confidence –If you want 90%, look up Z for 45% (  /2) –See Knoke, Figure 3.5 on page 87 for info

Small N Confidence Intervals Issue: What if N is not large? The sampling distribution may not be normal Z-distribution probabilities don’t apply… Standard CI formula doesn’t work Solution: Use the “T-Distribution” A different curve that accurately approximates the shape of the sampling distribution for small N Result: We can look up values in a “t-table” to determine probabilities associated with a # of standard deviations from the mean.

Confidence Intervals for Small N Small N C. I. Formula: Yields accurate results, even if N is not large Again, the standard error can be estimated by the sample standard deviation:

T-Distributions The T-distribution is a “family” of distributions In a T-Distribution table, you’ll find many T-distributions to choose from –Basically, the shape of sampling distribution varies with the size of your sample You need a specific t-distribution depending on sample size One t-distribution for each “degree of freedom” –Also called “df” or “DofF” Which T-distribution should you use? For confidence intervals: Use T-distribution for df = N - 1 Ex: If N = 15, then look at T-distribution for df = 14.

Looking Up T-Tables Choose the correct df (N-1) Choose the desired probability for  /2 Find t-value in correct row and column Interpretation is just like a Z-score = number of standard errors for C.I.!

Answering Questions… Knowledge of the standard error allows us to begin answering questions about populations Example: National educational standard requires all schools to maintain a test score average of 60 You observe that a sample (N=16, s=6) has a mean of 62 Question: Are you confident that the school population is above the national standard? We know Y-bar for the sample, but what about  for the whole school? Are we confident that  > 60?

Question: Is  > 60? Strategy 1: Construct a confidence interval around Y-bar And, see if the bounds fall above 60 Visually: Confident that  > 60: Y Visually:  might be 60 or less Y

Question: Is  > 60? Strategy 1: Construct a confidence interval around Y-bar –Let’s choose a desired confidence level of.95 –N of 16 is “small”… we must use the t-distribution, not the Z-distribution –Look up t=value for 15 degrees of freedom (N-1).

Looking Up T-Tables Choose the correct df (N-1)=15 Choose the desired probability for  /2 Find t-value in correct row and column Result: t = 2.131

Question: Is  > 60? Strategy 1: Construct a confidence interval around Y- bar Y CI is to 65.47! We aren’t confident  > 60

Question: Is  > 60? Note #1: Results would change if we used a different confidence level A 95% and 50% CIs yield different conclusions: Idea: Wouldn’t it be nice to know exactly which CI would describe the distance from Y-bar to  ? i.e., to calculate the exact probability of Y-bar falling a certain distance from  ? Y

Question: Is  > 60? Note #2: We typically draw CIs around Y-bar –But, we can also get the same result focusing on our comparison point (Y = 60) Example: If 60 is outside of CI around Y-bar Then, Y-bar is outside of the CI around Y Y

Question: Is  > 60? The critical issue is: How far is the distance between Y-bar and 60 –Is it “far” compared to the width of the sampling distribution? Ex: Y-bar is more than 2 Standard Errors from 60? In which case, the school probably exceeds the standard –Or, is it relatively close? Ex: Y-bar is only.5 Standard Errors from 60 In which case we aren’t confident… –Note: If we know the sampling distribution is normal (or t-distributed), we can convert SE’s to a probability

Question: Is  > 60? Strategy 2: Determine the probability of Y-bar = 62, if  is really 60 or less Procedure: –1. Use Y=60 as a reference point –2. Determine how far Y-bar is from 60, measured in Standard Errors Which we can convert to a probability –3. Issue: Is it likely to observe a Y-bar as high as 62? If this is common to observe, even when  = 60 (or less), then we can’t be confident that  > 60! But, if that is a rare event, we can be confident that  > 60!

Question: Is  > 60? Strategy 2: Look at sampling distribution Confident that  not 60 or less: Y Visually:  might easily be < Y  is unlikely to really be 60… because Y-bar usually falls near the center of the sampling distribution! In this case, it is common to get Y-bars of 62 or even higher

Question: Is  > 60? Issue: How do we tell where Y-bar falls within the sampling distribution? Strategy: Compute a Z-score Recall: Z-scores help locate the position of case within a distribution It can tell us how far a Y-bar falls from the center of the sampling distribution In units of “standard errors”! Probability can be determined from a Z-table Note: for small N, we call it a t-score, look up in a t-table.

Question: Is  > 60? Note: We use a slightly modified Z formula “Old” formula calculates # standard deviations a case falls from the sample mean From Y-sub-i to Y-bar New formula tells the number of standard errors a mean estimate falls from the population mean  Distance from Y-bar to  in the sampling distribution In this case we compare to hypothetical  = 60.

Question: Is  > 60? Let’s calculate how far Y-bar falls from  –Since N is small, we call it a “t-score” or “t-value” Y-bar is 1.33 standard errors above  !

Question: Is  > 60? Question: What is the probability of t>1.33 i.e., Y-bar falling or more standard errors from  ? Result: p = about.105 Note: Knoke t-table doesn’t contain this range… have to look it up elsewhere or use SPSS to calculate probability Y This area reflects the probability

Question: Is  > 60? Result: p =.105 In other words, if  = 60, we will observe Y-bar of 62 or greater about 10% of the time Conclusion: It is plausible that  is 60 or lower We are not 95% confident that  > 60 Conclusion matches result from confidence interval We have just tested a claim using inferential statistics!

Hypothesis Testing Hypothesis Testing: A formal language and method for examining claims using inferential statistics –Designed for use with probabilistic empirical assessments Because of the probabilistic nature of inferential statistics, we cannot draw conclusions with absolute certainty –We cannot “prove” our claims are “true” –However, improbable, we will occasionally draw an un-representative sample, even if it is random

Hypothesis Testing The logic of hypothesis testing: We cannot “prove” anything Instead, we will cast doubt on other claims, thus indirectly supporting our own Strategy: 1. We first state an “opposing” claim The opposite of what we want to claim 2. If we can cast sufficient doubt on it, we are forced (grudgingly) to accept our own claim.

Hypothesis Testing Example: Suppose we wish to argue that our school is above the national standard First we state the opposite: “Our school is not above the national standard” Next we state our alternative: “Our school is above the national standard” If our statistical analysis shows that the first claim is highly improbable, we can “reject” it, in favor of the second claim …“accepting” the claim that our school is doing well.

Hypothesis Testing: Jargon Hypotheses: Claims we wish to test Typically, these are stated in a manner specific enough to test directly with statistical tools –We typically do not test hypotheses such as “Marx was right” / “Marx was wrong” –Rather: The mean years of education for Americans is/is not above 18 years.

Hypothesis Testing: Jargon The hypothesis we hope to find support for is referred to as the alternate hypothesis The hypothesis counter to our argument is referred to as the null hypothesis Null and alternative hypotheses are denoted as: H 0 : School does not exceed the national standard H-zero indicates null hypothesis H 1 : School does exceed national standard H-1 indicates alternate hypotheses Sometimes called: “Ha”

Hypothesis Testing: More Jargon If evidence suggests that the null hypothesis is highly improbable, we “reject” it Instead, we “accept” the alternative hypothesis So, typically we: Reject H 0, accept H 1 –Or: Fail to reject H 0, do not find support for H 1 That was what happened in our example earlier today…

Hypothesis Testing In order to conduct a test to evaluate hypotheses, we need two things: 1. A statistical test which reflects on the probability of H 0 being true rather than H 1 Here, we used a z-score/t-score to determine the probability of H 0 being true 2. A pre-determined level of probability below which we feel safe in rejecting H 0 (  ) In the example, we wanted to be 95% confident…  =.05 But, the probability was.10, so we couldn’t conclude that the school met the national standard!

Hypothesis Test for the Mean Example: Laundry Detergent Suppose we work at the Tide factory We know the “cleaning power” of tide detergent, exactly: It is 73 on a continuous scale. “Cleaning Power” of Tide = 73 You conduct a study of a competitor. You buy 50 bottles of generic detergent and observe a mean cleaning power of 65 H0: Tide is no better than competitor (  >= 73) H1: Tide is better than competitor (  < 73)

Hypothesis Test: Example It looks like Tide is better: Cleaning power is 73, versus 65 for a sample of the competition Question: Can we reject the null hypothesis and accept the alternate hypothesis? Answer: No! It is possible that we just drew an atypical sample of generic detergent. The true population mean for generics may be higher.

Hypothesis Test: Example We need to use our statistical knowledge to determine: What is the probability of drawing a sample (N=50) with mean of 65 from a population of mean 73 (the mean for Tide) If that is a probable event, we can’t draw very strong conclusions… But, if the event is very improbable, it is hard to believe that the population of generics is as high as that of Tide… We have grounds for rejecting the null hypothesis.

Hypothesis Test: Example How would we determine the probability (given an observed mean of 65) that the population mean of generic detergent is really 73? Answer: We apply the Central Limit Theorem to determine the shape of the sampling distribution And then calculate a Z-value or T-value based on it If we chose an alpha (  ) of.05 If we observe a t-value with probability of only.0023, then we can reject the null hypothesis. If we observe a t-value with probability of.361, we cannot reject the null hypothesis

Hypothesis Test: Steps 1. State the research hypothesis (“alternate hypothesis), H 1 2. State the null hypothesis, H 0 3. Choose an  -level (alpha-level) –Typically.05, sometimes.10 or Look up value of test statistic corresponding to the  -level (called the “critical value”) Example: find the “critical” t-value associated with  =.05

Hypothesis Test: Steps 5. Use statistics to calculate a relevant test statistic. –T-value or Z-value –Soon we will learn additional ones 6. Compare test statistic to “critical value” –If test statistic is greater, we reject H 0 –If it is smaller, we cannot reject H 0

Hypothesis Test: Steps Alternate steps: 3. Choose an alpha-level 4. Get software to conduct relevant statistical test. –Software will compute test statistic and provide a probability… the probability of observing a test statistic of a given size. –If this is lower than alpha, reject H 0

Hypothesis Test: Errors Due to the probabilistic nature of such tests, there will be periodic errors. Sometimes the null hypothesis will be true, but we will reject it –Our alpha-level determines the probability of this Sometimes we do not reject the null hypothesis, even though it is false

Hypothesis Test: Errors When we falsely reject H 0, it is called a Type I error When we falsely fail to reject H 0, it is called a Type II error In general, we are most concerned about Type I errors… we try to be conservative.

Hypothesis Tests About a Mean What sorts of hypothesis tests can one do? 1. Test the hypothesis that a population mean is NOT equal to a certain value –Null hypothesis is that the mean is equal to that value. 2. Population mean is higher than a value –Null hypothesis: mean is equal or less than a value 3. Population mean is lower than a value –Null hypothesis: mean is equal or greater than a value Question: What are examples of each?

Hypothesis Tests About Means Example: Bohrnstedt & Knoke, section 3.93, pp N = 1015, Y-bar = 2.91, s=1.45 H0: Population mean  = 4 H1: Population mean  not = 4 Strategy: 1. Choose Alpha (let’s use.001) 2. Determine the Standard Error 3. Use S.E. to determine the range in which sample means (Y-bar) is likely to fall 99.9% of time, IF the population mean is If observed mean is outside range, reject H0

Example: Is  =4? Let’s determine how far Y-bar is from hypothetical  =4 In units of standard errors Y-bar is 24 standard errors below 4.0!

Hypothesis Tests About a Mean A Z-table (if N is large) or a T-table will tell us probabilities of Y-bar falling Z (or T) standard deviations from  In this example, the desired  =.001 Which corresponds to t=3.3 (taken from t-table) –That is:.001 (i.e,.1%) of samples (of size 1015) fall beyond 3.29 standard errors of the population mean –99.9% fall within 3.29 S.E.’s.

Hypothesis Tests About a Mean There are two ways to finish the “test” 1. Compare “critical t” to “observed t” –Critical t is 3.3, observed t = -24 We reject H0: t of +/-24 is HUGE, very improbable It is highly unlikely that  = 4 2. Actually calculate the probability of observing a t-value of 24, compare to pre-determined  If observed probability is below , reject H0 –In this case, probability of t=27 is … Very improbable. Reject H0!

Two-Tail Tests Visually: Most Y-bars should fall near  99.9% CI: –3.3 < t < 3.3, or 3.85 to 4.15 Sampling Distribution of the Mean Z=-3.3 Z=+3.3 Mean of 2.91 (t=24) is far into the red area (beyond edge of graph)

Hypothesis Tests About a Mean Note: This test was set up as a “two-tailed test” Meaning, that we reject H0 if observed Y-bar falls in either tail of the sampling distribution Ex: Very high Y-bar or very low Y-bar means reject H0 –Not all tests are done that way… Sometimes you only reject H0 if Y-bar falls in one particular tail.

Hypothesis Testing Definition: Two-tailed test: A hypothesis test in which the  -area of interest falls in both tails of a Z or T distribution. Example: H0: m = 4; H1: m ≠ 4 Definition: One-tailed test: A hypothesis test in which the  -area of interest falls in just one tail of a Z or T distribution. Example: H0:  > or = 4; H1:  < 4 This is called a “directional” hypothesis test.

Hypothesis Tests About Means A one-tailed test: H1:  < 4 Entire  -area is on left, as opposed to half (  /2) on each side. Also, critical t-value changes. 4

Hypothesis Tests About Means T-value changes because the alpha area (e.g., 5%) is all concentrated in one size of distribution, rather than split half and half. One tail vs. Two-tail:  

Hypothesis Tests About Means Use one-tailed tests when you have a directional hypothesis –e.g.,  > 5 Otherwise, use 2-tailed tests Note: In many instances, you are more likely to reject the null hypothesis when utilizing a one- tailed test –Concentrating the alpha area in one tail reduces the critical T-value needed to reject H0

Tests for Differences in Means A more useful and interesting application of these same ideas… Hypothesis tests about the means of two different groups –Up until now, we’ve focused on a single mean for a homogeneous group –It is more interesting to begin to compare groups –Are they the same? Different? We’ll do that next class!