Difference Between Means Test (“t” statistic)

Test statistics Independent and dependent variables are continuous Regression (r2 and R2) b statistic - interpreted as unit change in the DV for each unit change in the IV Independent variables are nominal or continuous; dependent variable is nominal Logistic regression, generates “b” and exp(b) (a.k.a. odds ratio) Independent and dependent variables are categorical Chi-Square (X2) Categorical dependent and continuous independent variables Difference between the means test (t statistic) Procedure Level of Measurement Statistic Interpretation Regression All variables continuous r2, R2 b Proportion of change in the dependent variable accounted for by change in the independent variable. Unit change in the dependent variable caused by a one-unit change in the independent variable Logistic regression DV nominal & dichotomous, IV’s nominal or continuous exp(B) (odds ratio) Don’t try - it’s on a logarithmic scale Odds that DV will change if IV changes one unit, or, if IV is dichotomous, if it changes its state. Chi-Square All variables categorical (nominal or ordinal) X2 Reflects difference between Observed and Expected frequencies. Difference between means IV dichotomous, DV continuous t Reflects magnitude of difference. Use table to determine if coefficient is sufficiently large to reject null hypothesis.

Difference Between Means (“t”) Test So far we’ve examined several statistics that can be used to test hypotheses: Chi-Square (X 2): All variables must be categorical Regression (b & R2): Continuous IV’s (some nominal IV’s OK) and a continuous DV Logistic regression (b & Exp b): Continuous and nominal IV’s and a nominal DV The difference between the means test, which yields the “t” statistic, is used to test hypotheses with categorical IV’s and a continuous DV Gender  height, Gender  cynicism (e.g., 1-5 scale) We compare the means of two randomly drawn samples The null hypothesis can be rejected if the difference, reflected in the size of the t statistic, goes so far beyond what would be expected by sampling error, that there are less than five chances in 100 (p> .05) that the relationship between the variables is due to chance This sampling error is called the “standard error of the difference between means” – the difference between all possible pairs of means, due to chance alone Must know if the hypothesis is 1-tailed (direction of effect predicted) or 2-tailed (not predicted) Major advantage: Remember that weak real-life effects can produce significant results? When comparing means, we know their actual values. This lets us recognize situations where the “real world” differences are trivial.

Calculating t 1. Obtain the “pooled sample variance” Sp2 (Simplified method – midpoint between the two sample variances) Compute the S.E. of the Diff. Between Means Compute the t statistic Compute the “degrees of freedom” df = n1 + n2 - 2 (total number of cases in both samples minus 2) Actual (“obtained”) difference between means Predicted difference due to sampling error The t is a ratio: the greater the difference between means, the smaller the predicted error, the larger the t coefficient The larger the t, the more likely we are to reject the null hypothesis. According to the null, any relationship between variables, any difference between means, is due to chance. Our actual difference between means must be “significantly” larger than the difference we would obtain by chance. We use a table to determine whether the t is large enough to reject the null hypothesis (see next slide). We can reject the null if the probability that the difference between means is due to chance is less than five in one-hundred (p< .05). If the probability that the difference between the means is due to chance is five in one-hundred or larger (p> .05), the null hypothesis is true.

Classroom exercise - Jay’s cop shop H1: Male officers more cynical than females H2: Officer gender determines cynicism Draw one sample of male officers, one of females Compute each sample’s variance, then obtain the pooled sample variance Compute the S.E. of the Difference Between Means Calculate the t coefficient Compute the degrees of freedom df = n1 + n2 - 2 Check the t table for significance. Confirm the original, “working” hypothesis if there are less than 5 chances in 100 that the null hypothesis is true. Be sure to use the correct significance row (one-tailed or two-tailed). Use the one-tailed test if the working hypothesis predicts the direction of the difference - that a specific gender (male or female) is significantly more cynical than the other. Use the two-tailed test if the working hypothesis predicts that there is a significant difference in cynicism between genders, but not which gender is more cynical. One-tailed hypotheses require a smaller t to reach statistical significance. t-table

1. Is hypothesis one-tailed (direction of change in the DV predicted) or two-tailed (direction not predicted)? H1: Males more cynical than females. That’s one-tailed, so use the top row. H2: Males and females differ in cynicism. That’s two-tailed, so use the second row. 2. df, “Degrees of Freedom” represents sample size – add the numbers of cases in both samples, then subtract two: df = n1 + n2 – 2 3. To call a t “significant” (confirm the working hypothesis, thus reject the null hypothesis) the coefficient must be as large or larger than what is required at the .05 level; that is, we cannot take more than 5 chances in 100 that the difference between means is due to chance. For a one-tailed test, use the top row, then slide over to the .05 column. For a two-tailed test, use the second row, then slide to .05 column. If the t is smaller than the number at the intersection of the .05 column and the appropriate df row, it is non-significant. If the t is that size or larger, it is significant. Slide to the right to see if it is large enough to be significant at a more stringent level.

Higher income  More expensive car Parking lot exercise Higher income  More expensive car Transfer your panel’s data from the other coding sheet Compute each sample’s variance, then calculate the pooled sample variance = = 1.49 Compute the S.E. of the Difference Between Means  1.49 ( 1 + 1 ) 10 10  1.49 (.2) =  .3 = .55 …continued on next slide 2.97 2 2 1 1 2 1.4 1.4 1.4 .6.4 .4 .4 .4.4.6.4.6.6 .36.16.16.16.16.16.36.16.36.36 2.4 .27 4 5 2 1 3.3 .7 1.7 1.3 2.3 .49 2.9 1.7 5.3 24.17 2.7

Yes! There are less than five chances in one-thousand that the null Calculate the t coefficient = = -3.5 Note: the sign, + or -, indicates the direction of the difference between groups. Keep that in mind! It turns out that, consistent with the hypothesis, the faculty lot has the more expensive cars. We had arbitrarily placed it second, so subtracting yields a negative t. Compute the df (degrees of freedom) df = n1 + n2 - 2 = 10 + 10 - 2 = 18 Check the t table for significance. Note: Use the one-tailed test if the working hypothesis predicts the direction of the difference (which parking lot would have more expensive cars). Use the two-tailed test if the hypothesis predicts there will be a difference in car values between the lots, but not which lot would have the more expensive cars. df = 18 t = -3.5 1.4 - 3.3 .55 -1.9 .55 Yes! There are less than five chances in one-thousand that the null hypothesis is true (one-tailed) or less than one in one-hundred that it is true (two-tailed)

More complex mean comparisons: Analysis of Variance

When there are more than two groups: Analysis of Variance Dependent variable: continuous Independent variable(s): categorical Example: does officer professionalism vary between cities? (scale 1-10) Calculate the “F” statistic, look up the table. An “F” statistic that is sufficiently large can overcome the null hypothesis that the differences between the means are due to chance. City L.A. S.F. S.D. Mean 8 5 3

“Two-way” Analysis of Variance Stratified independent variable(s) F statistic is a ratio of “between-group” to “within” group differences. To overcome the null hypothesis, the differences in scores between groups (between cities and, overall, between genders) should be much greater than the differences in scores within cities Between group variance (error + systematic effects of ind. variable) Within group variance (how scores disperse within each city) Between Within City L.A. S.F. S.D. Mean – M 10 7 5 Mean - F 6 3 2

Homework

Homework assignment Two random samples of 10 patrol officers from the XYZ Police Department, each officer tested for cynicism (continuous variable, scale 1-5) Sample 1 scores: 3 3 3 3 3 3 3 1 2 5 -- Variance = .99 Sample 2 scores: 2 1 1 2 3 3 3 3 4 2 -- Variance = .93

Pooled sample variance Sp2 Simplified method: midpoint between the two sample variances Sp2 = Standard error of the difference between means x1 -x2 x1 -x2 =  Sp2 ( ) T-Test for significance of the difference between means x1 -x2 t = -------------- x1 -x2 s21 + s22 2 1 1 n1 n2 +

CALCULATIONS Pooled sample variance: .96 Standard error of the difference between means: .44 t statistic: 1.14 df – degrees of freedom: (n1 + n2) – 2 = 18 Would you use a ONE-tailed t-test OR a TWO-tailed t-test? Depends on the hypothesis Two-tailed (does not predict direction of the change): Gender  cynicism One-tailed (predicts direction of the change): Males more cynical than females Can you reject the NULL hypothesis? (probability that the t coefficient could have been produced by chance must be less than five in a hundred) NO – For a ONE-tailed test need a t of 1.734 or higher NO – For a TWO-tailed test need a t of 2.101 or higher

Final exam practice questions (answers at the end)

Question 1 You will have to write two relatively brief, one-paragraph answers: 1. General question about hypotheses 2. Question about rejecting the null hypothesis Your responses to the above should be based on a thorough understanding of the material in slides 3-7 of the Inferential Stat Intro presentation

Question 2 Hypothesis is that alarm systems prevent crime Question 2 Hypothesis is that alarm systems prevent crime. To test this hypothesis we randomly sampled 120 businesses with an alarm system, and 90 businesses without an alarm system: With an alarm system: 50 were victims of crime, 70 were not victims of crime Without an alarm system: 50 were victims of crime, 40 were not victims of crime. 1-A: State the “null hypothesis” using plain English (2 pts.) 1-B. Draw a table with the Observed frequencies 1-C. Compute the Expected frequencies and place them in a table 1-D. Compute the Chi-Square 1-E. Compute the degrees of freedom df = (r-1) X (c-1) 1-F. Check the table. Do the results support the working hypothesis?

Question 3 Are male CJ majors significantly more cynical than female CJ majors? We randomly sampled five males and five females. Here are their test results: Males: 4, 5, 5, 3, 4 Variance (s2): 0.7 Females: 4, 3, 4, 4, 5 Variance (s2): 0.5 2-A: State the “null hypothesis” using plain English (2 pts.) 2-B. Pooled sample variance (midpoint between the two variances): _______ 2-C. Standard error of the differences between means ( ): _______ 2-D. t-statistic: ______ 2-E. df (degrees of freedom): _____ df = (n1+n2)-2 2-F. Check the t table. Can you reject the null hypothesis? YES ____ NO ____ 2-G. Justify your answer to 2-F using plain English. Avoid technical terms. Make sure to include the estimated accuracy.

Question 4 Hypothesis: Unstructured socializing and other factors  youth violence In which model does Age have the greatest effect? What is its numerical significance? Use words to explain #2 Use Odds Ratio (same as Exp b) to describe the percentage effect of Age on Violence in Model 1 What happens to Age as it moves from Model 2 to Model 3? What seems most responsible?

Answers to practice questions 2-4 (No answers for question 1 - please study the Inferential stat’s intro slide show, esp. slides 3-7)

ANSWERS TO QUESTION 2 Null hypothesis: No significant difference in crime between businesses with and without alarms Observed frequencies Expected frequencies Chi-Square = 3.82 Df = (r-1) X (c-1) = 1 Check the table. Do the results support the working hypothesis? No - Chi-Square must be at least 3.84 to reject the null hypothesis of no relationship between alarm systems and crime, with only five chances in 100 that it is true

ANSWERS TO QUESTION 3 Null hypothesis: No significant difference between cynicism of males and females Variance for males (provided): 0.7 Variance for females (provided): 0.5 Pooled sample variance = .6 SE of the difference between means = .49 t = .41 df = 8 Check the “t” table. Can you reject the null hypothesis? NO Describe conclusion using words: The t must be at least 1.86 (one-tailed test) to reject the null hypothesis of no significant difference in cynicism, with only five chances in 100 that it is true.

ANSWERS TO QUESTION 4 Hypothesis: Unstructured socializing and other factors  youth violence In which model does Age have the greatest effect? Model 1 What is its numerical significance? .001 Use words to explain #2 Less than one chance in 1,000 that the relationship between age and violence is due to chance Use Odds Ratio (same as Exp b) to describe the percentage effect of Age on Violence in Model 1 For each year that age increases, violence is seventeen percent more likely What happens to Age as it moves from Model 2 to Model 3? What seems most responsible? Age becomes non-significant. Most likely cause is introduction of variable Deviant Peers (it is the most significant of the four variables introduced in Model 3).