Inference and Tests of Hypotheses PA330 March 13, 2000

Homework review

Two new questions What is the probability that a particular finding arose by chance? Tests of statistical significance How strong is the relationship between an independent variable and a dependent variable? Measures of association comparison of means tests analysis of variance regression analysis

First, we need to introduce two new ideas the null hypothesis Type I and Type II error

Hypotheses A research hypothesis is the hypothesis that the researcher is studying. Also called the alternative hypothesis. A null hypothesis states that no relationship exists between two variables or that there is no difference between categories of the IV on values of the DV. H1: Male planners earn higher salaries than female planners. H0: Gender is not related to planners’ salaries.

Hypotheses More examples H1: Students with an MSW/MPA are more likely to obtain an administrative job than those with just an MSW. H0: Students with MSW/MPA degrees are equally likely as those with an MSW to obtain an administrative job. H1: Students who write clearly on exams are more likely to receive high grades. H0: Writing clarity is not related to the probability that a student will receive a high exam grade.

Type I and Type II error In hypothesis testing, the possibility exists for two types of error. Type I error Rejecting a true null hypothesis. Concludes from sample data that a research hypothesis is true when it is, in fact, untrue.

Type I and Type II error Type II error Failure to reject a a false null hypothesis Failure to detect an existing relationship Note: we use the terminology “failure to reject”. We don’t say “accept the null hypothesis.”

Type I and Type II error If we determine there is no difference and there is no difference, we are correct. If we determine there is a difference, and there is a difference, we are also correct. Either way, we have not committed an error. However ...

Type I and Type II error What if we reject the null hypothesis (no difference), and there is no difference. Type I error What if we fail to reject the null hypothesis (no difference), and there is a difference. Type II error Thus, we have four possible scenarios.

Possible outcomes for test of statistical significance

Example A researcher is interested in determining if participation in the school lunch program improves the nutritional status of children’s lunches. S/he takes a sample of children from 50 different schools in the U.S. The IV is participation in school lunch program. The DV is nutritional status of children’s lunches.

Example What is the research hypothesis? The null? H1: Children who participate in the school lunch program eat more nutritious lunches than those who do not participate. H0: There is no difference in the nutrition of lunches eaten by children who participate in the school lunch program and those who do not.

Example If there is no difference in nutritional value between school lunches and lunches kids bring from home, but we reject the null hypothesis (no difference) - Type I error If there is a difference but we fail to pick up on that difference (in other words, fail to reject the null hypothesis) - Type II error

Inferential statistics Inferential statistics - estimate population parameters based on sample data/statistics Remember, we often do not have data on an entire population. We have to use sample data to estimate the population parameters (due to cost, inaccessibility, etc.)

Tests of statistical significance Tests of statistical significance are a type of inferential statistic (infer findings from a sample to the population) learn the probability that the observed relationship between variables in a sample could have occurred if the two variables are randomly related (not causally related to each other) In other words, “could this relationship have occurred by chance?”

Tests of statistical significance Sampling error - every possible sample either over or underestimates the population parameters. Every sample will deviate from what actually exists in the population. Due to probability of selecting one unit rather than another. does not mean you used the wrong procedures to sample (called bias or non-sampling error).

Standard error If we took repeated samples, we could determine the difference between the population mean and each sample mean. The average (mean) of these distances is called the standard error. It follows the same logic as the standard deviation.

Confidence level The confidence that the researcher has that any single sample estimates the population parameters within an acceptable range. Usually expressed as a range, called the confidence interval. The higher the degree of confidence needed by the researcher, the larger the required sample size.

Confidence level In the same way we can use a distribution and standard deviation to determine how far a particular case is from the mean, we can use a distribution and standard error to determine how far the mean of a particular sample is from the population mean.

Two-tailed tests A two-tailed test means that an equal part of the standard error is in each “tail” of the distribution.

Confidence interval Using z scores, we can calculate the actual values represented by the boundary of the area in the tails. For example, if we want a 95% confidence limit, we find the values for which 2.5% of scores (half of 5%) fall below and 2.5% fall above.

Example See handout

Hypothesis testing with sample data Do two variables have a nonrandom relationship? Four basic steps State the null and research hypothesis Select a significance level (alpha ()) Select and compute a test statistic Make a decision by comparing to critical value of the test statistic.

Hypothesis testing Hypothesis testing relies on disconfirming evidence We do not directly assert that the data confirm the hypothesis (although it may well do so). We disconfirm the null hypothesis.

Hypothesis testing Why do we disconfirm? Although we may confirm a hypothesis by demonstrating causality, eliminating alternative explanations (IV’s), and replicating the results, the study still only represents a single sample. Sampling error may exist. Another sample from the same population may contain data that cannot confirm the research hypothesis. Therefore, we can “prove” neither the research nor the null hypothesis.

alpha () level recognizes that there is always uncertainty about the true population parameters when using a sample. Pre-determines what level of uncertainty is acceptable The alpha level is a number between 0 and 1 The most commonly used levels are: .05 (5% chance of committing a Type I error) .01 (1% chance of committing a Type I error) .001 (0.1% chance of committing a Type I error)

Selecting an alpha level Depends on the practical consequences of being wrong - of committing Type I and Type II error In general, =.05 is used in social sciences Also a trade off between power of the statistic, sample size, and size of the effect. If you have a small sample, =.05 may not detect changes.

Selecting a test statistic Researcher selects an appropriate test statistic to determine the probability that the hypothesized relationship in the population is random. Most commonly used in social science Difference of means tests (interval data) t-test to compare means of two groups (sample mean to population mean; two independent sample means) z test to compare sample mean to a population with a known mean and standard deviation chi-square (2) for nominal level data

Make a decision After selecting a test statistic, you calculate it. Then, you compare the calculated value to the critical value (from a table) and decide if your case or group fits within the uncertainty level () you have chosen.

Example - t test You are interested in knowing if the mean age of W.V. residents is higher than the mean age of the U.S. You take a sample of 144 W.V. residents and obtain the following data. For the U.S.  = 32 For W.V. (the sample) mean = 36 s = 14.5 The mean is higher for WV than the U.S. Is the difference statistically significant?

Example - t test State the research and null hypothesis. H1: The average age of West Virginians is higher than that of the U.S. H0:There is no difference in the average age of West Virginias and the entire U.S. Note: this is a directional hypothesis. It states that one is higher or lower than the other.

Example - t test Select an . In social sciences the general standard is  = .05 (remember, this equates to what percent of the time you are likely to be wrong). Choose a test statistic - the t test is an appropriate statistic since we are comparing the mean of a group (sample) to a known population mean, but we don’t know the pop standard deviation.

Another t test formula (than one in your book)

Example - t test Compute the statistic

Example - t test Find the critical t (2.58) Because this problem has a directional hypothesis, we will use a one-tailed test. The degrees of freedom are equal to the sample size minus 1. Using the t table in the back of your book, find the critical t for  = .05 and df = 143.

Example - t test So, the calculated t = 3.31 the critical t (t.05) = 2.58 Because the calculated t score is greater than the critical t, we reject the null hypothesis. There is a statistically significant difference in the mean age of WV and of the population.

Practice problems Handout