Hypothesis Testing making decisions using sample data

General Overview Our goal in conducting a study is often to estimate a particular parameter of interest with as little error as possible. Our goal in conducting a study is often to estimate a particular parameter of interest with as little error as possible. The % that report they will vote for a given candidate is a good example. The % that report they will vote for a given candidate is a good example.

General Overview We express our best estimate of this % in terms of both a point estimate and a Confidence Interval. We express our best estimate of this % in terms of both a point estimate and a Confidence Interval. The Confidence Interval gives a range of values that we are confident contains the true value in the population. The Confidence Interval gives a range of values that we are confident contains the true value in the population.

General Overview Sometimes our goal is to test a specific hypothesis. Sometimes our goal is to test a specific hypothesis. The hypothesis often involves differences between two groups in terms of average scores (means) or percentages. The hypothesis often involves differences between two groups in terms of average scores (means) or percentages.

General Overview Will a treatment group score higher than a control group? Will a treatment group score higher than a control group? Are males scoring differently on a particular test than females? Are males scoring differently on a particular test than females? Is the % of students with a particular disability higher among males than females? Is the % of students with a particular disability higher among males than females?

General Overview When we conduct studies to answer these types of questions, we create Confidence Intervals around our best guess of the amount of difference between the groups. When we conduct studies to answer these types of questions, we create Confidence Intervals around our best guess of the amount of difference between the groups. We also test hypotheses using the particular statistical strategy. We also test hypotheses using the particular statistical strategy.

The Process Craft a problem statement and specific research questions. Craft a problem statement and specific research questions. For each research question, translate it into an either or situation: the Null and Alternative hypotheses. For each research question, translate it into an either or situation: the Null and Alternative hypotheses.

The Process The Null Hypotheses The Null Hypotheses What we do not want to happen What we do not want to happen The conclusion we will draw unless compelling evidence convinces us otherwise. “The Show Me” state. The conclusion we will draw unless compelling evidence convinces us otherwise. “The Show Me” state. The no knowledge condition. The no knowledge condition. There is no difference between the groups. There is no difference between the groups.

The Process The Alternative Hypotheses The Alternative Hypotheses What we want to happen What we want to happen The conclusion we will draw if compelling evidence convinces us there is a difference. The conclusion we will draw if compelling evidence convinces us there is a difference. The new knowledge condition. The new knowledge condition. There is a difference between the groups. There is a difference between the groups.

The Process However, we can’t just accept any amount of difference between the sample groups and conclude that there is a difference between the populations. However, we can’t just accept any amount of difference between the sample groups and conclude that there is a difference between the populations. Why not? Why not?

The Process We want to know that the amount of difference we have observed is reasonably beyond what might be caused by sampling error. We want to know that the amount of difference we have observed is reasonably beyond what might be caused by sampling error. We use probability to help us be confident in our final decision. We use probability to help us be confident in our final decision.

The Process We choose a test statistic. We choose a test statistic. We determine what the sampling distribution of that test statistic is when the Null hypothesis is True. We determine what the sampling distribution of that test statistic is when the Null hypothesis is True.

The Process This is like saying, if there really is no difference in the population, how much difference might I get by sampling error alone? This is like saying, if there really is no difference in the population, how much difference might I get by sampling error alone? Given the size of the sample and the variability in the populations, what amount of difference can I expect even if there is no true difference? Given the size of the sample and the variability in the populations, what amount of difference can I expect even if there is no true difference?

The Process We next have to specify a region of the sampling distribution of the test statistic that represents an acceptable risk of rejecting the null hypothesis even when it is true. We next have to specify a region of the sampling distribution of the test statistic that represents an acceptable risk of rejecting the null hypothesis even when it is true. The Rejection Region The Rejection Region

The Process We set the size of this region to equal the acceptable probability of making this type of error. We set the size of this region to equal the acceptable probability of making this type of error. We call this probability Alpha We call this probability Alpha Typically set at.05 Typically set at.05

The Process After our study has been conducted and the test statistic has been calculated we apply a particular decision rule. After our study has been conducted and the test statistic has been calculated we apply a particular decision rule. Reject the null if the test statistic is large enough that it falls in the rejection region. Reject the null if the test statistic is large enough that it falls in the rejection region.

The Process Falling in the rejection region means that the test statistic is a rare event if the null hypothesis were true Falling in the rejection region means that the test statistic is a rare event if the null hypothesis were true It is unlikely that sampling error alone made the test statistic so big. It is more likely that there is a true difference between the groups It is unlikely that sampling error alone made the test statistic so big. It is more likely that there is a true difference between the groups

Can a picture help? Graphical display of the Type I and Type II regions Graphical display of the Type I and Type II regionsGraphical display of the Type I and Type II regionsGraphical display of the Type I and Type II regions Review of some definitions Review of some definitionsReview of some definitionsReview of some definitions

Some Examples Some examples of the 2x2 matrix Some examples of the 2x2 matrix Some examples of the 2x2 matrix Some examples of the 2x2 matrix