Hypothesis Testing (Statistical Significance)

Hypothesis Testing Goal: Make statement(s) regarding unknown population parameter values based on sample data Elements of a hypothesis test: Null hypothesis - Statement regarding the value(s) of unknown parameter(s). Typically will imply no association between explanatory and response variables in our applications (will always contain an equality) Alternative hypothesis - Statement contradictory to the null hypothesis (will always contain an inequality) The level of significant (Alpha) is the maximum probability of committing a type I error. P(type I error)= alpha

Definitions Rejection (alpha, α) Region: Represents area under the curve that is used to reject the null hypothesis Level of Confidence, 1 - alpha (a): Also known as fail to reject (FTR) region Represents area under the curve that is used to fail to reject the null hypothesis FTR H0H0 α/2

1 vs. 2 Sided Tests Two-sided test No a priori reason 1 group should have stronger effect Used for most tests Example H 0 : μ 1 = μ 2 H A : μ 1 ≠ μ 2 One-sided test Specific interest in only one direction Not scientifically relevant/interesting if reverse situation true Example H 0 : μ 1 ≤ μ 2 H A : μ 1 > μ 2

Example: It is believed that the mean age of smokers in San Bernardino is 47. Researchers from LLU believe that the average age is different than 47. Hypothesis H 0 :μ = 47 H A : μ ≠ 47 μ = 47 α /2 = 0.025Fail to Reject (FTR) α /2 = 0.025

Three Approaches to Reject or Fail to Reject A Null Hypothesis: 1a. Confidence interval  Calculate the confidence interval Decision Rule: a. If the confidence interval (CI) includes the null, then the decision must be to fail to reject the H 0. b. If the confidence interval (CI) does not include the null, then the decision must be to reject the H 0.

1b. Confidence interval to compare groups  Calculate the confidence interval for each group Decision Rule: a. If the confidence interval (CI) overlap, then the decision must be to fail to reject the H 0. b. If the confidence interval (CI) do not include the null, then the decision must be to reject the H 0.

2.Test Statistic Calculate the test statistic (TS) Obtain the critical value (CV) from the reference table Decision Rule: a. If the test statistic is in the FTR region, then the decision must be to fail to reject the H 0. b. If the test statistic is in the rejection region, then the decision must be to reject the H 0. FTR CV TS Since the test statistic is in the rejection region, reject the H 0 FTR CV Since the test statistic is in the fail to reject region, fail to reject the H 0 TS CV

3. P-Value Choose α Calculate value of test statistic from your data Calculate P- value from test statistic Decision Rule: a. If the p-value is less than the level of significance, α, then the decision must be to reject H 0. b. If the p-value is greater than or equal to the level of significance,α, then the decision must be to fail to reject H 0. FTR CV TS FTR CV TS P-value

Types of Errors!

Types of Errors Truth Hypothesis Testing Decision Based on a Random Sample 1-α (Correct Decision ) Type II error (β) Type I error (α) 1-β ( Power) (Correct Decision) Fail to Reject H 0 Reject H 0 The Null Hypothesis (H 0 ) is True The Null Hypothesis (H 0 ) is False

FTR CV H 0 is True t s Since the H 0 is true and we decide to accept it, we have thus made a correct decision Correct Decision

FTR tsts CV H 0 is True Since the H 0 is true and we decide to reject it, we have thus made an incorrect decision leading to Type I error Alpha (α) Error

tsts FTR CV H 0 is False Since the H 0 is False and we decide to reject it, we have thus made a correct decision Power

FTR t s CV H 0 is False Since the H 0 is False and we decide to accept it, we have thus made an incorrect decision leading to type II error. Beta, β, Error

Null Hypothesis TrueFail to reject Correct Decision Reject Type I Error FalseFail to Reject Type II Error Reject Correct Decision

How to Reduce Errors Alpha error is reduced by increasing the confidence interval or reducing bias Beta error is reduced by increasing the sample size Alpha and beta are inversely related

Example  What type of error was possibly committed in the above example?  How would you reduce the error?