Chapter 2 HYPOTHESIS TESTING BAE 5333 Applied Water Resources Statistics Biosystems and Agricultural Engineering Department Division of Agricultural Sciences and Natural Resources Oklahoma State University Source Dr. Dennis R. Helsel & Dr. Edward J. Gilroy 2006 Applied Environmental Statistics Workshop and Statistical Methods in Water Resources

Which Statistical Test to Use? Type of Variables Grouped and Continuous Qualitative and Quantitative Distribution of Data Normal Non-normal

Statistical Test Based on Variable Type

Parametric vs. Nonparametric Parametric Test Assumes Specific underlying distribution for the data Or invoke the Central Limit Theorem (see next slide) Computed with mean, standard deviation (i.e. uses parameters) Normal Distribution most common assumption. Data should look symmetric.

Central Limit Theorem (CLT) Given a distribution with a mean (μ) and variance (σ2), the sampling distribution of the mean approaches a normal distribution with a mean (μ) and a variance σ2/N as N, the sample size, increases. No matter what the shape of the original distribution, the sampling distribution of the mean approaches a normal distribution. For most distributions, a normal distribution is approached very quickly as N increases. Parametric tests use the sample mean, which the CLT tells us will be approximately normally distributed. Source: http://davidmlane.com/hyperstat/A14043.html

Central Limit Theorem Example Computer sampled N values from a uniform distribution and computed the mean. This procedure was performed 500 times for samples sizes of 1, 4, 7, and 10. Mean N=Sample Size Value (x) a b Probability Uniform Distribution Source: http://davidmlane.com/hyperstat/A14461.html

Central Limit Theorem Examples Central Limit Theorem Applet http://www.stat.sc.edu/~west/javahtml/CLT.html Demonstrates the CLT using dice rolling Large number of rolls  bell-shaped curve ONE Die  approximate uniform distribution The Central Limit Theorem - How to Tame Wild Populations http://intuitor.com/statistics/CentralLim.html

Central Limit Theorem, cont. A sampling distribution always has less variability than the population. σs = σp / n1/2 σs = standard deviation of sampling distribution σp = standard deviation of the population n = sample size n = 100  1/10 the variability of the population n = 10000  1/100 the variability of the population The sampling distribution will act more like a normal distribution as the sample size increases, for any type distribution. If the CLT did not exist, we could not use parametric statistics since we could not reliably estimate a parameter like the mean by using an average derived from a much smaller sample. Source: http://intuitor.com/statistics/CentralLim.html

Parametric vs. Nonparametric Nonparametric Test Does not assume a specific distribution for the data (Distribution Free) Uses median, percentiles Information extracted from positions (ranks) of the data Data may be symmetric or skewed

Choice Based on Data Distribution Close to a Normal Distribution? Yes Use nonparametric test or transform data Parametric or nonparametric test

How Hypothesis Tests Work Choose a test procedure based on type of variable and data distribution Establish hypothesis Null (Ho) Alternative (Ha) Select alpha (significance level) Compute test statistic and obtain p-value (measure of how much evidence we have against the null hypothesis) Compare p-value to alpha Reject Ho if p-value < alpha

Null vs. Alternative Hypothesis Null Hypothesis (Ho) Assumed true until found unlikely by data The null state: no difference between groups, no correlation, no trend, etc. Alternative Hypothesis (Ha) Is chosen if null hypothesis is not supported by data Two Types: one-sided, two-sided http://www.itl.nist.gov/

Total Suspended Solids (TSS) in the Cimarron River Null Hypothesis (Ho): TSS is the same in the Cimarron River and the Arkansas River Alternative Hypothesis (Ha) Two-sided: TSS is different in the two rivers One-sided: TSS is higher in the Cimarron River One-sided: TSS is lower in the Cimarron River Reject Null Hypothesis when p < alpha

Alternative Hypothesis One or Two-sided? Two Sided (not equal to) Change can be high or low “Has TSS changed?” “Are values different between the Cimarron River and the Arkansas River?” One Sided (< or >) Change is directional “Has TSS increased?” “Are values higher between the Cimarron River and the Arkansas River?”

Outliers (p value) in Red Two-sided Test Outliers (p value) in Red

Outliers (p value) in Red One-sided Test Outliers (p value) in Red

How Do You Decide an Alpha Level? Alpha (significance level) is a management decision What percent error can I live with? Set by scientist, legislation, or management Common: 5% (alpha=0.05) Alpha is the probability of rejecting the null hypothesis when it is actually true (Type I Error)

How Do You Decide an Alpha Level? Continued… Choosing an alpha level: strength of evidence needed depends on study objectives, and cost of rejecting null hypothesis Small alpha 0.05 Preponderance of evidence Smaller alpha 0.03 Clear and convincing Very small alpha 0.01 Beyond a reasonable doubt

Type I and Type II Errors Type I Error (α) – occurs if one rejects the null hypothesis when it is true Type II Error (β) – occurs if one does NOT reject the null hypothesis when it is false Ho True Ho False Type I Error α Correct Decision 1- α Type II β 1- β Reject Ho Do Not Reject Ho Source: Elementary Statistics, 6th ed., Allan Bluman, McGraw-Hill

Type III Error Correctly rejecting the null hypothesis for the wrong reason. Correctly rejecting the null hypothesis, but incorrectly attributing the cause. Correctly identifying an effect, but incorrectly attributing the cause of the effect. Source: http://en.wikipedia.org/wiki/Type_III_error

Reading Assignment Chapter 4 Hypothesis Testing (pages 97 to 116) Statistical Methods in Water Resources by D.R. Helsel and R.M. Hirsch