1 PRINCIPLES OF HYPOTHESIS TESTING

2 A Quick Review of Important Issues About Sampling: To examine the sample’s attributes (sample statistics) as ESTIMATES of the population’s characteristics (population parameters) use sample characteristics to make inferences about the population. Estimating, by definition, involves some error (i.e., sampling error/bias Resulting from the fact that the sample may not mirror characteristics of the population). Why Sampling?

3 HYPOTHESIS TESTING OFTEN INVOLVES: a.comparing groups regarding differences in means or proportions, or b.Examining strength and direction of relationships between two variables Common Types of Research Hypotheses and the Related Statistical Data Analysis Methods: a.Checking for Presence/Absence of Relationship(s) Among Variables (and direction/strength of the relationship) Bivariate (e.g., Pearson Correlation—r) Between one variable and another: Y = a + b 1 x 1 Multivariate (e.g., Multiple Regression Analysis) Between one dep. var. and an independent variable, while holding all other independent variables constant: Y = a + b 1 x 1 + b 2 x 2 + b 3 x 3 + … + b k x k b.Checking for Presence/Absence of Difference(s) Among Groups Difference(s) in Proportions (Chi Square Test—  2 ) Difference(s) in Means (Analysis of Variance)

NOTE: Statistical tests of hypotheses always report the result of testing the null hypothesis. The researcher will then have to restate the results in terms of finding/not finding support for the original research hypothesis. 4 HYPOTHESIS TESTING If we reject the null, the conclusion is that we have found a statistically significant relationship/difference. – When we don’t reject the null, we infer that...? “…any difference/relationship that may be apparent from sample data is likely to be the result of...?? QUESTIO QUESTION: When testing hypotheses regarding presence of a relationship/ difference, what does “NULL HYPOTHESIS” (H 0 ) refer to? artifact of the particular sample sampling error (i.e., an artifact of the particular sample being used)”. Null Hypothesis in most cases states: “There is no relationship or no difference.”

5 A Quick Review of Important Issues About Sampling: So, when using sample data to test hypotheses and make judgments about the population, there is always a chance for reaching erroneous conclusions about the population. What do we mean by erroneous conclusions?What do we mean by erroneous conclusions? What types of erroneous conclusions can we reach when testing hypotheses?What types of erroneous conclusions can we reach when testing hypotheses?

6 Important Notes About Sampling Two possible types of erroneous conclusions from sampling error, when testing hypotheses: TYPE I ERROR and TYPE II ERROR

7 Important Notes About Sampling Type I Error? Rejecting a true “null hypothesis” (erroneously) Rejecting the null when we should not (i.e., when the null is true) Null Hypothesis? States “There is NO relationship, there is NO difference, etc.” “Rejecting the null” refers to concluding…? Concluding that: “There is a significant relationship/ difference”

So, type I error (“Rejecting a True Null”) means? No relationship/difference exists, but from sample evidence we come to the conclusion that a significant relationship/ difference does exist. Example: A drug is really not effective, but we conclude it is. Conclusion: Type I Error involves finding “something” that does not really exist—i.e., a case of “False positive” 8 Important Notes About Sampling Sample

9 Important Notes About Sampling Type II Error? Accepting a false null hypothesis (or failing to reject a false null hypothesis) False Null means? “A relationship/difference does in fact exist”. So, “accepting a false null” (i.e., type II error) means? A relationship/difference does in fact exist, but from sample evidence we fail to detect it (fail to reject the null). –Come to the conclusion that there is no relationship/difference (in the population). Example; A drug is really effective, but our study shows it is not.

10 Important Notes About Sampling Type II Error: CONCLUSION: Type II Error represents failing to find “something” that does exist; it represents a case of “False Negative.” Sample

11 A Quick Review of Important Issues About Sampling: –Statistical tests of significance assess the likelihood of reaching an erroneous conclusion when using sample data. In fact, they always assess the likelihood of type I error. They assess the probability that the relationship/ difference we have found (using sample data) may simply be an artifact of the particular sample we have happened to end up with (i.e., is caused by sampling error). –In fact, when using data from the entire population (e.g., a census): No chance of sampling error exists No need for conducting statistical tests of significance. When testing hypotheses, what is the purpose of statistical testing (significance testing)?

12 HYPOTHESIS TESTING Statistical tests of significance alway assess the likelihood/probability of type I error (  ) when using sample data. Once a test is conducted and  is determined, we will have to decide if we are able/willing to tolerate  the risk involved in rejecting the null (and, thereby, to report what…?) … that the relationship/difference detected (from sample data), is too large to be attributed to chance/sampling error.  That is, decide whether we should consider the relationship/difference “statistically significant.” Sampl e

13 Important Notes About Sampling The complement of  : 1-  or confidence level. What does 1-  represent? probability of accepting (not rejecting) the null when it is true: –concluding NO relationship/difference exists, when indeed it DOES NOT exist (i.e., chance of not finding what does not exist)—a correct conclusion The probability of committing Type I Error is called:  (alpha) or significance level.

14 HYPOTHESIS TESTING Use of statistical tests of significance is entirely consistent with John Locke’s view who, in his theory of knowledge, acknowledges that: – It is often difficult/impossible to establish knowledge/truth with absolute certainty and, thus, – Truth is often a function of the strength of the supporting reason and evidence. That is, plausibility of a proposition should be judged based on the weight of the supporting evidence and, thus, on the basis of: – How wise it would be to accept a proposition even though it has “only probability and not certainty” in its favor. As Locke stated: – The “degree of assent” that we give to a proposition should depend on the grounds of probability in its favor. John Locke ( )

15 HYPOTHESIS TESTING It requires us to : a)Determinethe likelihood of null being true (  riskwe would be takingto reject the null a)Determine the likelihood of null being true (  and, thus, the risk (of being wrong) that we would be taking if we decide to reject the null ( , and b)Decide whetherwilling/able to tolerate b)Decide whether we are willing/able to tolerate that risk (   level) by actually rejecting the null… and reporting that we have found a “significant” relationship/difference. So, generally speaking, when should we be tempted to reject the null? So, generally speaking, when should we be tempted to reject the null? When (  is ___large or when it is ___small? So, testing the plausibility of hypothesis/propositions (i.e., decision to reject/not reject H 0 ) is a probabilistic decision... ?  So, testing the plausibility of hypothesis/propositions (i.e., decision to reject/not reject H 0 ) is a probabilistic decision... ?

16 HYPOTHESIS TESTING A small  means that… CHANCENULL TRUETOO SMALL ACCEPTING…the CHANCE of NULL being TRUE is TOO SMALL to warrant ACCEPTING it. …if we decide to reject the null (i.e., conclude that we have found a relationship/difference), we stand a relatively small chance of being wrong. …rejecting the null is a relatively safe bet. …the difference/relationship found is statistically significant.NOTE: Small  rejecting the null finding a statistically significant relationship/difference reporting that the relationship/difference found (from sample evidence) is too large to be attributed to chance/ sampling error

17 HYPOTHESIS TESTING BUT HOW DO YOU MEASURE  ?  How would you determine what the actual  level is (i.e., how much risk of being wrong you would actually be taking if you were to decide to reject the null?  ANSWER… (a)Look up the actual  from a table of probability distribution for the test statistic being used, OR (b) More conveniently, rely on your statistical software (e.g., SPSS) to compute and report the actual  “Sig.” or “Prob.”) level for you.

18 HYPOTHESIS TESTING DECIDING ON AN   that I CAN TOLERATE!!! BUT, WAIT... How am I supposed to know what odds of being wrong I should be willing/able to tolerate as I consider rejecting the null?  A SIMPLE ANSWER:  5% is conventionally considered to be a reasonable/small enough risk to be tolerable in most situations.

19 HYPOTHESIS TESTING IS THERE A RULE OF THUMB TO FOLLOW WHEN TESTING HYPOTHESIS? YES! WHAT IS IT? THE GOLDEN RULE: THE GOLDEN RULE: When testing a hypothesis, if the reported  (e.g., “sig.” in SPSS) turns out to be less than or equal to 0.05, reject the null and report a statistically significant relationship/difference Because the odds of being wrong would be tolerable). Otherwise, refrain from rejecting the null, on the grounds that the odds of committing an error (i.e., rejecting a true null) would be prohibitive. And, as a result, report... ?

20 HYPOTHESIS TESTING EXCEPTIONS TO THE RULE? Use a smaller  (e.g.,  < 0.01 ) when: 1.Sample size is relatively large 2.Consequence of committing type I error is serious/costly (i.e., False positive results are very costly) (e.g., H 0 : Capital punishment is not a strong deterrent for criminal behavior.) Use a larger  (e.g.,  < 0.10 ) when: 1.Sample size is relatively small 2.Conducting exploratory research whose results provide the basis for further research

21 HYPOTHESIS TESTING QUESTIONS OR COMMENTS ?