Scientific Methodology: The Hypothetico-Deductive Approach, the Test of Hypothesis, and Null Hypotheses BIOL January 2016

Null Hypothesis Prediction that no cause and effect will be evident in the data Prediction that no cause and effect will be evident in the data On the contrary, research hypothesis generally predicts there will be cause and effect On the contrary, research hypothesis generally predicts there will be cause and effect Specific to the conditions of the test—hence part of inductive reasoning Specific to the conditions of the test—hence part of inductive reasoning

Types of Data Meristic Meristic Mensural Mensural Variation in meristic and mensural data may be… Variation in meristic and mensural data may be… …discrete (few possible numerical values), or …discrete (few possible numerical values), or …continuous (many possible numerical values) …continuous (many possible numerical values) Categorical Categorical Meristic or mensural data may be made categorical (“bin sorting”); opposite transformation often not possible Meristic or mensural data may be made categorical (“bin sorting”); opposite transformation often not possible

Null Hypothesis First basic format: No difference: There is no difference in DV between or among groups or DV between or among groups or treatments treatments IV (groups) varies discretely Data commonly presented in bar graph bar graph Esp. common in controlled experiments experiments

Null Hypothesis Second basic format: No correlation: There is no correlation of DV with IV of DV with IV IV varies continuously Data commonly presented in X-Y scatterplot X-Y scatterplot Esp. common in data collection and analysis and analysis

Examples from Articles

Conclusions Regarding Null Hypotheses Reject H 0 : Find it to be false Reject H 0 : Find it to be false A difference or correlation does exist A difference or correlation does exist Thus, cause and effect exists Thus, cause and effect exists Do not reject H 0 : Do not find it to be false Do not reject H 0 : Do not find it to be false No difference or correlation is shown to exist No difference or correlation is shown to exist No cause and effect is established No cause and effect is established “Accept” implies a false sense of certainty “Accept” implies a false sense of certainty

Conclusions Regarding the Research Hypothesis Supported (data are consistent with predictions of the hypothesis) predictions of the hypothesis) Not supported (data are not consistent with predictions of the hypothesis; predictions of the hypothesis; examine other hypotheses) examine other hypotheses)

The Discussion Section of an Article Typically begins with summary of analyses Typically begins with summary of analyses Leads to conclusion re research hypothesis Leads to conclusion re research hypothesis Other interesting points are discussed Other interesting points are discussed Incongruent, ambiguous results Incongruent, ambiguous results Unexpected trends in data Unexpected trends in data Comparison to results of other studies Comparison to results of other studies New, as-yet untested hypotheses are offered New, as-yet untested hypotheses are offered Suggestions for further studies Suggestions for further studies

The Cyclical Nature of Science Fig. 1-4, p. 6

Examples from Articles

Sampling

Sampling Entire population of all possible data points cannot ever be collected Entire population of all possible data points cannot ever be collected Sampling is meant to represent the entire population, and generally will, if… Sampling is meant to represent the entire population, and generally will, if… …large enough …large enough Deviation from true means or true X-Y relationships lessen as N increases Deviation from true means or true X-Y relationships lessen as N increases …taken randomly …taken randomly Means every data point has an equal opportunity of being included in the sample Means every data point has an equal opportunity of being included in the sample …independent data points …independent data points

Variance and Standard Deviation of a Sample of Meristic or Mensural Data V = SD 2 V = SD 2 SD = √V SD = √V V = [  (x i  x) 2 ]/(N  1) V = [  (x i  x) 2 ]/(N  1) = sum of squared differences between the mean and each data point, divided by one less than sample size Not quite correct to say SD is average difference between mean and data points (but nearly so) Not quite correct to say SD is average difference between mean and data points (but nearly so) ____ __

Variance and Standard Deviation Text, p. 34

Statistical Inference

Using Probability Goal of any statistical test is to calculate p Goal of any statistical test is to calculate p p is the probability of getting sample data at least as different/correlated as yours, if… p is the probability of getting sample data at least as different/correlated as yours, if… …H 0 were actually true. …H 0 were actually true. …IV did not influence DV. …IV did not influence DV. …no cause and effect existed. …no cause and effect existed. p < 0.05: Reject H 0 as (probably) false p < 0.05: Reject H 0 as (probably) false p > 0.05: Do not reject H 0 ; it may be true p > 0.05: Do not reject H 0 ; it may be true

Calculating p Choose test and format of H 0, based on… Choose test and format of H 0, based on… …types of data for IV and DV …types of data for IV and DV …whether data are parametric or nonparametric …whether data are parametric or nonparametric Plug data into standard formulae for the test Plug data into standard formulae for the test Crunch numbers Crunch numbers Calculate test statistic Calculate test statistic Compare test statistic to standard values, based on degrees of freedom and desired level of confidence (  generally 0.05 or less) Compare test statistic to standard values, based on degrees of freedom and desired level of confidence (  generally 0.05 or less)

Calculating p If calculated test statistic > critical value to which it is compared, reject H 0, with p critical value to which it is compared, reject H 0, with p <  If calculated test statistic  If calculated test statistic 

Degrees of Freedom Can depend on… Can depend on… …sample size, N …sample size, N The smaller N is, the larger the test statistic has to be to exceed critical value, allowing rejection of H 0 The smaller N is, the larger the test statistic has to be to exceed critical value, allowing rejection of H 0 …number of comparisons being made …number of comparisons being made …number of independent variables …number of independent variables

Statistical Confidence If p < 0.05, reject H 0 with 95% confidence If p < 0.05, reject H 0 with 95% confidence Meaning: Were H 0 true, we would stand less than a 5% chance of getting data that would be so different/so correlated Meaning: Were H 0 true, we would stand less than a 5% chance of getting data that would be so different/so correlated Inference: More likely, H 0 was not true, so that is why the data that we have collected are so different/so correlated Inference: More likely, H 0 was not true, so that is why the data that we have collected are so different/so correlated

Statistical Confidence %Confidence + p(100) = 100 %Confidence = 100(1  p) p < 0.02—Reject H 0 with 98% confidence p < 0.01—Reject H 0 with 99% confidence p < 0.001—Reject H 0 with 99.9% confidence p < —Reject H 0 with 99.99% confidence p = 0.029—Reject H 0 with 97.1% confidence p = —Reject H 0 with 99.15% confidence

Statistical Confidence p = 0.128—Do not reject H 0 —to do so implies mere 87.2% confidence, not enough* to rule out chance as having influenced the data *Because we agreed, a priori (and somewhat arbitrarily), that we needed a minimum level of 95% confidence to reject H 0

Type I Error aka “False Positive” aka “False Positive” = Rejection of H 0, even though it was true = Rejection of H 0, even though it was true No cause and effect, but we are falsely led to believe there is cause and effect No cause and effect, but we are falsely led to believe there is cause and effect Occurs in 5% of all tests of true null hypotheses, if standard critical value (  ) of 0.05 is used—i.e.,  is the chance of a false positive Occurs in 5% of all tests of true null hypotheses, if standard critical value (  ) of 0.05 is used—i.e.,  is the chance of a false positive

Type II Error aka “False Negative” aka “False Negative” = Failure to reject H 0 even though it was untrue = Failure to reject H 0 even though it was untrue Cause and effect is missed by researcher Cause and effect is missed by researcher Statistical Power: N required to be able to identify cause and effect—i.e., correctly reject H 0 —with specified level of probability (  ) Statistical Power: N required to be able to identify cause and effect—i.e., correctly reject H 0 —with specified level of probability (  )

N, , and  N does not influence chance of false positive N does not influence chance of false positive Small N is taken into account via degrees of freedom, by increasing critical value of test statistic Small N is taken into account via degrees of freedom, by increasing critical value of test statistic N is related to chance of a false negative—the smaller N is, the greater this chance N is related to chance of a false negative—the smaller N is, the greater this chance Power is low when N is low Power is low when N is low