Testing Hypotheses

Basic Research Designs Descriptive Designs: Descriptive Studies: thoroughly describe a single variable in order to better understand it Correlational Studies: examine the relationships between two or more quantitative variables as they exist with no effort to manipulate them Inferential Designs: Quasi-Experimental Studies: make comparisons between naturally-occurring groups of individuals Experimental Studies: make comparisons between actively manipulated groups

Chain of Reasoning in Inferential Statistics Population With Parameters Sample With Statistics Random Selection Inference Sampling Distributions Of the Statistics Probability

Inferential Reasoning Population: group under investigation Sample: a smaller group representing the population A sample that has been randomly selected should be representative of the population Random Selection Inference

Hypothesis Testing Hypothesis Testing: the process of using inferential procedures to determine whether a hypothesis is supported by the results of a research study

Hypothesis Testing Conceptual Hypothesis: a general statement about the relationship between the independent and dependent variables Statistical Hypothesis: a mathematical statement that can be shown to be supported or not supported. It is designed to make inferences about a population or populations.

Hypothesis Testing In psychological research, no hypotheses can be proven to be true. Modus Tollens: a procedure of falsification that relies on the fact that a single observation can lead to the conclusion that the premise or prior statement is incorrect Null Hypothesis (H0): statements of equality (no relationship; no difference); statements of opposing difference Alternative (Research) Hypothesis (H1 or HA): a statement that there is a relationship or difference between levels of a variable; statements of inequality

Types of Research Hypotheses Nondirectional Research Hypothesis: reflects a difference between groups, but the direction of the difference is not specified (two-tailed test) H1: X ≠ Y Directional Research Hypothesis: reflects a difference between groups, and the direction of the difference is specified (one-tailed test) H1: X > Y H1: X < Y z = -1.96 µ z = 1.96 p = .025 p = .025 µ z = 1.645 p = .05

Rejecting the Null Hypothesis Alpha Level (α): the level of significance set by the researcher. It is the confidence with which the researcher can decide to reject the null hypothesis. Significance Level (p): the probability value used to conclude that the null hypothesis is an incorrect statement If p > α  cannot reject the null hypothesis If p ≤ α  reject the null hypothesis

Determining the Alpha Level Type I Error (α): the researcher rejects the null hypothesis when in fact it is true; stating that an effect exists when it really does not Type II Error (β): the researcher fails to reject a null hypothesis that should be rejected; failing to detect a treatment effect Type 1 error An error made by rejecting the null hypothesis even though it is really true; stating that an effect exists when it really does not. The odds of making a type 1 error are equal to the significance level we choose e.g. .05  5 times out of 100 we will reject the null when we shouldn’t Type 2 error An error made by failing to reject the null hypothesis even though it is really false; failing to detect a treatment effect. The more extreme the significance level, the more likely we are to make a type 2 error Probability of making a type 2 error is affected by the amount of overlap between the populations being sampled  the more overlap there is, the harder it will be to detect the effect of the IV Represented by the Greek letter beta 1-β = power of the statistical test (the odds of correctly rejecting the null hypothesis when it is false) Can reduce β by: Increasing sample size Reducing the variability between data sets Using more powerful statistical tests Accept a less extreme significance level When we make a type 1 error, we explain an effect that does not really exist, and this can often be a more serious error than failing to detect an effect (like putting an innocent person in jail) APA has suggested that statistical hypothesis testing should be supported by other techniques e.g. effect size estimates and confidence intervals

Determining the Significance Level (Probability) The distribution used to determine the probability of a specific score (or difference between scores) is determined by multiple factors. Regardless of the distribution used, the logic and process used to determine probability is essentially the same. All statistical distributions mimic the function of the standard normal distribution.

The Normal Curve Three Main Characteristics: Symmetrical: perfectly symmetrical about the mean; the two halves are identical Mean = Median = Mode Asymptotic Tail: the tails come closer and closer to the horizontal axis, but they never touch

The Normal Distribution and the Standard Deviation In the normal distribution… 68% of scores fall between +/-1 standard deviations 95% of scores fall between +/-2 standard deviations 99.7% of scores fall between +/- 3 standard deviations It is possible to determine the probability of obtaining any given score (or any differences between scores). The Normal Distribution and the Standard Deviation Understanding the relationship between the standard deviation and the normal distribution is crucial to understanding much of the information on standardized tests Another important property of the normal distribution is that the mean = median = mode The mean is at the center of the distribution, with 50% of scores above the mean and 50% below In the normal distribution, 68% of all scores will fall between one standard deviation above the mean and one standard deviation below the mean ; 95% between –2 and +2; and 99.7% between –3 and +3 Note: Not all assessments are normally distributed The normal distribution and the standard deviation also let us know at what percentile individual scores fall (e.g. scores at one standard deviation above the mean would be at the 84th percentile, with 84% of people scoring below that and 16% scoring above) Percentile score – represents the percent of people you do better than

The Normal Curve and Probability The normal distribution is the most commonly used distribution in behavioral science research. The scores of variables can be converted to standard z-scores, which can be used to determine the probability of a specific score. All probabilities are a number between 0.0 and 1.0, and given all possible outcomes of an event, the probabilities must equal 1.0. µ z = 1.645 µ z = 1.645

z-scores z-score: represents the distance between an observed score and the mean relative to the standard deviation; a score on an assessment expressed in standard deviation units Formula: z = X – M s z = X – µ σ

More Curves and Probability µ z = 2.326 p = .01 µ z = 1.282 p = .10 z = -1.645 µ p = .05 µ z = 1.645 p = .05