Chapter 4 Hypothesis Testing, Power, and Control: A Review of the Basics

From Question to Hypothesis Finding the TRUTH starts with asking a question that comes from Finding the TRUTH starts with asking a question that comes from –Curiosity –Necessity –Past Research As scientists we PREDICT the answer from As scientists we PREDICT the answer from –Theory –Past Research –Common Sense That prediction is EDUCATED not random That prediction is EDUCATED not random –An educated prediction is a HYPOTHESIS To ANSWER the question we TEST the HYPOTHESIS To ANSWER the question we TEST the HYPOTHESIS

Conceptual hypotheses Conceptual hypotheses –State expected relationships among concepts. Research hypotheses Research hypotheses –Concepts are operationalized so that they are measurable. Statistical hypotheses Statistical hypotheses –State the expected relationship between or among summary values of populations, called parameters.  Null hypothesis (H 0 )  Alternative hypothesis (H 1 ) Three levels of hypotheses

Example Question – What is the role of neurotransmitters in memory? Question – What is the role of neurotransmitters in memory? Conceptual – Increasing certain neurotransmitter will increase memory Conceptual – Increasing certain neurotransmitter will increase memory Research – Smoking 1 crack rock before testing will increase performance on a standard test of memory compared to placebo control Research – Smoking 1 crack rock before testing will increase performance on a standard test of memory compared to placebo control Statistical – H O : M T = M C H A : M T >M C Statistical – H O : M T = M C H A : M T >M C

Testing the null hypothesis Null hypothesis Null hypothesis –The hypothesis being statistically tested when you use inferential statistics. –The researcher hopes to show that the null is not likely to be true (i.e.. hopes to nullify it). Alternative hypothesis Alternative hypothesis –The hypothesis the researcher postulated at the outset of the study. –If the researcher can show that the null is not supported by the data, then he or she is able to accept the alternative hypothesis.

Testing the null hypothesis Steps in testing a research hypothesis: Steps in testing a research hypothesis: 1.State the null and the alternative. 2.Collect the data and conduct the appropriate statistical analysis. 3.Reject the null and accept the alternative or fail to reject the null. 4.State your inferential conclusion.

Statistical significance Statistical difference Statistical difference –The probability that the groups are the same is very low. Significance levels ( α ) Significance levels ( α ) –Alpha ( α ) is the level of significance chosen by the researcher to evaluate the null hypothesis. –5% or 1%

Inferential Errors: Type I and Type II Type I Error Type I Error –Rejecting a true null. –Probability is equal to alpha ( α ). Type II Error Type II Error –Failing to reject a false null. –Probability is beta ( β ). Power – our ability to reject false nulls. Power – our ability to reject false nulls.

Inferential Errors: Type I and Type II True State of Affairs Null is true Null is false Reject the null Type I error ( α ) Correct inference (power) Fail to reject the null Correct inference Type II error ( β ) Our decision

Why Power is Important A powerful test of the null is more likely to lead us to reject false nulls than a less powerful test. A powerful test of the null is more likely to lead us to reject false nulls than a less powerful test. Powerful tests are more sensitive than less powerful tests to differences between the actual outcome (what you found) and the expected outcome (null hypothesis). Powerful tests are more sensitive than less powerful tests to differences between the actual outcome (what you found) and the expected outcome (null hypothesis). Power, or the probability of rejecting a false null, is 1 – β. Power, or the probability of rejecting a false null, is 1 – β.

Power and How to Increase it How one measures variables How one measures variables –Interval or ratio scales are better  In testing the effects of alcohol intoxication on aggression… –Intoxication – BAC better than # of drinks –Aggression – Level of shock (1-10) as opposed to shock or no shock

Power and How to Increase it Use more powerful statistical analyses Use more powerful statistical analyses Parametric vs. Nonparametric Parametric vs. Nonparametric ANOVA vs. Chi-Square ANOVA vs. Chi-Square

Power and How to Increase it Use designs that provide good control over extraneous variables. Use designs that provide good control over extraneous variables. Remove unintended variation Remove unintended variation Experimental vs. Correlational Designs Experimental vs. Correlational Designs Laboratory vs. Field Laboratory vs. Field

Power and How to Increase it Restrict your sample to a specific group of individuals. Restrict your sample to a specific group of individuals. Use selection procedures to reduce nuisance variables Use selection procedures to reduce nuisance variables

Power and How to Increase it Increase your sample size  reduces error variance Increase your sample size  reduces error variance

Power and How to Increase it Maximize treatment manipulation Maximize treatment manipulation Precision Precision Separation Separation

Effect size – a measure of the strength of the relationship between/among variables. Effect size – a measure of the strength of the relationship between/among variables. Effect size helps us determine if differences are not only statistically significant, but also whether they are important. Effect size helps us determine if differences are not only statistically significant, but also whether they are important. Powerful tests should be considered to be tests that detect large effects. Powerful tests should be considered to be tests that detect large effects. Effect size

Ways to calculate effect size: Ways to calculate effect size: –Cohen’s d – use with t-tests. –Coefficient of determination (r 2 ) – use with correlations. –eta-squared ( η 2 ) – use with ANOVAs. –Cramer’s v – use with Chi-square analyses.

Power and the role of replication in research Power increases when we replicate findings in a new study with different participants in a different setting. Power increases when we replicate findings in a new study with different participants in a different setting.

External and internal validity External validity External validity –When the findings of a study can be generalized to other populations and settings. Internal validity Internal validity –Refers to the validity of the measures within the study. –The internal validity of an experiment is directly related to the researcher’s control of extraneous variables.

Confounding and extraneous variables Extraneous variable Extraneous variable –A variable that may affect the outcome of a study but was not manipulated by the researcher. Confounding variable Confounding variable –A variable that is systematically related to the independent and dependent variable. Spurious effect Spurious effect –An outcome that was influenced not by the independent variable itself but rather by a variable that was confounded with the independent variable.

Confounding and extraneous variables Controlled variable Controlled variable –A variable that the researcher takes into account when designing the research study or experiment. Nuisance variables Nuisance variables –Variables that contribute variance to our dependent measures and cloud the results.

Controlling extraneous variables Elimination Elimination –Get rid of the extraneous variables completely (e.g.. by conducting research in a lab). Constancy Constancy –Keep the various parts of the experiment constant (e.g.. instructions, measuring instruments, questions). Secondary variable as an IV Secondary variable as an IV –Make variables other than the primary IV secondary variables to study (e.g.. gender).

Controlling extraneous variables Randomization: Random assignment of participants to groups Randomization: Random assignment of participants to groups –Randomly assigning participants to each of the treatment conditions so that we can assume the groups are initially equivalent. Repeated measures Repeated measures –Use the same participants in all conditions. Statistical control Statistical control –Treat the extraneous variable as a covariate and use statistical procedures to remove it from the analysis.