1. Simple Correlation Scatterplots & r –scatterplots for continuous - binary relationships –H0: & RH: –Non-linearity Interpreting r Outcomes vs. RH: –Supporting vs. “contrary” results Outcomes vs. Population –Correct vs. Error results Differences between correlations in a group Difference between correlations in different groups

2. A scatterplot a graphical depiction of the relationship between two quantitative (or binary) variables each participant’s x & y values depicted as a point in x-y space Pearson’s correlation coefficient (r value) summarizes the direction and strength of the linear relationship between two quantitative variables into a single number (range from -1.00 to 1.00) you should always examine the scatterplot before considering the correlation between two variable NHST can be applied to test if the correlation in the sample is sufficiently large to reject H0: of no linear relationship between the variables in the population A linear regression formula allows us to take advantage of this relationship to estimate or predict the value of one variable (the criterion) from the other (the predictor). prediction should only be applied if the relationship between the variables is “linear” and “substantial”

3. Example of a “scatterplot” Age of Puppy (weeks) Amount Puppy Eats (pounds) 543210543210 4 8 12 16 20 24 Puppy Age (x) Eats (y) Sam Ding Ralf Pit Seff … Toby 8 20 12 4 24.. 16 2 4 2 1 4.. 3

4. When examining a scatterplot, we look for three things... relationship no relationship linear non-linear direction (if linear) positive negative strength strong moderate weak linear, positive, weak linear, negative, moderate No relationship Hi Lo linear, positive, strong Hi Lo nonlinear, strong Hi Lo

5. We can use correlation to examine the relationship between a quantitative predictor variable and a quantitative criterion variable. Y X A positive r tells us those higher X values tend to have higher Y values A negative r tells us those with lower X values tend to have higher Y values A nonsignificant r tells us there is no linear relationship between X & Y Y Y Y Y Y strong +weak ++1.00 strong -weak -.00 X X X X X

6. We can also use correlation to examine the relationship between a binary predictor variable and a quantitative criterion variable. Y grp 1 grp 2 A positive r tells us the group with the higher X code as the higher mean Y A negative r tells us the group with the lower X code as the higher mean Y A nonsignificant r tells us the groups have “equivalent” means on Y Y Y Y Y Y strong +weak ++1.00 strong -weak -.00

7. For each of the following show the envelope for the H0: and the RH: People who have more depression before therapy will be those who have more depression after therapy. Instructor isn’t related to practice. Depression before Depression after H0: RH: Study # Errors 0 1 Instructor Practice Those who study more have fewer errors on the spelling test H0: RH: H0: RH:

8. For each of the following show the envelope for the H0: and the RH: People who score better on the pretest will be those who tend to score worse on the posttest I predict that snapping turtles (coded 1) will eat more crickets than painted turtles (coded 0). Pretest Post-test H0: RH: # Sessions Depression 0 1 Species # Crickets You can’t predict depression from the number of therapy sessions H0: RH: H0: RH:

9. The Pearson’s correlation ( r ) summarizes the direction and strength of the linear relationship shown in the scatterplot r has a range from -1.00 to 1.00 1.00 a perfect positive linear relationship 0.00 no linear relationship at all -1.00 a perfect negative linear relationship r assumes that the relationship is linear if the relationship is not linear, then the r-value is an underestimate of the strength of the relationship at best and meaningless at worst For a non-linear relationship, r will be based on a “rounded out” envelope -- leading to a misrepresentative r

10. Extreme Non-linear relationship r value is “misinformative” actual scatterplot notice... there is an x-y relationship Scatterplot as correlation “sees it” regression line has 0 slope & r = 0 -- no linear relationship

11. Moderate Non-linear relationship r value is an underestimate of the strength of the nonlinear relationship actual scatterplot notice... there is an x-y relationship Scatterplot as correlation “sees it” regression line has non-0 slope & r ~= 0 but, the regression line not a great representation of the bivariate relationship

12. NHST testing with r H0: r = 0.00 is the same as r 2 = 0.00 get used to working with both r (the correlation between the 2 vars ) and r 2 (the “variance shared between the 2 vars)” Performing the significance test software will usually provide an exact p-value (use p <.05) a general formula is … r² N = sample size F = ------------------------ (1 - r²) / (N - 2) Find F-critical using df = 1 & N-2

13. What “retaining H0:” and “Rejecting H0:” means... When you retain H0: you’re concluding… is not –The linear relationship between these variables in the sample is not strong enough to allow me to conclude there is a linear relationship between them in the population represented by the sample. When you reject H0: you’re concluding… is –The linear relationship between these variables in the sample is strong enough to allow me to conclude there is a linear relationship between them in the population represented by the sample.

14. The p-value (value range 1.0 – 0) tells the probability of making a Type I error if you reject the H0: based on the sample data e.g., p =.10 means “if we reject H0: based on these data there is a 10% chance that there really is no relationship between the variables in the population represented by the sample” The usual “acceptable risk” is less than 5% or p <.05 r (range -1.0 – 1.0) tells strength and direction of the bivariate relationship between Y & X “large enough to be interesting” value vary across research areas, but a common guideline is.10 = small,.30 = medium and.50 = large r 2 (range 0 – 1.0) tells how much of the Y variability is “accounted for,” “predicted from” or “caused by” X e.g., r=.30 means that.30 2 (9%) of the Y variability is accounted for by X “large enough to be interesting” will vary across research areas, but a common guideline is 1% = small, 10% = medium and 25% = large effect significance vs. effect size vs. shared variance

15. Interpret each of the following (significance, strength & direction) For age & social skills: r =.25, p =.043. For practice and performance errors: r = -.52, p =.015 For age and performance: r = -.33, p =.231 For gender (m=1, f=2) and social skills: r =.14, p =.004 For gender (m=1, f=2) and performance: r = -.31, p =.029 For gender (m=1, f=2) and practice: r =.11, p =.098 Sig – medium – positive  Older adolescents tend to have higher social skills scores Sig – large – negative  Those who practiced more tended to have fewer errors Nonsig – medium? - negative ?  There is no linear relationship between age and performance Sig – small – positive  Females had higher mean on social skills scores Sig – medium – negative  Males had higher mean performance Nonsig – small? – positive?  No mean practice difference between males & females

16. Statistical Conclusion Errors In the population there are only three possibilities... In the Population -r r = 0 +r … and three possible statistical decisions Type I error Type II error Correctly retained H0: Correctly rejected H0: Type III error Please note that this is a different question than whether the results “match” the RH: This is about whether the results from the sample are “correct” – whether the results are “represent the population. This is about statistical conclusion validity Outcomes -r r = 0 +r

17. The 9 outcomes come in 5 types … Type I error -- “false alarm” - finding a significant mean difference between the conditions in the study when there really isn’t a difference between the populations Type II error -- “miss” - finding no difference between the conditions of the study when there really is a difference between the populations Type III error -- “misspecification” - finding a difference between the conditions of the study that is different from the the difference between the populations Correctly retained H0: -- finding no difference between the conditions of the study when there really is no difference between the populations Correctly rejected H0: -- finding a difference between the conditions of the study that is the same as the the difference between the populations

18. Practice with statistical decision errors... Type II Correct rejection Type III Type I Correct retention We found that students who did more homework problems tended to have higher exam scores, which is what the other studies have found. We found that students who did more homework problems tended to have lower exam scores. All other studies found the opposite effect. We found that students who did more homework problems and those who did fewer problems tended to have about the same exam scores, which is what the other studies have found. We found that students who did more homework problems tended to have lower exam scores. Ours is the only study with this finding, others find no relationship. We found that students who did more homework problems and those who did fewer problems tended to have about the same exam scores. Everybody else has found that homework helps. We found that students who did more homework problems tended to have lower exam scores. Ours is the only study with this finding. Can’t tell -- what DID the other studies find?

19. correlation RH: vs. outcomes Research Hypotheses -r r = 0 +r … and three possible statistical outcomes Outcomes -r r = 0 +r So, there are only 9 possible combinations of RH: & Outcomes … of 4 types “effect as expected” “unexpected null” “unexpected effect” “backward effect” ? There are only three possible Research Hypotheses Results contrary to RH:

20. Keep in mind that rejecting H0: does not guarantee support for the research hypothesis? Why not ??? The direction of the r might be opposite that of the RH: The RH: might be that’s there’s no correlation (RH: = H0:) ? Remember !!! Our purpose is not to “Reject the H0:” … nor even to “support our RH:” … Our real purpose is for our results to represent the relationship between the constructs in the target population !!!!!

21. A quick focus on the two that are most often confused … Type III Statistical Decision Error –When our significant findings have a direction or pattern different from that found in the population –A difference between “the effect we found” and “the effect we should have found” “Results contrary to our RH:” –When our findings have a direction or pattern different from what we had hypothesized –A difference between “the effect we found” and “the effect we hypothesized” A result can be BOTH!!!!! (Or neither, or one, or the other !!!)

22. RH:, statistical conclusions & statistical decision errors... RH: + direction/pattern H0: - direction/pattern Statistical Decision + direction/pattern (p <.05) H0: (p >.05) - direction/pattern (p <.05 Correct rejection, Type I or Type III Correct retention or Type II RH: supported Unexpected H0: Correct rejection, Type I or Type III Correct retention or Type II Unexpected effect “backward” Results

23. Lets practice … Our RH: was that there will be a negative correlation between performance on the GRE and cumulative GPA. We found r =.47, p =.016. These results are “contrary to our RH:” -- a significant relationship in the opposite direction from the RH: The consistent results of these other studies suggests that our finding was a correct rejection – what we found “does describe the relationship between these variables in the population”. A literature review revealed 105 other studies involving these two variables, each of which found a correlation between.43 and.61 (all p <.05). Our RH: was incorrect, not supported, but our results were right!!!

24. Another … Our RH: was that there will be a positive correlation between the severity of depression at the beginning of therapy and the amount of improvement a patient shows during the first six weeks of therapy. We found r =.27, p =.085. These results are “contrary to our RH:” -- a nonsignificant relationship isn’t the RH: +r The consistent findings of these other studies suggests that our finding was a Type II error – what we found “doesn’t describe the likely relationship between these variables in the population”. The 14 studies of these two variables which followed ours each found a correlation between -.33 and -.41 (all p <.05). Our RH: was incorrect, not supported & our results were wrong!!!

25. Try this one … Our RH: was that there will be a positive correlation between social skills and comfort in an unfamiliar social situation. We found r (82) =.37, p =.016. These results “support our RH:” - a significant relationship in the RH: direction The consistent results of these other studies suggests that our finding was a Type I error – what we found “does not describe the relationship between these variables in the population”. Our RH: was incorrect but supported & our results were wrong !!! A literature review revealed 22 other studies involving these two variables, each of which found a correlation between -.13 and.11 (all p >.05)

26. Last one … Our RH: is that there will be a positive correlation between how much a person likes to compliment people and the number of close friends a person reports. We found r (58) =.30, p <.05. These results “support our RH:” -- a significant, positive relationship, as hypothesized Our finding was consistent with earlier research! A literature review revealed 8 other studies of these two variables, each of which found a correlation between.25 and.32 (all p <.05). The “researchers Trifecta” RH: is correct & supported and the results are correct 1!!! Keep in mind … There are 27 combinations of RH: (+ 0 -), Results (+ 0 -) and Population value (+ 0 -). “Success” depends more on a consistent agreement of the last two than of the first two!

27. Comparing “Correlated Correlations” Are two correlations (that share a variable) different, within a single population/group? Common versions of this question include… “Which is better correlated with performance, practice or prior skill?” “Which is better correlated with practice, performance or confidence?” H0: The correlation between practice and performance is the same as between prior skill and performance. H0: r perf, prac = r perf, pskill Tested using Hotelling’s t-test, Steiger’s Z-test, or one of several variations…

28. Comparing Correlations across Populations/Groups Is a correlation different in one pop/group than in another? Common versions of this question include… “Is performance better correlated with practice for novices or for experts?” ““Is pracice better correlated with confidence for novices or for experts?” H0: The correlation between practice and performance is the same for novice and for experienced participants H0: r perf, prac for novices = r perf, pract for experts Tested using Fisher’s Z-test

29. Identify the kinds of “correlation question” for each … Is age a better predictor of social skills for boys than girls? Is age a better predictor of social skills than SES? Does age predict social skills? Does IQ predict school performance? Does IQ predict school performance better than SES? Does SES predict IQ better for children or adults? “Is a correlation different across two populations?” “Are two correlations different, within a single population?” “Is a correlation different across two populations?” “Are two correlations different, within a single population?” “Are two variables correlated, within a specific population?”

30. 1st moment of caution when comparing correlations! You have to decide if you are going to compare …. the “correlations” of the two predictors (including sign + or -) or the “strength”, r 2, |r|, or “predictive utility” of the two predictors (ignoring the sign) For example: r(98) =.35 for # correct and confidence ratings r(98) = -.25 for # correct and time to complete the task (r = -.45 for confidence and time to complete) Comparing.35 & -.25 yields Z = 3.55, p <.01  different r Comparing.35 &.25 yields Z =.63, p >.05  same r 2 Notice that these questions are equivalent if the signs of the two correlations are the same!

31. 2nd moment of caution when comparing correlations! Don’t confuse asking… if each variable is significantly correlated with the criterion vs. if the variables are differentially correlated with the criterion Example… r(28) =.37, p <.05 for # correct and time to complete the task r(28) =.33, p >.05 for confidence and time to complete the task Although # correct is significantly correlated with time to complete the task and confidence is not significantly correlated with time to complete, it is a different question to ask if the two correlations are significantly different! Said differently  There may not be a significant difference between a significant correlation and a non-significant correlation.

32. An important variation of comparing correlations… While it is most common to apply these models to ask which of two variables is the better predictor of a given criterion… … it is possible to apply them to ask for which criterion a given variable is the better predictor. Often we collect multiple variables that are considered “outcome” or criterion variables. If so, when we talk about how good a predictor is, it is important to know if the effectiveness of the predictor depends upon the criterion we are using. Remember – like in the other applications of these models … it is different to say that a predictor is correlated with one criterion and not correlated with another, than to say it is differentially correlated with the two! it is different to ask if two correlations are significantly different than to ask if the two r 2 or |r| are different