1. Lecture 16 Correlation and Coefficient of CorrelationBy
Aziza Munir

2. Learning Objectives What is Correlation What does it indicates
What is the purpose of correlation if regression is already there/
What does coefficient of Correlation Indicates
Linear, multiple and Partial correlation

3. Introduction
Correlation a LINEAR association between two random variables
Correlation analysis show us how to determine both the nature and strength of relationship between two variables
When variables are dependent on time correlation is applied
Correlation lies between +1 to -1

4. A zero correlation indicates that there is no relationship between the variables
A correlation of –1 indicates a perfect negative correlation
A correlation of +1 indicates a perfect positive correlation

5. Types of Correlation Types Type 1 Type 2 Type 3
There are three types of correlation
Types
Type 1
Type 2
Type 3

6. Type1 Positive Negative No Perfect
If two related variables are such that when one increases (decreases), the other also increases (decreases).
If two variables are such that when one increases (decreases), the other decreases (increases)
If both the variables are independent

7. Type 2 Linear Non – linear
When plotted on a graph it tends to be a perfect line
When plotted on a graph it is not a straight line

9. Type 3 Simple Multiple Partial
Two independent and one dependent variable
One dependent and more than one independent variables
One dependent variable and more than one independent variable but only one independent variable is considered and other independent variables are considered constant

10. Methods of Studying Correlation
Scatter Diagram Method
Karl Pearson Coefficient Correlation of Method
Spearman’s Rank Correlation Method

11. Correlation: Linear Relationships Strong relationship = good linear fit
Very good fit
Moderate fit
Points clustered closely around a line show a strong correlation. The line is a good predictor (good fit) with the data. The more spread out the points, the weaker the correlation, and the less good the fit. The line is a REGRESSSION line (Y = bX + a)

12. Coefficient of Correlation
A measure of the strength of the linear relationship between two variables that is defined in terms of the (sample) covariance of the variables divided by their (sample) standard deviations
Represented by “r”
r lies between +1 to -1
Magnitude and Direction

13. -1 < r < +1
The + and – signs are used for positive linear correlations and negative linear correlations, respectively

14. Shared variability of X and Y variables on the top
Individual variability of X and Y variables on the bottom

15. Interpreting Correlation Coefficient r
strong correlation: r > .70 or r < –.70
moderate correlation: r is between .30 & .70 or r is between –.30 and –.70
weak correlation: r is between 0 and .30 or r is between 0 and –.30 .

16. Coefficient of Determination
Coefficient of determination lies between 0 to 1
Represented by r2
The coefficient of determination is a measure of how well the regression line represents the data
If the regression line passes exactly through every point on the scatter plot, it would be able to explain all of the variation
The further the line is away from the points, the less it is able to explain

17. r 2, is useful because it gives the proportion of the variance (fluctuation) of one variable that is predictable from the other variable
It is a measure that allows us to determine how certain one can be in making predictions from a certain model/graph 
The coefficient of determination is the ratio of the explained variation to the total variation
The coefficient of determination is such that 0 <  r 2 < 1,  and denotes the strength of the linear association between x and y   

18. The Coefficient of determination represents the percent of the data that is the closest to the line of best fit
For example, if r = 0.922, then r 2 = 0.850
Which means that 85% of the total variation in y can be explained by the linear relationship between x and y (as described by the regression equation)
The other 15% of the total variation in y remains unexplained

19. Spearmans rank coefficient
A method to determine correlation when the data is not available in numerical form and as an alternative the method, the method of rank correlation is used. Thus when the values of the two variables are converted to their ranks, and there from the correlation is obtained, the correlations known as rank correlation.

20. Computation of Rank Correlation
Spearman’s rank correlation coefficient ρ can be calculated when
Actual ranks given
Ranks are not given but grades are given but not repeated
Ranks are not given and grades are given and repeated

21. Testing significance of correlation
Test for the significance of relationships between two CONTINUOUS variables
We introduced Pearson correlation as a measure of the STRENGTH of a relationship between two variables
But any relationship should be assessed for its SIGNIFICANCE as well as its strength.
A general discussion of significance tests for relationships between two continuous variables.
Factors in relationships between two variables
The strength of the relationship:
is indicated by the correlation coefficient: r
but is actually measured by the coefficient of determination: r2
The significance of the relationship
is expressed in probability levels: p (e.g., significant at p =.05)
This tells how unlikely a given correlation coefficient, r, will occur given no relationship in the population
NOTE! NOTE! NOTE! The smaller the p-level, the more significant the relationship
BUT! BUT! BUT! The larger the correlation, the stronger the relationship

22. Consider the classical model for testing significance
It assumes that you have a sample of cases from a population
The question is whether your observed statistic for the sample is likely to be observed given some assumption of the corresponding population parameter.
If your observed statistic does not exactly match the population parameter, perhaps the difference is due to sampling error
The fundamental question: is the difference between what you observe and what you expect given the assumption of the population large enough to be significant -- to reject the assumption?
The greater the difference -- the more the sample statistic deviates from the population parameter -- the more significant it is
That is, the lessl ikely (small probability values) that the population assumption is true.
 The classical model makes some assumptions about the population parameter:
Population parameters are expressed as Greek letters, while corresponding sample statistics are expressed in lower-case Roman letters:
r = correlation between two variables in the sample
(rho) = correlation between the same two variables in the population
A common assumption is that there is NO relationship between X and Y in the population: = 0.0
Under this common null hypothesis in correlational analysis: r = 0.0 

23. Testing for the significance of the correlation coefficient, r
When the test is against the null hypothesis: r xy = 0.0
What is the likelihood of drawing a sample with r xy ­ 0.0?
The sampling distribution of r is
approximately normal (but bounded at -1.0 and +1.0) when N is large
and distributes t when N is small.
The simplest formula for computing the appropriate t value to test significance of a correlation coefficient employs the t distribution:
 The degrees of freedom for entering the t-distribution is N - 2
Example: Suppose you obsserve that r= .50 between literacy rate and political stability in 10 nations
Is this relationship "strong"?
Coefficient of determination = r-squared = .25
Means that 25% of variance in political stability is "explained" by literacy rate
Is the relationship "significant"?
That remains to be determined using the formula above r = .50 and N=10

24. Comments set level of significance (assume .05)
determine one-or two-tailed test (aim for one-tailed)
For 8 df and one-tailed test, critical value of t = 1.86
We observe only t = 1.63
It lies below the critical t of 1.86
So the null hypothesis of no relationship in the population (r = 0) cannot be rejected  
Comments
Note that a relationship can be strong and yet not significant
Conversely, a relationship can be weak but significant
The key factor is the size of the sample.
For small samples, it is easy to produce a strong correlation by chance and one must pay attention to signficance to keep from jumping to conclusions: i.e.,
rejecting a true null hypothesis,
which meansmaking a Type I error.
For large samples, it is easy to achieve significance, and one must pay attention to the strength of the correlation to determine if the relationship explains very much

25. Correlation summary
Most common form (Pearson) used with two continuous variables, in a linear association
Spearman used with curvilinear associations
Point-biserial used whenever an independent samples t-test can be used
Phi used when a chi square for goodness of fit (with just 2 levels/variable) can be used
Can vary between -1 and +1
Does not tell anything about causation

26. Difference between Correlation and Regression
Correlation Coefficient, R, measures the strength of bivariate association   
The regression line is a prediction equation that estimates the values of y for any given x

27. Back to the idea of prediction
With correlation, you can predict the value of one variable based on the value of another variable
If you know someone’s marital problems, you can predict that person’s level of satisfaction
But, if you knew more about that person you could do an even better job predicting satisfaction
 regression: used to predict one quantitative variable from a whole mess of quantitative variables

28. Building up to regression
First, the equation for a line?
Y = bX + a
AKA: Y = mX + b
In both, have intercept and slope
Intercept = predicted value of Y when X is zero
Slope = how much Y is predicted to change as X changes
Goal of regression line: Minimize the discrepancy between predicted and actual values of Y

29. Linking this to correlation
Correlation = slope of the regression line, if the scores are in z-scores
 predicted z score for Y variable = correlation value * z-score for X variable

30. Difference between regression and correlation
Correlation is a special case of regression, with just one predictor variable
Regression lets you add in more predictor variables to:
Figure out how much of the Y variable is explained by a whole mess of predictor variables
Figure out how much each predictor variable uniquely tells about the Y variable
 two tests for significance – for whole model, and for each individual variable

31. Limitations of the correlation coefficient
Though R measures how closely the two variables approximate a straight line, it does not validly measures the strength of nonlinear relationship 
When the sample size, n, is small we also have to be careful with the reliability of the correlation
Outliers could have a marked effect on R
Causal Linear Relationship

32. Conclusion Correlation and regression Types of correlations
Coefficient of correlation and its interpretation
Difference between regression and correlation