Download presentation
Presentation is loading. Please wait.
Published byArron Payne Modified over 6 years ago
1
Association between quantitative variables: Correlation and regression
2
Correlation Examining relationships between two variables
paired by time paired by person or team paired by region or country Leads on to regression
3
Is Third World Aid a Good Thing?
Argument for: ( Infant mortality Govt health expenditure Paired by country
4
Is Third World Aid a Good Thing?
Argument against: Aid GDP Time
5
Is Third World Aid a Good Thing?
Paired by year
6
Is Third World Aid a Good Thing?
How good a relationship between Government Health Expenditure & Infant Mortality rate does there have to be to convince you that they really are linked?
7
Key Questions: When does a relationship start to suggest that there really is an association, rather than it just being a coincidence? Can we measure how strong the association is? How do we interpret the results
8
Relationships between 2 variables
First draw a scatter graph. Each point represents an x value and a y value. y x
9
Relationships between 2 variables
Perfect positive correlation can draw a straight line through all points, with a positive slope. As x increases, so does y – a positive relationship y x
10
Relationships between 2 variables
Perfect negative correlation can draw a straight line through all points, with a negative slope. As x increases, y decreases – a negative relationship y x
11
Relationships between 2 variables
No correlation an increase or decrease in x doesn't appear to have any effect on the value of y. y x
12
Relationships between 2 variables
Non linear relationship a bit more complicated than the straight lines we saw earlier. y x
13
Relationships between 2 variables
Some correlation but how much? As x increases, then y tends to increase, but not always.. y x
14
Outliers Unusual value Investigate and find out the reason
May need to remove from data y x outlier
15
Important note about x and y
x is the explanatory (or independent) variable y is the response (or dependent) variable Important to get these the right way round Ways of thinking about this: y is the variable we are trying to predict x is the variable we can control (sometimes) we think that x might cause y (but see later: the data can’t prove this)
16
Describing scatter plots
Direction: does the trend go up or down? Curvature: is the pattern linear or curved? Variation: are the points tightly clustered around the trend? Outliers: is there something unexpected?
17
Measuring association
We would like to measure the amount of association Measure using covariance and correlation
18
Example A company has varied promotional marketing expenditure over the last 6 months to see the effect on sales. x variable is marketing, y variable is sales Data is paired by month (assume marketing has the effect in the same month, e.g. direct promotions)
19
Scatter chart
20
Covariance = Σ (xi – x)(yi – y) Like the variance: n – 1
= Σ (xi – x)(xi – x) n – 1 As for the standard deviation and variance, population version divides by n rather than n – 1
21
Covariance x below mean, x above mean, y above mean, y above mean,
so covariance –ve x above mean, y above mean, so covariance +ve x below mean, y below mean, so covariance +ve x above mean, y below mean, so covariance –ve
22
Covariance for our data
Covariance = 15.2 (units are £0002) But: If we measure our data in £ rather than £000 then: covariance = 15,200,000 (units are £2) Issues: Can’t tell whether number indicates strong or weak association Value depends on the units
23
Correlation Usually denoted r
Often called Pearson’s correlation coefficient Dividing by the standard deviations takes away the effects of the size of the numbers and the number of points – i.e. it makes r scale invariant Correlation doesn’t have any units
24
Interpretation of r – 1 r 1 (ALWAYS!!)
r = 1 means perfect positive correlation r = – 1 means perfect negative correlation r = 0 means no correlation closer to 1 or – 1 means stronger correlation if we calculate r greater than 1 or smaller than –1 , there is a mistake!
25
Correlations (Stine and Foster, Statistics for Business p113)
26
Scatter chart For our data, r = 0.946
27
Critical values (CV) for a hypothesis test on correlation at significance level α – there is evidence of correlation at significance level α if the absolute value of r is greater than CV n CV (α=0.05) CV (α=0.01) 4 .950 .999 5 .878 .959 6 .811 .917 7 .754 .875 8 .707 .834 9 .666 .798 10 .632 .765 11 .602 .735 12 .576 .708 13 .553 .684 14 .532 .661 15 .514 .641 16 .497 .623 17 .482 .606 18 .468 .590 19 .456 .575 n CV (α=0.05) CV (α=0.01) 20 .444 .561 25 .396 .505 30 .361 .463 35 .335 .430 40 .312 .402 45 .294 .378 50 .279 60 .254 .330 70 .236 .305 80 .220 .286 90 .207 .269 100 .196 .256
28
Hypothesis testing If the absolute value of r is greater than the critical value then it is “statistically significant” (at significance level α) α = 0.05 is commonly used but really just an arbitrary cut-off level just means that the likelihood of this result occurring by chance is 5% (i.e. fairly unlikely) i.e. if there is really no relationship the chance of getting this result is 5% (1 in 20) Therefore “statistically significant” just means reasonably strong evidence of a pattern in the data sometimes occurs when there really isn’t a relationship sometimes doesn’t occur when there really is a relationship
29
Hypothesis testing - lottery
n = 13, correlation = –0.836, CV (0.05) = 0.553, CV (0.01) = Significant at the 1% level! Does the Saturday draw have some influence on the draw on the next Wednesday?
30
Hypothesis testing If we do lots of studies and analyse them separately then there is very likely to be one that is “statistically significant” just by chance lottery example: 47 quarters, 3 are statistically significant at the 5% level Need to combine studies together lottery example: 630 pairs, r = 0.025, CV (5%) = 0.078, not statistically significant Hence: one small scientific study with a “statistically significant” result doesn’t prove much Also “statistically significant” does not mean causal relationship (see next slide) Note: media generally doesn’t understand these points (and nor do some scientists!!)
31
Where do babies come from?
Old European folk stories explain that babies are delivered to new parents by storks
32
Where do babies come from?
High correlation does NOT imply cause and effect NOT
33
Other examples of correlations
Predict top authors by hand size? Hand size and writing ability CO2 and football ticket prices Taking medicine and being ill Ice cream and deaths from drowning More CO2 makes people prepared to pay more? Taking medicine causes illness? Ban ice cream?
34
High correlation High correlation (positive or negative) just means there is an interesting pattern in the data. Possible reasons are: changes in x cause changes in y changes in y cause changes in x changes in z cause changes in both x and y where z is a 3rd factor apparent correlation is just caused by chance particularly if not much data can test for statistical significance
35
Regression
36
Modelling We have our data – probably a sample representing a much larger population We've drawn a scatter diagram We've calculated the correlation coefficient What do we do now? want a model estimate of what happens more generally can apply model to other hypothetical situations Assuming we believe there is a causal relationship
37
Regression model A linear regression model is a mathematical equation of a straight line involving y and x Formula is y = a + bx Not exact: usually points won't all lie exactly on a straight line We need to choose the line of best fit where a and b are constants
38
Equation of a straight line
y = a + bx (where a and b are constants) a is the intercept on the y axis b is the gradient or slope (negative if the line slopes downwards) y H b = H / L L a x
39
Line of best fit Assumes: x is explanatory variable, y is response y x
40
Line of best fit y We are trying to predict the y values. We will measure how well the line fits using the error distance in the y direction x Regression minimises the sum of the squared differences
41
Regression line “Least squares” regression line: minimises Σ(y – yL)2 where y is the y value of the data point and yL is the y value of the line Squaring the differences stops the positive and negative values cancelling out Squaring the distances penalise large distances from the line otherwise tends to follow greatest concentration of points rather than going through the middle i.e. corresponds to what a well fitting line should look like Important: putting x and y the other way round alters the equation of the regression line
42
Previous example
43
Regression in Excel
44
Plot of regression line
y = x
45
Plot of regression line
y = x Predict the sales if we spend £4500 on marketing Note: this assumes a causal relationship
46
Comments Line looks sensible
always plot to check it looks o.k. We can only be reasonably confident in the regression line within the range of the data here: between £3k and £7k advertising If we believe there is a causal relationship then we can make predictions: if we spend £4,500 on marketing we will get sales of £74,200 Data cannot prove causation
47
Comments Interpretation of constants:
intercept, a = 40: if the line applies down to x = 0, this is the sales if no advertising slope, b = 7.6: extra sales for each unit of advertising
48
Extrapolation y x
49
The regression model How "good" is the model?
r2 (where r = correlation coefficient) gives an indication of how closely the straight line fits the points r2 measures the proportion of variation in the data that can be explained by the regression equation but it's not the only possible measure of the model adjusted r2 hypothesis tests
50
Correlation & regression - summary
Draw scatter graph of data x value is explanatory variable; y response variable look at the pattern of the data look for outliers and decide whether to discard them Relationship is strong if: r is close to 1 or -1 (r2 close to 1) AND many data points can do a hypothesis test Correlation does not necessarily imply cause and effect
51
Correlation & regression - summary
Regression line gives the line of “best” fit minimises squared errors in y Can use regression line for prediction as long as confident there is a relationship dangerous to extrapolate regression line beyond range of data don’t use regression line after circumstances change
52
Further regression topics
Multiple regression Transforming variables to deal with non-linear variables Choosing the best model Hypothesis testing
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.