i.e. How to get an A on the big project I’m about to assign you… Regression Notes i.e. How to get an A on the big project I’m about to assign you…

Bivariate (2-variable) Statistics PHASE I. Scatter-Plots and Bivariate (2-variable) Statistics

One Variable Descriptors: Two-Variable Descriptors: Describing Variables One Variable Descriptors: SHAPE CENTER SPREAD anything unusual? Two-Variable Descriptors: LINEAR? DIRECTION SCATTER anything unusual?

What do you see in these scatter plots? Mean January Air Temperatures for 30 U.S. Locations 20 19 18 Temperature (°C) 17 16 LINEAR TREND NEGATIVE ASSOCIATION 15 CONSTANT SCATTER 14 ANYTHING UNUSUAL? 35 40 45 Latitude (°N)   

What do you see in these scatter plots? 10 20 30 40 GDP per capita (thousands of dollars) 50 60 70 80 Internet Users (%) % of population who are Internet Users vs GDP per capita for 202 Countries NON-LINEAR TREND POSITIVE ASSOCIATION NON-CONSTANT SCATTER and an OUTLIER!!!

What do you see in these scatter plots? Year 1990 1980 1970 1960 1950 1940 1930 30 28 26 24 22 20 Age Average Age Americans are First Married 2 SEPARATE, NON-LINEAR TRENDS …gap in data in 1940s? NEGATIVE ASSOCIATION TIL ~1970, THEN POSITIVE NO SCATTER

What to look for in scatter plots Trend Linear or non-linear?

What to look for in scatter plots Trend Positive or negative association?

What to look for in scatter plots 2. Scatter Strong or weak relationship?

What to look for in scatter plots 2. Scatter Constant or non-constant scatter?

What to look for in scatter plots 3. Anything unusual Outlier

What to look for in scatter plots 3. Anything unusual Groupings

Rank relationships: weakest (1) to strongest (4) 4 2 1 3

Correlation Coefficient r

Correlation Coefficient little r – what is it? r is the correlation coefficient between y and x r measures the strength of a linear relationship r is a multiple of the slope

Only use r if the scatter plot is linear r – when can it be used? Only use r if the scatter plot is linear Don’t use r if the scatter plot is non-linear! x y * r = 0.99

How close the points in the scatter plot come to lying on the line r – what does it tell you? How close the points in the scatter plot come to lying on the line r = 0.99 x y * r = 0.57 Difficult Ones

Playing with Outliers (1)… Playing with Outliers (1)… What will happen to the correlation coefficient if we remove the tallest 12th grader? bigger or smaller Hint: …correlation measures how linear the data is ----------------- and an OUTLIER!!! LINEAR TREND POSITIVE ASSOCIATION MOSTLY CONSTANT SCATTER See for yourself HERE

Playing with Outliers (2)… Playing with Outliers (2)… What will happen to the correlation coefficient if we remove the elephant? bigger or smaller Hint: …make your brain zoom in on that main cluster of points ----------------- and an OUTLIER!!! LINEAR TREND POSITIVE ASSOCIATION CONSTANT SCATTER See for yourself HERE

Guess which are correlated with Test Scores? 4 are… 4 aren’t… Highly educated parents Mom’s age >30 at birth Mom stays home until Kindergarten Intact family (live with mom and dad) Attended Head Start program Parents have money Move to a better neighborhood Low birthweight (including premature)

Guess which are correlated with Test Scores? 3 are… 4 aren’t… Parents speak English Family goes to museums, zoos, concerts… Parents involved in PTA Child spanked regularly Watches TV a lot Parents own a lot of books Child is read to every day

Life Expectancy Example Life Expectancy and Availability of Doctors for a Sample of 40 Countries 80 Can you suggest how to increase life expectancy in a country? - Non-linear trend - Negative Association - Fairly Constant Scatter 70 Life Expectancy Get fewer people per doctor! Duh! 60 50 10000 20000 30000 40000 People per Doctor

Life Expectancy Example Life Expectancy and Availability of Televisions for a Sample of 40 Countries 80 Can you suggest how to increase life expectancy in a country? 70 Get fewer people per TV?!? Life Expectancy BEWARE LURKING VARIABLES!!! 60 50 100 200 300 400 500 600 People per Television

Kinds of Lurking Variables (1) CAUSATION “People who take showers have better organizational skills.” perceived correlation Organized y Shower x Maybe changes in x CAUSE changes in y

Kinds of Lurking Variables (2) CAUSATION again... but in reverse “People who take showers have better organizational skills.” perceived correlation Organized y Shower X Maybe changes in y CAUSE changes in x

Kinds of Lurking Variables (3) COMMON RESPONSE “People who take showers have better organizational skills.” perceived correlation Organized y Shower x Good Habits in General z Maybe something else z is causing changes in both X and Y at the same time!

Kinds of Lurking Variables (4) CONFOUNDING …we don’t know which variable is causing the changes “People who take showers have better organizational skills.” perceived correlation Organized y Shower x Good Habits in General z

LURKING VARIABLES Heh, Heh, Heh…

(Clearly children can hold their drink better than adults) How Regression gets you in Trouble… Famous examples of strong correlations: Instances of drunkenness in those below 18 years of age are significantly lower than for those above. (Clearly children can hold their drink better than adults)

(eating ice cream makes you tastier?) How Regression gets you in Trouble… Famous examples of strong correlations: Whenever ice cream sales rise, so do the number of shark attacks. (eating ice cream makes you tastier?)

(learning words make you hungry?) How Regression gets you in Trouble… Famous examples of strong correlations: As vocabulary in infancy rises, so does appetite. (learning words make you hungry?)

(firetrucks cause damage?) How Regression gets you in Trouble… Famous examples of strong correlations: The more fire trucks you send to a fire, the worse the damage is. (firetrucks cause damage?)

The more you pay teachers in a town, the more expensive alcohol is. How Regression gets you in Trouble… Famous examples of strong correlations: The more you pay teachers in a town, the more expensive alcohol is.

How Regression gets you in Trouble… How Regression gets you in Trouble… Famous examples of strong correlations: In Scandinavia, storks appear more often on the rooftops of families with more babies.

Deer and cattle, orient themselves along a north/south axis when grazing.

Correlation is not Causation How Regression gets you in Trouble… Famous examples of strong correlations: Correlation is not Causation

The story: The smoking ban in Wales "caused" a 13% fall in heart attacks from October to December 2007, compared with the same period in 2006. The flaw: The ban began in April. We also observed a 13% fall in heart attacks in April. And presumably it "caused" me to spill my coffee. For that happened during the smoking ban too.

!! TRADE UNIONS SECURE BETTER PAY !! See? Union membership can get you as much as 30% more pay!! 39

!! TRADE UNIONS SECURE BETTER PAY !! perceived correlation Union Membership x Better Pay! y Education level of employee z Experience Level of employee z Age of employee z 40

Gapminder 41

Least Square Regression Lines (LSRL) PHASE II. Residuals and Least Square Regression Lines (LSRL)

Residuals = Actual – Predicted Residuals = Actual – Predicted 4 = prediction line y = 5 + 2x The actual point is (8, 25) (8, 25) 25 The predicted point is (8, 21) 21 (8, 21) 17 4 6 8 10 12 Residuals = Actual – Predicted 4 = 25 – 21

“Actual” – “Predicted” 7 -3 17 1 -10 -4

Σ (Resids)2 = 439.2988 We’ll try to get the Least Squares Least Squares Regression: We’ll try to get the Least Squares Σ (Resids)2 = 439.2988

Least Squares Regression Line Facts: There is one and only one LSRL for every set of bivariate data. Σ Residuals = 0 (just like with st.dev) Your calculator will give you the one equation with the “least” amount of squares… “Least Squares Regression Line” (LSRL) The LSRL must go through the point You’ll only have to calculate the LSRL by hand once (…heh, heh)

PHASE III. Your Project

Example of analysis: “Going Crackers” An example of the type of work you’ll be doing for your REGRESSION ASSIGNMENT You start with some raw data… 1. Predict the energy content of a cracker with 25% fat content. 2. If you reduced the salt content by 100g, how would the fat change?

Example of analysis: “Going Crackers” 1. Predict the energy content of a cracker with 25% fat content. ENERGY FAT (ENERGY) = 380 + 4.98 (FAT) = 380 + 4.98 (25) = 504.5

Example of analysis: “Going Crackers” 2. If you reduced the salt content by 100g, how would the fat change? FAT SALT (FAT) = - 2.7 + 0.0237 (SALT) The fat content would drop by 0.0237 mg.

Problem 2 Analysis The data suggest a linear trend. The association is positive with constant scatter about the trend line. It is reasonable to do a linear regression.

Problem 2 Analysis The LSRL is y = -2.7 + 0.0237x. The slope of the fitted line is 0.0237 which tells us, on average, each 100mg decrease in salt content is associated with a decrease in total fat content by 2.4% The moderate relationship (r = 0.69) means that predicting such a drop will not necessarily be highly accurate.