回归预测 （Regression and Forecasting）

时间序列预测模型 Two important features: Uses historical data for the phenomenon we wish to forecast. We seek a routine calculation to apply to a large number of cases and that may be automated, without relying on qualitative information about the underlying phenomena. Two types of forecasting: Short Time and Long Term Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Three Components of Time Series Behavior Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

移动平均模型 The n-period moving average builds a forecast by averaging the observations in the most recent n periods: where xt represents the observation made in period t, and At denotes the moving average calculated after making the observation in period t. Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Convention We adopt the following convention for the steps in forecasting: Make the observation in period t Carry out the necessary calculations Use the calculations to forecast period (t + 1) Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Worksheet for Calculating Moving Averages Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

What Number of Periods to Include in Moving Average? There is no definitive answer, but there is a trade-off to consider. Suppose the mean of the underlying process remains stable: If we include very few data points, then the moving average exhibits more variability than if we include a larger number of data points. In that sense, we get more stability from including more points. Suppose there is an unanticipated change in the mean of the underlying process: If we include very few data points, our moving average will tend to track the changed process more closely than if we include a larger number of data points. In that case, we get more responsiveness from including fewer points. Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Moving-Average Calculations in a Stylized Example Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Comparison of 4-week and 6-week Moving Averages Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

MEASURES OF FORECAST ACCURACY MSE: the Mean Squared Error between forecast and actual MAD: the Mean Absolute Deviation between forecast and actual MAPE: the Mean Absolute Percent Error between forecast and actual Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Comparison of Measures of Forecast Accuracy The MAD calculation and the MAPE calculation are similar: one is absolute, the other is relative. MAPE is usually reserved for comparisons in which the magnitudes of two cases are different. Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

The Exponential Smoothing Model Exponential smoothing weighs recent observations more than older ones. Where α (the smoothing constant) is some number between zero and one. St is the smoothed value of the observations (our “best guess” as to the value of the mean) Our forecasting procedure sets the forecast Ft+1 = St. Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Comparison of Weights Placed on k-year-old Data Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Worksheet for Exponential Smoothing Calculations Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Comparison of Smoothed and Averaged Forecasts Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Trend Model Calculations with a Trend in the Data Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

HOLT’S METHOD This more flexible procedure uses two smoothing constants, as shown in the following formulas: Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Holt's Method with a Trend in the Data Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Forecasting With Linear Regression

Linear Regression Rectangular coordinate Two quantitative variables One variable is called independent (X) and the second is called dependent (Y) Points are not joined No frequency table

Example

Scatter diagram of weight and systolic blood pressure

Scatter diagram of weight and systolic blood pressure

Scatter plots The pattern of data is indicative of the type of relationship between your two variables: positive relationship negative relationship no relationship

Positive relationship

Negative relationship Reliability Age of Car

No relation

Correlation Coefficient Statistic showing the degree of relation between two variables

Simple Correlation coefficient (r) It is also called Pearson's correlation or product moment correlation coefficient. It measures the nature and strength between two variables of the quantitative type.

The sign of r denotes the nature of association while the value of r denotes the strength of association.

If the sign is +ve this means the relation is direct (an increase in one variable is associated with an increase in the other variable and a decrease in one variable is associated with a decrease in the other variable). While if the sign is -ve this means an inverse or indirect relationship (which means an increase in one variable is associated with a decrease in the other).

The value of r ranges between ( -1) and ( +1) The value of r denotes the strength of the association as illustrated by the following diagram. strong intermediate weak weak intermediate strong -1 -0.75 -0.25 0.25 0.75 1 indirect Direct perfect correlation perfect correlation no relation

If r = Zero this means no association or correlation between the two variables. If 0 < r < 0.25 = weak correlation. If 0.25 ≤ r < 0.75 = intermediate correlation. If 0.75 ≤ r < 1 = strong correlation. If r = l = perfect correlation.

How to compute the simple correlation coefficient (r)

Example: A sample of 6 children was selected, data about their age in years and weight in kilograms was recorded as shown in the following table . It is required to find the correlation between age and weight. Weight (Kg) Age (years) serial No 12 7 1 8 6 2 3 10 5 4 11 13 9

These 2 variables are of the quantitative type, one variable (Age) is called the independent and denoted as (X) variable and the other (weight) is called the dependent and denoted as (Y) variables to find the relation between age and weight compute the simple correlation coefficient using the following formula:

Y2 X2 xy Weight (Kg) (y) Age (years) (x) Serial n. 144 49 84 12 7 1 64 36 48 8 6 2 96 3 100 25 50 10 5 4 121 66 11 169 81 117 13 9 ∑y2= 742 ∑x2= 291 ∑xy= 461 ∑y= ∑x= 41 Total

r = 0.759 strong direct correlation

EXAMPLE: Relationship between Anxiety and Test Scores Test score (Y) X2 Y2 XY 10 2 100 4 20 8 3 64 9 24 81 18 1 7 49 5 6 25 36 30 ∑X = 32 ∑Y = 32 ∑X2 = 230 ∑Y2 = 204 ∑XY=129

Calculating Correlation Coefficient Indirect strong correlation

exercise

Regression Analyses Regression: technique concerned with predicting some variables by knowing others The process of predicting variable Y using variable X

Regression Uses a variable (x) to predict some outcome variable (y) Tells you how values in y change as a function of changes in values of x

Correlation and Regression Correlation describes the strength of a linear relationship between two variables Linear means “straight line” Regression tells us how to draw the straight line described by the correlation

Regression Calculates the “best-fit” line for a certain set of data The regression line makes the sum of the squares of the residuals smaller than for any other line Regression minimizes residuals

By using the least squares method (a procedure that minimizes the vertical deviations of plotted points surrounding a straight line) we are able to construct a best fitting straight line to the scatter diagram points and then formulate a regression equation in the form of: b

Regression Equation Regression equation describes the regression line mathematically Intercept Slope

Linear Equations 28

Hours studying and grades

Regressing grades on hours Predicted final grade in class = 59.95 + 3.17*(number of hours you study per week)

Predict the final grade of… Predicted final grade in class = 59.95 + 3.17*(hours of study) Predict the final grade of… Someone who studies for 12 hours Final grade = 59.95 + (3.17*12) Final grade = 97.99 Someone who studies for 1 hour: Final grade = 59.95 + (3.17*1) Final grade = 63.12

Exercise A sample of 6 persons was selected the value of their age ( x variable) and their weight is demonstrated in the following table. Find the regression equation and what is the predicted weight when age is 8.5 years.

Weight (y) Age (x) Serial no. 12 8 10 11 13 7 6 5 9 1 2 3 4

Answer Y2 X2 xy Weight (y) Age (x) Serial no. 144 64 100 121 169 49 36 25 81 84 48 96 50 66 117 12 8 10 11 13 7 6 5 9 1 2 3 4 742 291 461 41 Total

Regression equation

we create a regression line by plotting two estimated values for y against their X component, then extending the line right and left.

Exercise 2 B.P (y) Age (x) 128 136 146 124 143 130 121 126 123 46 53 60 20 63 43 26 19 31 23 120 141 134 132 140 144 58 70 The following are the age (in years) and systolic blood pressure of 20 apparently healthy adults.

Find the correlation between age and blood pressure using simple and Spearman's correlation coefficients, and comment. Find the regression equation? What is the predicted blood pressure for a man aging 25 years?

x2 xy y x Serial 400 2400 120 20 1 1849 5504 128 43 2 3969 8883 141 63 3 676 3276 126 26 4 2809 7102 134 53 5 961 3968 31 6 3364 7888 136 58 7 2116 6072 132 46 8 8120 140 9 4900 10080 144 70 10

x2 xy y x Serial 2116 5888 128 46 11 2809 7208 136 53 12 3600 8760 146 60 13 400 2480 124 20 14 3969 9009 143 63 15 1849 5590 130 43 16 676 3224 26 17 361 2299 121 19 18 961 3906 126 31 529 2829 123 23 41678 114486 2630 852 Total

= =112.13 + 0.4547 x for age 25 B.P = 112.13 + 0.4547 * 25=123.49 = 123.5 mm hg

Multiple Regression Multiple regression analysis is a straightforward extension of simple regression analysis which allows more than one independent variable.

Multiple Regression Model The equation that describes how the dependent variable y is related to the independent variables x1, x2, . . . xp and an error term is: y = b0 + b1x1 + b2x2 + . . . + bpxp + e where: b0, b1, b2, . . . , bp are the parameters, and e is a random variable called the error term

Multiple Regression Equation The equation that describes how the mean value of y is related to x1, x2, . . . xp is: E(y) = 0 + 1x1 + 2x2 + . . . + pxp

Estimated Multiple Regression Equation ^ y = b0 + b1x1 + b2x2 + . . . + bpxp A simple random sample is used to compute sample statistics b0, b1, b2, . . . , bp that are used as the point estimators of the parameters b0, b1, b2, . . . , bp.

Estimation Process b0, b1, b2, . . . , bp Sample statistics are Multiple Regression Model E(y) = 0 + 1x1 + 2x2 +. . .+ pxp + e Multiple Regression Equation E(y) = 0 + 1x1 + 2x2 +. . .+ pxp Unknown parameters are b0, b1, b2, . . . , bp Sample Data: x1 x2 . . . xp y . . . . Estimated Multiple Regression Equation Sample statistics are b0, b1, b2, . . . , bp b0, b1, b2, . . . , bp provide estimates of

Least Squares Method Least Squares Criterion Computation of Coefficient Values The formulas for the regression coefficients b0, b1, b2, . . . bp involve the use of matrix algebra. We will rely on computer software packages to perform the calculations.

Multiple Regression Model Example: Programmer Salary Survey A software firm collected data for a sample of 20 computer programmers. A suggestion was made that regression analysis could be used to determine if salary was related to the years of experience and the score on the firm’s programmer aptitude test. The years of experience, score on the aptitude test, and corresponding annual salary ($1000s) for a sample of 20 programmers is shown on the next slide.

Multiple Regression Model Exper. Score Salary Exper. Score Salary 4 7 1 5 8 10 6 78 100 86 82 84 75 80 83 91 24.0 43.0 23.7 34.3 35.8 38.0 22.2 23.1 30.0 33.0 9 2 10 5 6 8 4 3 88 73 75 81 74 87 79 94 70 89 38.0 26.6 36.2 31.6 29.0 34.0 30.1 33.9 28.2 30.0

Multiple Regression Model Suppose we believe that salary (y) is related to the years of experience (x1) and the score on the programmer aptitude test (x2) by the following regression model: y = 0 + 1x1 + 2x2 +  where y = annual salary ($1000) x1 = years of experience x2 = score on programmer aptitude test

Solving for the Estimates of 0, 1, 2 Least Squares Output Input Data x1 x2 y 4 78 24 7 100 43 . . . 3 89 30 Computer Package for Solving Multiple Regression Problems b0 = b1 = b2 = R2 = etc.

Solving for the Estimates of 0, 1, 2 Excel’s Regression Equation Output Note: Columns F-I are not shown.

Estimated Regression Equation SALARY = 3.174 + 1.404(EXPER) + 0.251(SCORE) Note: Predicted salary will be in thousands of dollars.

Interpreting the Coefficients In multiple regression analysis, we interpret each regression coefficient as follows: bi represents an estimate of the change in y corresponding to a 1-unit increase in xi when all other independent variables are held constant.

Interpreting the Coefficients b1 = 1.404 Salary is expected to increase by $1,404 for each additional year of experience (when the variable score on programmer attitude test is held constant).

Interpreting the Coefficients b2 = 0.251 Salary is expected to increase by $251 for each additional point scored on the programmer aptitude test (when the variable years of experience is held constant).

Logistic Regression In many ways logistic regression is like ordinary regression. It requires a dependent variable, y, and one or more independent variables. Logistic regression can be used to model situations in which the dependent variable, y, may only assume two discrete values, such as 0 and 1. The ordinary multiple regression model is not applicable.

Logistic Regression Logistic Regression Equation The relationship between E(y) and x1, x2, . . . , xp is better described by the following nonlinear equation.

Logistic Regression Interpretation of E(y) as a Probability in Logistic Regression If the two values of y are coded as 0 or 1, the value of E(y) provides the probability that y = 1 given a particular set of values for x1, x2, . . . , xp.

Logistic Regression Estimated Logistic Regression Equation A simple random sample is used to compute sample statistics b0, b1, b2, . . . , bp that are used as the point estimators of the parameters b0, b1, b2, . . . , bp.

Logistic Regression Example: Simmons Stores Simmons’ catalogs are expensive and Simmons would like to send them to only those customers who have the highest probability of making a $200 purchase using the discount coupon included in the catalog. Simmons’ management thinks that annual spending at Simmons Stores and whether a customer has a Simmons credit card are two variables that might be helpful in predicting whether a customer who receives the catalog will use the coupon to make a $200 purchase.

Logistic Regression Example: Simmons Stores Simmons conducted a study by sending out 100 catalogs, 50 to customers who have a Simmons credit card and 50 to customers who do not have the card. At the end of the test period, Simmons noted for each of the 100 customers: 1) the amount the customer spent last year at Simmons, 2) whether the customer had a Simmons credit card, and 3) whether the customer made a $200 purchase. A portion of the test data is shown on the next slide.

Logistic Regression x1 x2 Simmons Test Data (partial) y Annual Spending ($1000) 2.291 3.215 2.135 3.924 2.528 2.473 2.384 7.076 1.182 3.345 Simmons Credit Card 1 $200 Purchase 1 Customer 1 2 3 4 5 6 7 8 9 10

Logistic Regression Simmons Logistic Regression Table (using Minitab) Predictor Coef SE Coef Z p Odds Ratio 95% CI Lower Upper Constant Spending Card -2.1464 0.3416 1.0987 0.5772 0.1287 0.4447 -3.72 2.66 2.47 0.000 0.008 0.013 1.41 3.00 1.09 1.25 1.81 7.17 Log-Likelihood = -60.487 Test that all slopes are zero: G = 13.628, DF = 2, P-Value = 0.001

Logistic Regression Simmons Estimated Logistic Regression Equation

Logistic Regression Using the Estimated Logistic Regression Equation For customers that spend $2000 annually and do not have a Simmons credit card: For customers that spend $2000 annually and do have a Simmons credit card: