Forecasting Based on supervised learning/classification

Forecasting Based on supervised learning/classification

Probability in Trees When the system is asked by whether Bob is late today, what will be the answer?

Forecasting on Time Series data Based on supervised learning/classification

What to Predict Behavior of the Inhabitants
Location Tasks / goals Actions Behavior of the Environment Device behavior (e.g. heating, AC) Interactions Visit course website

Example: Location Prediction
Where will Bob go next? Locationt+1 = f(x) Input data x: Locationt, Locationt-1, … Time, date, day of the week Sensor data

Example: Location Prediction
Time Date Day Locationt Locationt+1 6:30 02/25 Monday Bedroom Bathroom 7:00 Kitchen 7:30 Garage 17:30 18:00 18:10 Living room 22:00 22:10 02/26 Tuesday

Time series sequence Suppose: Bedroom -> 1 Bathroom -> 2
Kitchen -> 3 Garage -> 4 Living room -> 5 The given time series dataset is: Or:

Prediction Techniques
Classification-Based Approaches Nearest Neighbor Neural Networks Bayesian Classifiers Decision Trees Sequential Behavior Modeling Hidden Markov Models Temporal Belief Networks

Markov Model For a series of finite states:{Xn, n = 0, 1, … n}
If we assume: Then the time series chain:{Xn, n = 0, 1, … n} is called a Markov chain. ,…,

Forecasting based on Training data
Forecasting usually involves time (or date), e.g.:

Forecasting based on Training data
From the given dataset, we learn that: For any neighboring two days: Sunny -> Rainy : 1/2 Rainy -> Sunny: 1/3 Suppose there are only two types of weathers: Sunny or Rainy.

Simple Questions Suppose 2016.5.1 is a Sunny day.
What is the probability that is a rainy day? What is the probability that is a sunny day?

Hidden Markov Model (HMM)

一个东京的朋友每天根据天气{下雨，天晴}决定当天的活动{公园散步,购物,清理房间}中的一种，我每天只能在twitter上看到她发的推“啊，我前天公园散步、昨天购物、今天清理房间了！”那么我可以根据她发的推特推断东京这三天的天气

Question: the most probable weathers
We know the twitter sequence: (W, S, C) What is the most probable weather sequence?

Forecasting on Time Series Datasets using Regression

Forecasting with Time-Series Models
Two important features: Uses historical data for the phenomenon we wish to forecast. We seek a routine calculation to apply to a large number of cases and that may be automated, without relying on qualitative information about the underlying phenomena. Two types of forecasting: Short Time and Long Term Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Three Components of Time Series Behavior
Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

The Moving-Average Model
The n-period moving average builds a forecast by averaging the observations in the most recent n periods: where xt represents the observation made in period t, and At denotes the moving average calculated after making the observation in period t. Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Convention We adopt the following convention for the steps in forecasting: Make the observation in period t Carry out the necessary calculations Use the calculations to forecast period (t + 1) Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Worksheet for Calculating Moving Averages

What Number of Periods to Include in Moving Average?
There is no definitive answer, but there is a trade-off to consider. Suppose the mean of the underlying process remains stable: If we include very few data points, then the moving average exhibits more variability than if we include a larger number of data points. In that sense, we get more stability from including more points. Suppose there is an unanticipated change in the mean of the underlying process: If we include very few data points, our moving average will tend to track the changed process more closely than if we include a larger number of data points. In that case, we get more responsiveness from including fewer points. Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Moving-Average Calculations in a Stylized Example

Comparison of 4-week and 6-week Moving Averages

MEASURES OF FORECAST ACCURACY
MSE: the Mean Squared Error between forecast and actual MAD: the Mean Absolute Deviation between forecast and actual MAPE: the Mean Absolute Percent Error between forecast and actual Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Comparison of Measures of Forecast Accuracy
The MAD calculation and the MAPE calculation are similar: one is absolute, the other is relative. MAPE is usually reserved for comparisons in which the magnitudes of two cases are different. Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

The Exponential Smoothing Model
Exponential smoothing weighs recent observations more than older ones. Where α (the smoothing constant) is some number between zero and one. St is the smoothed value of the observations (our “best guess” as to the value of the mean) Our forecasting procedure sets the forecast Ft+1 = St. Chapter 7 Copyright © 2013 John Wiley & Sons, Inc.

Exponential Smoothing Calculations in a Stylized Example

Comparison of Smoothed and Averaged Forecasts

Trend Model Calculations with a Trend in the Data

Holt's Method with a Trend in the Data

Forecasting With Regression Methods

Linear Regression Rectangular coordinate Two quantitative variables
One variable is called independent (X) and the second is called dependent (Y) Points are not joined No frequency table

Example

Scatter diagram of weight and systolic blood pressure

Scatter plots The pattern of data is indicative of the type of relationship between your two variables: positive relationship negative relationship no relationship

Positive relationship

Negative relationship
Reliability Age of Car

No relation

Correlation Coefficient
Statistic showing the degree of relation between two variables

Simple Correlation coefficient (r)
It is also called Pearson's correlation or product moment correlation coefficient. It measures the nature and strength between two variables of the quantitative type.

The sign of r denotes the nature of association
while the value of r denotes the strength of association.

If the sign is +ve this means the relation is direct (an increase in one variable is associated with an increase in the other variable and a decrease in one variable is associated with a decrease in the other variable). While if the sign is -ve this means an inverse or indirect relationship (which means an increase in one variable is associated with a decrease in the other).

The value of r ranges between ( -1) and ( +1)
The value of r denotes the strength of the association as illustrated by the following diagram. strong intermediate weak weak intermediate strong -1 -0.75 -0.25 0.25 0.75 1 indirect Direct perfect correlation perfect correlation no relation

If r = Zero this means no association or correlation between the two variables.
If 0 < r < 0.25 = weak correlation. If 0.25 ≤ r < 0.75 = intermediate correlation. If 0.75 ≤ r < 1 = strong correlation. If r = l = perfect correlation.

How to compute the simple correlation coefficient (r)

Example: A sample of 6 children was selected, data about their age in years and weight in kilograms was recorded as shown in the following table . It is required to find the correlation between age and weight. Weight (Kg) Age (years) serial No 12 7 1 8 6 2 3 10 5 4 11 13 9

These 2 variables are of the quantitative type, one variable (Age) is called the independent and denoted as (X) variable and the other (weight) is called the dependent and denoted as (Y) variables to find the relation between age and weight compute the simple correlation coefficient using the following formula:

Y2 X2 xy Weight (Kg) (y) Age (years) (x) Serial n. 144 49 84 12 7 1 64 36 48 8 6 2 96 3 100 25 50 10 5 4 121 66 11 169 81 117 13 9 ∑y2= 742 ∑x2= 291 ∑xy= 461 ∑y= ∑x= 41 Total

r = 0.759 strong direct correlation

EXAMPLE: Relationship between Anxiety and Test Scores
Test score (Y) X2 Y2 XY 10 2 100 4 20 8 3 64 9 24 81 18 1 7 49 5 6 25 36 30 ∑X = 32 ∑Y = 32 ∑X2 = 230 ∑Y2 = 204 ∑XY=129

Calculating Correlation Coefficient
Indirect strong correlation

exercise

Regression Analyses Regression: technique concerned with predicting some variables by knowing others The process of predicting variable Y using variable X

Regression Uses a variable (x) to predict some outcome variable (y)
Tells you how values in y change as a function of changes in values of x

Correlation and Regression
Correlation describes the strength of a linear relationship between two variables Linear means “straight line” Regression tells us how to draw the straight line described by the correlation

Regression Calculates the “best-fit” line for a certain set of data
The regression line makes the sum of the squares of the residuals smaller than for any other line Regression minimizes residuals

By using the least squares method (a procedure that minimizes the vertical deviations of plotted points surrounding a straight line) we are able to construct a best fitting straight line to the scatter diagram points and then formulate a regression equation in the form of: b

Regression Equation Regression equation describes the regression line mathematically Intercept Slope

Linear Equations 28

Hours studying and grades

Regressing grades on hours
Predicted final grade in class = *(number of hours you study per week)

Predict the final grade of…
Predicted final grade in class = *(hours of study) Predict the final grade of… Someone who studies for 12 hours Final grade = (3.17*12) Final grade = 97.99 Someone who studies for 1 hour: Final grade = (3.17*1) Final grade = 63.12

Exercise A sample of 6 persons was selected the value of their age ( x variable) and their weight is demonstrated in the following table. Find the regression equation and what is the predicted weight when age is 8.5 years.

Weight (y) Age (x) Serial no. 12 8 10 11 13 7 6 5 9 1 2 3 4

Answer Y2 X2 xy Weight (y) Age (x) Serial no. 144 64 100 121 169 49 36 25 81 84 48 96 50 66 117 12 8 10 11 13 7 6 5 9 1 2 3 4 742 291 461 41 Total

Regression equation

we create a regression line by plotting two estimated values for y against their X component, then extending the line right and left.

Exercise 2 B.P (y) Age (x) 128 136 146 124 143 130 121 126 123 46 53 60 20 63 43 26 19 31 23 120 141 134 132 140 144 58 70 The following are the age (in years) and systolic blood pressure of 20 apparently healthy adults.

Find the correlation between age and blood pressure using simple and Spearman's correlation coefficients, and comment. Find the regression equation? What is the predicted blood pressure for a man aging 25 years?

x2 xy y x Serial 400 2400 120 20 1 1849 5504 128 43 2 3969 8883 141 63 3 676 3276 126 26 4 2809 7102 134 53 5 961 3968 31 6 3364 7888 136 58 7 2116 6072 132 46 8 8120 140 9 4900 10080 144 70 10

x2 xy y x Serial 2116 5888 128 46 11 2809 7208 136 53 12 3600 8760 146 60 13 400 2480 124 20 14 3969 9009 143 63 15 1849 5590 130 43 16 676 3224 26 17 361 2299 121 19 18 961 3906 126 31 529 2829 123 23 41678 114486 2630 852 Total

= = x for age 25 B.P = * 25= = mm hg

Multiple Regression Multiple regression analysis is a straightforward extension of simple regression analysis which allows more than one independent variable.

Multiple Regression Model
The equation that describes how the dependent variable y is related to the independent variables x1, x2, xp and an error term is: y = b0 + b1x1 + b2x bpxp + e where: b0, b1, b2, , bp are the parameters, and e is a random variable called the error term

Multiple Regression Equation
The equation that describes how the mean value of y is related to x1, x2, xp is: E(y) = 0 + 1x1 + 2x pxp

Estimated Multiple Regression Equation
^ y = b0 + b1x1 + b2x bpxp A simple random sample is used to compute sample statistics b0, b1, b2, , bp that are used as the point estimators of the parameters b0, b1, b2, , bp.

Estimation Process b0, b1, b2, . . . , bp Sample statistics are
Multiple Regression Model E(y) = 0 + 1x1 + 2x pxp + e Multiple Regression Equation E(y) = 0 + 1x1 + 2x pxp Unknown parameters are b0, b1, b2, , bp Sample Data: x1 x xp y Estimated Multiple Regression Equation Sample statistics are b0, b1, b2, , bp b0, b1, b2, , bp provide estimates of

Least Squares Method Computation of Coefficient Values
Least Squares Criterion Computation of Coefficient Values The formulas for the regression coefficients b0, b1, b2, bp involve the use of matrix algebra. We will rely on computer software packages to perform the calculations.

Example: Programmer Salary Survey A software firm collected data for a sample of 20 computer programmers. A suggestion was made that regression analysis could be used to determine if salary was related to the years of experience and the score on the firm’s programmer aptitude test. The years of experience, score on the aptitude test, and corresponding annual salary ($1000s) for a sample of 20 programmers is shown on the next slide.

Exper. Score Salary Exper. Score Salary 4 7 1 5 8 10 6 78 100 86 82 84 75 80 83 91 24.0 43.0 23.7 34.3 35.8 38.0 22.2 23.1 30.0 33.0 9 2 10 5 6 8 4 3 88 73 75 81 74 87 79 94 70 89 38.0 26.6 36.2 31.6 29.0 34.0 30.1 33.9 28.2 30.0

Suppose we believe that salary (y) is related to the years of experience (x1) and the score on the programmer aptitude test (x2) by the following regression model: y = 0 + 1x1 + 2x2 +  where y = annual salary ($1000) x1 = years of experience x2 = score on programmer aptitude test

Solving for the Estimates of 0, 1, 2
Least Squares Output Input Data x1 x2 y Computer Package for Solving Multiple Regression Problems b0 = b1 = b2 = R2 = etc.

Solving for the Estimates of 0, 1, 2
Excel’s Regression Equation Output Note: Columns F-I are not shown.

Estimated Regression Equation
SALARY = (EXPER) (SCORE) Note: Predicted salary will be in thousands of dollars.

Interpreting the Coefficients
In multiple regression analysis, we interpret each regression coefficient as follows: bi represents an estimate of the change in y corresponding to a 1-unit increase in xi when all other independent variables are held constant.

b1 = 1.404 Salary is expected to increase by $1,404 for each additional year of experience (when the variable score on programmer attitude test is held constant).

b2 = 0.251 Salary is expected to increase by $251 for each additional point scored on the programmer aptitude test (when the variable years of experience is held constant).

Multiple Coefficient of Determination
Relationship Among SST, SSR, SSE SST = SSR SSE = + where: SST = total sum of squares SSR = sum of squares due to regression SSE = sum of squares due to error

Excel’s ANOVA Output SSR SST

R2 = SSR/SST R2 = / =

Adjusted Multiple Coefficient
of Determination

Assumptions About the Error Term 
The error  is a random variable with mean of zero. The variance of  , denoted by 2, is the same for all values of the independent variables. The values of  are independent. The error  is a normally distributed random variable reflecting the deviation between the y value and the expected value of y given by 0 + 1x1 + 2x pxp.

Testing for Significance
In simple linear regression, the F and t tests provide the same conclusion. In multiple regression, the F and t tests have different purposes.

Testing for Significance: F Test
The F test is used to determine whether a significant relationship exists between the dependent variable and the set of all the independent variables. The F test is referred to as the test for overall significance.

Testing for Significance: t Test
If the F test shows an overall significance, the t test is used to determine whether each of the individual independent variables is significant. A separate t test is conducted for each of the independent variables in the model. We refer to each of these t tests as a test for individual significance.

Testing for Significance: F Test
Hypotheses H0: 1 = 2 = = p = 0 Ha: One or more of the parameters is not equal to zero. Test Statistics F = MSR/MSE Rejection Rule Reject H0 if p-value < a or if F > F, where F is based on an F distribution with p d.f. in the numerator and n - p - 1 d.f. in the denominator.

Testing for Significance: t Test
Hypotheses Test Statistics Rejection Rule Reject H0 if p-value < a or if t < -tor t > t where t is based on a t distribution with n - p - 1 degrees of freedom.

Testing for Significance: Multicollinearity
The term multicollinearity refers to the correlation among the independent variables. When the independent variables are highly correlated (say, |r | > .7), it is not possible to determine the separate effect of any particular independent variable on the dependent variable.

Testing for Significance: Multicollinearity
If the estimated regression equation is to be used only for predictive purposes, multicollinearity is usually not a serious problem. Every attempt should be made to avoid including independent variables that are highly correlated.

Using the Estimated Regression Equation for Estimation and Prediction
The procedures for estimating the mean value of y and predicting an individual value of y in multiple regression are similar to those in simple regression. We substitute the given values of x1, x2, , xp into the estimated regression equation and use the corresponding value of y as the point estimate.

Using the Estimated Regression Equation for Estimation and Prediction
The formulas required to develop interval estimates for the mean value of y and for an individual value of y are beyond the scope of the textbook. ^ Software packages for multiple regression will often provide these interval estimates.

Qualitative Independent Variables
In many situations we must work with qualitative independent variables such as gender (male, female), method of payment (cash, check, credit card), etc. For example, x2 might represent gender where x2 = 0 indicates male and x2 = 1 indicates female. In this case, x2 is called a dummy or indicator variable.

Example: Programmer Salary Survey As an extension of the problem involving the computer programmer salary survey, suppose that management also believes that the annual salary is related to whether the individual has a graduate degree in computer science or information systems. The years of experience, the score on the programmer aptitude test, whether the individual has a relevant graduate degree, and the annual salary ($1000) for each of the sampled 20 programmers are shown on the next slide.

Exper. Score Degr. Salary Exper. Score Degr. Salary 4 7 1 5 8 10 6 78 100 86 82 84 75 80 83 91 No Yes 24.0 43.0 23.7 34.3 35.8 38.0 22.2 23.1 30.0 33.0 9 2 10 5 6 8 4 3 88 73 75 81 74 87 79 94 70 89 Yes No 38.0 26.6 36.2 31.6 29.0 34.0 30.1 33.9 28.2 30.0

Estimated Regression Equation
y = b0 + b1x1 + b2x2 + b3x3 ^ where: y = annual salary ($1000) x1 = years of experience x2 = score on programmer aptitude test x3 = 0 if individual does not have a graduate degree 1 if individual does have a graduate degree x3 is a dummy variable

Excel’s Regression Equation Output Note: Columns F-I are not shown. Not significant

More Complex Qualitative Variables
If a qualitative variable has k levels, k - 1 dummy variables are required, with each dummy variable being coded as 0 or 1. For example, a variable with levels A, B, and C could be represented by x1 and x2 values of (0, 0) for A, (1, 0) for B, and (0,1) for C. Care must be taken in defining and interpreting the dummy variables.

More Complex Qualitative Variables
For example, a variable indicating level of education could be represented by x1 and x2 values as follows: Highest Degree x x2 Bachelor’s 0 0 Master’s 1 0 Ph.D

Residual Analysis For simple linear regression the residual plot against and the residual plot against x provide the same information. In multiple regression analysis it is preferable to use the residual plot against to determine if the model assumptions are satisfied.

Standardized Residual Plot Against
Standardized residuals are frequently used in residual plots for purposes of: Identifying outliers (typically, standardized residuals < -2 or > +2) Providing insight about the assumption that the error term e has a normal distribution The computation of the standardized residuals in multiple regression analysis is too complex to be done by hand Excel’s Regression tool can be used

Excel Value Worksheet Note: Rows are not shown.

Excel’s Standardized Residual Plot Outlier

Logistic Regression In many ways logistic regression is like ordinary regression. It requires a dependent variable, y, and one or more independent variables. Logistic regression can be used to model situations in which the dependent variable, y, may only assume two discrete values, such as 0 and 1. The ordinary multiple regression model is not applicable.

Logistic Regression Logistic Regression Equation
The relationship between E(y) and x1, x2, , xp is better described by the following nonlinear equation.

Logistic Regression Interpretation of E(y) as a
Probability in Logistic Regression If the two values of y are coded as 0 or 1, the value of E(y) provides the probability that y = 1 given a particular set of values for x1, x2, , xp.

Logistic Regression Estimated Logistic Regression Equation
A simple random sample is used to compute sample statistics b0, b1, b2, , bp that are used as the point estimators of the parameters b0, b1, b2, , bp.

Logistic Regression Example: Simmons Stores
Simmons’ catalogs are expensive and Simmons would like to send them to only those customers who have the highest probability of making a $200 purchase using the discount coupon included in the catalog. Simmons’ management thinks that annual spending at Simmons Stores and whether a customer has a Simmons credit card are two variables that might be helpful in predicting whether a customer who receives the catalog will use the coupon to make a $200 purchase.

Logistic Regression Example: Simmons Stores
Simmons conducted a study by sending out 100 catalogs, 50 to customers who have a Simmons credit card and 50 to customers who do not have the card. At the end of the test period, Simmons noted for each of the 100 customers: 1) the amount the customer spent last year at Simmons, 2) whether the customer had a Simmons credit card, and 3) whether the customer made a $200 purchase. A portion of the test data is shown on the next slide.

Logistic Regression x1 x2 Simmons Test Data (partial) y
Annual Spending ($1000) 2.291 3.215 2.135 3.924 2.528 2.473 2.384 7.076 1.182 3.345 Simmons Credit Card 1 $200 Purchase 1 Customer 1 2 3 4 5 6 7 8 9 10

Logistic Regression Simmons Logistic Regression Table (using Minitab)
Predictor Coef SE Coef Z p Odds Ratio 95% CI Lower Upper Constant Spending Card 0.3416 1.0987 0.5772 0.1287 0.4447 -3.72 2.66 2.47 0.000 0.008 0.013 1.41 3.00 1.09 1.25 1.81 7.17 Log-Likelihood = Test that all slopes are zero: G = , DF = 2, P-Value = 0.001

Logistic Regression Simmons Estimated Logistic Regression Equation

Logistic Regression Using the Estimated Logistic Regression Equation
For customers that spend $2000 annually and do not have a Simmons credit card: For customers that spend $2000 annually and do have a Simmons credit card:

Logistic Regression Testing for Significance Hypotheses
Ha: One or both of the parameters is not equal to zero. Test Statistics z = bi/sbi Reject H0 if p-value < a Rejection Rule

Logistic Regression Testing for Significance Conclusions
For independent variable x1: z = 2.66 and the p-value = .008. Hence, b1 = 0. In other words, x1 is statistically significant. For independent variable x2: z = 2.47 and the p-value = .013. Hence, b2 = 0. In other words, x2 is also statistically significant.

Logistic Regression With logistic regression is difficult to interpret the relationship between the variables because the equation is not linear so we use the concept called the odds ratio. The odds in favor of an event occurring is defined as the probability the event will occur divided by the probability the event will not occur. Odds in Favor of an Event Occurring Odds Ratio

Logistic Regression Estimated Probabilities
Annual Spending $ $ $ $ $ $ $7000 Credit Card Yes No Computed earlier

Logistic Regression Comparing Odds
Suppose we want to compare the odds of making a $200 purchase for customers who spend $2000 annually and have a Simmons credit card to the odds of making a and do not have a Simmons credit card.

Forecasting Based on supervised learning/classification

Similar presentations

Presentation on theme: "Forecasting Based on supervised learning/classification"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Forecasting Based on supervised learning/classification

Similar presentations

Presentation on theme: "Forecasting Based on supervised learning/classification"— Presentation transcript:

Similar presentations

About project

Feedback