1. Non-Linear Models Tractable non-linearity Intractable non-linearity
Equation may be transformed to a linear model.
Intractable non-linearity
No linear transform exists

2. Tractable Non-Linear Models
Several general Types
Polynomial
Power Functions
Exponential Functions
Logarithmic Functions
Trigonometric Functions

3. Polynomial Models Linear Parabolic Cubic & higher order polynomials
All may be estimated with OLS – simply square, cube, etc. the independent variable.

4. Power Functions Simple exponents of the Independent Variable
Estimated with

5. Exponential and Logarithmic Functions
Common Growth Curve Formula
Estimated with
Note that the error terms are now no longer normally distributed!

6. Logarithmic Functions

7. Trigonometric Functions
Sine/Cosine functions
Fourier series

8. Intractable Non-linearity
Occasionally we have models that we cannot transform to linear ones.
For instance a logit model
Or an equilibrium system model

9. Intractable Non-linearity
Models such as these must be estimated by other means.
We do, however, keep the criteria of minimizing the squared error as our means of determining the best model

10. Estimating Non-linear models
All methods of non-linear estimation require an iterative search for the best fitting parameter values.
They differ in how they modify and search for those values that minimize the SSE.

11. Methods of Non-linear Estimation
There are several methods of selecting parameters
Grid search
Steepest descent
Marquardts algorithm

12. Grid search estimation
In a grid search estimation, we simply try out a set of parameters across a set of ranges and calculate the SSE.
We then ascertain where in the range (or at which end) the SSE was at a minimum.
We then repeat with either extending the range, or reducing the range and searching with smaller grid around the estimated SSE
Try the spreadsheet
Try this for homework!

13. Regression Diagnostics
Some cases are very influential in regression models
There are two ways to describe the influence that case may exert
Residual
Leverage
Examination of these particular cases may lead us to theoretical insight.

14. Regression Diagnostics
Residual
Outliers – extreme measures weaken the goodness of fit indexes and hypothesis tests
Studentized residuals
where hii is the hat diagonal

15. Residual Diagnostics (cont.)
RStudent

16. Residual Diagnostics (cont.)
Leverage – influential observation
Hat diagonal – a measure of the observations “remoteness” in X-space.
Hat diagonal;s greater than 2 times the number of coefficients in the model divided by the number of observations are considered significant.

17. Residual Diagnostics (cont.)
Leverage (cont.)
Cook’s D
If D is greater than 1, then the observation is influential.

18. Residual Diagnostics (cont.)
Leverage (cont.)
dffits
If dffits is greater than 1, then the observation is influential.

19. Residual Diagnostics (cont.)
Leverage (cont.)
Dfbetas – an indicator of how much a given observation influences each regression coefficient