Chem Math 252 Chapter 5 Regression

Linear & Nonlinear Regression Linear regression –Linear in the parameters –Does not have to be linear in the independent variable(s) –Can be solved through a system of linear equations Nonlinear –Nonlinear in parameters –Usually requires linearization and iteration

Linear Least-Squares Regression Residual Sum of Square Residuals Want to minimize Z

Linear Least-Squares Regression

Example

Linear Least-Squares Regression Uncertainties in Parameters Example

Linear Least-Squares Regression Regression on “y” Treat x as y and y as x Choose x as variable with smallest error Can also be determined by equation

Linear Least-Squares Regression

Example – Vapour Pressure of Cadmium

Linear Least-Squares Regression Uncertainties in Parameters

Nonlinear Least-Squares Regression This results in a system of nonlinear equations Linearize & solve iteratively Need initial estimate of parameters

Nonlinear Least-Squares Regression - Example Van der Waals parameters for nitrogen p/atmT/KV m /(L mol -1 )p/atmT/KV m /(L mol -1 )

Weighted Least-Squares Regression may not always want to give equal weight to each point Applies to linear and nonlinear case

Drawbacks of Iterative Matrix Method Local minima can cause problems Can be sensitive to initial guess Derivatives must be evaluated for each iteration

Simplex Method Simplex has one more vertex than dimension of space –2D – Triangle m parameters – m+1 vertices Simplex Method used to optimize a set of parameters –Find optimal set of  ’s such that Z is minimum More robust than previous iterative procedure –Often slower

Simplex Method 1.Evaluate Z at m+1 unique sets of parameters 2.Identify Z B (best, smallest) and Z W (worst, largest) 3.Calculate Centroid of all but worst (average of different sets of parameters ignoring worst set) 4.Reflect worst point through Centroid

Simplex Method 5.Replace Worst point: a.If Z R 1 <Z B (reflected point is better than previous best) calculate i.If Z R 2 <Z R 1 replace W with R 2 ii.Otherwise replace W with R 1 b.If Z B <Z R 1 <Z W replace W with R 1 c.If Z R 1 >Z W a contracted point id calculated i.If Z R 3 <Z W replace W with R 3 ii.Otherwise move all points closer to the best point 6.Repeat until converged or maximum number of iterations have been performed

Simplex Regression - Example Van der Waals parameters for nitrogen p/atmT/KV m /(L mol -1 )p/atmT/KV m /(L mol -1 )

Simplex program

Simplex - Example Iteration 1: Response betaResponse Best Worst Centroid First reflected point Second reflected point Iteration 2: Response betaResponse Worst Best Centroid First reflected point Second reflected point Iteration 3: Response betaResponse Best Worst Centroid First reflected point Iteration 4: Response betaResponse Best Worst Centroid First reflected point Contracted point Iteration 31: Response betaResponse Best Worst Centroid First reflected point Contracted point Iteration 32: Response betaResponse Worst Best Centroid First reflected point Contracted point Iterations converged. R^ Final Converged Parameters kbeta

Simplex – Example (Iteration 1) B W C R1R1 R2R2

Simplex – Example (Iteration 2) B W C R1R1 R2R2

Simplex – Example (Iteration 3) B W C R1R1

Simplex – Example (Iteration 4) B W C R1R1 Contracted

Simplex – Example (Iteration 32) B W C R1R1 Contracted

Comparing Models Often have more than 1 equation that can be used to represent the data If two equations (models) have the same number of parameters the one with smaller Z is a better representation (fit) If two models have different number of parameters then can not do a direct comparison –Need to use F distribution & Confidence level –Model A – fewer number of parameters Model B – larger number of parameters

Comparing Models Model B is a better model if (and only if) Usually lookup F in Table and compare ratios With Maple can calculate confidence level for which B is a better model than A