1. Chem 302 - Math 252 Chapter 5 Regression

2. Linear & Nonlinear Regression Linear regression –Linear in the parameters –Does not have to be linear in the independent variable(s) –Can be solved through a system of linear equations Nonlinear –Nonlinear in parameters –Usually requires linearization and iteration

3. Linear Least-Squares Regression Residual Sum of Square Residuals Want to minimize Z

4. Linear Least-Squares Regression

5. Example

6. Linear Least-Squares Regression Uncertainties in Parameters Example

7. Linear Least-Squares Regression Regression on “y” Treat x as y and y as x Choose x as variable with smallest error Can also be determined by equation

8. Linear Least-Squares Regression

9. Example – Vapour Pressure of Cadmium

10. Linear Least-Squares Regression Uncertainties in Parameters

11. Nonlinear Least-Squares Regression This results in a system of nonlinear equations Linearize & solve iteratively Need initial estimate of parameters

12. Nonlinear Least-Squares Regression - Example Van der Waals parameters for nitrogen p/atmT/KV m /(L mol -1 )p/atmT/KV m /(L mol -1 ) 1223.1518.283405373.156.13064 5223.153.6343620373.151.53844 10223.151.8038950373.150.621118 20223.150.8897485473.157.77970 1273.1522.404610473.153.89744 10273.152.2317420473.151.95651 20273.151.1118950473.150.792572 50273.150.44191

13. Weighted Least-Squares Regression may not always want to give equal weight to each point Applies to linear and nonlinear case

14. Drawbacks of Iterative Matrix Method Local minima can cause problems Can be sensitive to initial guess Derivatives must be evaluated for each iteration

15. Simplex Method Simplex has one more vertex than dimension of space –2D – Triangle m parameters – m+1 vertices Simplex Method used to optimize a set of parameters –Find optimal set of  ’s such that Z is minimum More robust than previous iterative procedure –Often slower

16. Simplex Method 1.Evaluate Z at m+1 unique sets of parameters 2.Identify Z B (best, smallest) and Z W (worst, largest) 3.Calculate Centroid of all but worst (average of different sets of parameters ignoring worst set) 4.Reflect worst point through Centroid

17. Simplex Method 5.Replace Worst point: a.If Z R 1 <Z B (reflected point is better than previous best) calculate i.If Z R 2 <Z R 1 replace W with R 2 ii.Otherwise replace W with R 1 b.If Z B <Z R 1 <Z W replace W with R 1 c.If Z R 1 >Z W a contracted point id calculated i.If Z R 3 <Z W replace W with R 3 ii.Otherwise move all points closer to the best point 6.Repeat until converged or maximum number of iterations have been performed

18. Simplex Regression - Example Van der Waals parameters for nitrogen p/atmT/KV m /(L mol -1 )p/atmT/KV m /(L mol -1 ) 1223.1518.283405373.156.13064 5223.153.6343620373.151.53844 10223.151.8038950373.150.621118 20223.150.8897485473.157.77970 1273.1522.404610473.153.89744 10273.152.2317420473.151.95651 20273.151.1118950473.150.792572 50273.150.44191

19. Simplex program

20. Simplex - Example Iteration 1: Response 0.344652 betaResponse 1.3000000.0500000.425437 1.3260000.0505000.344652Best 1.3130000.0510000.579697Worst 1.3130000.050250Centroid 1.3130000.0495000.229741First reflected point 1.3130000.0487500.116962Second reflected point Iteration 2: Response 0.116962 betaResponse 1.3000000.0500000.425437Worst 1.3260000.0505000.344652 1.3130000.0487500.116962Best 1.3195000.049625Centroid 1.3390000.0492500.076378First reflected point 1.3585000.0488750.011665Second reflected point Iteration 3: Response 0.0116649 betaResponse 1.3585000.0488750.011665Best 1.3260000.0505000.344652Worst 1.3130000.0487500.116962 1.3357500.048812Centroid 1.3455000.0471250.041013First reflected point Iteration 4: Response 0.0116649 betaResponse 1.3585000.0488750.011665Best 1.3455000.0471250.041013 1.3130000.0487500.116962Worst 1.3520000.048000Centroid 1.3910000.0472500.195042First reflected point 1.3325000.0483750.027212Contracted point Iteration 31: Response 0.00543252 betaResponse 1.3934870.0496240.005433 1.3933400.0496190.005433Best 1.3932200.0496160.005433Worst 1.3934130.049621Centroid 1.3936070.0496270.005433First reflected point 1.3933170.0496190.005433Contracted point Iteration 32: Response 0.00543252 betaResponse 1.3934870.0496240.005433Worst 1.3933400.0496190.005433 1.3933170.0496190.005433Best 1.3933280.049619Centroid 1.3931700.0496130.005433First reflected point 1.3934080.0496210.005433Contracted point Iterations converged. R^2 0.999999 Final Converged Parameters kbeta 01.39332 10.0496186

21. Simplex – Example (Iteration 1) B W C R1R1 R2R2

22. Simplex – Example (Iteration 2) B W C R1R1 R2R2

23. Simplex – Example (Iteration 3) B W C R1R1

24. Simplex – Example (Iteration 4) B W C R1R1 Contracted

25. Simplex – Example (Iteration 32) B W C R1R1 Contracted

26. Comparing Models Often have more than 1 equation that can be used to represent the data If two equations (models) have the same number of parameters the one with smaller Z is a better representation (fit) If two models have different number of parameters then can not do a direct comparison –Need to use F distribution & Confidence level –Model A – fewer number of parameters Model B – larger number of parameters

27. Comparing Models Model B is a better model if (and only if) Usually lookup F in Table and compare ratios With Maple can calculate confidence level for which B is a better model than A