1. Lecture (14,15) More than one Variable, Curve Fitting, and Method of Least Squares

2. Two Variables Often two variables are in some way connected. Observation of the pairs: X Y X1 Y1 X2 Y2. Xn Yn

3. Covariance The covariance gives the some information about the extent to which the two random variables influence each other.

4. Example Covariance What does this number tell us?

5. Pearson’s R Covariance does not really tell us anything –Solution: standardise this measure Pearson’s R: standardise by adding std to equation:

6. Correlation Coefficient

7. Correlation Coefficient (Cont.) Correlation Coefficient (Cont.)

8. Procedure of Best Fitting (Step 1) How to find out the relation between the two variables? 1. Make observation of the pairs: X Y X1 Y1 X2 Y2. Xn Yn

9. Procedure of Best Fitting (Step 2) 2. Make plot of the observations. It is always difficult to decide whether a curved line fits nicely to a set of data. Straight lines are preferable. We change the scale to obtain straight lines.

10. Method of Least Square (Step 3) 3. Specify a straight line relation. Y=a+bX We need to find a and b that minimises the square of the differences between the line and the observed data.

11. Step 3 (cont.)  find best fit of a line in a cloud of observations: Principle of least squares ε y = ax + b ε = residual error =, true value =, predicted value

12. Method of Least Square (Step 4)

13. Method of Least Square (Step 5)

14. Method of Least Square (Step 6)

15. Method of Least Square (Step 7)

16. Example Xy 11 32 44 64 85 97 118 149 We have the following eight pairs of observations:

17. Example (Cont.) xiyiXi^2xi.yiYi^2 11111 32964 4416 64362416 85644025 97816349 1181218864 14919612681 5640524364256 7565.545.532 Construct the least square line:  1/n  N=8

18. Example (Cont.) xiyiXi^ 2 xi.yiYi^2 11111 32964 4416 64362416 85644025 97816349 1181218864 14919612681 5640524364256 7565.545.532

19. Example (Cont.) Equation Y = 0.545+ 0.636 * X Number of data points used = 8 Average X = 7 Average Y = 5

20. i12345 xixi 2.106.227.1710.513.7 yiyi 2.903.835.985.717.74 Example (2)

21. Example (3)

22. Excel Application See Excel

23. Covariance and the Correlation Coefficient Use COVAR to calculate the covariance Cell =COVAR(array1, array2) –Average of products of deviations for each data point pair –Depends on units of measurement Use CORREL to return the correlation coefficient Cell =CORREL(array1, array2) –Returns value between -1 and +1 Also available in Analysis ToolPak

24. Analysis ToolPak Descriptive Statistics Correlation Linear Regression t-Tests z-Tests ANOVA Covariance

25. Descriptive Statistics Mean, Median, Mode Standard Error Standard Deviation Sample Variance Kurtosis Skewness Confidence Level for Mean Range Minimum Maximum Sum Count kth Largest kth Smallest

26. Correlation and Regression Correlation is a measure of the strength of linear association between two variables –Values between -1 and +1 –Values close to -1 indicate strong negative relationship –Values close to +1 indicate strong positive relationship –Values close to 0 indicate weak relationship Linear Regression is the process of finding a line of best fit through a series of data points –Can also use the SLOPE, INTERCEPT, CORREL and RSQ functions

27. Polynomial Regression Minimize the residual between the data points and the curve -- least-squares regression Must find values of a 0, a 1, a 2, … a m Linear Quadratic Cubic General

28. Polynomial Regression Residual Sum of squared residuals Minimize by taking derivatives

29. Polynomial Regression Normal Equations

30. Example x01.01.52.32.54.05.16.06.57.08.19.0 y0.20.82.5 3.54.33.05.03.52.41.32.0 x9.311.011.312.113.114.015.516.017.517.819.020.0 y-0.3-1.3-3.0-4.0-4.9-4.0-5.2-3.0-3.5-1.6-1.4-0.1

31. Example x01.01.52.32.54.05.16.06.57.08.19.0 y0.20.82.5 3.54.33.05.03.52.41.32.0 x9.311.011.312.113.114.015.516.017.517.819.020.0 y-0.3-1.3-3.0-4.0-4.9-4.0-5.2-3.0-3.5-1.6-1.4-0.1

32. Example Regression Equation y = - 0.359 + 2.305x - 0.353x 2 + 0.012x 3

33. Nonlinear Relationships If relationship is an exponential function To make it linear, take logarithm of both sides To make linear, take logarithm of both sides Now it’s a linear relation between ln(y) and x Now it’s a linear relation between ln(y) and ln(x) If relationship is a power function

34. Examples Quadratic curve –Flow rating curve: q = measured discharge, H = stage (height) of water behind outlet Power curve –Sediment transport: c = concentration of suspended sediment q = river discharge –Carbon adsorption: q = mass of pollutant sorbed per unit mass of carbon, C = concentration of pollutant in solution

35. Example – Log-Log xyX=Log( x) Y=Log( y) 1.22.10.180.74 2.811.51.032.44 4.328.11.463.34 5.441.91.693.74 6.872.31.924.28 7.991.42.074.52 x vs y X=Log(x) vs Y=log(y)

36. Example – Log-Log Using the X’s and Y’s, not the original x’s and y’s

37. Example – Carbon Adsorption q = pollutant mass sorbed per carbon mass C = concentration of pollutant in solution, K = coefficient n = measure of the energy of the reaction

38. Example – Carbon Adsorption Linear axes: K = 74.702, and n = 0.2289

39. Example – Carbon Adsorption Logarithmic axes: logK = 1.8733, K = 10 1.6733 = 74.696, n = 0.2289

40. Multiple Regression Y 1 = x 11  1 + x 12   +…+ x 1n  n +  1 Y 2 = x 21  1 + x 22   +…+ x 2n  n +  2 : Y m = x m1  1 + x m2   +…+ x mn  n  +  m. Regression model                                       m 2 1 m1 21 11 m 2 1     x x x y y y  nn 12 x 22 x 2n x 1n x m2 x mn x Multiple regression model In matrix notation

41.                                       m 2 1 m1 21 11 m 2 1     x x x y y y  nn 12 x 22 x 2n x 1n x m2 x mn x Multiple Regression (cont.) Observed data = design matrix * parameters + residuals