REGRESI 1

2 Regresi adalah Diberikan sejumlah n buah data Yg dimodelkan oleh persamaan Model yg paling baik (best fit) secara umum adalah model yg meminimalkan jumlah kuadrat residual Figure. Basic model for regression Jumlah kuadrat residual

3 REGRESI LINIER

4 Regresi Linier (Kriteria 1) Diberikan sejumlah n data Best fit dimodelkan dalam bentuk persamaan x y Figure. Linear regression of y vs. x data showing residuals at a typical point, x i.

5 Contoh Kriteria 1 xy Diberikan sejumlah titik (2,4), (3,6), (2,6) and (3,8), best fit dimodelkan dalam bentuk persamaan garis lurus Figure. Data points for y vs. x data. Table. Data Points

6 xyy predicted ε = y - y predicted Table. Residuals at each point for regression model y = 4x – 4. Figure. Regression curve for y=4x-4, y vs. x data Dengan menggunakan persamaan y=4x-4 maka diperoleh kurva regresi

7 xyy predicted ε = y - y predicted Table. Residuals at each point for y=6 Figure. Regression curve for y=6, y vs. x data Persamaan y=6

8 Kedua persamaan y=4x-4 and y=6 memiliki residual minimum tetapi memiliki model regresi yang tidak unik. Oleh karena itu kriteria 1 merupakan kriteria yang buruk

9 Regresi linier (Kriteria 2) x y Figure. Linear regression of y vs. x data showing residuals at a typical point, x i. Meminimalkan dengan memberikan harga mutlak

10 xyy predicted |ε| = |y - y predicted | Table. The absolute residuals employing the y=4x-4 regression model Figure. Regression curve for y=4x-4, y vs. x data Dengan menggunakan persamaan y=4x-4

11 xyy predicted |ε| = |y – y predicted | Table. Absolute residuals employing the y=6 model Figure. Regression curve for y=6, y vs. x data Dengan persamaan y=6

12 Can you find a regression line for whichand has unique regression coefficients? for both regression models of y=4x-4 and y=6. The sum of the errors has been made as small as possible, that is 4, but the regression model is not unique. Hence the above criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion.

13 Least Squares Criterion Kriteria Least Squares meminimalkan jumlah kuadrat residual dari model Persamaan. x y Figure. Linear regression of y vs. x data showing residuals at a typical point, x i.

14 Finding Constants of Linear Model Minimize the sum of the square of the residuals: To find giving andwe minimizewith respect toand.

15 Finding Constants of Linear Model Solving for and directly yields,

16 Example 1 The torque, T needed to turn the torsion spring of a mousetrap through an angle, is given below. Angle, θ Torque, T RadiansN-m Table: Torque vs Angle for a torsional spring Find the constants for the model given by Figure. Data points for Angle vs. Torque data

17 Example 1 cont. The following table shows the summations needed for the calculations of the constants in the regression model. RadiansN-mRadians 2 N-m-Radians Table. Tabulation of data for calculation of important Using equations described for N-m/rad summations andwith

18 Example 1 cont. Use the average torque and average angle to calculate Using, N-m

19 Example 1 Results Figure. Linear regression of Torque versus Angle data Using linear regression, a trend line is found from the data Can you find the energy in the spring if it is twisted from 0 to 180 degrees?

20 Example 2 StrainStress (%)(MPa) To find the longitudinal modulus of composite, the following data is collected. Find the longitudinal modulus, Table. Stress vs. Strain data using the regression model and the sum of the square of the residuals. Figure. Data points for Stress vs. Strain data

21 Example 2 cont. Residual at each point is given by The sum of the square of the residuals then is Differentiate with respect to Therefore

22 Example 2 cont. iεσε 2 εσ ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − × ×10 − ×10 8 Table. Summation data for regression model With and Using

23 Example 2 Results The equation Figure. Linear regression for Stress vs. Strain data describes the data.

REGRESI NON LINIER

Nonlinear Regression Some popular nonlinear regression models: 1. Exponential model: 2. Power model: 3. Saturation growth model: 4. Polynomial model: 25

Nonlinear Regression Given n data pointsbest fit to the data, whereis a nonlinear function of. Figure. Nonlinear regression model for discrete y vs. x data 26

Regression Exponential Model 27

Exponential Model Givenbest fitto the data. Figure. Exponential model of nonlinear regression for y vs. x data 28

Finding Constants of Exponential Model The sum of the square of the residuals is defined as Differentiate with respect to a and b 29

Finding Constants of Exponential Model Rewriting the equations, we obtain 30

Finding constants of Exponential Model Substituting a back into the previous equation The constant b can be found through numerical methods such as bisection method. Solving the first equation for a yields 31

Example 1-Exponential Model t(hrs) Many patients get concerned when a test involves injection of a radioactive material. For example for scanning a gallbladder, a few drops of Technetium-99m isotope is used. Half of the techritium-99m would be gone in about 6 hours. It, however, takes about 24 hours for the radiation levels to reach what we are exposed to in day-to-day activities. Below is given the relative intensity of radiation as a function of time. Table. Relative intensity of radiation as a function of time. 32

Example 1-Exponential Model cont. Find: a) The value of the regression constantsand b) The half-life of Technium-99m c) Radiation intensity after 24 hours The relative intensity is related to time by the equation 33

Plot of data 34

Constants of the Model The value of λ is found by solving the nonlinear equation 35

Setting up the Equation in MATLAB t (hrs) γ

Setting up the Equation in MATLAB t=[ ] gamma=[ ] syms lamda sum1=sum(gamma.*t.*exp(lamda*t)); sum2=sum(gamma.*exp(lamda*t)); sum3=sum(exp(2*lamda*t)); sum4=sum(t.*exp(2*lamda*t)); f=sum1-sum2/sum3*sum4; 37

Calculating the Other Constant The value of A can now be calculated The exponential regression model then is 38

Plot of data and regression curve 39

Relative Intensity After 24 hrs The relative intensity of radiation after 24 hours This result implies that only radioactive intensity is left after 24 hours. 40

Homework What is the half-life of technetium 99m isotope? Compare the constants of this regression model with the one where the data is transformed. Write a program in the language of your choice to find the constants of the model. 41

Polynomial Model Givenbest fit to a given data set. Figure. Polynomial model for nonlinear regression of y vs. x data 42

Polynomial Model cont. The residual at each data point is given by The sum of the square of the residuals then is 43

Polynomial Model cont. To find the constants of the polynomial model, we set the derivatives with respect to whereequal to zero. 44

Polynomial Model cont. These equations in matrix form are given by The above equations are then solved for 45

Example 2-Polynomial Model Temperature, T ( o F) Coefficient of thermal expansion, α (in/in/ o F) ×10 − ×10 −6 −405.72×10 −6 − ×10 −6 − ×10 −6 − ×10 −6 − ×10 −6 Regress the thermal expansion coefficient vs. temperature data to a second order polynomial. Table. Data points for temperature vs Figure. Data points for thermal expansion coefficient vs temperature. 46

Example 2-Polynomial Model cont. We are to fit the data to the polynomial regression model The coefficientsare found by differentiating the sum of the square of the residuals with respect to each variable and setting the values equal to zero to obtain 47

Example 2-Polynomial Model cont. The necessary summations are as follows Temperature, T ( o F) Coefficient of thermal expansion, α (in/in/ o F) ×10 − ×10 −6 −405.72×10 −6 − ×10 −6 − ×10 −6 − ×10 −6 − ×10 −6 Table. Data points for temperature vs. 48

Example 2-Polynomial Model cont. Using these summations, we can now calculate Solving the above system of simultaneous linear equations we have The polynomial regression model is then 49

Linearization of Data To find the constants of many nonlinear models, it results in solving simultaneous nonlinear equations. For mathematical convenience, some of the data for such models can be linearized. For example, the data for an exponential model can be linearized. As shown in the previous example, many chemical and physical processes are governed by the equation, Taking the natural log of both sides yields, Letand (implying)with We now have a linear regression model where 50

Linearization of data cont. Using linear model regression methods, Onceare found, the original constants of the model are found as 51

Example 3-Linearization of data t(hrs) Many patients get concerned when a test involves injection of a radioactive material. For example for scanning a gallbladder, a few drops of Technetium- 99m isotope is used. Half of the technetium-99m would be gone in about 6 hours. It, however, takes about 24 hours for the radiation levels to reach what we are exposed to in day-to-day activities. Below is given the relative intensity of radiation as a function of time. Table. Relative intensity of radiation as a function of time Figure. Data points of relative radiation intensity vs. time 52

Example 3-Linearization of data cont. Find: a) The value of the regression constantsand b) The half-life of Technium-99m c) Radiation intensity after 24 hours The relative intensity is related to time by the equation 53

Example 3-Linearization of data cont. Exponential model given as, Assuming,andwe obtain This is a linear relationship betweenand 54

Example 3-Linearization of data cont. Using this linear relationship, we can calculate and where 55

Example 3-Linearization of Data cont − − − − − − − − − − −2.8778− Summations for data linearization are as follows Table. Summation data for linearization of data model With 56

Example 3-Linearization of Data cont. Calculating Since also 57

Example 3-Linearization of Data cont. Resulting model is Figure. Relative intensity of radiation as a function of temperature using linearization of data model. 58

Example 3-Linearization of Data cont. The regression formula is then b) Half life of Technetium 99 is when 59

Example 3-Linearization of Data cont. c) The relative intensity of radiation after 24 hours is then This implies that onlyof the radioactive material is left after 24 hours. 60

Comparison Comparison of exponential model with and without data linearization: With data linearization (Example 3) Without data linearization (Example 1) A λ− − Half-Life (hrs) Relative intensity after 24 hrs ×10 − ×10 −2 Table. Comparison for exponential model with and without data linearization. The values are very similar so data linearization was suitable to find the constants of the nonlinear exponential model in this case. 61

62 ADEQUACY OF REGRESSION MODELS

Data

Is this adequate? Straight Line Model

Quality of Fitted Data Does the model describe the data adequately? How well does the model predict the response variable predictably?

Linear Regression Models Limit our discussion to adequacy of straight-line regression models

Four checks 1. Plot the data and the model. 2. Find standard error of estimate. 3. Calculate the coefficient of determination. 4. Check if the model meets the assumption of random errors.

Example: Check the adequacy of the straight line model for given data T (F) α (μin/in/F)

1. Plot the data and the model

Data and model T (F) α (μin/in/F)

2. Find the standard error of estimate

Standard error of estimate

Standard Error of Estimate

Standard Error of Estimate

Scaled Residuals 95% of the scaled residuals need to be in [-2,2]

Scaled Residuals TiTi αiαi Residual Scaled Residual

3. Find the coefficient of determination

Coefficient of determination

Sum of square of residuals between data and mean y x

Sum of square of residuals between observed and predicted y x

Limits of Coefficient of Determination

Calculation of S t

Calculation of S r

Coefficient of determination

Caution in use of r 2 Increase in spread of regressor variable (x) in y vs. x increases r 2 Large regression slope artificially yields high r 2 Large r 2 does not measure appropriateness of the linear model Large r 2 does not imply regression model will predict accurately

Final Exam Grade

Final Exam Grade vs Pre-Req GPA

4. Model meets assumption of random errors

Model meets assumption of random errors Residuals are negative as well as positive Variation of residuals as a function of the independent variable is random Residuals follow a normal distribution There is no autocorrelation between the data points.

Therm exp coeff vs temperature Tα Tα Tα

Data and model

Plot of Residuals

Histograms of Residuals

Check for Autocorrelation Find the number of times, q the sign of the residual changes for the n data points. If (n-1)/2-√(n-1) ≤ q ≤ (n-1)/2+√(n-1), you most likely do not have an autocorrelation.

Is there autocorrelation?

y vs x fit and residuals n=40 Is 13.3≤21≤ 25.7? Yes! (n-1)/2-√(n-1) ≤p≤ (n-1)/2+√(n-1)

y vs x fit and residuals (n-1)/2-√(n-1) ≤p≤ (n-1)/2+√(n-1) Is 13.3≤2≤ 25.7? No! n=40

What polynomial model to choose if one needs to be chosen?

First Order of Polynomial

Second Order Polynomial

Which model to choose?

Optimum Polynomial

Effect of an Outlier

Effect of Outlier