Design and Analysis of Experiments Dr. Tai-Yue Wang Department of Industrial and Information Management National Cheng Kung University Tainan, TAIWAN, ROC This is a basic course blah, blah, blah…

Response Surface Methodology Dr. Tai-Yue Wang Department of Industrial and Information Management National Cheng Kung University Tainan, TAIWAN, ROC This is a basic course blah, blah, blah…

Outline Introduction The Method of Steepest Ascent Analysis of A Second-order Response Experimental Designs for Fitting Response Surfaces Experiment with Computer Models Mixture Experiments Evolutionary Operation

Introduction A collection of mathematical and statistical techniques useful for modeling and analysis of problems in which a response of interest is influenced by several variables. The objective is to minimize this response. Example: a chemical process is to maximize its yield(y) and its inputs, temperature(x1) and pressure(x2), that is, where ε is the noise observed in the response, y

Introduction If we denote the expected response by Then is called a Response Surface The response surface is usually plotted graphically

Introduction Response graph and its contour

Introduction A polynomial function certainly will not work well for the entire surface But it works fine in small region. Steps in RSM Find a suitable approximation for y = f(x) using a low order polynomial Move towards the region of the optimum When curvature is found find a new approximation for y = f(x) {generally a higher order polynomial} and perform the “Response Surface Analysis”

Introduction

Introduction RSM dates from the 1950s; early applications in chemical industry Modern applications of RSM span many industrial and business settings

The Method of Steepest Ascent Generally the initial estimate of the optimum operating conditions for the system will be far from the actual optimum  to move to general vicinity of the optimum rapidly When we are remote from the optimum, we usually assume that first order model is an adequate approximation to the true surface in a small region of the x’s

The Method of Steepest Ascent The method of steepest ascent is a procedure for moving sequentially in the direction of the maximum increase in the response. If the minimization is desired, it is called the method of steepest descent. The fitted first order model is

The Method of Steepest Ascent The first order response surface, the contours of , is a series of parallel lines. The direction of the steepest ascent is the direction the increases most rapidly This direction, the path of steepest ascent, is normal to the fitted response surface The steps along the path are proportional to the regression coefficients .

The Method of Steepest Ascent Experiments are conducted along the path of the steepest ascent until no further increase in response is observed.

The Method of Steepest Ascent Then a new first order model is may be fit and a new path of the steepest ascent is determined and the procedure continues The experimenter will arrive at the vicinity of optimum indicated by lack of fit of a first order model. Additional experiments are conducted to obtain a more precise estimate of the optimum.

The Method of Steepest Ascent – Example 1-(1/15) A process to maximize process yield Two controllable variables: reaction time and reaction temperature

The Method of Steepest Ascent – Example 1-(2/15) The engineer is currently operating the process with a reaction time of 35 minutes and a temperature of 155∘F. This operating condition yields a bout 40 percent and is unlikely in the region of optimum.

The Method of Steepest Ascent – Example 1-(2/15) The engineer decides to explore the region for the first order model starting from(30, 40) for reaction time and (150, 160) reaction temperature. The variables are coded as ξis natural variable and is pronounced as xi

The Method of Steepest Ascent – Example 1-(3/15) Stat>DOE>Factorial>Create Factorial Design Number of factors  2 Designs>Number of center points per block  5 Factors  OK, OK

The Method of Steepest Ascent – Example 1-(3/15)

The Method of Steepest Ascent – Example 1-(3/15) Experiments are run in 22 factorial with 5 center points

The Method of Steepest Ascent – Example 1-(4/15) STAT>DOE>Analyze Factorial Design Response  yield Terms  2 OK Factorial Fit: Yield versus x2, x1 Estimated Effects and Coefficients for Yield (coded units) Term Effect Coef SE Coef T P Constant 40.4250 0.1037 389.89 0.000 x2 0.6500 0.3250 0.1037 3.13 0.035 x1 1.5500 0.7750 0.1037 7.47 0.002 x2*x1 -0.0500 -0.0250 0.1037 -0.24 0.821 Ct Pt 0.0350 0.1391 0.25 0.814 S = 0.207364 PRESS = * R-Sq = 94.27% R-Sq(pred) = *% R-Sq(adj) = 88.54%

The Method of Steepest Ascent – Example 1-(5/15) Analysis of Variance for Yield (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 2 2.82500 2.82500 1.41250 32.85 0.003 2-Way Interactions 1 0.00250 0.00250 0.00250 0.06 0.821 Curvature 1 0.00272 0.00272 0.00272 0.06 0.814 Residual Error 4 0.17200 0.17200 0.04300 Pure Error 4 0.17200 0.17200 0.04300 Total 8 3.00222

The Method of Steepest Ascent – Example 1-(6/15) Download Macros from website and save it Tools>Options>set Macros location OK

The Method of Steepest Ascent – Example 1-(6/15) Edit > Command Line editor  %ASCENT C7 C5-C6; STORE C9-C10; step 1; base C6; RUNS 13. Submit command

The Method of Steepest Ascent – Example 1-(7/15) Executing from file: C:\Program Files (x86)\Minitab 15\English\Macros\ASCENT.MAC Path of Steepest Ascent Overview Total # of Runs 13 Total # of Factors 2 Base Factor Name x2 Step Size Base Factor by 1.000 Coded Coefficient of Base Factor 0.325 Factor Name Coded Coef. Low Level High Level x2 0.325 150 160 x1 0.775 30 40

The Method of Steepest Ascent – Example 1-(8/15) Results X.1 X.2 155.00 35 157.10 40 159.19 45 161.29 50 163.39 55 165.48 60 167.58 65 169.68 70 171.77 75 173.87 80 175.97 85 178.06 90 180.16 95

The Method of Steepest Ascent – Example 1-(9/15)

The Method of Steepest Ascent – Example 1-(10/15)

The Method of Steepest Ascent – Example 1-(11/15) A new first order model is fit around (85, 175) Another 22 factorial design with 5 center points is performed

The Method of Steepest Ascent – Example 1-(12/15)

The Method of Steepest Ascent – Example 1-(13/15) Factorial Fit: Yield versus x2, x1 Estimated Effects and Coefficients for Yield (coded units) Term Effect Coef SE Coef T P Constant 77.7500 0.1151 675.45 0.000 x2 1.0000 0.5000 0.1151 4.34 0.012 x1 2.0000 1.0000 0.1151 8.69 0.001 x2*x1 0.5000 0.2500 0.1151 2.17 0.096 Ct Pt 2.1900 0.1544 14.18 0.000 S = 0.230217 PRESS = * R-Sq = 98.68% R-Sq(pred) = *% R-Sq(adj) = 97.37% Analysis of Variance for Yield (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 2 5.0000 5.0000 2.5000 47.17 0.002 2-Way Interactions 1 0.2500 0.2500 0.2500 4.72 0.096 Curvature 1 10.6580 10.6580 10.6580 201.09 0.000 Residual Error 4 0.2120 0.2120 0.0530 Pure Error 4 0.2120 0.2120 0.0530 Total 8 16.1200 The first order model maybe inappropriate!!

The Method of Steepest Ascent – Example 1-(14/15) General procedure for determining the coordinates of a point on the path of steepest ascent.

The Method of Steepest Ascent – Example 1-(15/15) In example 1, x1 has the largest regression coefficient, we select x1 as the variable in step 1 Suppose 5 minutes is selected as the step size, Then

Analysis of a Second-order Response Surface When the experimenter is relatively close to the optimum, a model that incorporates curvature is usually required to approximate the response. In most case, second order model is adequate

Analysis of a Second-order Response Surface –Location of Stationary point A point, if exists, will be the set of x1, x2, …, xk for which the partial derivatives is called stationary point. The stationary point could represent a point of maximum response, a point of minimum response, or a saddle point.

Analysis of a Second-order Response Surface –Location of Stationary point

Analysis of a Second-order Response Surface –Location of Stationary point

Analysis of a Second-order Response Surface –Location of Stationary point

Analysis of a Second-order Response Surface –Location of Stationary point The general mathematical solution First order Second order

Analysis of a Second-order Response Surface –Location of Stationary point Derivatives: The stationary point is

Analysis of a Second-order Response Surface –Location of Stationary point Once we have found the stationary point, it is usually necessary to characterize the response surface in the immediate vicinity of this point. That is, is the stationary point a max. or min. or saddle point? Canonical analysis, use saddle point as the new origin and rotate the coordinate system until they are parallel to the principal axes of the fitted response surface.

Analysis of a Second-order Response Surface –Location of Stationary point Canonical analysis, use saddle point as the new origin and rotate the coordinate system until they are parallel to the principal axes of the fitted response surface.

Analysis of a Second-order Response Surface –Location of Stationary point The fitted model is The above equation is in canonical form and λi is the eigenvalues of matrix B If all λi are positive  minimum If all λi are negative  maximum If λi have different signs  saddle point

Analysis of a Second-order Response Surface –Location of Stationary point The surface is steepest in the wi direction for which | λi | is the greatest For example, | λ1 |> | λ2 | and λi are negative. The response is maximum.

Analysis of a Second-order Response Surface– Example 2-(1/11) From example 1. A second order model cannot fit the data in Table 11.4 Four additional points are furnished(Axial points) DOE>Modify design>Axial points Two additional response are of interest.

Analysis of a Second-order Response Surface– Example 2-(2/11)

Analysis of a Second-order Response Surface– Example 2-(3/11)

Analysis of a Second-order Response Surface– Example 2-(4/11) Response Surface Regression: Yield versus x2, x1 The analysis was done using coded units. Estimated Regression Coefficients for Yield Term Coef SE Coef T P Constant 79.9400 0.11896 671.997 0.000 x2 0.5152 0.09405 5.478 0.001 x1 0.9950 0.09405 10.580 0.000 x2*x2 -1.0013 0.10085 -9.928 0.000 x1*x1 -1.3763 0.10085 -13.646 0.000 x2*x1 0.2500 0.13300 1.880 0.102 S = 0.266000 PRESS = 2.34577 R-Sq = 98.28% R-Sq(pred) = 91.84% R-Sq(adj) = 97.05%

Analysis of a Second-order Response Surface– Example 2-(5/11) Response Surface Regression: Yield versus x2, x1 The analysis was done using coded units. Analysis of Variance for Yield Source DF Seq SS Adj SS Adj MS F P Regression 5 28.2478 28.2478 5.64956 79.85 0.000 Linear 2 10.0430 10.0430 5.02148 70.97 0.000 Square 2 17.9548 17.9548 8.97741 126.88 0.000 Interaction 1 0.2500 0.2500 0.25000 3.53 0.102 Residual Error 7 0.4953 0.4953 0.07076 Lack-of-Fit 3 0.2833 0.2833 0.09443 1.78 0.290 Pure Error 4 0.2120 0.2120 0.05300 Total 12 28.7431

Analysis of a Second-order Response Surface– Example 2-(6/11) Response Surface Regression: Yield versus x2, x1 Estimated Regression Coefficients for Yield using data in uncoded units Term Coef Constant -1430.52 x2 13.2705 x1 7.80749 x2*x2 -0.0400500 x1*x1 -0.0550500 x2*x1 0.0100000

Analysis of a Second-order Response Surface– Example 2-(7/11) Contour

Analysis of a Second-order Response Surface– Example 2-(8/11)

Analysis of a Second-order Response Surface– Example 2-(9/11) Findings from above two plots This is a maximum problem Optimum is around 1750F and 85 minutes roughly The contour shows that it is more sensitive to reaction time than temperature One needs to optimize the surface DOE>Response Surface>Response Optimizer

Analysis of a Second-order Response Surface– Example 2-(10/11)

Analysis of a Second-order Response Surface– Example 2-(11/11) By adjusting the red line, one can find the convenient settings that meet your target.

Analysis of a Second-order Response Surface– Multiple Responses Many response surface problems involve the analysis of several responses. Simultaneous consideration of the multiple responses involves first building an appropriate response surface model for each response and then trying to find a set of operating conditions that in some sense optimizes all responses or at least keeps them in desired ranges.

Analysis of a Second-order Response Surface– Multiple Responses In Example 2, one may obtain models for viscosity and molecular weight responses.

Analysis of a Second-order Response Surface– Multiple Responses In coded units:

Analysis of a Second-order Response Surface– Multiple Responses Contour and surface plots: Viscosity

Analysis of a Second-order Response Surface– Multiple Responses Contour and surface plots: Weight

Analysis of a Second-order Response Surface– Multiple Responses Overlay the contour plots DOE>Response surface>overlay contour plot

Analysis of a Second-order Response Surface– Multiple Responses Overlay the contour plots If one wants the response y1>78.5, y2 between 62 and 68, and y3<3400. DOE> Response surface> overlay contour plot

Analysis of a Second-order Response Surface– Multiple Responses Overlay the contour plots

Analysis of a Second-order Response Surface– Multiple Responses The unshaded area is the portion that a number of combinations of time and temperature will result in a satisfactory process. The larger area could be the area to be investigated. Use constrained optimization to model the problem.

Analysis of a Second-order Response Surface– Multiple Responses Problem

Analysis of a Second-order Response Surface– Multiple Responses Or use response optimizer

Analysis of a Second-order Response Surface– Multiple Responses Desirability: convert each response yi into an individual desirability function di that varies over the range. And the design variables are chosen to maximize the overall desirability

Analysis of a Second-order Response Surface– Multiple Responses The individual desirability functions are structured as maximize

Analysis of a Second-order Response Surface– Multiple Responses minimize

Analysis of a Second-order Response Surface– Multiple Responses Between L and U

Analysis of a Second-order Response Surface– Multiple Responses

Analysis of a Second-order Response Surface– Multiple Responses Solution

Experimental Design for Fitting Response Surfaces Some features of a desirable design are as follow when selecting a response surface design: Provides a reasonable distribution of data points throughout the region of interest Allows model adequacy ,including lack of fit, to be investigated Allows experiments to be performed in blocks

Experimental Design for Fitting Response Surfaces Allows designs of higher order to be built up sequentially Provides an internal estimate of error Provides precise estimates of the models coefficient. Provides a food profile of the prediction variance throughout the experimental Provides reasonable robustness against outlier or missing values

Experimental Design for Fitting Response Surfaces Does not require a large number of runs Does not require too many levels of the independent variables Ensures simplicity of calculation of the model parameters

Experimental Design for Fitting Response Surfaces -- Design for fitting the first order model Suppose we wish to fit the first order model in k variables The orthogonal first order designs minimizes the variance of the regression coefficients, Including 2k factorial design and fractions of the 2k series with no main effect is aliased with other effects..

Experimental Design for Fitting Response Surfaces -- Design for fitting the first order model Another design is the simplex. That is, k+1 variables in k dimension.

Experimental Design for Fitting Response Surfaces -- Design for fitting the second order model Central Composite Design(CCD): 2k factorial (or fraction with resolution IV) with nF factorial runs, 2k axial or star runs, and nC center runs.

Experimental Design for Fitting Response Surfaces -- Design for fitting the second order model Two parameters to be determined: the distance α and the number of center point nC . To have good rotatability, To have sphere CCD Center points: two to five