1. Design and Analysis of Experiments
Dr. Tai-Yue Wang
Department of Industrial and Information Management
National Cheng Kung University
Tainan, TAIWAN, ROC
This is a basic course blah, blah, blah…

2. Response Surface Methodology
Dr. Tai-Yue Wang
Department of Industrial and Information Management
National Cheng Kung University
Tainan, TAIWAN, ROC
This is a basic course blah, blah, blah…

3. Outline Introduction The Method of Steepest Ascent
Analysis of A Second-order Response
Experimental Designs for Fitting Response Surfaces
Experiment with Computer Models
Mixture Experiments
Evolutionary Operation

4. Introduction
A collection of mathematical and statistical techniques useful for modeling and analysis of problems in which a response of interest is influenced by several variables.
The objective is to minimize this response.
Example: a chemical process is to maximize its yield(y) and its inputs, temperature(x1) and pressure(x2), that is,
where ε is the noise observed in the response, y

5. Introduction If we denote the expected response by Then
is called a Response Surface
The response surface is usually plotted graphically

6. Introduction
Response graph and its contour

7. Introduction
A polynomial function certainly will not work well for the entire surface
But it works fine in small region.
Steps in RSM
Find a suitable approximation for y = f(x) using a low order polynomial
Move towards the region of the optimum
When curvature is found find a new approximation for y = f(x) {generally a higher order polynomial} and perform the “Response Surface Analysis”

8. Introduction

9. Introduction
RSM dates from the 1950s; early applications in chemical industry
Modern applications of RSM span many industrial and business settings

10. The Method of Steepest Ascent
Generally the initial estimate of the optimum operating conditions for the system will be far from the actual optimum
 to move to general vicinity of the optimum rapidly
When we are remote from the optimum, we usually assume that first order model is an adequate approximation to the true surface in a small region of the x’s

11. The Method of Steepest Ascent
The method of steepest ascent is a procedure for moving sequentially in the direction of the maximum increase in the response.
If the minimization is desired, it is called the method of steepest descent.
The fitted first order model is

12. The Method of Steepest Ascent
The first order response surface, the contours of , is a series of parallel lines.
The direction of the steepest ascent is the direction the increases most rapidly
This direction, the path of steepest ascent, is normal to the fitted response surface
The steps along the path are proportional to the regression coefficients

13. The Method of Steepest Ascent
Experiments are conducted along the path of the steepest ascent until no further increase in response is observed.

14. The Method of Steepest Ascent
Then a new first order model is may be fit and a new path of the steepest ascent is determined and the procedure continues
The experimenter will arrive at the vicinity of optimum indicated by lack of fit of a first order model.
Additional experiments are conducted to obtain a more precise estimate of the optimum.

15. The Method of Steepest Ascent – Example 1-(1/15)
A process to maximize process yield
Two controllable variables: reaction time and reaction temperature

16. The Method of Steepest Ascent – Example 1-(2/15)
The engineer is currently operating the process with a reaction time of 35 minutes and a temperature of 155∘F. This operating condition yields a bout 40 percent and is unlikely in the region of optimum.

17. The Method of Steepest Ascent – Example 1-(2/15)
The engineer decides to explore the region for the first order model starting from(30, 40) for reaction time and (150, 160) reaction temperature.
The variables are coded as
ξis natural variable and is pronounced as xi

18. The Method of Steepest Ascent – Example 1-(3/15)
Stat>DOE>Factorial>Create Factorial Design
Number of factors  2
Designs>Number of center points per block  5
Factors 
OK, OK

19. The Method of Steepest Ascent – Example 1-(3/15)

20. The Method of Steepest Ascent – Example 1-(3/15)
Experiments are run in 22 factorial with 5 center points

21. The Method of Steepest Ascent – Example 1-(4/15)
STAT>DOE>Analyze Factorial Design
Response  yield
Terms  2 OK
Factorial Fit: Yield versus x2, x1
Estimated Effects and Coefficients for Yield (coded units)
Term Effect Coef SE Coef T P
Constant x x
x2*x
Ct Pt
S = PRESS = *
R-Sq = 94.27% R-Sq(pred) = *% R-Sq(adj) = 88.54%

22. The Method of Steepest Ascent – Example 1-(5/15)
Analysis of Variance for Yield (coded units)
Source DF Seq SS Adj SS Adj MS F P
Main Effects
2-Way Interactions
Curvature
Residual Error
Pure Error
Total

23. The Method of Steepest Ascent – Example 1-(6/15)
Download Macros from website and save it
Tools>Options>set Macros location OK

24. The Method of Steepest Ascent – Example 1-(6/15)
Edit > Command Line editor
 %ASCENT C7 C5-C6;
STORE C9-C10;
step 1;
base C6;
RUNS 13.
Submit command

25. The Method of Steepest Ascent – Example 1-(7/15)
Executing from file: C:\Program Files (x86)\Minitab 15\English\Macros\ASCENT.MAC Path of Steepest Ascent Overview Total # of Runs 13 Total # of Factors 2 Base Factor Name x2 Step Size Base Factor by Coded Coefficient of Base Factor Factor Name Coded Coef. Low Level High Level x x

26. The Method of Steepest Ascent – Example 1-(8/15)
Results
X.1 X.2

27. The Method of Steepest Ascent – Example 1-(9/15)

28. The Method of Steepest Ascent – Example 1-(10/15)

29. The Method of Steepest Ascent – Example 1-(11/15)
A new first order model is fit around (85, 175)
Another 22 factorial design with 5 center points is performed

30. The Method of Steepest Ascent – Example 1-(12/15)

31. The Method of Steepest Ascent – Example 1-(13/15)
Factorial Fit: Yield versus x2, x1
Estimated Effects and Coefficients for Yield (coded units)
Term Effect Coef SE Coef T P
Constant x x
x2*x
Ct Pt
S = PRESS = *
R-Sq = 98.68% R-Sq(pred) = *% R-Sq(adj) = 97.37%
Analysis of Variance for Yield (coded units)
Source DF Seq SS Adj SS Adj MS F P
Main Effects
2-Way Interactions
Curvature
Residual Error
Pure Error
Total
The first order model maybe inappropriate!!

32. The Method of Steepest Ascent – Example 1-(14/15)
General procedure for determining the coordinates of a point on the path of steepest ascent.

33. The Method of Steepest Ascent – Example 1-(15/15)
In example 1, x1 has the largest regression coefficient, we select x1 as the variable in step 1
Suppose 5 minutes is selected as the step size,
Then

34. Analysis of a Second-order Response Surface
When the experimenter is relatively close to the optimum, a model that incorporates curvature is usually required to approximate the response.
In most case, second order model is adequate

35. Analysis of a Second-order Response Surface –Location of Stationary point
A point, if exists, will be the set of x1, x2, …, xk for which the partial derivatives
is called stationary point.
The stationary point could represent a point of maximum response, a point of minimum response, or a saddle point.

36. Analysis of a Second-order Response Surface –Location of Stationary point

37. Analysis of a Second-order Response Surface –Location of Stationary point

38. Analysis of a Second-order Response Surface –Location of Stationary point

39. Analysis of a Second-order Response Surface –Location of Stationary point
The general mathematical solution
First order
Second order

40. Analysis of a Second-order Response Surface –Location of Stationary point
Derivatives:
The stationary point is

41. Analysis of a Second-order Response Surface –Location of Stationary point
Once we have found the stationary point, it is usually necessary to characterize the response surface in the immediate vicinity of this point.
That is, is the stationary point a max. or min. or saddle point?
Canonical analysis, use saddle point as the new origin and rotate the coordinate system until they are parallel to the principal axes of the fitted response surface.

42. Analysis of a Second-order Response Surface –Location of Stationary point
Canonical analysis, use saddle point as the new origin and rotate the coordinate system until they are parallel to the principal axes of the fitted response surface.

43. Analysis of a Second-order Response Surface –Location of Stationary point
The fitted model is
The above equation is in canonical form and λi is the eigenvalues of matrix B
If all λi are positive  minimum
If all λi are negative  maximum
If λi have different signs  saddle point

44. Analysis of a Second-order Response Surface –Location of Stationary point
The surface is steepest in the wi direction for which | λi | is the greatest
For example, | λ1 |> | λ2 | and λi are negative. The response is maximum.

45. Analysis of a Second-order Response Surface– Example 2-(1/11)
From example 1.
A second order model cannot fit the data in Table 11.4
Four additional points are furnished(Axial points) DOE>Modify design>Axial points
Two additional response are of interest.

46. Analysis of a Second-order Response Surface– Example 2-(2/11)

47. Analysis of a Second-order Response Surface– Example 2-(3/11)

48. Analysis of a Second-order Response Surface– Example 2-(4/11)
Response Surface Regression: Yield versus x2, x1
The analysis was done using coded units.
Estimated Regression Coefficients for Yield
Term Coef SE Coef T P
Constant x x
x2*x
x1*x
x2*x
S = PRESS =
R-Sq = 98.28% R-Sq(pred) = 91.84% R-Sq(adj) = 97.05%

49. Analysis of a Second-order Response Surface– Example 2-(5/11)
Response Surface Regression: Yield versus x2, x1
The analysis was done using coded units.
Analysis of Variance for Yield
Source DF Seq SS Adj SS Adj MS F P
Regression
Linear
Square
Interaction
Residual Error
Lack-of-Fit
Pure Error
Total

50. Analysis of a Second-order Response Surface– Example 2-(6/11)
Response Surface Regression: Yield versus x2, x1
Estimated Regression Coefficients for Yield using data in uncoded units
Term Coef
Constant x x
x2*x
x1*x
x2*x

51. Analysis of a Second-order Response Surface– Example 2-(7/11)
Contour

52. Analysis of a Second-order Response Surface– Example 2-(8/11)

53. Analysis of a Second-order Response Surface– Example 2-(9/11)
Findings from above two plots
This is a maximum problem
Optimum is around 1750F and 85 minutes roughly
The contour shows that it is more sensitive to reaction time than temperature
One needs to optimize the surface
DOE>Response Surface>Response Optimizer

54. Analysis of a Second-order Response Surface– Example 2-(10/11)

55. Analysis of a Second-order Response Surface– Example 2-(11/11)
By adjusting the red line, one can find the convenient settings that meet your target.

56. Analysis of a Second-order Response Surface– Multiple Responses
Many response surface problems involve the analysis of several responses.
Simultaneous consideration of the multiple responses involves first building an appropriate response surface model for each response and then trying to find a set of operating conditions that in some sense optimizes all responses or at least keeps them in desired ranges.

57. Analysis of a Second-order Response Surface– Multiple Responses
In Example 2, one may obtain models for viscosity and molecular weight responses.

58. Analysis of a Second-order Response Surface– Multiple Responses
In coded units:

59. Analysis of a Second-order Response Surface– Multiple Responses
Contour and surface plots: Viscosity

60. Analysis of a Second-order Response Surface– Multiple Responses
Contour and surface plots: Weight

61. Analysis of a Second-order Response Surface– Multiple Responses
Overlay the contour plots
DOE>Response surface>overlay contour plot

62. Analysis of a Second-order Response Surface– Multiple Responses
Overlay the contour plots
If one wants the response y1>78.5, y2 between 62 and 68, and y3<3400.
DOE>
Response surface>
overlay contour plot

63. Analysis of a Second-order Response Surface– Multiple Responses
Overlay the contour plots

64. Analysis of a Second-order Response Surface– Multiple Responses
The unshaded area is the portion that a number of combinations of time and temperature will result in a satisfactory process.
The larger area could be the area to be investigated.
Use constrained optimization to model the problem.

65. Analysis of a Second-order Response Surface– Multiple Responses
Problem

66. Analysis of a Second-order Response Surface– Multiple Responses
Or use response optimizer

67. Analysis of a Second-order Response Surface– Multiple Responses
Desirability: convert each response yi into an individual desirability function di that varies over the range.
And the design variables are chosen to maximize the overall desirability

68. Analysis of a Second-order Response Surface– Multiple Responses
The individual desirability functions are structured as
maximize

69. Analysis of a Second-order Response Surface– Multiple Responses
minimize

70. Analysis of a Second-order Response Surface– Multiple Responses
Between L and U

71. Analysis of a Second-order Response Surface– Multiple Responses

72. Analysis of a Second-order Response Surface– Multiple Responses
Solution

73. Experimental Design for Fitting Response Surfaces
Some features of a desirable design are as follow when selecting a response surface design:
Provides a reasonable distribution of data points throughout the region of interest
Allows model adequacy ,including lack of fit, to be investigated
Allows experiments to be performed in blocks

74. Experimental Design for Fitting Response Surfaces
Allows designs of higher order to be built up sequentially
Provides an internal estimate of error
Provides precise estimates of the models coefficient.
Provides a food profile of the prediction variance throughout the experimental
Provides reasonable robustness against outlier or missing values

75. Experimental Design for Fitting Response Surfaces
Does not require a large number of runs
Does not require too many levels of the independent variables
Ensures simplicity of calculation of the model parameters

76. Experimental Design for Fitting Response Surfaces -- Design for fitting the first order model
Suppose we wish to fit the first order model in k variables
The orthogonal first order designs minimizes the variance of the regression coefficients,
Including 2k factorial design and fractions of the 2k series with no main effect is aliased with other effects..

77. Experimental Design for Fitting Response Surfaces -- Design for fitting the first order model
Another design is the simplex. That is, k+1 variables in k dimension.

78. Experimental Design for Fitting Response Surfaces -- Design for fitting the second order model
Central Composite Design(CCD): 2k factorial (or fraction with resolution IV) with nF factorial runs, 2k axial or star runs, and nC center runs.

79. Experimental Design for Fitting Response Surfaces -- Design for fitting the second order model
Two parameters to be determined: the distance α and the number of center point nC .
To have good rotatability,
To have sphere CCD
Center points: two to five