1. DOE: Optimization Response Surface Methods
6BV04
DOE: Optimization
Response Surface Methods

2. Contents Optimisation steps Box method Steepest ascent method
Practical example
Response surface designs
Multiple responses
EVOP
Software
Literature

3. Optimisation steps
Optimisation is achieved by going through the following phases:
screening (determine which factors really influence the outcome; tool: screening designs like fractional factorial)
improvement (approach optimum by repeated change of factor settings; tools: Box/simplex or steepest ascent approach)
determination of optimum (find optimal settings of factor settings; tool: response surface designs like CCD or Box-Behnken + analysis of response surface using eigenvalues)

4. optimum
improvement
current settings

5. Regression models used in optimisation
Statistical techniques for optimisation assume the following (often reasonably satisfied in practice):
“Far away” from the optimum a first order model often suffices. for example:
Y = ß0 + ß1x1 + ß2x2 + 
“Near” the optimum often a quadratic (second order) model suffices. For example:
Y = ß0 + ß1x1 + ß2x2 + ß12x1x2 + ß11x12 + ß22x22 +
Lack-of-fit techniques must be applied in order to check whether these models are appropriate, since we cannot directly see whether we are near the optimum (cf. next slides).

6. Models
Far away from optimum:
first order model

7. fitting a first order model
Models
Near optimum:
fitting a first order model
shows lack-of-fit
(curvature)

8. Models
Near optimum:
second order model

9. Improvement
In order to efficiently move from current factor settings to factor setting that yield near-optimal values, 2 methods are available:
Box/Simplex method
idea: form new full factorials in direction of largest increase in current full factorial
simple; no statistics needed for implementation
not efficient
Steepest ascent/descent method
idea: use 1st order regression model from fractional factorial to obtain direction of largest increase (“steepest ascent”)
perform single runs in direction of largest increase until increase stops
advanced
recommended since it is more efficient

10. Box method direction of largest increase 41.2 41.3 40.6 41.9 41.8
39.3
40.9
41.5
40.0
stop if one has to return to previous settings

11. Steepest ascent method
perpendicular
to contour line
direction of
steepest ascent
contour lines of
first-order model
region where
1eorder-model
has been determined

12. full factorial + centre points
Optimization scheme
start
end
screening
accept
stationary point no
yes
stationary
point
optimum?
stationary
point
nearby?
1st order
model OK? no
full factorial + centre points
yes
RSM design
(CCD, ...)
fit 2nd order model
Bij Statlab: als stationair punt te ver weg is, opnieuw 1e orde model fitten, dus geen 2e orde model bij het geschatte optimum.
yes no
single observation
in direction
steepest ascent
go to
stationary point
yes no
better
observation?

13. Practical example goal: maximise yield of chemical reactor
significant factors obtained after screening experiment:
reaction time
reaction temperature
current factor setting: time = 35 min. temp = 155 °C
current yield: 40 %

14. Steepest ascent 22-design with 5 centre points:
time: min; temp: °C
results: montgomery14-1.sfx
there is no significant interaction
there is no significant lack-of-fit
the regression model is significant
Hence, we are not near the optimum.
Interactie plot maken
Lack of fit toets doen

15. Steepest ascent path direction path: normal vector
outcome analysis of measurement:
yield = *time *temp
with coding:
x1= (time-35)/5 x2 = (temp-155)/5
yield = *x *x2
direction path: normal vector
De vector van steepest ascent is altijd de vector van regressie-coefficienten.
Laat zien dat Statgraphics ook via table options het path van steepest ascent kan laten zien!!!
step size: 5 min reaction time (choice of chemical engineer!)
coded step size temp (= 2.1°C)

16. Steepest ascent path experiments
Further experiments with factor settings of experiment nr. 10.

17. Near the optimum Settings experiment 10: time = 85 min
temperature = 175 °C
A 22 design with 5 centre points is executed.
results: montgomery14-4.sfx
Lack-of-fit indicates curvature. Hence, we now are probably near the optimum.

18. Quadratic models
In order to fit a quadratic model (suitable when we are near the optimum), we must vary the factors at 3 levels.
A 2p-design with centre points does not suffice, because then all quadratic factors are confounded.
A 3p-design is possible, but not to be recommended:
number of runs grows fast
uses more runs than necessary to fit quadratic model.

19. Response surface designs
The following designs are widely used for fitting a quadratic model:
Central Composite Design (uniform precision of effect estimates)
Box-Behnken Design (almost uniform precision of effect estimates, but usually fewer runs required than for CCD)
The choice between these models is usually decided by the availability of these designs for a given number of runs and number of factors.
Note that there are other suitable designs (usually available in statistical software that supports DOE).

20. Central Composite Design
A CCD consists of 3 parts:
factorial points
centre points
axial points
A CCD is often executed by adding
points to an already performed
2p-design (highly efficient, but beware
of blocking!).

21. Rotatability In a CCD there are 2 possible choices:
number of centre points
location axial points
By choosing the axial points at the locations (,0,…,0) etc. with  = (# factorial points)¼ , the design becomes rotatable, i.e. the precision (variance) of the model depends on the distance to the origin only. In other words, one has the same precision for all factor estimates.

22. Box-Behnken designs These are designs that consists of
combinations from 2p-designs.
Properties:
efficient (few runs)
(almost) rotatable
no corner points of hypercube (these are extreme conditions which are often hard to set)

23. Stationary point Near the optimum usually a quadratic model suffices:
How do find the optimum after we correctly estimated the parameters using a response surface design (CCD or Box-Behnken)?
The next slides show the tools to derive optimal settings and the pitfalls that have to be avoided.

24. Recap: optimisation in dimension1
necessary condition for extremum: 1st derivative = 0
not sufficient: “point of inflection”
extra sufficient condition: 2nd derivative  0

25. Zero first derivatives: saddlepoint vs. maximum
maximum (favourable)
saddle point (unfavourable)

26. Determination of type of optimum
Graphically: make contourplot (if 2 factors)
Analytically:
matrix notation:
Note: B must be chosen
as symmetric matrix,
see example:

27. Stationarity and matrix analysis
stationary point (zero first-order derivatives):
characterisation through eigenvalues of matrix B:
all eigenvalues positive: min
all eigenvalues negative: max
eigenvalues different signs: saddle point
(the l’s are sometimes called “parameters of canonical form”)

28. Stationarity and matrix analysis
In StatGraphics:
augment design
add star points
Please note that additional centre points are added and a block variable.
We can remove the centre points from the data set and ignore the block variable in the analysis.
StatGraphics results: montgomery14-6.sfx

29. Stationarity and matrix analysis
Use Matlab to avoid manual computations:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> B = [ /2 ; 0.5/ ] / 2
Analysis Summary
File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfx
Comment: Montgomery table 14-4
Estimated effects for opbrengst
average = /
A:tijd = /
B:temperatuur = /
AA = /
AB = /
BB = /
Standard errors are based on total error with 7 d.f.

30. Stationarity and matrix analysis
In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> B = [ /2 ; 0.5/ ] / 2
B =

31. Stationarity and matrix analysis
In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> eig(B)

32. Stationarity and matrix analysis
In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> eig(B)
ans =
both negative → maximum

33. Stationarity and matrix analysis
In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> b = [ ; ] /2
Analysis Summary
File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfx
Comment: Montgomery table 14-4
Estimated effects for opbrengst
average = /
A:tijd = /
B:temperatuur = /
AA = /
AB = /
BB = /
Standard errors are based on total error with 7 d.f.

34. Stationarity and matrix analysis
In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> b = [ ; ] /2
b =
0.9950
0.5152

35. Stationarity and matrix analysis
In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> spcoded = -0.5 * inv(B) * b

36. Stationarity and matrix analysis
In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> spcoded = -0.5 * inv(B) * b
spcoded =
0.3893
0.3059
< (distance star point)
→ inside experimental region

37. Stationarity and matrix analysis
In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> sporiginal = spcoded .* [5 ; 5] + [85 ; 175]

38. Stationarity and matrix analysis
In Matlab:
create matrix B and vector b
compute eigenvalues and location of stationary point
>> sporiginal = spcoded .* [5 ; 5] + [85 ; 175]
sporiginal =

39. full factorial + centre points
Optimization scheme
start
end
screening
accept
stationary point no
yes
stationary
point
optimum?
stationary
point
nearby?
1st order
model OK? no
full factorial + centre points
yes
RSM design
(CCD, ...)
fit 2nd order model
Bij Statlab: als stationair punt te ver weg is, opnieuw 1e orde model fitten, dus geen 2e orde model bij het geschatte optimum.
yes no
single observation
in direction
steepest ascent
go to
stationary point
yes no
better
observation?

40. Multiple responses
If more than 1 response variable needs to be optimised, then a graphical way of optimising may be achieved by overlaying contour plots in case there are only 2 independent variables.

41. Evolutionary Operation (EVOP)
Optimisation of a running production process is not always possible or may not be allowed because of costs:
involves interruption
may (temporarily) yield low quality products
An alternative is Evolutionary Operation:
experimentation within running operation
frequent execution of 2k-designs, starting at current settings
high and low setting of factors are close to each other, thus no risk of low quality products

42. Software design experiment with pre-defined catalogue
StatLab optimisation: Interactive software for teaching DOE through cases
Box: Game-like demonstration of Box method
Matlab virtual reactor: Statistics toolbox -> Demos -> Empirical Modeling -> RSM demo
Statgraphics: menu choice Special -> Experimental Design
design experiment with pre-defined catalogue
analysis of experiments with ANOVA

43. Literature
J. Trygg and S. Wold. Introduction to Experimental Design – What is it? Why and Where is it Useful?, Homepage of Chemometrics, editorial August 2002:
DOE booklet from Umetrics:
Introduction to DOE from moresteam.com
StatSoft Electronic Statistics Textbook, chapter on experimental design
NIST Engineering Statistics Handbook: