1. / department of mathematics and computer science 1212 1 2DS01 Statistics 2 for Chemical Engineering lecture 3 http://www.win.tue.nl/~sandro/2DS01

2. / department of mathematics and computer science 1212 2 Contents Optimisation steps Box method Steepest ascent method Practical example Response surface designs Multiple responses EVOP Software Literature

3. / department of mathematics and computer science 1212 3 Optimisation steps Optimisation is achieved by going through the following phases: screening (which factors are of importance) improvement (approach optimum as fast as possible: steepest ascent/ simplex) determination of optimum (response surface designs)

4. / department of mathematics and computer science 1212 4 current settings improvement optimum

5. / department of mathematics and computer science 1212 5 Models Far away from the optimum a first order model often suffices. for example: Y = ß 0 + ß 1 x 1 + ß 2 x 2 +  Near the optimum often a quadratic (second order) model suffices. For example: Y = ß 0 + ß 1 x 1 + ß 2 x 2 + ß 12 x 1 x 2 + ß 11 x 1 2 + ß 22 x 2 2 + 

6. / department of mathematics and computer science 1212 6 Models Far away from optimum: first order model

7. / department of mathematics and computer science 1212 7 Models Near optimum: fitting a first order model shows lack-of- fit (curvature)

8. / department of mathematics and computer science 1212 8 Models Near optimum: second order model

9. / department of mathematics and computer science 1212 9 Improvement In order to quickly find factor settings which yield near- optimal values, 2 methods are available: Box method (simple) Steepest ascent/descent method (advanced)

10. / department of mathematics and computer science 1212 10 Box method 40.641.9 41.8 41.2 41.3 39.340.9 41.5 40.0 direction of largest increase direction of largest increase stop if one has to return to previous settings

11. / department of mathematics and computer science 1212 11 Steepest ascent method direction of steepest ascent contour lines of first order model perpendicular to contour line region where 1 e order-model has been determined

12. / department of mathematics and computer science 1212 12 Practical example goal: maximise yield of chemical reactor significant factors: reaction time reaction temperature current factor setting: time = 35 min. temp = 155 °C current yield: 40 %

13. / department of mathematics and computer science 1212 13 Steepest ascent 2 2 -design with 5 centre points: time: 30 - 40 min; temp: 150 - 160 °C results: montgomery14-1.sfxmontgomery14-1.sfx there is no significant interaction there is no significant lack-of-fit the regression model is significant Hence, we are not near the optimum.

14. / department of mathematics and computer science 1212 14 Steepest ascent path outcome analysis of measurement: yield = 24.94 + 0.155*time + 0.065*temp with coding: x 1 = (time-35)/5 x 2 = (temp-155)/5 yield = 40.44 + 0.775*x 1 + 0.325*x 2 direction path: normal vector step size: 5 min reaction time (choice of chemical engineer!) coded step size temp (= 2.1 °C)

15. / department of mathematics and computer science 1212 15 Steepest ascent path experiments Further experiments with factor settings of experiment nr. 10.

16. / department of mathematics and computer science 1212 16 Settings experiment 10: time = 85 min temperature = 175 °C A 2 2 design with 5 centre points is executed. results: montgomery14-4.sfxmontgomery14-4.sfx Now we are probably near the optimum.

17. / department of mathematics and computer science 1212 17 Quadratic models In order to fit a quadratic model, we must vary the factors at 3 levels. A 2 p -design with centre points does not suffice,because then all quadratic factors are confounded. A 3 p -design is possible, but not to be recommended: number of runs grows fast uses more runs than necessary to fit quadratic model.

18. / department of mathematics and computer science 1212 18 Response surface designs The following designs are widely used for fitting a quadratic model: Central Composite Design Box-Behnken Design However, there are other suitable designs.

19. / department of mathematics and computer science 1212 19 Central Composite Design A CCD consists of 3 parts: factorial points axial points centre points A CCD is often executed by adding points to an already performed 2 p -design (but beware of blocking!).

20. / department of mathematics and computer science 1212 20 Rotatability In a CCD there are 2 possible choices: number of centre points location axial points By choosing the axial points at the locations ( ,0,…,0) etc. with  = (# factorial points) ¼, the design becomes rotatable, i.e. the precision (variance) of the model depends on the distance to the origin only.

21. / department of mathematics and computer science 1212 21 Box-Behnken designs These are designs that consists of combinations from 2 p -designs. Properties: efficient (few runs) (almost) rotatable no corner points of hypercube (these are often extreme conditions and hence often hard to set)

22. / department of mathematics and computer science 1212 22 Stationary point Near the optimum usually a quadratic model suffices: How do find the optimum after we correctly estimated the parameters?

23. / department of mathematics and computer science 1212 23 One-dimensional case necessary condition for extremum: 1 st derivative = 0 not sufficient: “point of inflection” extra sufficient condition: 2 nd derivative  0

24. / department of mathematics and computer science 1212 24 Saddlepoint vs. maximum saddle point maximum

25. / department of mathematics and computer science 1212 25 Analysis response surface model Graphically: make contourplot (if 2 factors) Analytically: matrix notation: Note: B must be chosen as symmetric matrix, see example:

26. / department of mathematics and computer science 1212 26 Stationarity and matrix analysis stationary point: characterisation through eigenvalues of matrix B: all eigenvalues positive: min all eigenvalues negative: max eigenvalues different signs: saddle point (the ’s are sometimes called “parameters of canonical form”)

27. / department of mathematics and computer science 1212 27 Stationarity and matrix analysis In StatGraphics: augment design add Star points Please note that additional centre points are added and a block variable. We can remove the centre points from the data set and ignore the block variable in the analysis. StatGraphics results: montgomery14-6.sfxmontgomery14-6.sfx

28. / department of mathematics and computer science 1212 28 Stationarity and matrix analysis In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> B = [-2.75247 0.5/2 ; 0.5/2 -2.00253] / 2 Analysis Summary ---------------- File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfx Comment: Montgomery table 14-4 Estimated effects for opbrengst ---------------------------------------------------------------------- average = 79.94 +/- 0.11896 A:tijd = 1.98994 +/- 0.188092 B:temperatuur = 1.03033 +/- 0.188093 AA = -2.75247 +/- 0.201705 AB = 0.5 +/- 0.266003 BB = -2.00253 +/- 0.20171 ---------------------------------------------------------------------- Standard errors are based on total error with 7 d.f.

29. / department of mathematics and computer science 1212 29 Stationarity and matrix analysis In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> B = [-2.75247 0.5/2 ; 0.5/2 -2.00253] / 2 B = -1.3762 0.1250 0.1250 -1.0013

30. / department of mathematics and computer science 1212 30 Stationarity and matrix analysis In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> eig(B)

31. / department of mathematics and computer science 1212 31 Stationarity and matrix analysis In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> eig(B) ans = -1.4141 -0.9634 both negative → maximum

32. / department of mathematics and computer science 1212 32 Stationarity and matrix analysis In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> b = [1.98994 ; 1.03033] /2 Analysis Summary ---------------- File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfx Comment: Montgomery table 14-4 Estimated effects for opbrengst ---------------------------------------------------------------------- average = 79.94 +/- 0.11896 A:tijd = 1.98994 +/- 0.188092 B:temperatuur = 1.03033 +/- 0.188093 AA = -2.75247 +/- 0.201705 AB = 0.5 +/- 0.266003 BB = -2.00253 +/- 0.20171 ---------------------------------------------------------------------- Standard errors are based on total error with 7 d.f.

33. / department of mathematics and computer science 1212 33 Stationarity and matrix analysis In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> b = [1.98994 ; 1.03033] /2 b = 0.9950 0.5152

34. / department of mathematics and computer science 1212 34 Stationarity and matrix analysis In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> spcoded = -0.5 * inv(B) * b

35. / department of mathematics and computer science 1212 35 Stationarity and matrix analysis In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> spcoded = -0.5 * inv(B) * b spcoded = 0.3893 0.3059 < 1.414 (distance star point) → inside experimental region

36. / department of mathematics and computer science 1212 36 In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> sporiginal = spcoded.* [5 ; 5] + [85 ; 175] Stationarity and matrix analysis

37. / department of mathematics and computer science 1212 37 Stationarity and matrix analysis In Matlab: create matrix B and vector b compute eigenvalues and location of stationary point >> sporiginal = spcoded.* [5 ; 5] + [85 ; 175] sporiginal = 86.9463 176.5293

38. / department of mathematics and computer science 1212 38 start screening RSM design (CCD,...) single observation in direction steepest ascent full factorial + centre points 1 st order model OK? better observation? stationary point optimum? stationary point nearby? go to stationary point yes no yes no yes no accept stationary point end fit 2 nd order model yes Optimization scheme

39. / department of mathematics and computer science 1212 39 Multiple responses If more than 1 response variable needs to be optimised, then a graphical way of optimising may be achieved by overlaying contour plots in case there are only 2 independent variables.

40. / department of mathematics and computer science 1212 40 Evolutionary Operation (EVOP) Optimisation of a running production process is costly: involves interruption may (temporarily) yield low quality products An alternative is Evolutionary Operation: frequent execution of 2 k -designs high and low setting of factors are close to each other

41. / department of mathematics and computer science 1212 41 Software StatLab optimisation: http://www.win.tue.nl/statlab Interactive software for teaching DOE through cases http://www.win.tue.nl/statlab Box: http://www.win.tue.nl/~marko/box/box.html Game-like demonstration of Box methodhttp://www.win.tue.nl/~marko/box/box.html Matlab virtual reactor: Statistics toolbox -> Demos -> Empirical Modeling - > RSM demo Better beer brewery DOE simulation (with extra chemical information): http://www.moresteam.com/betterbeer.cfmextra chemical information http://www.moresteam.com/betterbeer.cfm Statgraphics: menu choice Special -> Experimental Design –design experiment with pre-defined catalogue –analysis of experiments with ANOVA

42. / department of mathematics and computer science 1212 42 Literature J. Trygg and S. Wold. Introduction to Experimental Design – What is it? Why and Where is it Useful?, Homepage of Chemometrics, editorial August 2002: http://www.acc.umu.se/~tnkjtg/Chemometrics/editorial/aug20 02.html http://www.acc.umu.se/~tnkjtg/Chemometrics/editorial/aug20 02.html StatSoft Electronic Statistics Textbook, chapter on experimental designchapter on experimental design NIST Engineering Statistics Handbook: http://www.itl.nist.gov/div898/handbook/ http://www.itl.nist.gov/div898/handbook/