1. Rethinking Steepest Ascent for Multiple Response Applications Robert W. Mee Jihua Xiao University of Tennessee

2. Outline Overview of RSM Strategy Steepest Ascent for an Example Efficient Frontier Plots Paths of Improvement (POI) Regions

3. Sequential RSM Strategy Box and Wilson (JRSS-B, 1951) 1. Initial design to estimate linear main effects 2. Exploration along path of steepest ascent 3. Repeat step 1 in new optimal location – If main effects are still dominant, repeat step 2; if not, go to step 4 4. Augment to complete a 2nd-order design 5. Optimization based on fitted second-order model

4. Multiple Responses RSM Literature Del Castillo (JQT 1996), "Multiresponse Optimization…” Construct confidence cones for path of steepest ascent (i.e., maximum improvement) for each response – Use very large 1-  for responses of secondary importance, e.g. 99%-99.9% confidence – Use 95%-99% confidence for more critical responses Identify directions x falling inside every confidence cone If no such x exists, choose a convex combination of the paths of steepest ascent, giving greater weight for responses that are well estimated – Constrain the solution to reside inside the confidence cones for the most critical responses.

5. Multiple Responses RSM Literature Desirability Functions (Derringer and Suich, JQT 1980) Score each response with a function between 0 and 1. The geometric mean of the scores is the overall desirability Recent enhancements use score functions that are “smooth” (i.e., differentiable).

6. An Example with Multiple Responses Vindevogel and Sandra (Analytical Chem. 1991) 2 5-2 fractional factorial design using micellar electrokinetic chromatography Higher surfactant levels required to separate two of four esters, but this increases the analysis time Response variables include: – Resolution for separation of 2 nd and 3 rd testosterone esters – Time for process, t IV – Four other responses of lesser importance

8. Reaction Time vs. Reaction Rate Rate = 1 / Time

9. Fitted First-Order Models for Resolution and Reaction Rate Good news: Both models have R 2 > 99% Bad news: Improvement for resolution and rate point in opposite directions – Authors recommend a compromise: – Lower x 1 (pH) and x 5 (buffer) to increase rate – Lower x 2 (SHS%) and x 3 (Acet.) and increase x 4 (surfactant) to increase resolution.

10. What about Modeling Desirability? First-order model for Desirability

11. What we just tried was a bad idea! Even when first-order models fit each response well, the desirability function for two or more responses will require a more complicated model Following an initial two-level design, one cannot model desirability directly. It is better to maximize desirability based on predicted response values from simple models for each response

12. Maximizing Predicted Desirability for the Vindevogel and Sandra Example JMP’s default finds the maximum within a hypercube This does not identify a useful path for exploration

13. Software Should Maximize Desirability Within a Hypersphere

14. Confidence Cone for Path of Steepest Ascent (Box and Draper) Define  b , the angle between least squares estimator b and true coefficient vector  Pivotal quantity: Upper confidence bound for sin 2  b   Assuming  b  < 90 o,

15. 95% Confidence Cone for Paths of Steepest Ascent for Resolution & Rate 95% Confidence Cones for Paths of Steepest Ascent – Resolution (Y 1 ):  b  < 14.4 o – Rate (Y 2 ):  b  < 32.7 o These confidence cones do not overlap, since the angle between b Resolution and b Rate is 141.5 o ! What compromise is best?

16. Efficient Frontier Notation J larger-the-better response variables First-order model in k factors for each response Notation – b j : vector of least squares estimates for j th response – T j : corresponding vector of t statistics for j th response Convex combinations for two responses – For 0≤c≤1: x C =(1-c)T 1 + cT 2

17. Efficient Frontier for Two Responses Let x N denote a vector that is not a convex combination of T 1 and T 2 There exists a convex combination x C, with |x C | = |x N |, such that x C ‘b j ≥ x N ‘b j (j = 1,2) Proof by contradiction. I.e., suppose not. Then So one only need consider convex combinations of the paths of steepest ascent.

18. Efficient Frontier for Resolution and Rate Predicted Resolution and Rate for x’x=7.49 Grid lines match Predicted Y @ design center One quadrant shows gain in both Y j ’s

19. Efficient Frontier for Resolution and Rate No change in Rate (Y 2 ): x C =(1-c 1 )T 1 + c 1 T 2 c 1 =0.63 x c = [-0.48, -0.10, -2.68, -0.10, -0.22] @ x c ’x c =7.49 Resolution = 1.76 Rate =.084

20. Efficient Frontier for Resolution and Rate No change in Resolution (Y 1 ): x C =(1-c 2 )T 1 + c 2 T 2 c 2 =0.7355 x c = [-1.39, 0.46, -1.85, -1.22, -0.65] @ x c ’x c =7.49 Resolution =.86 Rate =.11

21. Improving Both Responses If T 1 ’T 2 > 0, all convex combinations of T 1 and T 2 increase the predicted Y for both responses If T 1 ’T 2 < 0, all x c with c 1 < c < c 2 increase the predicted Y for both responses For our example,.63 < c <.735 increase predicted Resolution and Rate

22. Efficient Frontier @ x’x=5 versus Factorial Points Factorial pts. = 8 directions, none on the efficient frontier What about sampling error?

23. Attaching Confidence to Improvement Lower confidence limit for E[Y(x)], given x Lower confidence limit for change in E[Y(x)], given x where

24. Efficient Frontier @ x’x=7.49 with 90% Lower Confidence Bound for E(Y)-  0

25. Paths of Improvement (POI) Region POI Region = The POI Region is a cone about the path of steepest ascent, containing all x such that the angle Using t 2,.10 = 1.886, the upper bound for θ xb is 86.9 o for Resolution, and 83.3 o for Rate For simultaneous (in x) confidence region, replace t df,  with (kF k,df,  ) 1/2 or [(k-1)F k-1,df,  ] 1/2

26. Paths of Improvement vs. Path of Steepest Ascent “Path of Steepest Ascent” b is perpendicular to contours for predicted Y – The path of steepest ascent is not scale invariant – Contours are invariant to the scaling of the factors Paths of improvement contours are complementary to the confidence cone for steepest ascent path – Assuming  b  < 90 o, 100(1-  )% confidence cone for steepest ascent – Assuming  b  < 90 o, 100(1-  )% confidence cone for paths of improvement

27. Scale Dependence for Path of Steepest Ascent If the experiment uses a small range for one factor, steepest ascent will neglect that factor Suppose Y =  0 + X1 + X2 Experiment 1 – X1: [-2,2] – X2: [-1,1] – Path of S.A.: [4,1] Experiment 2 – X1: [-1,1] – X2: [-2,2] – Path of S.A.: [1,4] Contour [1,-1] for both

28. Complementary Regions As precision improves, the confidence cone for  shrinks, while the paths of improvement region expands toward half of R k

29. Common Paths of Improvement Using predicted values, convex combinations x C =(1-c 2 )T 1 + c 2 T 2 yield improvement in both responses for c 1 < c < c 2 – For our example,.63 < c <.735 Using lower confidence bounds, a smaller set of directions yield “certain” improvement in both responses – For our example using t 2,.10 = 1.886, we are sure of improvement for.651 < c <.727

30. Extensions to J > 2 Responses The efficient frontier for more than two responses is the set of directions x that are a convex combination of all J vectors of steepest ascent – If some directions of steepest ascent are interior to this set, they are not binding Overlaying contour plots can show the predicted responses for each direction x on the efficient frontier.

31. Is Simultaneous Improvement Really Possible? Can we reject H o :      = 180 o ? – An approximate F test based on the difference in SSE for regression of Y 2 on X and regression of Y 2 on predicted Y 1. – For our example, F = 25.45 vs. F 4, 2 (p =.04) Can we construct an upper confidence bound for this angle? – No solution at present – The larger this angle, the further one must extrapolate in these k factors to achieve gain in both responses.

32. Questions?