24 September 2004David Drain1 Experiment Design Suppose you want to do some experiments to improve a manufacturing process you think may be influenced by a dozen different factors –What do you really want from the experiment – what does “optimal” mean? –How can you most economically get the information you want?

24 September 2004David Drain2 An Experiment Three “factors” (potential influences) Seven “runs” – the process is run at this combination of settings and data is collected

24 September 2004David Drain3 Noise Variables and Process Targets A robust process target gives good results regardless of noise variables Control-noise interaction terms measure robustness

24 September 2004David Drain4 Constraints in Industrial Design Problems Number of runs Mandatory center points Blocks Constraints on control variables

24 September 2004David Drain5 Contour Plot of PEV Plot of prediction error variance for modified CCD with 2 control and 2 noise variables:

24 September 2004David Drain6 Sensitivity of a Classic Design FCC with 23 runs No noiseNoise parameters of magnitude 0.18

24 September 2004David Drain7 Design Discovery Methods Try the classic designs (CCD,…) Exchange methods –Choose candidates, then an algorithm (DETMAX, for example) selects a “best” subset –Candidate set must include optimal runs to get optimal design –Selecting too many candidates makes the algorithm computationally intractable Heuristic methods

24 September 2004David Drain8 Hybrid Heuristics A hybrid heuristic is some combination of basic heuristics Starting a GA with a random population is a simple hybrid: scattersearch followed by GA Talbi (2002) gives a complete taxonomy of hybrid heuristics

24 September 2004David Drain9 The No Free Lunch Theorem Scattersearch is just as good as an efficient hillclimbing algorithm, if their performance is averaged over all possible problems. No heuristic can be applied successfully to all problems: the heuristic must be chosen to match the problem being solved. “Roughly speaking, the average performance of any pair of algorithms across all possible problems is identical.” Wolpert and Macready (1997)

24 September 2004David Drain10 Genetic Algorithm Based on evolution: genetic material from two parents is combined to produce offspring, and some mutation is allowed Parent AParent BChild Crossover

24 September 2004David Drain11 Simulated Annealing Based on physical annealing processes Early in the process, while at a high “temperature”, relatively large changes from local optimum are allowed. –Disadvantageous changes are allowed with a probability inversely related to temperature Requires multiple starting points because of tendency to become trapped at local optima

24 September 2004David Drain12 GASA: Genetic Algorithm – Simulated Annealing Hybrid Integrated hybrid of GA and SA –Exploration with temperature-controlled GA –Exploitation with SA within each GA step –Discrete changes with GA (entire runs) –Continuous changes with SA –Temperature parameters tunable for computational efficiency and effectiveness

24 September 2004David Drain13 GASA Algorithm

24 September 2004David Drain14 GASA Initialization Initialize population from candidate list Read test vectors, or generate random test vectors Define objective function –Desirability functions are supported Read starting parameters Define rules for parameter modification by iteration

24 September 2004David Drain15 GASA Parameters Number of iterations Population size Number of elites Cross-breeding requirements  Roulette rate  Mutation rate  Number of test vectors used for objective function evaluation  Distance and probability rules for SA Fixed Iteration Dependent

24 September 2004David Drain16 Using GASA Effectively Parameter choices have a significant effect on algorithm effectiveness and computational efficiency Some parameter settings can be determined through the use of designed experiments (these must estimate interactions) Ideal parameter values seem to depend on the stage of optimization: –Early in the process, high mutation rates make sense –Late in the process, more intense SA may be needed –Dynamic parameter adjustment is a necessity

24 September 2004David Drain17 Using GASA Effectively Use the best starting points available Ensure sufficient genetic diversity –Unintentional cloning occurs otherwise –Large populations and forced crossbreeding enhance genetic diversity Choose discriminating objective functions –No population member should achieve a perfect score, and most members should not receive a failing score –Objective function parameters might need to be refined as optimum is approached

24 September 2004David Drain18 GASA Results GASA was validated on a problem recently solved by Borkowski (2003) –D-optimal design for a quadratic model in a square region using six runs GASA can improve the best known designs Greatest benefits are in slope estimate variance reduction and rotatability, rather than prediction error variance reduction Smaller designs exhibit the most improvement

24 September 2004David Drain19 Rotatability Improvement

24 September 2004David Drain20 Future Research Parameter setting optimization Dynamic and adaptive parameter determination “Neighborhood” definition (for SA) based on design properties rather than Euclidean distance Find alternatives to simulated annealing for exploitation mechanism –Simple hill-climbing did not work well –Gravitating to areas of high variance did not work well

24 September 2004David Drain21 References Borkowski, J.J. (2003). "Using a Genetic Algorithm to Generate Small Exact Response Surface Designs". Journal of Probability and Statistical Science 1, pp Borror, C. M.; Montgomery, D. C. and Myers, R. H. (2002). “Evaluation of Statistical Designs for Experiments Involving Noise Variables”. Journal of Quality Technology, 34, pp Box, G. E. P. and Wilson, L. B. (1951). “On the Experimental Attainment of Optimum Conditions”. Journal of the Royal Statistical Society B, 13, pp Forouraghi, B. (2000). "A Genetic Algorithm for Multiobjective Robust Design". Applied Intelligence 12, pp Haines, L. M. (1987). "The Application of the Annealing Algorithm to the Construction of Exact Optimal Designs for Linear-Regression Models". Technometrics 29, pp Hamada, M.; Martz, H.F.; Reese,C.S. and Wilson, A.G. (2001). "Finding Near-Optimal Bayesian Experimental Designs via Genetic Algorithms". The American Statistician 55, pp Heredia-Langner, A.; Carlyle, W.M.; Montgomery, D.C.; Borror, C.M. and Runger, G.C. (2003). “Genetic Algorithms for the Construction of D-Optimal Designs”. Journal of Quality Technology, 35, pp Kragelund, L. V. (1997). "Solving a Timetabling Problem using Hybrid Genetic Algorithms". Software- Practice and Experience, 27, pp

24 September 2004David Drain22 References (cont.) Li, W. D.; Ong, S. K. and Nee, Y. C. (2002). "Hybrid genetic algorithm and simulated annealing approach for the optimization of process plans for prismatic parts". International Journal of Production Research, 40, pp Myers, R. H.; Khuri, A. I. and Vining, G. (1992). "Response Surface Alternatives to the Taguchi Robust Parameter Design Approach". American Statistician, 46, pp Parkinson, D.B. (2000). "Robust Design Employing a Genetic Algorithm". Quality and Reliability Engineering International, 16, pp Taguchi, G. (1986). Introduction to Quality Engineering: Designing Quality into Products and Processes. Kraus International Publications, White Plains, NY. Talbi, E.-G. (2002). "A Taxonomy of Hybrid Metaheuristics". Journal of Heuristics 8, pp Welch, W.J. (1982). "Branch-and-Bound Search for Experimental Designs Based on D Optimality and Other Criteria". Technometrics 24, pp Wolpert, D.H. and Macready, W.G. (1997). "No Free Lunch Theorems for Optimization". IEEE Transactions on Evolutionary Computation, 1, pp