1. Adventures in industry Sue Lewis Southampton Statistical Sciences Research Institute University of Southampton sml@maths.soton.ac.uk

2. Outline Experiments on many factors - with Jaguar Cars - using two-stage group screening - to find the important factors Experiments on assembled mechanical products - where values of factors cannot be set - with Hosiden Besson, Sauer Danfoss, Goodrich Software for implementing the methods

3. Cold Start Optimisation

4. Factors Affecting Performance Control (or design) factors – can be set by the engineers Noise factors - cannot be controlled in use eg ambient temperature - can be controlled in an experiment Aim: find the control factor settings that Optimise the performance (engine starts - resistance) Minimize variability in performance - due to the varying noise factors - Deming, Taguchi

5. control x noise interactions For conventional factorial designs large number of factors  large number of runs Also main effects and control x control interactions Want to detect

6. Classical Solution Run an experiment to estimate only main effects - identify the important factors For the important factors, run an experiment - to estimate both main effects and interactions Disadvantage: could miss factors that interact with noise

7. Arrange the factors in groups Label the factor levels high - larger response anticipated low - smaller response anticipated For each group define a new grouped factor with two levels high - all factors in group high low - all factors in group low Experiment on the grouped factors Grouping factors

8. Stage 1: perform an experiment on the grouped factors to decide which groups are important - estimate main effects and/or interactions Stage 2: dismantle those groups found to be important and experiment on their individual factors - estimate both main effects and interactions Two Stage Group Screening

9. Gathering Information from Experts Opinions on Factors that might be included in the experiment - and their levels The likely importance of each factor The direction of each main effect Any insights/experience on interactions Local brainstorming – but experts often at different sites

10. Web-based System (GISEL) Gathers opinions/suggestions on factors and their levels - via a dynamic questionnaire - with free form comments Keeps a record of opinions, experiments and results Guides factor groupings via software that - explores the resources needed for various strategies and factor groupings - estimates the risk of missing important factors through simulation of experiments

11. Factors under Consideration

13. Summary of Opinions on Air to Fuel Ratio

14. Making a decision on groupings Assess possible grouping strategies - resource required - risk of missing an important factor Individual factors are classified as Very likely to be active Less likely to be active Not worth including Probabilities assigned eg 0.7 and 0.2

15. Ten Factors for the Experiment Control – very likely Noise Plug type* Temperature Plug gap* Injector tip leakage Air fuel ratio Injection timing Control – less likely Spark during crank Spark time during run-up Higher idle speed Idle flare * hard-to-change: grouped together

16. Investigation of different groupings

17. Plan for the First Stage (10 factors) Control factors: Group 1: Plug type* & Plug gap* Group 2: Air to fuel ratio & Injection timing Group 3: Spark time during crank & During run-up Group 4: Higher idle speed & Idle flare Noise factors: Group 5: Injector tip leakage Group 6: Temperature Design: Half-replicate (I=123456) in 4 sessions of 8 runs

18. Results of First Stage Experiment Included large interactions (Afr & Injection timing) x Temperature (Higher idle speed & Idle flare) x Injector tip leakage - both grouped control x noise interactions  6 factors to investigate at the Second Stage Experiment

19. Second Stage Experiment Design Half-replicate in 32 runs (I = ABCDEF) - for the individual factors - could have been smaller Preliminary findings include Air to Fuel Ratio x Temperature is large Possible three factor interaction

20. Experiments on assembled products Aim: mean sound output close to target with reduced variation armature diaphragm magnet front case Acoustic sounder Hosiden Besson

21. Gear pump Aim: reduce mean leakage and variation in leakage - under varying pressure and speed gear pack

22. Possible approaches Factorial experiments -set factors to values specified in the design Obtain parts with required factor values by - making special components - measuring large samples and using components with required factor values For our examples: too slow and costly Disassembly/reassembly experiments (Shainin) In our examples: cannot reuse components

23. Our Approach Take a sample of each kind of component from production Measure the relevant component variables Assemble the components to form a set of products for testing –to maximise information on the factors of interest

24. Factors Directly measurable on a component - eg permeability of the armature in the sounder Formed or derived as a function of measured quantities on two or more components - eg gaps between components in the assembled product - cannot be handled by conventional designs Factors that can be set - eg the skill of the operator in making certain adjustments during the manufacture of the sounder

25. To design the experiment -must decide which set of products to assemble There is a huge number of possibilities Eg For 4 components (pump gear pack) and sufficient parts to assemble 12 products - the number of possibilities is ~ 12x10 35 Needs a non-standard search algorithm that - finds an efficient set of assemblies - allows for the non-reuse of components - accommodates conventional factors

26. Finding a design Use a specially developed search algorithm with - a low order polynomial to describe the response - a design chosen for accurate estimation of the coefficients of the model (D-optimality) Software (DEAP) has been developed that - assists with product and component definition - provides access to the design algorithm

27. Software to Implement the Methods (DEAP)

29. Results from the studies The most important factors for improving the product performance were: For the sounder : the pip height and skill of operator For the pump: positioning of the cover and the alignment of gears

30. Conclusions Tools and methods developed in collaboration with industry for two kinds of experiments - large numbers of factors - assembled products Software at the beta testing stage - freely available

31. Some related references Atkinson, A.C. and Donev, A.N. (1992) Optimum Experimental Designs. Oxford: Oxford University Press. Dean, A.M. and Lewis, S.M. (2002) Comparison of group screening strategies for factorial experiments. Computational Statistics and Data Analysis, 39, 287- 297. Deming, W.E. (1986) Out of the Crisis. Cambridge: C.U.P. Dupplaw, D., Brunson, D., Vine, A.E., Please, C.P., Lewis, S.M., Dean, A.M., Keane, A.J. and Tindall, M.J. (2004) A web-based knowledge elicitation system (GISEL) for planning and assessing group screening experiments for product development. To appear in J. of Computing and Information Science in Engineering (ASME). Harville, D. A. (1974) Nearly optimal allocation of experimental units using observed covariate values. Technometrics 16, 589-599.

32. Some related references O’Neill, J.C., Borror, C.M., Eastman, P.Y., Fradkin, D.G., James, M.P., Marks, A.P. and Montgomery, D.C. (2000) Optimal assignment of samples to treatments for robust design. Qual. Rel. Eng. Int. 16, 417-421. Lewis, S.M. and Dean, A.M. (2001) Detection of Interactions in Experiments with large numbers of factors (with discussion). J. Roy. Statist. Soc. B, 63, 633-672. Sexton, C.J., Lewis, S.M. and Please, C.P. (2001) Experiments for derived factors with application to hydraulic gear pumps J. Roy. Statist. Soc. C, 50, 155-170. Shainin, R.D. (1993) Strategies for technical problem solving. Qual. Eng., 433-448. Taguchi, G. (1987) System of Experimental Design. New York: Kraus.