Two stage group screening with noise factors, unequal group sizes and differing probabilities of active effects Anna Vine University of Southampton, UK Susan Lewis, Angela Dean Funded by EPSRC

Summary Two-stage group screening –for control and noise factors –Interaction group screening Criteria for comparing strategies Example –Software to guide experimenters

Screening in industrial experiments Response can depend on a large number of different factors of two types: –control: can be set by engineers during manufacturing –noise: can’t be controlled in manufacturing/use but can be controlled in experiments Aim of screening experiments is to identify important factors –judged by engineers to give a substantial improvement, Δ > 0, in product performance

Two-stage group screening Individual two-level factors divided into groups: –F groups of control factors with sizes –N groups of noise factors with sizes Define a grouped factor for each group by setting all factors within a group to their high (low) level Two stages of experimentation –Stage I on grouped factors –Stage II on factors in groups found to be important at stage I

Active factors In interaction group screening main effects and interactions of the grouped factors are estimated at stage I A grouped control factor is declared active –if it has a detected main effect –if it is in a detected control x control interaction –if it is in a detected control x noise interaction A grouped noise factor is only declared active if in a detected interaction with a control factor Detection of active grouped factors is not certain

Probabilities of effects being active The total number of effects S that require estimation over the two-stage experiment is a random variable Based on experts’ opinions assign prior probabilities –to each individual main effect being active –to each individual interaction being active –or use heredity principle (Chipman 1996) Calculate probabilities of each grouped effect being active from individual probabilities

Criteria for choice of groupings Aim to 1.minimise expected total number of effects that require estimation, E(S) 2.minimise the probability of exceeding a target number u of factorial effects requiring estimation, P(S > u) 3.minimise the risk of failing to detect important main effects and interactions Conflict between aims Lewis and Dean (2001) and Dean and Lewis (2002)

Formulating the criteria Expected total number of individual effects requiring estimation Probability distribution of the total number of individual effects requiring estimation

Practical situation Partition individual control factors into two types: very likely and less likely Control factors believed very likely to be active are assigned the same high probability of a main effect being active Assign very likely and less likely control factors to separate groups

Example To illustrate use of criteria consider: 7 individual control factors - main effects probabilities 1 8 individual control factors - main effects probabilities individual noise factors - main effects probabilities 0.3 individual control x noise interaction probabilities 0.07 individual control x control interaction probabilities 0.05

Distribution of S Risk of exceeding a target

Software Interactive web based software –allows elicitation of information from experts –enables comparison of different groupings under criteria 1 and 2 –allows simulations of group screening experiments to be run for criterion 3

Use of methods/software Planned a group screening experiment on engine cold start optimisation at Jaguar Cars

Investigation of groupings Very likely control Less likely control Noise E(S)sd(S)P(S>120)P(S>150)P(S>180) Grouping 782, ,62, ,52, ,42, ,2,42, ,3,32, ,2,2,22, ,582, ,52,62, ,53,52, ,54,42, ,52,2,42, ,52,3,32, ,52,2,2,22, ,482, ,42,62, ,43,52, ,44,42, ,42,2,42, ,42,3,32, ,42,2,2,22, ,2,382, ,2,32,62, ,2,33,52, ,2,34,42, ,2,32,2,42, ,2,32,3,32, ,2,32,2,2,22,

Investigation of groupings - CGS Very likely control Less likely control Noise E(S)sd(S)P(S>80)P(S>110)P(S>140) Grouping 782, ,62, ,52, ,42, ,2,42, ,3,32, * 72,2,2,22, ,2,382, ,2,32,62, ,2,33,52, ,2,34,42, ,2,32,2,42, ,2,32,3,32, ,2,32,2,2,22,

Classical group screening Distribution of S Risk of exceeding a target

Choosing a strategy Is CGS a better option? IGS is more likely to exceed a target

Choosing a strategy – Simulation software Provides values of active and non-active effects sampled from specified distributions Forms the groups of factors at random For each selection of effect values, software simulates an experiment Calculates proportions of active individual main effects and interactions MISSED Process repeated 500 times Assesses risk of missing active effects

Choosing a strategy - Simulation results Strategy CGSIGS control main effects 6 – 171 – 11 control x control interactions 47 – 7327 – 63 control x noise interactions 71 – Percentage of active individual effects missed Conclude: CGS misses more active individual effects than IGS

Software Results - CGS Very likely control Less likely control Noise E(S)sd(S)P(S>80)P(S>110)P(S>140) Grouping 782, ,62, ,52, ,42, ,2,42, ,3,32, ,2,2,22, ,582, ,52,62, ,53,52, ,54,42, ,52,2,42, ,52,3,32, ,52,2,2,22, ,482, ,42,62, ,43,52, ,44,42, ,42,2,42, ,42,3,32, ,42,2,2,22, ,2,382, ,2,32,62, ,2,33,52, ,2,34,42, ,2,32,2,42, ,2,32,3,32, ,2,32,2,2,22,

Investigation of groupings 4 noise factors grouped in pairs Very likely control Less likely control E(S)sd(S)P(S>120)P(S>150)P(S>180) Grouping 2,2,3 * 2,2,3 8 2,6 3,5 4,4 2,2,4 2,3,3 2,2,2, Noise factors in two groups of size 2 was consistently more economical

Conclusions Good grouping strategies for minimising E(S): –Use groups as small and as equal in size as possible –Group together factors with higher probabilities of their main effects being active