Diploma in Statistics Design and Analysis of Experiments Lecture 4.11 © 2010 Michael Stuart Design and Analysis of Experiments Lecture Review of Lecture More on Variance Components 3.Measurement System Analysis

Diploma in Statistics Design and Analysis of Experiments Lecture 4.12 © 2010 Michael Stuart Minute Test: How Much

Diploma in Statistics Design and Analysis of Experiments Lecture 4.13 © 2010 Michael Stuart Minute Test: How Fast

Diploma in Statistics Design and Analysis of Experiments Lecture 4.14 © 2010 Michael Stuart Homework Process improvement study, reduced model: Y =  + B + C + D + BC +  Set up a "design matrix" with columns for the significant effects, headed by the effect coefficients. Calculate a fitted value for each design point by applying the rows of signs to the effect coefficients and adding the overall mean. Crosscheck with the fitted values calculated by Minitab

Diploma in Statistics Design and Analysis of Experiments Lecture 4.15 © 2010 Michael Stuart Homework Minitab provides estimated effects: Term Effect NaOHCon Speed Temp NaOHCon*Speed Model? Y =  + B + C + D + BC +  Coef Excel

Diploma in Statistics Design and Analysis of Experiments Lecture 4.16 © 2010 Michael Stuart Homework Minitab provides fitted values, residuals,  estimate via ANOVA Analysis of Variance for Impurity (coded units) Source DF SS MS F P Main Effects Way Interactions Residual Error Total

Diploma in Statistics Design and Analysis of Experiments Lecture 4.17 © 2010 Michael Stuart Comparison of fits All effect estimates are the same; SE's vary. Lenth:s = 2.25, PSE = Reduced:s = 1.74, SE(effect) = 0.87 Projected:s = 1.87, SE(effect) = 0.94 "Projected" model has 3 interactions missing from the "Reduced" model.

Diploma in Statistics Design and Analysis of Experiments Lecture 4.18 © 2010 Michael Stuart Degrees of freedom "Error" degrees of freedom relevant for t –check ANOVA table –count estimated effects –use replication structure t 5,.05 = 2.57 t 8,.05 = 2.31 t 11,.05 = 2.20 s = 2.25 s = 1.87 s = 1.74 Ref: EM Notes Ch. 4 pp. 3, 6 Ref: Extra Notes, Models for Experiments and Lab 2 Feedback

Diploma in Statistics Design and Analysis of Experiments Lecture 4.19 © 2010 Michael Stuart Review of Lecture 3.2 Introduction to Fractional Factorial Designs Each row gives design points for a 4-factor experiment Fourth column estimates D main effect. Fourth column also estimates ABC interaction effect. In fact, fourth column estimates D + ABC in D=

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Fractional factorial designs Full 2 4 requires 16 runs Half the full 2 4 requires 8 runs Saves resources, including time Sacrifices high order interactions via aliasing

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Fractional factorial designs Check D = ABC, design generator Derive ABC from first principles. D aliased with ABC 4th column estimates D + ABC, = D if ABC = 0

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Fractional factorial designs Alias List A = BCD B = ACD C = ABD D = ABC AB = CD AC = BD AD = BC ABCD =

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Part 2: More on Variance Components Identifying sources of variation Hierarchical design for variance component estimation Hierarchical ANOVA

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart  SS TT eBeB eSeS BB eTeT e = e B + e S + e T Sources of variation in moisture content Batch variation Sampling variation Testing variation y

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Components of Variance Recall basic model: Y =  + e B + e S + e T Components of variance:

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Hierarchical Design for Variance Component Estimation

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Minitab Nested ANOVA Analysis of Variance for Test Source DF SS MS F P Batch Sample Error Total Variance Components % of Source Var Comp. Total StDev Batch Sample Error Total

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Model for Nested ANOVA Y ijk = m + B i + S j(i) + T k(ij) SS(TO) = SS(B) + SS(S) + SS(T) 59 =

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Minitab Nested ANOVA Expected Mean Squares 1 Batch 1.00(3) (2) (1) 2 Sample 1.00(3) (2) 3 Error 1.00(3) Translation: EMS(Batch) = EMS(Sample) = EMS(Test) =

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Calculation = EMS(Test) = ½[EMS(Sample) – EMS(Test)] = ¼[EMS(Batch) – EMS(Sample)] Estimation = MS(Test) = ½[MS(Sample) – MS(Test)] = ¼[MS(Batch) – MS(Sample)]

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Conclusions from Variance Components Analysis Variance Components % of Source Var Comp. Total StDev Batch Sample Error Total Sampling variation dominates, testing variation is relatively small. Investigate sampling procedure.

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Part 3: Measurement System Analysis Accuracy and Precision Repeatability and Reproducibility Components of measurement variation Analysis of Variance Case study:the MicroMeter

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart The MicroMeter optical comparator

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart The MicroMeter optical comparator Place object on stage of travel table Align cross-hair with one edge Move and re-align cross-hair with other edge Read the change in alignment Sources of variation: –instrument error –operator error –parts (manufacturing process) variation

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Precise Biased Accurate Characterising measurement variation; Accuracy and Precision Imprecise

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Characterising measurement variation; Accuracy and Precision Centre and Spread Accurate means centre of spread is on target; Precise means extent of spread is small; Averaging repeated measurements improves precision, SE =  /√n –but not accuracy; seek assignable cause.

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Accuracy and Precision: Example Each of four technicians made six measurements of a standard (the 'true' measurement was 20.1), resulting in the following data: Technician Data Exercise:Make dotplots of the data. Assess the technicians for accuracy and precision

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Accuracy and Precision: Example

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Repeatability and Reproducability Factors affecting measurement accuracy and precision may include: –instrument –material –operator –environment –laboratory –parts (manufacturing)

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Repeatability and Reproducibility Repeatability: precision achievable under constant conditions: –same instrument –same material –same operator –same environment –same laboratory How variable is measurement under these conditions Reproducibility: precision achievable under varying conditions: –different instruments –different material –different operators –changing environment –different laboratories How much more variable is measurement under these conditions

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Measurement Capability of the MicroMeter 4 operators measured each of 8 parts twice, with random ordering of parts, separately for each operator. Three sources of variation: –instrument error –operator variation –parts(manufacturing process) variation. Data follow

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Measurement Capability of the MicroMeter

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Quantifying the variation Each measurement incorporates components of variation from –Operator error –Parts variation –Instrument error and also –Operator by Parts Interaction

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Measurement Differences

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Graphical Analysis of Measurement Differences

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Average measurements by Operators and Parts

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Graphical Analysis of Operators & Parts

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Graphical Analysis of Operators & Ordered Parts

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Quantifying the variation Notation:  E :SD of instrument error variation  P :SD of parts (manufacturing process) variation  O :SD of operator variation  OP :SD of operator by parts interaction variation  T :SD of total measurement variation N.B.: so

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Calculating s E sum = 18.6sum = 7.0 s 2 = ( )/32 = 0.8 s E = 0.89

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Analysis of Variance Analysis of Variance for Diameter Source DF SS MS F P Operator Part Operator*Part Error Total S =

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Basis for Random Effects ANOVA F-ratios in ANOVA are ratios of Mean Squares Check:F(O) = MS(O) / MS(O*P) F(P) = MS(P) / MS(O*P) F(OP) = MS(OP) / MS(E) Why? MS(O) estimates  E  OP  O 2 MS(P) estimates  E  OP  P 2 MS(OP) estimates  E  OP 2 MS(E) estimates  E 2

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Variance Components Estimated Standard Source Value Deviation Operator Part Operator*Part Error

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Diagnostic Analysis

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Diagnostic Analysis

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Measurement system capability  E   P means measurement system cannot distinguish between different parts. Need  E <<  P. Define  TP = sqrt(  E 2 +  P 2 ). Capability ratio =  TP /  E should exceed 5

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Repeatability and Reproducibility Repeatabilty SD =  E Reproducibility SD = sqrt(  O 2 +  OP 2 ) Total R&R= sqrt(  O 2 +  OP 2 +  E 2 )

Diploma in Statistics Design and Analysis of Experiments Lecture © 2010 Michael Stuart Reading EM §5.7, §7.5, §8.2 BHH, Ch. 5, §§ , §9.3