1. Diploma in Statistics Design and Analysis of Experiments Lecture 4.11 © 2010 Michael Stuart Design and Analysis of Experiments Lecture 4.1 1.Review of Lecture 3.2 2.More on Variance Components 3.Measurement System Analysis

2. Diploma in Statistics Design and Analysis of Experiments Lecture 4.12 © 2010 Michael Stuart Minute Test: How Much

3. Diploma in Statistics Design and Analysis of Experiments Lecture 4.13 © 2010 Michael Stuart Minute Test: How Fast

4. Diploma in Statistics Design and Analysis of Experiments Lecture 4.14 © 2010 Michael Stuart Homework 3.2.1 Process improvement study, reduced model: Y =  + B + C + D + BC +  Set up a "design matrix" with columns for the significant effects, headed by the effect coefficients. Calculate a fitted value for each design point by applying the rows of signs to the effect coefficients and adding the overall mean. Crosscheck with the fitted values calculated by Minitab

5. Diploma in Statistics Design and Analysis of Experiments Lecture 4.15 © 2010 Michael Stuart Homework 3.2.1 Minitab provides estimated effects: Term Effect NaOHCon -17.250 Speed 9.750 Temp 21.750 NaOHCon*Speed -7.500 Model? Y =  + B + C + D + BC +  Coef 49.750 -8.625 4.875 10.875 -3.750 Excel

6. Diploma in Statistics Design and Analysis of Experiments Lecture 4.16 © 2010 Michael Stuart Homework 3.2.1 Minitab provides fitted values, residuals,  estimate via ANOVA Analysis of Variance for Impurity (coded units) Source DF SS MS F P Main Effects 3 3462.75 1154.25 381.86 0.000 2-Way Interactions 1 225.00 225.00 74.44 0.000 Residual Error 11 33.25 3.02 Total 15 3721.00

7. Diploma in Statistics Design and Analysis of Experiments Lecture 4.17 © 2010 Michael Stuart Comparison of fits All effect estimates are the same; SE's vary. Lenth:s = 2.25, PSE = 1.125 Reduced:s = 1.74, SE(effect) = 0.87 Projected:s = 1.87, SE(effect) = 0.94 "Projected" model has 3 interactions missing from the "Reduced" model.

8. Diploma in Statistics Design and Analysis of Experiments Lecture 4.18 © 2010 Michael Stuart Degrees of freedom "Error" degrees of freedom relevant for t –check ANOVA table –count estimated effects –use replication structure t 5,.05 = 2.57 t 8,.05 = 2.31 t 11,.05 = 2.20 s = 2.25 s = 1.87 s = 1.74 Ref: EM Notes Ch. 4 pp. 3, 6 Ref: Extra Notes, Models for Experiments and Lab 2 Feedback

9. Diploma in Statistics Design and Analysis of Experiments Lecture 4.19 © 2010 Michael Stuart Review of Lecture 3.2 Introduction to Fractional Factorial Designs Each row gives design points for a 4-factor experiment Fourth column estimates D main effect. Fourth column also estimates ABC interaction effect. In fact, fourth column estimates D + ABC in 2 4-1. D=

10. Diploma in Statistics Design and Analysis of Experiments Lecture 4.110 © 2010 Michael Stuart Fractional factorial designs Full 2 4 requires 16 runs Half the full 2 4 requires 8 runs Saves resources, including time Sacrifices high order interactions via aliasing

11. Diploma in Statistics Design and Analysis of Experiments Lecture 4.111 © 2010 Michael Stuart Fractional factorial designs Check D = ABC, design generator Derive ABC from first principles. D aliased with ABC 4th column estimates D + ABC, = D if ABC = 0

12. Diploma in Statistics Design and Analysis of Experiments Lecture 4.112 © 2010 Michael Stuart Fractional factorial designs Alias List A = BCD B = ACD C = ABD D = ABC AB = CD AC = BD AD = BC ABCD =

13. Diploma in Statistics Design and Analysis of Experiments Lecture 4.113 © 2010 Michael Stuart Part 2: More on Variance Components Identifying sources of variation Hierarchical design for variance component estimation Hierarchical ANOVA

14. Diploma in Statistics Design and Analysis of Experiments Lecture 4.114 © 2010 Michael Stuart  SS TT eBeB eSeS BB eTeT e = e B + e S + e T Sources of variation in moisture content Batch variation Sampling variation Testing variation y

15. Diploma in Statistics Design and Analysis of Experiments Lecture 4.115 © 2010 Michael Stuart Components of Variance Recall basic model: Y =  + e B + e S + e T Components of variance:

16. Diploma in Statistics Design and Analysis of Experiments Lecture 4.116 © 2010 Michael Stuart Hierarchical Design for Variance Component Estimation

17. Diploma in Statistics Design and Analysis of Experiments Lecture 4.117 © 2010 Michael Stuart Minitab Nested ANOVA Analysis of Variance for Test Source DF SS MS F P Batch 14 1216.2333 86.8738 1.495 0.224 Sample 15 871.5000 58.1000 64.556 0.000 Error 30 27.0000 0.9000 Total 59 2114.7333 Variance Components % of Source Var Comp. Total StDev Batch 7.193 19.60 2.682 Sample 28.600 77.94 5.348 Error 0.900 2.45 0.949 Total 36.693 6.058

18. Diploma in Statistics Design and Analysis of Experiments Lecture 4.118 © 2010 Michael Stuart Model for Nested ANOVA Y ijk = m + B i + S j(i) + T k(ij) SS(TO) = SS(B) + SS(S) + SS(T) 59 = 14 + 15 + 30

19. Diploma in Statistics Design and Analysis of Experiments Lecture 4.119 © 2010 Michael Stuart Minitab Nested ANOVA Expected Mean Squares 1 Batch 1.00(3) + 2.00(2) + 4.00(1) 2 Sample 1.00(3) + 2.00(2) 3 Error 1.00(3) Translation: EMS(Batch) = EMS(Sample) = EMS(Test) =

20. Diploma in Statistics Design and Analysis of Experiments Lecture 4.120 © 2010 Michael Stuart Calculation = EMS(Test) = ½[EMS(Sample) – EMS(Test)] = ¼[EMS(Batch) – EMS(Sample)] Estimation = MS(Test) = ½[MS(Sample) – MS(Test)] = ¼[MS(Batch) – MS(Sample)]

21. Diploma in Statistics Design and Analysis of Experiments Lecture 4.121 © 2010 Michael Stuart Conclusions from Variance Components Analysis Variance Components % of Source Var Comp. Total StDev Batch 7.193 19.60 2.682 Sample 28.600 77.94 5.348 Error 0.900 2.45 0.949 Total 36.693 6.058 Sampling variation dominates, testing variation is relatively small. Investigate sampling procedure.

22. Diploma in Statistics Design and Analysis of Experiments Lecture 4.122 © 2010 Michael Stuart Part 3: Measurement System Analysis Accuracy and Precision Repeatability and Reproducibility Components of measurement variation Analysis of Variance Case study:the MicroMeter

23. Diploma in Statistics Design and Analysis of Experiments Lecture 4.123 © 2010 Michael Stuart The MicroMeter optical comparator

24. Diploma in Statistics Design and Analysis of Experiments Lecture 4.124 © 2010 Michael Stuart The MicroMeter optical comparator Place object on stage of travel table Align cross-hair with one edge Move and re-align cross-hair with other edge Read the change in alignment Sources of variation: –instrument error –operator error –parts (manufacturing process) variation

25. Diploma in Statistics Design and Analysis of Experiments Lecture 4.125 © 2010 Michael Stuart Precise Biased Accurate Characterising measurement variation; Accuracy and Precision Imprecise

26. Diploma in Statistics Design and Analysis of Experiments Lecture 4.126 © 2010 Michael Stuart Characterising measurement variation; Accuracy and Precision Centre and Spread Accurate means centre of spread is on target; Precise means extent of spread is small; Averaging repeated measurements improves precision, SE =  /√n –but not accuracy; seek assignable cause.

27. Diploma in Statistics Design and Analysis of Experiments Lecture 4.127 © 2010 Michael Stuart Accuracy and Precision: Example Each of four technicians made six measurements of a standard (the 'true' measurement was 20.1), resulting in the following data: Technician Data 120.219.920.120.420.220.4 219.920.219.520.420.619.4 320.620.520.720.620.821.0 420.119.920.219.921.120.0 Exercise:Make dotplots of the data. Assess the technicians for accuracy and precision

28. Diploma in Statistics Design and Analysis of Experiments Lecture 4.128 © 2010 Michael Stuart Accuracy and Precision: Example

29. Diploma in Statistics Design and Analysis of Experiments Lecture 4.129 © 2010 Michael Stuart Repeatability and Reproducability Factors affecting measurement accuracy and precision may include: –instrument –material –operator –environment –laboratory –parts (manufacturing)

30. Diploma in Statistics Design and Analysis of Experiments Lecture 4.130 © 2010 Michael Stuart Repeatability and Reproducibility Repeatability: precision achievable under constant conditions: –same instrument –same material –same operator –same environment –same laboratory How variable is measurement under these conditions Reproducibility: precision achievable under varying conditions: –different instruments –different material –different operators –changing environment –different laboratories How much more variable is measurement under these conditions

31. Diploma in Statistics Design and Analysis of Experiments Lecture 4.131 © 2010 Michael Stuart Measurement Capability of the MicroMeter 4 operators measured each of 8 parts twice, with random ordering of parts, separately for each operator. Three sources of variation: –instrument error –operator variation –parts(manufacturing process) variation. Data follow

32. Diploma in Statistics Design and Analysis of Experiments Lecture 4.132 © 2010 Michael Stuart Measurement Capability of the MicroMeter

33. Diploma in Statistics Design and Analysis of Experiments Lecture 4.133 © 2010 Michael Stuart Quantifying the variation Each measurement incorporates components of variation from –Operator error –Parts variation –Instrument error and also –Operator by Parts Interaction

34. Diploma in Statistics Design and Analysis of Experiments Lecture 4.134 © 2010 Michael Stuart Measurement Differences

35. Diploma in Statistics Design and Analysis of Experiments Lecture 4.135 © 2010 Michael Stuart Graphical Analysis of Measurement Differences

36. Diploma in Statistics Design and Analysis of Experiments Lecture 4.136 © 2010 Michael Stuart Average measurements by Operators and Parts

37. Diploma in Statistics Design and Analysis of Experiments Lecture 4.137 © 2010 Michael Stuart Graphical Analysis of Operators & Parts

38. Diploma in Statistics Design and Analysis of Experiments Lecture 4.138 © 2010 Michael Stuart Graphical Analysis of Operators & Ordered Parts

39. Diploma in Statistics Design and Analysis of Experiments Lecture 4.139 © 2010 Michael Stuart Quantifying the variation Notation:  E :SD of instrument error variation  P :SD of parts (manufacturing process) variation  O :SD of operator variation  OP :SD of operator by parts interaction variation  T :SD of total measurement variation N.B.: so

40. Diploma in Statistics Design and Analysis of Experiments Lecture 4.140 © 2010 Michael Stuart Calculating s E sum = 18.6sum = 7.0 s 2 = (18.6 + 7.0)/32 = 0.8 s E = 0.89

41. Diploma in Statistics Design and Analysis of Experiments Lecture 4.141 © 2010 Michael Stuart Analysis of Variance Analysis of Variance for Diameter Source DF SS MS F P Operator 3 32.403 10.801 6.34 0.003 Part 7 1193.189 170.456 100.02 0.000 Operator*Part 21 35.787 1.704 2.13 0.026 Error 32 25.600 0.800 Total 63 1286.979 S = 0.894427

42. Diploma in Statistics Design and Analysis of Experiments Lecture 4.142 © 2010 Michael Stuart Basis for Random Effects ANOVA F-ratios in ANOVA are ratios of Mean Squares Check:F(O) = MS(O) / MS(O*P) F(P) = MS(P) / MS(O*P) F(OP) = MS(OP) / MS(E) Why? MS(O) estimates  E 2 + 2  OP 2 + 16  O 2 MS(P) estimates  E 2 + 2  OP 2 + 8  P 2 MS(OP) estimates  E 2 + 2  OP 2 MS(E) estimates  E 2

43. Diploma in Statistics Design and Analysis of Experiments Lecture 4.143 © 2010 Michael Stuart Variance Components Estimated Standard Source Value Deviation Operator 0.5686 0.75 Part 21.0939 4.59 Operator*Part 0.4521 0.67 Error 0.8000 0.89

44. Diploma in Statistics Design and Analysis of Experiments Lecture 4.144 © 2010 Michael Stuart Diagnostic Analysis

45. Diploma in Statistics Design and Analysis of Experiments Lecture 4.145 © 2010 Michael Stuart Diagnostic Analysis

46. Diploma in Statistics Design and Analysis of Experiments Lecture 4.146 © 2010 Michael Stuart Measurement system capability  E   P means measurement system cannot distinguish between different parts. Need  E <<  P. Define  TP = sqrt(  E 2 +  P 2 ). Capability ratio =  TP /  E should exceed 5

47. Diploma in Statistics Design and Analysis of Experiments Lecture 4.147 © 2010 Michael Stuart Repeatability and Reproducibility Repeatabilty SD =  E Reproducibility SD = sqrt(  O 2 +  OP 2 ) Total R&R= sqrt(  O 2 +  OP 2 +  E 2 )

48. Diploma in Statistics Design and Analysis of Experiments Lecture 4.148 © 2010 Michael Stuart Reading EM §5.7, §7.5, §8.2 BHH, Ch. 5, §§6.1 - 6.3, §9.3