1. Diploma in Statistics Design and Analysis of Experiments Lecture 4.21 Design and Analysis of Experiments Lecture 4.2 Part 1: Components of Variation –identifying sources of variation –hierarchical design for variance component estimation –hierarchical ANOVA Part 2: Measurement System Analysis –Accuracy and Precision –Repeatability and Reproducibility –Components of measurement variation –Analysis of Variance –Case study:the MicroMeter

2. Diploma in Statistics Design and Analysis of Experiments Lecture 4.22 An invalid comparison Comparing standard process, A, with modified process, B A B 58.3 63.2 57.1 64.1 59.7 62.4 59.0 62.7 58.6 63.6 Means: 58.54 63.20 St. Devs: 0.96 0.68

3. Diploma in Statistics Design and Analysis of Experiments Lecture 4.23 An invalid comparison Process:batch manufacture of pigment paste Key variable:moisture content Sampling plan:single sample from single batch Measurements:5 repetitions s measures measurement error no measure of variation within batch no measure of variation between batches

4. Diploma in Statistics Design and Analysis of Experiments Lecture 4.24 Sources of variation in moisture content Batch, subject to Process variation Sample from batch, subject to within batch variation Measurement, subject to Test variation Model for variation in moisture content: Y =  + e P + e S + e T

5. Diploma in Statistics Design and Analysis of Experiments Lecture 4.25 ePeP PP Sources of variation in moisture content Process variation 

6. Diploma in Statistics Design and Analysis of Experiments Lecture 4.26 ePeP eSeS SS PP Sources of variation in moisture content Process variation Sampling variation 

7. Diploma in Statistics Design and Analysis of Experiments Lecture 4.27  SS TT ePeP eSeS PP eTeT e = e P + e P + e P Sources of variation in moisture content Process variation Sampling variation Testing variation y

8. Diploma in Statistics Design and Analysis of Experiments Lecture 4.28 Components of Variance Recall basic model: Y =  + e P + e S + e T Components of variance:

9. Diploma in Statistics Design and Analysis of Experiments Lecture 4.29 Conclusions for process testing Process measurements are subject to a hierarchy of variation sources. Several measurements on a single sample from a single batch do not reflect overall variation. Several batches and several samples from each batch are necessary to capture overall variation. Comparison of process methods must be referred to the relevant level of variation

10. Diploma in Statistics Design and Analysis of Experiments Lecture 4.210 Hierarchical Design for Variance Component Estimation A batch of pigment paste consists of 80 drums of material. 15 batches were available for testing 2 drums were selected at random from each batch and a sample was taken from each drum. 2 tests for moisture content were run on each sample. The results follow

11. Diploma in Statistics Design and Analysis of Experiments Lecture 4.211 Hierarchical Design for Variance Component Estimation

12. Diploma in Statistics Design and Analysis of Experiments Lecture 4.212 Nested ANOVA: Test versus Batch, Sample Analysis of Variance for Test Source DF SS MS F P Batch 14 1216.2333 86.8738 1.495 0.224 Sample 15 871.5000 58.1000 64.556 0.000 Error 30 27.0000 0.9000 Total 59 2114.7333 Variance Components % of Source Var Comp. Total StDev Batch 7.193 19.60 2.682 Sample 28.600 77.94 5.348 Error 0.900 2.45 0.949 Total 36.693 6.058

13. Diploma in Statistics Design and Analysis of Experiments Lecture 4.213 Nested ANOVA: Test versus Batch, Sample Expected Mean Squares 1 Batch 1.00(3) + 2.00(2) + 4.00(1) 2 Sample 1.00(3) + 2.00(2) 3 Error 1.00(3) Translation: EMS(Batch) = EMS(Sample) = EMS(Error) =

14. Diploma in Statistics Design and Analysis of Experiments Lecture 4.214 Calculation = EMS(Error) = ½[EMS(Sample) – EMS(Error)] = ¼[EMS(Batch) – EMS(Sample)]

15. Diploma in Statistics Design and Analysis of Experiments Lecture 4.215 Theory Model: Y ijk =  +  i +  i(j) +  ijk Y ij. =  +  i +  i(j) +  ij. Y i.. =  +  i +  i(.) +  i.. Decomposition: (Y ijk – Y... ) = (Y i.. – Y... ) + (Y ij. – Y i.. ) + (Y ijk – Y ij. ) (Y ij. – Y i.. ) = (  i(j) –  i(.) ) + (  ij. –  i.. ) EMS involves and

16. Diploma in Statistics Design and Analysis of Experiments Lecture 4.216 Conclusions from Variance Components Analysis Variance Components % of Source Var Comp. Total StDev Batch 7.193 19.60 2.682 Sample 28.600 77.94 5.348 Error 0.900 2.45 0.949 Total 36.693 6.058 Sampling variation dominates, testing variation is relatively small. Investigate sampling procedure.

17. Diploma in Statistics Design and Analysis of Experiments Lecture 4.217 Sampling procedure Standard: select 5 drums from batch at random, sample all levels of each drum using a specially constructed sampling tube thoroughly mix all samples take a sample from the mixture for analysis Actual take a sample from a drum for analysis

18. Diploma in Statistics Design and Analysis of Experiments Lecture 4.218 Another Example Testing drug treatments for pregnant women 22 women, 10 treatment A, 7 treatment B, 5 "control". Placentas examined for "irregularities": 5 locations, 10 slices, on microscope slides, 5 measurements (counts of "irregularities") per slide, 5,500 measurements in all. No significant treatment effect (10 vs 7)

19. Diploma in Statistics Design and Analysis of Experiments Lecture 4.219 Yet Another Example Comparing schools on student performance Schools Classes within schools Students within classes

20. Diploma in Statistics Design and Analysis of Experiments Lecture 4.220 Design and Analysis of Experiments Lecture 4.2 Part 2: Measurement System Analysis –Accuracy and Precision –Repeatability and Reproducibility –Components of measurement variation –Analysis of Variance –Case study:the MicroMeter

21. Diploma in Statistics Design and Analysis of Experiments Lecture 4.221 The MicroMeter optical comparator

22. Diploma in Statistics Design and Analysis of Experiments Lecture 4.222 The MicroMeter optical comparator Place object on stage of travel table Align cross-hair with one edge Move and re-align cross-hair with other edge Read the change in alignment Sources of variation: –instrument error –operator error –parts (manufacturing process) variation

23. Diploma in Statistics Design and Analysis of Experiments Lecture 4.223 Precise Biased Accurate Characterising measurement variation; Accuracy and Precision Imprecise

24. Diploma in Statistics Design and Analysis of Experiments Lecture 4.224 Characterising measurement variation; Accuracy and Precision Centre and Spread Accurate means centre of spread is on target; Precise means extent of spread is small; Averaging repeated measurements improves precision, SE =  /√n –but not accuracy; seek assignable cause.

25. Diploma in Statistics Design and Analysis of Experiments Lecture 4.225 Accuracy and Precision: Example Each of four technicians made six measurements of a standard (the 'true' measurement was 20.1), resulting in the following data: Technician Data 120.219.920.120.420.220.4 219.920.219.520.420.619.4 320.620.520.720.620.821.0 420.119.920.219.921.120.0 Exercise:Make dotplots of the data. Assess the technicians for accuracy and precision

26. Diploma in Statistics Design and Analysis of Experiments Lecture 4.226 Accuracy and Precision: Example

27. Diploma in Statistics Design and Analysis of Experiments Lecture 4.227 Repeatability and Reproducability Factors affecting measurement accuracy and precision may include: –instrument –material –operator –environment –laboratory –parts (manufacturing)

28. Diploma in Statistics Design and Analysis of Experiments Lecture 4.228 Repeatability and Reproducibility Repeatability: precision achievable under constant conditions: –same instrument –same material –same operator –same environment –same laboratory Reproducibility: precision achievable under varying conditions: –different instruments –different material –different operators –changing environment –different laboratories

29. Diploma in Statistics Design and Analysis of Experiments Lecture 4.229 Measurement Capability of the MicroMeter 4 operators measured each of 8 parts twice, with random ordering of parts, separately for each operator. Three sources of variation: –instrument error –operator variation –parts(manufacturing process) variation. Data follow

30. Diploma in Statistics Design and Analysis of Experiments Lecture 4.230 Measurement Capability of the MicroMeter

31. Diploma in Statistics Design and Analysis of Experiments Lecture 4.231 Quantifying the variation Each measurement incorporates components of variation from –Operator error –Parts variation –Instrument error and also –Operator by Parts Interaction

32. Diploma in Statistics Design and Analysis of Experiments Lecture 4.232 Measurement Differences

33. Diploma in Statistics Design and Analysis of Experiments Lecture 4.233 Graphical Analysis of Measurement Differences

34. Diploma in Statistics Design and Analysis of Experiments Lecture 4.234 Average measurements by Operators and Parts

35. Diploma in Statistics Design and Analysis of Experiments Lecture 4.235 Graphical Analysis of Operators & Parts

36. Diploma in Statistics Design and Analysis of Experiments Lecture 4.236 Graphical Analysis of Operators & Ordered Parts

37. Diploma in Statistics Design and Analysis of Experiments Lecture 4.237 Quantifying the variation Notation:  E :SD of instrument error variation  P :SD of parts (manufacturing process) variation  O :SD of operator variation  OP :SD of operator by parts interaction variation  T :SD of total measurement variation N.B.: so

38. Diploma in Statistics Design and Analysis of Experiments Lecture 4.238 Calculating s E sum = 18.6sum = 7.0 s 2 = (18.6 + 7.0)/32 = 0.8 s E = 0.89

39. Diploma in Statistics Design and Analysis of Experiments Lecture 4.239 Analysis of Variance Analysis of Variance for Diameter Source DF SS MS F P Operator 3 32.403 10.801 6.34 0.003 Part 7 1193.189 170.456 100.02 0.000 Operator*Part 21 35.787 1.704 2.13 0.026 Error 32 25.600 0.800 Total 63 1286.979 S = 0.894427

40. Diploma in Statistics Design and Analysis of Experiments Lecture 4.240 Basis for Random Effects ANOVA F-ratios in ANOVA are ratios of Mean Squares Check:F(O) = MS(O) / MS(O*P) F(P) = MS(P) / MS(O*P) F(OP) = MS(OP) / MS(E) Why? MS(O) estimates  E 2 + 2  OP 2 + 16  O 2 MS(P) estimates  E 2 + 2  OP 2 + 8  P 2 MS(OP) estimates  E 2 + 2  OP 2 MS(E) estimates  E 2

41. Diploma in Statistics Design and Analysis of Experiments Lecture 4.241 Variance Components Estimated Standard Source Value Deviation Operator 0.5686 0.75 Part 21.0939 4.59 Operator*Part 0.4521 0.67 Error 0.8000 0.89

42. Diploma in Statistics Design and Analysis of Experiments Lecture 4.242 Diagnostic Analysis

43. Diploma in Statistics Design and Analysis of Experiments Lecture 4.243 Diagnostic Analysis

44. Diploma in Statistics Design and Analysis of Experiments Lecture 4.244 Measurement system capability  E   P means measurement system cannot distinguish between different parts. Need  E <<  P. Define  TP = sqrt(  E 2 +  P 2 ). Capability ratio =  TP /  E should exceed 5

45. Diploma in Statistics Design and Analysis of Experiments Lecture 4.245 Repeatability and Reproducibility Repeatabilty SD =  E Reproducibility SD = sqrt(  O 2 +  OP 2 ) Total R&R= sqrt(  O 2 +  OP 2 +  E 2 )

46. Diploma in Statistics Design and Analysis of Experiments Lecture 4.246 Laboratory 1, Part 2 A four factor process improvement study Low (–)High (+) A: catalyst concentration (%),57, B: concentration of NaOH (%),4045, C: agitation speed (rpm), 1020, D: temperature (°F), 150180. The current levels are 5%, 40%, 10rpm and 180°F, respectively.

47. Diploma in Statistics Design and Analysis of Experiments Lecture 4.247 Design and Results

48. Diploma in Statistics Design and Analysis of Experiments Lecture 4.248 Pros and Cons of omitting "insignificant" terms pro: the model is simplified the error term has more degrees of freedom so that s is more precisely estimated in small samples, comparisons are more precisely made

49. Diploma in Statistics Design and Analysis of Experiments Lecture 4.249 Pros and Cons of omitting "insignificant" terms con: statistical insignificance does not imply substantive insignificance, so that –when the excluded term has some effect below statistically significant level, the residual standard deviation is likely to increase, giving less precise comparisons, –(although this may be a pro if conservative conclusions are valued) predictions may be slightly biased.

50. Diploma in Statistics Design and Analysis of Experiments Lecture 4.250 Reading EM §1.5.3, §7.5, §8.2.1 MS Introduction to Measurement Systems Analysis BHH, §9.3