1. Some New Ideas for Reliability Studies Stefan Steiner, Jock MacKay and Richard Cook University of Waterloo shsteiner@uwaterloo.ca Quality and Productivity Research Conference Minneapolis, Minnesota

2. May 19, 20052 Outline Background and Motivation Idea 1 – adding extra observations Idea 2 – selecting units Tentative conclusions

3. May 19, 20053 Measurement reliability studies Medical examples reliability of the ultrasonographers for the measurement of stenosis in the carotid artery clinical assessment of patients with psoriatic arthritis Industrial examples Piston head diameters Parking brake tightness

4. May 19, 20054 Standard Measurement Reliability (Gage R&R) Study Goal is to estimate or

5. May 19, 20055 General Context Two sources of variation measurement model general two component model: X vs. Rest (within)(between)

6. May 19, 20056 Ideas Suppose we have complete or partial knowledge of the distribution of Y: What is the gain if we know (i.e. ) or use supplementary data from the baseline investigation? Motivation: Gage R&R study on piston head diameters What units should we select? Motivation: Disassembly/Reassembly investigation for door closing effort

7. May 19, 20057 Idea 1: Add Baseline Data Add Baseline data: Standard reliability study

8. May 19, 20058 Estimating Look at as a function of n, k, m and

9. May 19, 20059 n=10, k=3, m=1,2,….,

10. May 19, 200510 n=4, k=6, m=1,2,….,

11. May 19, 200511 n=2, k=11, m=1,2,….,

12. May 19, 200512 Conclusions for Idea 1 Effects of adding extra observations are greatest when is large, i.e. most variation comes from the parts when n is small There are some design implications

13. May 19, 200513 Idea 2: Select Specify Units Suppose the baseline distribution of Y is known, e.g. N(0,1). Motivation: Consider a problem where we wish to determine whether variation in y is due to assembly or components? In an investigation we repeatedly disassembly and reassembly two car doors, measuring the closing force each time. Standard practice is to select two units with extreme and opposite values of y

14. May 19, 200514 Assessing Idea 2 Select one unit with Disassemble & reassemble, measure Model Estimate the intra-class correlation

15. May 19, 200515 Why Use Extreme Units? Use two extremesUse two randomly chosen units What happens when all the variation is due to Assembly (measurement)? Components (process/parts)?

16. May 19, 200516 When all Variation is Due to Assembly (or Measurement) We get a lot of information from the initial (extreme) values observed in the baseline Use two extremesUse two randomly chosen units

17. May 19, 200517 Assessing Idea 2 Simulation study (use 5000 trials) Measure one unit k times Compare results for different true intraclass corrrelations Compare the approaches Using the extreme unit (initial value z) – determine the maximum likelihood estimate for Using a random unit – estimate the measurement variation with the sample stdev, then

18. May 19, 200518 = 0.2, k = 20, z = 2

19. May 19, 200519 = 0.2, k = 20, z = 4

20. May 19, 200520 = 0.2, k = 5, z = 2

21. May 19, 200521 = 0.8, k = 20, z = 2

22. May 19, 200522 Conclusions for Idea 2 Selection provides large advantages in some cases Better than random selection when is small. Good precision even with very small samples Advantage greater when number of repeated measurements is small we use a more extreme unit Selection can be used to advantage in reliability studies if we use order statistic to select units for repeated measurement.

23. May 19, 200523 Tentative Overall Conclusions For problems with two variance components, we get relatively small benefit adding extra observations on the total large benefit by selecting units extreme in the total Remaining Problems more than two components, e.g. different labs, operators etc. selection based on order statistics