1. 1NSWC Corona-MS Interval DJ June 2002 Dr. Dennis Jackson 909-273-4492 DSN 933-4492 JacksonDH@Corona.Navy.Mil

2. 2NSWC Corona-MS Interval DJ June 2002 CALIBRATION INTERVAL ANALYSIS: CURRENT AND FUTURE Dr. Dennis Jackson MS30A1 June 2002

3. 3NSWC Corona-MS Interval DJ June 2002 Overview l Current Calibration Interval Methods l Interval Analysis Results l New Approaches to Calibration Interval Estimation

4. 4NSWC Corona-MS Interval DJ June 2002 Current Methods: What Is a Calibration? l Compare the measurement values from a UUT with the measurement values from a calibrator. –Deviation = UUT Measurement – Calibrator Measurement l A UUT is considered in tolerance if: –Lower Tolerance < Deviation < Upper Tolerance l Measurement Reliability is the probability of being in tolerance. l A Calibration Interval is the amount of time between calibrations that will meet a measurement reliability target (keeps the UUT in tolerance).

5. 5NSWC Corona-MS Interval DJ June 2002 Current Methods: Calibration Interval Determination 0 6 12 18 24 30 36 42 48 Test Equipment Reliability vs. Calibration Interval Calibration Interval (Months) 100 90 80 70 60 50 40 30 20 10 0 Measurement Reliability (%) 72% EOP Reliability for GPTE 85% EOP Reliability for Safety-of-Flight and Mission Critical

6. 6NSWC Corona-MS Interval DJ June 2002 Current Methods: Stages of the Calibration Interval Process Engineering Interval Est. No Further Review Gather Relevant Data Statistical Interval Est. Integrated Interval Est? DivisionReview PolicyReview TR-6 QA Yes METRL 1 23 4 5 No

7. 7NSWC Corona-MS Interval DJ June 2002 Interval Analysis Results: NAVSEA Interval Changes INTERVAL ACTION COUNT IN PROCESS148 INITIAL INTERVALS332 EXTENSIONS113 DECREASES24 NO CHANGE361 TOTAL978 (FY 2002 through April 2002)

8. 8NSWC Corona-MS Interval DJ June 2002 Interval Analysis Results: Annual Calibration Cost Avoidance NAVSEANAVY EXTENSIONS $153K 1918 (M/H) $372K 4644 (M/H) DECREASES -$40K -495 (M/H) -$60K -749 (M/H) COST AVOIDANCE $113K 1423 (M/H) $312K 3895 (M/H) (Based on changes made in FY 2002 Through April 2002)

9. 9NSWC Corona-MS Interval DJ June 2002 New Approaches to Calibration Interval Estimation l Near Term - Binomial Calibration Interval Estimation Methods –More accurate interval estimates –Alternative reliability models –Visual analysis methods l Long Term - Variables Data Calibration Interval Estimation Methods –Fixes data problems –More information on measurement characteristics –Less data required –MEASURE 2 capability with automated data

10. 10NSWC Corona-MS Interval DJ June 2002 Traditional Reliability Methods Assumptions: You know when the failure occurs. R = 1.0 at time 0. Data:Failure Times. Exponential Model: R = exp(- t)

11. 11NSWC Corona-MS Interval DJ June 2002 Tolerance Testing Data Characteristics: The failure occurs during an interval. R < 1.0 at time 0. Note: The points on this graph are observed in tolerance proportions.

12. 12NSWC Corona-MS Interval DJ June 2002 Using Traditional Methods On Tolerance Testing Data Problem: The estimates don’t match the data because the intercept must go through 1.0.

13. 13NSWC Corona-MS Interval DJ June 2002 Reliability Methods For Tolerance Testing Data Assumptions: The failure occurs during an interval. R < 1.0 at time 0. Data:Success/Failure (Binomial) Intercept Exponential Model R = R o exp(- t) = exp(  0 +  1 t)

14. 14NSWC Corona-MS Interval DJ June 2002 Current Status of Near Term Efforts l 2002 MSC Paper: “Calibration Intervals – New Models and Techniques” –Binomial Analysis, New Models, Reliability Intercepts, Initial Variables Methods l Binomial Calibration Interval Analysis System

15. 15NSWC Corona-MS Interval DJ June 2002 Benefits of Binomial Calibration Interval Estimation Methods l The use of Binomial estimation methods provides more accurate calibration interval estimates based on current statistical estimation theory. l Binomial estimation methods allow for alternative measurement reliability models, including intercept and multivariable models. l Better graphical tools provide more understanding of test equipment behavior.

16. 16NSWC Corona-MS Interval DJ June 2002 Long Term Approach: Variables Calibration Data

17. 17NSWC Corona-MS Interval DJ June 2002 Calibration Intervals Based on Variables Data l Compute a Drift Trend. l Compute a Variability Trend using residuals from the drift trend. l Obtain a Reliability Curve using the drift and variability trends. l Determine the Calibration Interval from the reliability curve. l Predict the Measurement Uncertainty using the drift and variability trends.

18. 18NSWC Corona-MS Interval DJ June 2002 Drift Trend Analysis E(d) = B 0 + B 1 t(Weighted Linear Regression on d)

19. 19NSWC Corona-MS Interval DJ June 2002 Variability Trend Analysis E(res 2 ) = C 0 + C 1 t(Linear Regression on res 2 )

20. 20NSWC Corona-MS Interval DJ June 2002 A Basis for Increasing Variability Generally, a single serial number does not show increasing variability

21. 21NSWC Corona-MS Interval DJ June 2002 A Basis for Increasing Variability However, several serial numbers could have slightly different slopes and intercepts:

22. 22NSWC Corona-MS Interval DJ June 2002 A Basis for Increasing Variability The overall effect is one of increasing variability for the population

23. 23NSWC Corona-MS Interval DJ June 2002 Reliability Curve Analysis

24. 24NSWC Corona-MS Interval DJ June 2002 Determining Calibration Intervals From Variables Data

25. 25NSWC Corona-MS Interval DJ June 2002 Current Status of Long Term Efforts l 2002 MSC Paper: “Calibration Intervals – New Models and Techniques” –Binomial Analysis, New Models, Reliability Intercepts, Initial Variables Methods l 2003 MSC Paper: “Calibration Intervals and Measurement Uncertainty Based on Variables Data” –NPSL, SCE l Variables Analysis Excel Tool –Estimates Trends, Calibration Intervals, Measurement Uncertainty l MEASURE 2 –Automated/Electronic data

26. 26NSWC Corona-MS Interval DJ June 2002 Benefits of Using Variables Data l MEASURE data is often suspect –In-Tolerance data is difficult to verify (success/failure) –Engineering review required for nearly all calibration interval determinations l Variables data is more trustworthy –This could significantly increase the number of interval analyses l Variables data provides much more information –Requires fewer calibrations to accurately determine a calibration interval than In-Tolerance data l Development of automated/electronic data recording could reduce calibration time.

27. 27NSWC Corona-MS Interval DJ June 2002 Summary l Calibration intervals minimize the amount of calibration effort required to keep test equipment adequately in tolerance. l Recent adjustments to calibration intervals will result in significant cost avoidance. l Near-term improvements using Binomial methods will provide better visual analysis and more accurate estimation techniques. l Long-term improvements using variables data methods will: –Fix data problems –Provide faster analyses with less data –Possibly reduce administrative part of calibration time