1. Guide to the Expression of Uncertainty in Measurement Keith D. McCroan MARLAP Uncertainty Workshop October 24, 2005 Stateline, Nevada

2. Introduction  Guide to the Expression of Uncertainty in Measurement was published by the International Organization for Standardization in 1993 in the name of 7 international organizations  Corrected and reprinted in 1995  Usually referred to simply as the “GUM”

3. The Seven Sponsors  International Bureau of Weights and Measures (BIPM)  International Electrotechnical Commission (IEC)  International Federation of Clinical Chemistry (IFCC)  International Organization for Standardization (ISO)  International Union of Pure and Applied Chemistry (IUPAC)  International Union of Pure and Applied Physics (IUPAP)  International Organization of Legal Metrology (OIML)

4. Stated Purposes  Promote full information on how uncertainty statements are arrived at  Provide a basis for the international comparison of measurement results

5. Benefits  Much flexibility in the guidance  Provides a conceptual framework for evaluating and expressing uncertainty  Promotes the use of standard terminology and notation  All of us can speak and write the same language when we discuss uncertainty

6. What Is Measurement Uncertainty?  “parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand” – GUM, VIM  Examples: A standard deviation (1 sigma) or a multiple of it (e.g., 2 or 3 sigma) The half-width of an interval having a stated level of confidence

7. The Measurand  In any measurement, the measurand is defined as the “particular quantity subject to measurement”  For example, if you’re trying to determine the massic activity of 239 Pu in a specified sample of soil as of a specified date and time, that is the measurand

8. Error of Measurement  In metrology the error of a measurement is the difference between the result and the actual value of the measurand  The error is treated as a random variable With mean and standard deviation True even for systematic error (discussed later)

9. Error vs. Uncertainty  In metrology, error is primarily a theoretical concept, because its value is unknowable  Uncertainty is a more practical concept  Evaluating uncertainty allows you to place a bound on the likely size of the error  It is a critical aspect of metrology  A measured value without some indication of its uncertainty is useless

10. Random & Systematic Errors  Error can be decomposed into random and systematic parts  The random error varies when a measurement is repeated under the same conditions (e.g., radiation counting)  The systematic error remains fixed when the measurement is repeated under the same conditions (e.g, error in a γ-ray emission probability)

11. Correcting for Systematic Error  If you know that a substantial systematic error exists and you can estimate its value, include a correction (additive) or correction factor (multiplicative) in the model to account for it  Remember: the correction term or factor itself has uncertainty  A small residual systematic error generally remains after all known corrections have been applied

12. Uncertainty  Uncertainty of measurement accounts for random error and systematic error  Does not account for blunders or other spurious errors, such as those caused by equipment failure  Spurious errors represent loss of statistical control of the measurement process

13. The Measurement Model  Usually the final result of a measurement is not measured directly, but is calculated from other measured quantities through a functional relationship  We’ll call this function a “measurement model”  The model might involve several equations, but we’ll follow the GUM and represent it abstractly as a single equation:

14. Example: Radiochemistry In radiochemistry, a simple model might look like where a denotes massic activity (the measurand), C s the sample count, t s the sample count time, ε the detection efficiency, etc.

15. Input and Output Quantities  In the generic model Y = f(X 1,…,X N ), the measurand is denoted by Y  Also called the output quantity  The quantities X 1,…,X N are called input quantities  The value of the output quantity (measurand) is calculated from the values of the input quantities using the measurement model

16. Input and Output Estimates  When one performs a measurement, one obtains estimated values x 1,x 2,…,x N for the input quantities X 1,X 2,…,X N  These estimated values may be called input estimates  One plugs input estimates into the model and calculates an estimated value for the output quantity  The calculated estimate may be called an output estimate

17. Propagation of Uncertainty  When a measurement model is used to estimate the value of the measurand, the uncertainty of the output estimate is usually obtained by mathematically combining the uncertainties of the input estimates  The mathematical operation of combining the uncertainties is called propagation of uncertainty

18. Standard Uncertainty  Before propagating uncertainties of input estimates, you must express them in comparable forms  The commonly used approach is to express each uncertainty in the form of an estimated standard deviation, called a standard uncertainty  The standard uncertainty of an input estimate x i is denoted by u(x i )  Radiochemists traditionally called this “one sigma” uncertainty

19. Combined Standard Uncertainty  The standard uncertainty of an output estimate obtained by uncertainty propagation is called the combined standard uncertainty  The combined standard uncertainty of the output estimate y is denoted by u c (y)

20. Methods for Uncertainty Evaluation  The GUM classifies methods of uncertainty evaluation (for input estimates) as either Type A or Type B  Type A: method of evaluation by statistical analysis of series of observations  Type B: method of evaluation by any means other than statistical analysis of series of observations  If it isn’t Type A, it’s Type B

21. Combining Uncertainties  All uncertainty components are treated alike for the purpose of uncertainty propagation  One does not distinguish between Type A uncertainties and Type B uncertainties when propagating them to obtain the combined standard uncertainty

22. Random & Systematic  Twenty years ago, it was common to call a Type A uncertainty a “random uncertainty” and a Type B uncertainty a “systematic uncertainty”  The GUM explicitly disparages those terms now  So avoid them  But recall that the terms “random error” and “systematic error” are still accepted (when referring to error, not uncertainty)

23. Examples: Type A  Make a series of observations of an input quantity X i  Let x i be the arithmetic mean and let u(x i ) be the experimental standard deviation of the mean (the “standard error” of the mean)  Least-squares regression can also be a Type A method  If there is a well-defined number of “degrees of freedom” (number of observations minus number of parameters estimated), it’s probably a Type A method of evaluation

24. Examples: Type B  Often a Type B evaluation involves estimating a bound, a, for the largest possible error in the estimate, x i, and dividing by an appropriate constant based on an assumed distribution for the error  For example, if you believe the true value lies within ±a of the estimated value, x i, but you know nothing more than that, assume a rectangular distribution, and divide a by to obtain u(x i )  Example: Uncertainty associated with rounding on a digital display

25. Rectangular Distribution xixi x i + ax i − a

26. Triangular Distribution  Sometimes you can estimate a bound, a, for the error, but you believe that values near x i are more likely than those farther away  In this case, you might assume a triangular distribution for the error  If so, you divide a by to obtain u(x i )  Example: Capacity of a pipette, with a specified nominal volume and tolerance

27. Triangular Distribution xixi x i + ax i − a

28. Imported Values  There are many other possible Type B methods  E.g., using the value and standard uncertainty of the half-life of a radionuclide published by NNDC  A calibration certificate for a standard might provide a confidence interval for the value with some specified level of confidence, such as 95 % Assume a normal distribution and derive standard uncertainty from percentiles of that distribution (e.g., if the confidence level is 95 %, divide the half-width of the confidence interval by 1.96)

29. What about Counting Uncertainty?  Make a radiation counting measurement, where C counts are observed  Let x i = C and u(x i ) =  What type of uncertainty evaluation is this: Type A or Type B?  This method of evaluation presumes Poisson counting statistics Beware – Sometimes the distribution isn’t Poisson Note – Counting uncertainty isn’t the total uncertainty

30. Correlations  An issue sometimes neglected in uncertainty evaluation is the fact that some input estimates may be correlated with each other  May either increase or decrease the uncertainty of the final result  One common example is the correlation that often exists between the parameters for a calibration curve fit by least squares

31. Notation for Correlations  If you know there is a correlation between two input estimates x i and x j, you should evaluate it and propagate it  Estimated correlation coefficient (a number between −1 and +1) is denoted by r(x i,x j )  The estimated covariance of x i and x j is denoted by u(x i,x j )  u(x i,x j ) = r(x i,x j ) × u(x i ) × u(x j )

32. Uncertainty Propagation Formula  Most commonly used equations for uncertainty propagation are based on the general equation shown below, which the GUM calls the “law of propagation of uncertainty”  MARLAP prefers “uncertainty propagation formula”

33. Sensitivity Coefficients  The partial derivatives ∂f/∂x i that appear in the uncertainty propagation formula are called sensitivity coefficients  These derivatives are evaluated at the measured values of the input estimates  OK to approximate them – You don’t necessarily have to calculate them using formulas from calculus

34. Components of Uncertainty  The term component of uncertainty means several things, but one definition is explicit in the GUM  The component of the combined standard uncertainty, u c (y), generated by the standard uncertainty u(x i ) is the product of the absolute value of the sensitivity coefficient ∂f/∂x i and u(x i ), which may be denoted by u i (y)

35. Uncertainty Propagation  Uncertainty propagation formula is derived from a first-order Taylor-polynomial approximation of f  It is commonly used, but the approximation is not great in some situations (e.g., dividing one value by another value with a very large relative uncertainty)

36. Automatic Uncertainty Propagation  Many find the uncertainty propagation formula intimidating, but it is actually straightforward  Simple enough to be done automatically in most cases by software libraries  In the presenter’s opinion, uncertainty propagation is one of the easiest aspects of uncertainty evaluation  The hard part is understanding the measurement process well enough to recognize and evaluate uncertainties that ought to be propagated

37. Expanded Uncertainty  It is common to multiply the combined standard uncertainty, u c (y), by a factor, k, chosen so that the interval y ± ku c (y) has a specified high probability of containing the true value of the measurand  GUM calls product U = k×u c (y) an expanded uncertainty  Factor k is called a coverage factor (often k=2 or 3)  The probability that y ± U contains the true value is called the coverage probability, p

38. Summary of Steps  Define the measurand and construct the mathematical model of the measurement  Obtain estimates, x i, of the input quantities  Evaluate the standard uncertainties u(x i ), by Type A or Type B methods, and evaluate the covariance u(x i,x j ) for each pair of correlated input estimates x i and x j  Apply the model to evaluate the output estimate, y

39. Summary of Steps…Continued  Propagate the standard uncertainties u(x i ) and covariances u(x i,x j ) to obtain the combined standard uncertainty u c (y)  Optionally, multiply u c (y) by a coverage factor, k, to obtain an expanded uncertainty, U  Report the result, y, with either the combined standard uncertainty, u c (y), or the expanded uncertainty, U  Explain the uncertainty clearly

40. Questions?