DQOs and the Development of MQOs Carl V. Gogolak USDOE Environmental Measurements Lab

2 Data Quality Objectives DQOs define the performance criteria that limit the probabilities of making decision errors by:   considering the purpose of collecting the data;   defining the appropriate type of data needed;   and specifying tolerable probabilities of making decision errors.

3 Measurement Quality Objectives MQOs are statements of performance objectives or requirements for a particular analytical method performance characteristic. MQOs are acceptance criteria for quality attributes, usually measured by project DQIs. One of the most important of these is the analytical measurement uncertainty (accuracy).

4 Uncertainty Uncertainty is defined in the ISO Guide to the Expression of Uncertainty in Measurement (GUM) as: “a parameter associated with the result of a measurement that characterizes the dispersion of values that could reasonably be attributed to the measurand.”

5 Uncertainty The uncertainty of a measured value is typically expressed as an estimated standard deviation, called a standard uncertainty. The standard uncertainty of a calculated result is usually obtained by propagating the standard uncertainties of a number of other measured values. In this case it is called a combined standard uncertainty.

6 DQOs and Uncertainty No measurement program or sampling plan can be adequately designed without some estimate of the uncertainty in the data relative to the action level.

7 Uncertainty and the Action Level Relatively large uncertainty can be tolerated: Either more accuracy or more samples are needed:

8 Connecting the MQOs to the DQOs  Decision errors are made because there is uncertainty in the data  One component of the uncertainty is analytical measurement uncertainty  To limit decision errors, the analytical measurement uncertainty should be limited to a level appropriate to the DQOs

9 MARLAP emphasizes Method Uncertainty as an MQO. Method Uncertainty refers to the predicted uncertainty of a measured value that would be calculated if the method were applied to a hypothetical laboratory sample with a specified analyte concentration. The Method Uncertainty is a characteristic of the analytical method and the measurement process.

10 Developing MQOs for Method Uncertainty Data are collected so that decisions can be made... Decisions can be made about individual samples …as for bioassays Decisions can be made about the mean of a sampled population …as for MARSSIM final status surveys

11 Decisions made about individual samples H 0 : Sample contains no radioactivity H a : Sample contains radioactivity Type I error: Decide there is radioactivity when there isn’t Type II error: Decide there is no radioactivity when there is This is the familiar framework for LLD calculations.

12 Desired Limits on Decision Errors Concentration Probability of Detection AL 0 Gray Region: The probability of detection decreases with decreasing concentration Limit the probability of missing a sample above the action level Limit the probability of flagging a blank sample

13 Lower Limit of Detection

14 The probability of detection increases with increasing concentration

15 Power Curve Concentration Probability of Detection LDLD LCLC 0 Gray Region

16 The MQO for Method Uncertainty The width of the gray region is  = AL – 0 = AL. To limit the probability of decision errors,  and , to 0.05, we have to limit the standard deviation of the analytical method,  M So that  /  M > = The performance requirement is that the upper bound for the measurement standard deviation is  MR =  / 3.29 = AL / This is essentially the same as requiring that the minimum detectable concentration (MDC) not exceed the action level.

17 Required Method Uncertainty  The true standard deviation of the measurement process,, is a theoretical quantity and is never known exactly.  The true standard deviation of the measurement process,  M, is a theoretical quantity and is never known exactly.  The laboratories estimate of  The laboratories estimate of  M is denoted u M, the method uncertainty.   The value  MR is the required upper bound for the unknown  M.   Therefore in practice  MR is an upper bound on u M, the method uncertainty.   Used this way,  MR is called the required method uncertainty and is denoted by u MR.

18 Decisions made about the mean of a sampled population H 0 : The mean exceeds the action level. H a : The mean is below the action level. Type I error: Decide mean exceeds AL when it doesn’t. Type II error: Decide mean does not exceed AL when it does.

19 Desired Limits on Decision Errors Mean Concentration Probability of Deciding Mean < AL AL DL0 Limit the probability of missing a mean above the action level Limit the probability of missing a mean below the DL Gray Region The width of the gray region, , represents the smallest concentration difference that it is important to detect.

20 Power Curve Mean Concentration Probability of Deciding Mean < AL AL DL0 Gray Region

21 The MQO for Method Uncertainty The width of the gray region is  = AL – DL. The total variance of the data is  The sampling standard deviation,  S, depends on the variability in the spatial distribution of the analyte concentrations and other factors having to do with how the sampling is performed.  The analytical standard deviation,  M, is affected by laboratory sample preparation, subsampling and analysis procedures.

22 The MQO for Method Uncertainty The sample size needed to conduct the hypothesis test with specified limits on  and  depends on the relative shift,  / . To keep sample sizes reasonable,  should be such that 1<  /  < 3. Ideally,  /   3. The cost in samples rises rapidly when  /  < 1, but there is little benefit from increasing  /  above 3.

23 The MQO for Method Uncertainty Generally it is easier to control  M than  S. If  S is large, then the best one can do is make  M small relative to  S. If  M ~  S /3, then the analytical method variance is contributing less than 10% to the total variance  2. Reducing it further will not reduce  very much. This implies that the upper bound for  M should be

24 Required Method Uncertainty If the lower bound of the gray region is zero, then the required method uncertainty is This is essentially the same as requiring that the relative standard deviation of the measurements near the action level be 10%. In other words, the minimum quantifiable concentration MQC should be no larger than the action level.

25 Required Method Uncertainty  The required method uncertainty is specified at the Action Level  At concentrations below the action level, the bound on the standard deviation  Req =  MR is constant.  At concentrations above the action level, the bound on the relative standard deviation  Req =  MR =  MR /AL is constant.  The combined standard uncertainty u c of each result may be compared to these bounds.

26

27 Required Method Uncertainty The required method uncertainty, u MR, and the required method relative uncertainty,  MR, can be used for both method selection and to develop acceptance criteria for QC sample results.