The DL Blues: The Continuing Saga of Detection and Quantitation Limits Richard R. Rediske, Ph.D. Annis Water Resources Institute Grand Valley State University

Problems with the MDL MDLs vary by day, lab., analyst & method Precision at low levels is usually poor Designed for false-positives; false negatives may be reported 50% of the time Inherent uncertainties make enforcing permit MDL values impossible

Problems with the MDL Not conservative for limiting false positives.  Since replicates often processed over short time (e.g., single batch), long-term variability (s) is under estimated.  MDL is a “prediction limit. FP error small for only one future sample. For batch of 20 samples, Probability of FP = 1 - (0.99) 20  0.2. Standard deviation (s) estimated from small set of replicates, but its variability (even over a short time) is not taken into account.

Problems with the MDL MDL depends on precision (s) only and does not take bias into account.  Positive Bias - Concentration of trace contamination in “blanks” can exceed MDL. MDL does not minimize false negatives (even when negative analytical bias is absent) !  Cannot reliably report non- detects as “< MDL,” contrary to EPA and DOD guidance.  FN error at MDL is at least 50%.

MDL = 3 s  0 False Negatives at MDL  FNProbability (H 0 :  = MDL) FP Probability (H 0 :  = 0) Limit to report non-detects

LQLQ LDLD LCLC Can’t differentiate from background False Positive Error rate too high LCLC LDLD LQLQ Critical Level Limit of Detection Limit of Quantitation +

LQLQ LDLD LCLC Can differentiate from background False Positive Acceptable + + LCLC LDLD LQLQ Critical Level Limit of Detection Limit of Quantitation

LQLQ LDLD LCLC False Negative error rate too high to detect with confidence - LCLC LDLD LQLQ Critical Level Limit of Detection Limit of Quantitation

LQLQ LDLD LCLC -- LCLC LDLD LQLQ Critical Level Limit of Detection Limit of Quantitation False Negative error rate acceptable

LQLQ LDLD LCLC The analyte is detectable but the quantity is uncertain P&B LCLC LDLD LQLQ Critical Level Limit of Detection Limit of Quantitation

LQLQ LDLD LCLC The analyte is quantifiable with a known level of Precision and Bias P&B LCLC LDLD LQLQ Critical Level Limit of Detection Limit of Quantitation

Current State of the Art

History Seminal work on detection and quantitation is by Lloyd Currie Published in Analytical Chemistry in 1968 (40, p586) –Introduced terms of “critical level” (L C ), “critical value” (CRV); the “detection decision”; with a 50% confidence level “minimum detectable value” (MDV), “detection limit” (L D ) with a 99% confidence level “determination limit”, “minimum quantifiable value” (MQV); limit of quantitation” (LOQ); commonly “quantitation limit” (L Q ) required precision, accuracy, false negative error rate and qualitative identification criteria for the intended purpose.

History On December 3, 1979, EPA proposed the 600- Series organic methods (44 FR 69463) –GC/MS Methods 624 and 625 contained a “limit of detection” for each compound In Method 624, the LOD was defined as defined as the “minimum level at which entire system must recognizable mass spectra and acceptable calibration points” In Method 625, the LOD was defined as the “minimum level at which the analytical system must give mass spectral confirmation.” –The LOD in Methods 624 and 625 Were estimates of the lowest level that could be measured and the basis for the minimum level of quantitation (ML)

History Method detection limit (MDL) was first published in a paper by John Glaser and others at EPA’s laboratory in Cincinnati in 1981 in Environmental Science and Technology (15, p1426) –MDL based on Currie’s work –Employs low-level spikes rather than backgrounds –Uses Student’s t-test to allow for varying number of replicates –Has remained largely unchanged since publication

MDL and ML Promulgation On October 26, 1984 (49 FR 43234), EPA promulgated: –The 600- and Series organic methods at 40 CFR 136, appendix A The 600-Series methods contained MDLs as detection limits Methods 1624 and 1625 contained MLs as detection limits –The MDL procedure at 40 CFR 136, appendix B –Method for metals at 40 CFR 136, appendix C Method contained MDLs as detection limits

WQBEL Guidance In 1994, EPA published draft “National Guidance for the Permitting, Monitoring, and Enforcement of Water Quality-based Effluent Limitations Set Below Analytical Detection/Quantitation Levels” (WQBEL Guidance) –Guidance suggested use of the ML as a compliance evaluation threshold when the ambient water quality criterion was below the detection limit of the most sensitive analytical method Concerns –Industry - ML too low –States - ML not as protective as the MDL. EPA did not complete the WQBEL Guidance, but it precipitated further discussions of detection and quantitation limits

Refinement of ML In support of the draft WQBEL Guidance, EPA refined the ML to be consistent with the ISO/IUPAC LOQ (at the time) and the ACS LOD –ML = 10 times the standard deviation of 7 replicates ML = 3.18 times the MDL

ASTM IDE and IQE In the mid- to late- 1990s, Robert Gibbons, David Coleman, Nancy Grams, and others worked through IIAG and within ASTM Committee D19 to develop the “interlaboratory detection estimate” (IDE) and “interlaboratory quantitation estimate” (IQE) –IDE and IQE used a model of standard deviation or relative standard deviation (RSD) as a function of concentration as the basis for establishing detection and quantitation estimates

EPA Method 1631 for Mercury On June 8, 1999 (64 FR 30417), EPA promulgated Revision B to EPA Method 1631 for determination of mercury by atomic fluorescence –The Alliance of Automobile Manufacturers, Inc. and others brought suit over, among other things, the detection and quantitation limits in EPA Method 1631 and how they were developed

Settlement Agreement In Clause 6 of a Settlement Agreement signed October 19, 2000, EPA agreed to: –Re-assess procedures for determining detection and quantitation limits –Peer review the re-assessment –Provide opportunity for comment on the re- assessment –Publish a notice and invite comment on the re-assessment by February 28, 2003 –Publish a notice of final action on the re- assessment by September 30, 2004

Evaluation of Concepts Evaluation criteria for detection and quantitation –The concept must be scientifically valid It can and has been tested It has been subjected to peer review and publication It is supported by a well-defined procedure It has been accepted by the scientific community The error rate is known or can be estimated –The concept must include routine variability –The concept must be applicable in a single laboratory –The detection limit concept should identify the concentration at which there is 99% confidence that the analyte is present –The quantitation limit should identify the concentration at which the reliability of the measurement is consistent with the capabilities of a method practiced by an experienced staff in a well-operated laboratory

Withdrawal of Proposed Changes to CFR EPA published a revised procedure for comment in 2003 EPA received 136 comments that they could do better Withdrawal of proposal in November 2004 EPA announced plans to work with stakeholders to address concerns about calculation and use of detection and quantitation limits in CWA programs This action did not change the way EPA calculates MDLs and MLs

Federal Advisory Committee Formation On January 26, 2005, EPA held a public meeting to report findings in the “Situation Assessment Report on Detection and Quantitation Approaches and Uses in [EPA’s] Clean Water Act Programs” and formation of a Federal Advisory Committee On May 24, 2005 (70 FR 29743), EPA announced the first meeting of the “Federal Advisory Committee on Detection and Quantitation Approaches and Uses in Clean Water Act (CWA) Programs”

Is this too long?

FAC-FACDQ Four Members from each Group Environmental Labs Industry Water Utilities States Environmental Groups (One resigned) One EPA Member Consensus is defined as all members agree or not opposed

Key Issues Procedure vs method Many regulatory standards near or below detection limits (PCBs, Mercury, TCDD, PPCPs) Lab credibility often associated with detection limits (Mine is lower than yours) Maze of tests and matrices make it difficult to find a universally applicable technique Calculating detection limits can be expensive

Key Issues The procedures do not incorporate and apply Data Quality Objectives for bias, precision, representativeness, and comparability for lab and method performance at the detection and quantitation limits used in CWA programs, at all levels and frequencies of operations that can influence data use and interpretation relative to detection and quantitation limits.

Accomplishments Pilot Study Three detection and three quantitation procedures were studied –Detection ACIL MDL USEPA OGWDW Hubaux-Vos Yc ASTM Yc/Lc (from IDE procedure) –Quantitation ACIL ML USEPA OGWDW LCMRL ASTM IQE (at 20% and 30% RSD) Five methods studied –Inorganic ions by EPA Method (ion chromatography) –Total cyanide by spectrophotometry (EPA Method 335.4) –Metals by EPA Method (ICP-OES) –Organochlorine pesticides by GC/ECD (EPA Method 608) –Semivolatile organics by GC/MS (EPA Method 625) Between 6 and 8 labs performed Pilot Study analyses for each method

Accomplishments Single Lab Procedure Modified ACIL Procedure Demonstrates the laboratories performance at a specified level. Determines the lowest possible value achievable by the laboratory while meeting the Measurement Quality Objectives. Recognition of Method Types Censored method – methods that produce no quantitative response below a certain signal threshold (Chromatographic methods) Uncensored method – methods that produce a quantitative response for each measurement regardless of concentration (ICP)

Single Lab Procedure Does the procedure provide that qualitative identification criteria defined in the analytical method are met at the determined detection and quantitation limits? –Yes qualitative identification criteria are required to be met for any results above the DL Does the procedure adequately represent routine variability in lab performance? –Yes, the procedure uses routine method blanks and spikes generated over a period of time. Does the procedure perform on-going verification of estimates? –Yes, both false positives (through blanks) and false negatives (through spikes) are checked and the DL and QL are adjusted if the rates are too high

Single Lab Procedure Does the procedure use only data that results from test methods conducted in their entirety? –Yes, this is explicitly required Does the procedure explicitly adjust or account for situations where method blanks always return a non- zero result/response? –Yes, the mean blank value is added to the DL estimate Does the procedure capable of calculating limits using matrices other than lab reagent grade water? –Yes, it is straightforward to apply the procedure to other matrices, and there is a blank/QL spike check incorporated into the procedure for individual matrices such as a specific wastewater.

Single Lab Procedure Does the procedure assess multi-laboratory and inter-laboratory variability when data from more than one lab is used? –In a multi laboratory setting, the QL would be set at a level achieved by a specified proportion of the participant laboratories Is the procedure applicable to all users and test methods? –Yes, we believe so, any test method for which spiking is feasible Is the procedure clearly written with enough detail so that most users can understand and implement them? –We believe so – the procedure is similar to that used in the pilot study

Single Lab Procedure Is the procedure cost effective? –Yes – the procedure is more expensive than a MDL that is only performed once, but less expensive than a MDL that must be repeated each year. In addition, good QL level bias and precision information is obtained. For uncensored tests, easy to collect the blank data for DL Existing MDL data can be used for the initial estimate of QL (and censored method DL) Ongoing periodic spikes required, but No need to start over every year

Single Lab Procedure Vote on Procedure Vote: 14 Agree, 1 Not Opposed, 5 Disagree Not Approved States: 4 Agree Labs: 3 Agree, 1 Disagree Industry: 4 Agree Public Utilities: 2 Agree, 1 Not Opposed, 1 Disagree EPA: 1 Disagree Environmental Community: 1 Agree, 2 Disagree

Measurement Quality Objectives The FACDQ recommends that EPA establish quantitative MQO bounds for relevant Data Quality Indicators (DQIs) that define Quantitation for intended CWA uses. These bounds would be offered for public comment by EPA. Vote: 9 Agree, 7 Not Opposed, 1 Disagree, 1 Absent Not Approved Precision ≤ 30% RSD Accuracy (measured as recovery for single determination) = % False Negative rate ≤ 10% Ratio of Accuracy to Precision must be no less than 1.0 Example: 40% Recovery / 20% RSD = 2 O.K., Example: 20% Recovery / 30% RSD =.66 Not Acceptable Vote: 12 Agree, 5 Not Opposed, 1 Disagree, 1 Absent Not Approved

EPA use the Data Quality Objective process to set target Measurement Quality Objectives for National Quantitation Limits for use in NPDES permit compliance testing. A minimum of 6-7 labs be used to set National Quantitation Limits. Data be collected, at a minimum, over 3- 6 months. A minimum of 20 spikes be used in the calculation of each Laboratory Quantitation Limit. The data and lab be evaluated for validity prior to acceptance. An appropriate outlier test then is applied to the dataset. The data are evaluated for normality, using standard statistical tests. If the data are normally distributed then calculate the upper 95% confidence limit, which becomes the Quantitation Limit. If the data are non-normally distributed then the 95th percentile of the Laboratory Quantitation Limit data becomes the Quantitation Limit. Vote: 9 Agree, 8 Not Opposed, 1 Disagree,

December 2008 EPA proposes rule. Assuming a FACDQ recommendation that EPA accepts, propose to amend –Part 136 (Analytical Methods) to add new approach(es). –Part 122 (EPA Administered Permit Programs: The National Pollutant Discharge Elimination System) to add uses provisions. EPA –Takes comments and continues rulemaking accordingly –Conducts training/outreach for states, permittees, labs States –Comments on proposed EPA rules –Participates in EPA training/outreach –Continues existing approaches or transitions –Begins planning if federal rules are implementable with or without state rules Permittees and Labs –Comments on proposed EPA rules

December 2009 Final rule. –All DLs and QLs promulgated after this date would be required to use the new approach(es). All previously promulgated MDLs or MLs would still be valid unless re-promulgated using the new approach(es). Preamble to this final rule could contain guidance to stakeholders or it could be a separate document issued at the same time. Rules should contain dates by which entities need to have accomplished certain tasks Time lag for states to modify rules to fully implement regulations Labs prepare to implement the new procedures EPA –EPA publishes final rule and announces effective date ( ) days before actual effective date –Implements new rules or oversight of states where delegated –Begins promulgation of National Quantitation Limits based on priority

December 2009 States –Begin implementing federal rules or begin state rule promulgation (if rules are necessary, it may take another year or two) –May need to maintain duplicate system for methods/analytes with National QLs versus those without –Plan training/outreach to permittees Permittees –Newly issued permits may specify procedures to be used to set DL and QL and other steps resulting from new rules –Existing permits may be modified by states to contain new procedures –Existing permits may automatically signal changes because of language that anticipates rule-making –If state does not modify permits or have automatic change language in permits, some permits could go 5 years under the current requirements Laboratories –Begin using new procedure –May need to maintain duplicate procedures or nomenclature

Future December 2010 Date by which time labs must have generated QLs and DLs using the new procedure December 2011 Date by which time delegated states must have fully implemented procedures that comply with federal requirements

We must chose wisely!