EPA’s WET Interlaboratory Variability Study Findings Marion Kelly USEPA Office of Science & Technology Office of Water

Interlaboratory Variability Study Design Overview — National Study of WET test methods which demonstrated that the methods were scientifically sound — Test methods -12 of the 17 approved WET test methods evaluated in the study — Laboratories - 56 different laboratories involved, each in an average of 3 methods Base study design = 9 EPA-sponsored labs, up to 11 non-EPA- sponsored labs Extended study design = additional non-EPA-sponsored labs

Office of Water Study Design Overview (cont.) — Samples - over 700 samples analyzed Participant laboratories each tested 3 or 4 blind samples Sample types included: effluents, receiving waters, blanks, and reference toxicants — Objectives - study assessed the following parameters for each of the 12 WET test methods: Test completion rate False positive rate Interlaboratory precision

Office of Water Peer Review Process — Peer review involvement Study plan - independently peer reviewed 10/9/ /9/1998 Study results and report – independently peer reviewed 1/01 – 4/01 — Peer reviewers Three experts in the field of aquatic toxicology and biometrics Identity blinded to EPA Experts not associated with the generation of the WET methods final rule — Peer review charge Ensure that study design and results are scientifically acceptable within the context of the intended use

Office of Water WET Test Methods Evaluated Acute — Freshwater acute fathead minnow, Pimephales promelas cladoceran, Ceriodaphnia dubia — Marine acute sheepshead minnow, Cyprinodon variegatus inland silverside minnow, Menidia beryillina mysid, Holmesimysis costata Chronic — Freshwater chronic fathead minnow, Pimephales promelas cladoceran, Ceriodaphnia dubia green algae, Selenastrum capricornutum — Marine chronic sheepshead minnow, Cyprinodon variegatus inland silverside minnow, Menidia beryillina mysid, Mysidopsis bahia red macroalga, Champia parvula

Office of Water Participant Laboratory Procurement — Solicitation 319 labs solicited for participation in the study — Prequalification All participant laboratories were required to prequalify Prequalification required documentation of capacity and capabilities, experience and proficiency, and quality assurance / quality control Only 8 laboratories failed to prequalify — Selection / level of participation Dependent upon bid cost, sponsorship, and random selection Base study design = 9 EPA sponsored labs, up to 11 non-EPA sponsored labs Extended study design = additional non-EPA sponsored labs

Office of Water Participant Laboratories (freshwater) Test MethodEPA-sponsoredNon-EPA-sponsoredTotal Base Extended Fathead acute Fathead chronic Ceriodaphnia acute Ceriodaphnia chronic Selenastrum chronic92011

Office of Water Participant Laboratories (marine) Test MethodEPA-sponsoredNon-EPA-sponsoredTotal Base Extended Silverside acute9009 Silverside chronic91010 Sheepshead acute7007 Sheepshead chronic7007 Mysidopsis chronic92011 Homsesimysis acute2Interlaboratory testing canceled Champia chronic1Interlaboratory testing canceled

Developed Study Plan Funded Study Managed Study STUDY PLAN STUDY RESULTS SOPs Collected, Prepared, & Shipped Samples Conducted Preliminary Testing Procured Labs Managed Day-to-Day Activities Tracked Samples Reviewed & Analyzed Study Data TEST DATA (Over 700 tests) Conducted WET Tests on 3 or 4 Blind Samples 7 to 35 Participant Labs per Method PARTICIPANT LABS SAMPLES Blank Reference Toxicant Effluent Receiving Water REFEREE LAB WET Variability Study Design

Office of Water Roles and Responsibilities — EPA Assemble WET technical workgroup to develop study plan and to provide technical oversight during the study Provide overall study management — Sample Control Center (SCC) operated by DynCorp I&ET Procure referee and participant laboratories Coordinate and provide day-to-day management of referee and participant laboratories Track sample shipment and receipt Review, validate, and analyze study data

Office of Water Roles and Responsibilities (cont.) — Referee Laboratories Conduct preliminary testing to determine the appropriateness of samples for interlaboratory testing Collect, prepare, package and ship test samples to participant laboratories — Participant Laboratories Conduct WET tests on blind samples during interlaboratory testing

Office of Water Interlaboratory Variability Study Timeline TaskDate Settlement agreement signed7/24/98 Preliminary study plan published8/24/98 Expert peer review conducted on preliminary study plan10/9/98 – 12/9/98 Final study plan published6/11/99 Interlaboratory testing conducted9/28/99 – 4/4/00 Preliminary study results published10/10/00 Expert peer review conducted on preliminary study results 1/01 – 4/01 Final study report and proposed rule published9/28/01

Office of Water Peer Review Process — Peer review involvement Study plan - independently peer reviewed 10/9/ /9/1998 Study results and report – independently peer reviewed 1/01 - 4/01 — Peer reviewers Three experts in the field of aquatic toxicology and biometrics Identity blinded to EPA Experts not associated with the generation of the WET methods final rule — Peer review charge Ensure that study design and results are scientifically acceptable within the context of the intended use

Office of Water Study QA — Laboratory prequalification - EPA required that laboratories prequalify for participation in the study — Test-specific QC criteria - standard quality control measures for WET testing were required in the study, including test acceptability criteria as stated in the methods manual, reference toxicant testing, and test condition monitoring — Data reporting standards - EPA required that laboratories submit all bench-level data electronically in pre-designed standard reporting templates — Independent result recalculation - EPA independently recalculated all test results from reviewed electronic data

Office of Water Results - Successful Test Completion Rates Test MethodTest completion rate Ceriodaphnia dubia acute95.2% Ceriodaphnia dubia chronic82.0% Fathead minnow acute100% Fathead minnow chronic98.0% Selenastrum capricornutum chronic (with EDTA)63.6% Mysidopsis bahia chronic97.7% Sheepshead minnow acute100% Sheepshead minnow chronic100% Inland Silverside minnow acute94.4% Inland Silverside minnow chronic100%

Office of Water Results – False Positive Rates Test MethodSurvival Endpoint Sublethal Endpoint Ceriodaphnia dubia acute0.00%NA Ceriodaphnia dubia chronic0.00%3.70% Fathead minnow acute0.00%NA Fathead minnow chronic0.00%4.35% Selenastrum capricornutum chronic (with EDTA)NA0.00% Mysidopsis bahia chronic0.00% Sheepshead minnow acute0.00%NA Sheepshead minnow chronic0.00% Inland Silverside minnow acute0.00%NA Inland Silverside minnow chronic0.00%

Office of Water Results – Interlaboratory Precision (CV%) Test MethodLC50IC25 Ceriodaphnia dubia acute29.0%NA Ceriodaphnia dubia chronic21.5%35.1% Fathead minnow acute20.0%NA Fathead minnow chronic13.4%20.9% Selenastrum capricornutum chronic (with EDTA)NA34.3% Mysidopsis bahia chronic31.2%41.3% Sheepshead minnow acute26.0%NA Sheepshead minnow chronic8.72%10.6% Inland Silverside minnow acute38.5%NA Inland Silverside minnow chronic40.6%43.8%

Office of Water Study Results Summary — Successful test completion rate - 8 of 10 methods had successful test completion rates greater than 90% — False positive rate - All 10 methods had false positive rates less than 5% (0.00% %) — Interlaboratory variability - Ranged from 8.72% to 40.6% for LC50s and from 10.6% to 43.8% for IC25 values Comparable to chemical methods promulgated at 40 CFR Part 136 Below interlaboratory variability previously cited for the WET methods

Comparison of WET Method Variability at Method Promulgation and Reported in the WET Variability Study Reported in WET Variability StudyReported at time of promulgation Interlaboratory Variability (%CV) Ceriodaphnia dubia acute Ceriodaphnia dubia chronic Fathead minnow acute Fathead minnow chronic Sheepshead minnow acute Sheepshead minnow chronic Inland silverside acute 44.2% 42% 35% 34% 42% 44.2% 42.2% 29.0% 35.0% 20.0% 20.9% 26.0% 10.5% 38.5%

Office of Water For More Information — Contact: Marion Kelly U.S. EPA Office of Water Engineering and Analysis Division (4303T) 1200 Pennsylvania Ave., N.W. Ariel Rios Building EPA West Washington, DC (202)