1 of 36 Managing Uncertainty with Systematic Planning for Environmental Decision Making 3-Day DOE DQO Training Day 1

2 of 36 Presenter Sebastian Tindall DQO Coordinator Bechtel Hanford Inc George Washington Way MS H9-03; Room 49 Richland, WA (509)

3 of 36 History of the DQO Training Funding provided by DOE Headquarters in 1998 to develop the DQO training Development of the presentation Funding provided by EPA Headquarters to train Region personnel to present Funding provided by other interested parties (i.e., States, USACE) for training

4 of 36 DOE EPA Other London, England

5 of 36 Number of People Trained CompanyNumber of People DOE - HQ770 EPA - OSWER542 Other390 TOTAL1702 As of October 6, 2004 By Funding entity:

6 of 36 Introduction to Course and Module Objectives, Agenda, and Key Concepts Presenter: Sebastian Tindall 15 minutes DQO Training Course Day 1 Module 0

7 of 36 The Problem in a Nutshell Without systematic planning, your agency/company or your client may be subject to huge financial liability

8 of 36 Federal Facility Projects n Under the Comprehensive Environmental Response, Compensation, and Liability Act of 1980 (CERCLA) –Joint, Several, and Strict Liability –Local entities “buy” contamination FOREVER –Deep Pockets n City, County, State attorneys have issued opinions that failure to conduct an independent environmental assessment of the Site could leave them liable for the cleanup of unexpected contamination

9 of 36 Deficiencies and Inconsistencies in Environmental Decision Making n Exist because an integrated decision-making process is not being used to make environmental decisions n Result from the lack of systematic planning n Result from failure to manage uncertainty n Result from lack of standard documentation n Result in decisions that are not scientifically, technically, or legally defensible n Result in vulnerability to third-party challenges

10 of 36 What’s In It For Me (and my Organization)? n Understand that there is inherent uncertainty in all environmental decisions that use data n Reduce possibility of third-party challenges n Reduce re-work n Reduce unnecessary clean up n Cost-effective budget allocation in time of diminishing budgets

11 of 36 Course Objectives n Learn the concepts behind the jargon n Learn to use systematic planning to manage uncertainty n Use the FAM approach for more cost- effective data collection n Use the tools to generate the documentation and perform calculations

12 of 36 Uncertainty is Inherent in Data and Must be Managed n We are forced to make environmental decisions based on estimates n Estimates always involve errors n Errors in estimates are not mistakes n If unmanaged, errors in estimates CAN lead to decision errors which ARE MISTAKES n Decision errors must be managed –Acknowledge –Identify –Quantify n Severe consequences of decision errors mandate a statistical basis

13 of 36 Uncertainty is additive! Analytical + Sampling & Sub-sampling + Natural heterogeneity of the site = Total Uncertainty

14 of 36 Learn the Jargon t-test UCL - upper confidence limit AL - action level N - target population n - population units sampled  - population mean x - sample mean  - population standard deviation s - sample standard deviation H 0 - null hypothesis  - alpha error rate  - beta error rate  - width of gray region

15 of 36 Use Systematic Planning to Manage Uncertainty n The Data Quality Objectives (DQO) Process is a type of Systematic Planning –Step 1: State the Problem –Step 2: Identify the Decisions –Step 3: Identify the Inputs –Step 4: Specify the Boundaries –Step 5: Define the Decision Rules –Step 6: Specify Error Tolerances –Step 7: Optimize Sample Design

16 of 36 The Managing Uncertainty Approach Systematic Planning Dynamic Work Plan Real-Time Measurement Technologies

17 of 36 Insert M-cubed graphic

18 of 36 Foundation of Defensible Decisions n Guidance –EPA QA/G-4 (DQOs) –EPA QA/G-9 (Data Quality Assessment - DQA) –Complete documents on web n Training –Days 1 (foundation), 2 (how to) and 3 (how to) –Visual Sample Plan (VSP) n Standardized documentation –DQO e-Workbook (electronic template)

19 of 36 Foundation (cont.) - Tools n DQO Implementation Work Process Flow Chart n Scoping Checklist n DQO Web Site –DQO tools and materials –Latest version of all of today’s slides n Visual Sample Plan –Download free software n Data Quality Assessment (coming!)

20 of 36 How Many Samples do I Need? REMEMBER: HETEROGENEITY IS THE RULE!

21 of 36 The following slides show today’s agenda with the content and times of each module

22 of 36 Module 1 Evolution of the DQO Concept n Module Objective –To understand how the DQO Process has matured over time from a qualitative concept to practical implementation 20 minutes (10 minute break)

23 of 36 Module 2 EPA Inspector General Audit Reports n Module Objective: –To highlight the general findings from Environmental Protection Agency (EPA) Inspector General Audit Reports and the problems created when the DQO Process is not followed 20 minutes

24 of 36 Module 3 How Many Samples do I Need? Part 1 n Module Objective: –To describe the sources of uncertainty that can lead to errors when making decisions based on environmental sampling results and to discuss how these errors can be managed and controlled 60 minutes

25 of 36 Module 4 Key Concepts Underlying DQOs and VSP n Module Objective: –To gain a basic understanding of the key statistical concepts needed to develop defensible sampling plans 60 minutes (75 min lunch break)

26 of 36 Module 5 How Many Samples do I Need? Part 2 n Module Objective: –To describe the sources of uncertainty that can lead to errors when making decisions based on environmental sampling results and to discuss how these errors can be managed and controlled 60 minutes (5 minute “stretch” break)

27 of 36 Module 6 How Many Samples do I Need? Part 3 n Module Objective: –To describe the sources of uncertainty that can lead to errors when making decisions based on environmental sampling results and to discuss how these errors can be managed and controlled 50 minutes (5 minute “stretch” break)

28 of 36 Module 7 Introduction to The EPA 7-Step DQO Process Steps n Module Objective: –To understand the role of Steps 1-4 in the EPA 7-Step DQO Process 30 minutes

29 of 36 Module 8 Introduction to The EPA 7-Step DQO Process Steps n Module Objective: –To understand the role of Steps 5-7 in the EPA 7-Step DQO Process 25 minutes (5 minute “stretch” break)

30 of 36 Module 9 DQO Implementation Process Flow Chart n Module Objective: –To highlight the elements of the flow diagram for implementing the DQO Process 30 minutes

31 of 36 Appendix A : –Thomas Grumbly DOE DQO directive, September 1994 Appendix B : –Timothy Fields EPA OSWER directive, June 1999 Appendix C : –Expedited Site Characterization Field Quality Assurance Meeting minutes, January 1994 Note: Appendices have been removed from the binders but can be found on our web site:

32 of 36 Appendix D : Effective Data –Dr. Bart Simmons, CAL/EPA DTSC Using Field Methods: Court decisions –Dr. Al Robbat, Tufts University Dynamic Work Plans & Field Analytics –Sebastian Tindall, BHI ESC M-Cubed Approach Culture Change Flow Chart

33 of 36 Appendix E : –List of Acronyms and Symbols used in course materials Appendix F : –U.S. EPA ORDER A2 CHG 2 May 5, 2000 Appendix G: –Science Advisory Board Report to EPA: Report Enclosure D (http)://

34 of 36 We want to help you: Do it! (Get the job done - right) Prove it! (Document what/why/how) Certified

35 of 36

36 of 36 End of Module 0 Thank you Questions?