1. REDUCING BURDEN WHILE INCREASING QUALITY AT A GOVERNMENT AGENCY David A. Marker (Westat), Mary K. Dingwall (Westat), and Marla D. Smith (U.S. EPA) Presented at Q2008 Rome Italy July 9, 2008 Contact: davidmarker@westat.com

2. 2 U.S. EPA Quality System Began in 1979 Primary Goal  Ensure that the environmental data are of sufficient quantity and quality to support the data's intended use Quality Assurance Project Plans (QAPP)  Describe and document primary data collection, secondary data usage, and data processing (incl. modeling)  Describe what is done at each step to assure quality, including who is responsible  Required for each project  Required from each contractor Data Quality Objectives (DQO)  What analyses are planned, and how accurate do they need to be  What size sample is needed to meet those objectives

3. 3 Improve and Streamline Quality Planning EPA Office of Water staff, 1 st line supervisors, and Westat  Staff were particularly motivated to improve the process Focus on activities that truly affect quality Reduce burden on both contractor and government staff Ideally: improve quality, improve transparency of data and decisions, standardize QAPP requirements to require less staff time and less cost

4. 4 Quality Improvement Process 1.Meet with representatives throughout the organization  Management and others  Identify strengths and weaknesses of existing process 2.Describe existing process – flow chart 3.Identify key quality questions at each step 4.Conduct one-on-one meetings with employees to understand their concerns

5. 5 Quality Improvement Process (cont.) 5. Conduct focus groups with contractor staff for their viewpoint 6.Develop draft revised process 7.Post for comments by all other staff (Hallway exercise) - Large pieces of paper in common hall -Management support 8.Present revised process to management for approval to implement

6. 6 New Process QAPP needs to only address steps that are relevant Address each of the quality questions associated with that box Initial QAPP due 30 days after kick-off meeting Revise QAPP as procedures change or new procedures are identified Final report should include any changes since the most recent QAPP

7. 7 Team Summary: Effluent Guidelines Development Process 1 Industry Designated in Effluent Guidelines Plan 2Review Available Data - Identify Data Needs 3Secondary Data Collection (non-EPA data sources) 4Survey Questionnaire -Engineering -Economic -Environmental Assessment 5Identify Candidates for Site Visits 6Studies on Process Modifications And Treatability 7Engineering, Economic, Statistical & Environmental Analysis of Responses 8Site Visits/Sampling Visits - Process & Treatment Assessment - In-Process & Stream Sampling

8. 8 9Site Reports on Technology 10Lab Analysis of Field Samples 11Analysis of Field Data 12Environmental Assessment (Baseline & Delta) - Modeling 14 Develop Regulatory Options 15 Calculate Effluent Limitations 16 Select Preferred Option 17 Propose Regulation Technology Costing & Loads a. Subcategor- ization b. Economic Analysis c. Cost Effectiveness d. 13 Assess Technology Technology Performance, Water Use, Costs, Residuals, Modeling, etc. Team Summary: Effluent Guidelines Development Process

9. 9 Example of QA Components for One Step #3 Secondary Data Collection  Which EPA databases can be used  Search the Web  Clarify the acceptance criteria  Include economic assessments  Define scope of other data sources

10. 10 Example of QA Components for One Step (cont.) What documentation is available about data quality from secondary source -Are there any criticisms of that source -Anyone else in EPA use that source (and what was their experience) Define QC requirements for secondary data sources (e.g., facility sampling data) Use Office of Water Quality Management Plan requirements concerning use of secondary data Data collection plan needs to obtain best data available for economic and environmental analyses

11. 11 Lessons Learned Involve all parts of the organization, especially the Quality Assurance Officer Involve customers and suppliers Give all employees opportunity to comment on draft  Be prepared for more meetings than originally planned Get final management buy-in