The Road to Improved Quality Management Q-Town The Road to Improved Quality Management 31st National Energy & Environmental Conference American Society for Quality September 12-15, 2004

Framework for Improved Quality Management Policy requiring quality systems Implementation of quality systems Evidence of improved quality Procedures within quality systems ?? Evidence of functioning quality systems ??

Where We Are Now Success: Policies are in place. i.e., administrative aspects requiring quality systems and the procedures they dictate Success: All Program offices and Regions have approved quality management plans.

Metrics for Policy/Procedures Currently able to track activities through review of annual reporting submissions Resources QA Assistance Training Assessments (technical and management) Innovations Resources: FTE and $ Assistance: review of QMPs or QA Project Plans, planning Training: needs and courses provided and taken Assessments: project-specific audits, data quality assessments, performance evaluations, and assessments of quality systems

Are Quality Systems Supporting Defensible Decisions? Issue: Only tracking procedural elements, not implementation of quality systems or evidence of improved quality. Issue: Management systems assessments currently focused on procedural aspect of individual Quality Management Plans. Shift to implementation and evidence needed.

Goal Establish new metrics that demonstrate quality system implementation with decisions supported by appropriate data quality.

Expanding Our Understanding of Quality: Policy/Procedures Improve current annual reporting requirements template Better standardization of definitions Identify categories for types of training, assessments, assistance, etc. and their impacts Align with business processes for decision support

Expanding Our Understanding of Quality: Implementation Track # of QA Project Plans written out of total # of activities that should have had a QA Project Plan Track # of grants, contracts, etc., that incorporate quality system requirements out of total # that should have incorporated quality system requirements 3rd bullet added by Patricia (she suggested deleting 2nd bullet)

Expanding Our Understanding of Quality: Evidence Look at challenges to data quality From Information Quality Guidelines In regulatory development, permits, Superfund remediation decisions, enforcement actions Identify target population; count # of actions with No challenge (successful implementation) Challenge unsuccessfully defended Challenge successfully defended

Expanding Our Understanding of Quality: Evidence (cont.) Look at achievement of quality measures identified in: Annual Performance Reports Government Performance and Results Act (GPRA) measures

Questions in need of Answers Can the right set of metrics replace or reduce the need for management assessments? Are a standard set of metrics flexible enough to measure diverse lines of business? Have programs matured to a level to improve and sustain performance under a results-based regime?

Reggie Cheatham (202) 564-7713 cheatham.reggie@epa.gov Q-Town Reggie Cheatham (202) 564-7713 cheatham.reggie@epa.gov