Quality Assurance Program Presenter: Erin Mustain 1

2 Recommendation 5 Benchmarks   Data quality issues have been categorized and quantified.   A detailed plan exists for addressing sources of continuing errors and correcting historical errors.   The plan has been validated with representative data samples   Substantive progress has been made toward correcting major categories of errors.   The Steering Committee agrees that progress is being made and that there is a high probability that existing data problems will be resolved.

33 Progress   Institutionalized Quality Assurance   Business Rules   Error chart Error chart   Steering committee and User groups collaboration   Eliminated system-generated violations for paper tracking SMRs

4 Total Study Error Data PopulationData Generation MigrationManual Data Entry System Limitations Training SMRs Lab errors Sampling Errors Intentional manual errors Field not populated Field auto- populated incorrectly Field not appearing Data doesn’t follow business rules Data entered into SWIM incorrectly Non-numeric data (ND, QND, etc.) not handled by eSMR System can’t handle unique orders (several facilities under one permit) No place to store data (enrollee history) Training Manuals don’t follow Business Rules Instruction not consistent Doesn’t enforce all of the Business Rules Lack of Training Manual Errors in data entry form Calculation errors Can’t easily delete records System generated duplicates (SMARTS) Business Rules not followed Typos Difficulty Data Mining Selecting the wrong link Duplicate entry

55 Current vs. Historical Data

6 Data Validation QA Reports Audits SOPs QAP DQOs What decisions will be made with this data? CIWQS QA Program 6 Training Data Verification Data Cleanup Business rules Corrective Action Integration with State Water Board QMP Communication

77 Plan and Procedures   Quality Assurance Plan Scope Roles and responsibilities Data quality indicators Quality objectives Assurance activities Problem reporting and corrective action Audits Migration and future projects

8 Plan and Procedures   Standard Operating Procedures Data Cleanup Training Document Management Corrective Action Quality Assurance Reports Audit End-user-layer enhancements and testing Database enhancement prioritization Report prioritization

9 Plan and Procedures   DIT Standard Procedures Document Maintenance & Documentation Requirements System Environment Data model, database, data integration, & maintenance Application source code integration & maintenance Application and database source code & scripts repository CVS Database tools & scripts standards Issue routing Maintenance implementation

1010 External Audit Onsite audit of Regional Boards and State Board programs Onsite audit of Division of Information Technology Data audit using stratified random design approach Policies and procedures for data entry Security, performance, and policies and procedures Accuracy of records

1111 Next Steps   Quantifying the data quality issues – Audit   Correcting historical data – recommendations to management after audit results   Validate the QAP and SOPs with representative data samples   Implement training program   Continuing process improvement at all levels (QA Program is not static)   Establish a mechanism for communication between QA Program and panel