Don’t Waste My Time: Here’s Why Our Data Look Bad and What We Really Need to Improve the Quality of the Data NCES MIS Conference 2012

Agenda Why did we propose this session? Informal poll grading data quality (DQ) Identify current resources to support DQ Feedback/discussion – What would it take to REALLY improve DQ?

Why this Session? We: – believe responsible use of data is powerful – are concerned about the quality of some of the data we steward – are concerned about how data are being used* – feel a sense of urgency to get the data right…so there is a chance the data will be used responsibly, proactively, and often!

How We Want to Look When We Review the Data…

How We Actually Look When We Review the Data… Photo(s) are stock photos. Release for web use of all photos on file.

Data Quality Assessment If you did a DQ assessment today…what’s the health of your data? Photo(s) are stock photos. Release for web use of all photos on file.

Health of IDEA Data At First Submission to ED “C” “C+” Incomplete, missing elements, computational errors, impossible, improbable Photo(s) are stock photos. Release for web use of all photos on file.

What Does Healthy Data Look Like? “Fit for use” EDGB DQ elements – Timeliness – Accuracy – Completeness – Validity – Usability Usable…Dependable Photo(s) are stock photos. Release for web use of all photos on file.

Current Resources to Improve DQ Federal State From participants

Federal Resources for States Technical Assistance – General, targeted, or intensive Pre-submission tools – Instructions! Q&As! Definitions! – Webinars – conference sessions - calls Submission tools – Reports of “impossible” data errors – Reports that look like legacy reports Post-submission tools

Resources Within ED EDFacts Data Governance!!!! – It ALL goes back to relationships. EQuIP

State/District/School Resources Georgia What resources do you use to ensure data quality? – What is data verification? – What is data validation? – What happens at the SEA and LEA levels?

Do We Agree There’s a Gap Between Reality and Desired Data Quality?

What would it take to REALLY improve DQ? Systems changes? Personnel? Tools? ??????????

So How Do We Take the Next Step? School District State Federal

From the Participants During the Session Currently Doing to Improve DQ: -Creating reports for intended audience -Coaching -Validation checks -Coding -Providing definitions -Guidance about improvement Challenges to Improve DQ: -Late data submissions -Resource allocations -Format violations -Source input errors -Early system maturity related to SLDS

From Participants During Session: What’s the Difference between Verification and Validation? Verification -Human checks -Superintendent sign-off -Audits – monitor using independent source -Personnel training about data: here’s what you should know Validation -Something automated -Built via IT -Catch outliers -Year-to-year change reports -Business rules -Verify reliability

From Participants During Session: What Can USDOE Do to Impact State Data Quality? Minimize change – keep data elements stable! 2-year advance notice for any data changes Audit paper & electronic files Use the data or lose the quality Communicate with states and locals about our data use More public relations about benefits to using Common Education Data Standards (CEDS) Less reporting flexibility Get LEA input about data ALL Conferences: What’s collected and WHY…Build shared responsibility about the data Usability testing: Talk about how usable the data are Include the vendor community in the discussions Quality control the EDFacts file specifications Understandable descriptions of data elements (not just for techies)

Thank You Bonnie Dye: