1. MI School Data May 2012 1

2. MI School Data – Functionality Overview District/School Summary Quick Facts Openings/Closings School data file Assessment and Accountability Dashboard and Report Card MEAP, MME, MI-Access, and ACT College Readiness Indicator (ACT scores) Students not tested report Assessment revised cut scores Student Graduation/Dropout Non-resident Report Student Count Staffing/Financial Educator Effectiveness Effectiveness Ratings (Principals only 2010/11) Evaluation Factors Postsecondary Reports by High School/District Enrollment/Credit Accumulation Remedial Coursework 2

3. MI School Data – Current Work Earliest Priorities:  Migration of Data for Student Success (D4SS) Dynamic Inquiries  Additional dashboard metrics (Best Practices)  K-3 Pupil Teacher Ratio, General Fund Balance, Salaries, Days of Instruction  Additional displays/reports from MSLDS data sources:  Pupil Attendance, Retention in Grade, Pupil Mobility  Usability improvements  “Front Page,” Location Selection, “Sticky Settings”  User Administration Improvements Early Childhood  More stakeholder discussion required Additional K-12  Finance - Source: FID  Staffing - Source: REP  Special Education public reporting and data portrait queries  Top to Bottom Listing of Schools Postsecondary  Enrollment, Credit Accumulation, & Remediation - User interface  By High School  By Institution of Higher Education  Requirements initiated for additional reports  More stakeholder discussion required Workforce Reports  Workforce supply/demand study 3

4. MI School Data – Improved Location Set 4

5. Improved Location Set Sort Order 5

6. Multiple Parameter Display 6

7. A New Home Page? 7

8. CEPI Data Quality Overview May 2012 8

9. CEPI Data Quality – Overview “YOUR DATA ARE NOT NECESSARILY WRONG!” The goal of our data quality process is finding ANOMALIES, not ERRORS An ERROR is: “a deviation from accuracy or correctness” An ANOMALY is: “an odd, peculiar or strange condition, situation, quality, etc.” (definitions from Dictionary.com) 9

10. CEPI Data Quality – Applications CEPI has several data collection applications  The Michigan Student Data System (MSDS)  Graduation and Dropout Application (GAD)  Title I Supplemental Education Services (SES)  The Financial Information Database (FID)  The Educational Entity Master (EEM)  The Registry of Educational Personnel (REP)  The School Infrastructure Database (SID) We will be focusing primarily on the last three databases (REP, SID and EEM) 10

11. CEPI Data Quality – Applications 11

12. CEPI Data Quality – Collection Windows Data are submitted for each of our CEPI Applications during Collection Windows (except the EEM, which is always open for updates) REP has two collections per year  The End-of-Year (EOY) REP collection is open from April 1 through June 30  The Fall REP collection is open from September 1 through the first business day in December The SID collection is once a year from April 1 through June 30 12

13. CEPI Data Quality – Process The data quality process is similar across the applications in the School Data Quality unit Data Quality runs are completed at three points in the collection  Before the collection opens (pre)  During the collection (mid)  After the collection closes (post) Started by checking 10-20 items in EOY 2007 Expanded to over 300 in the REP collection alone for Fall 2011 13

14. CEPI Data Quality – PRE collection Analyzes data from the PRIOR collection Prior collection data cannot be modified in the current collection window Identifies data elements that can be improved upon in the current collection Each district’s authorized users are informed of the findings via e-mail shortly after the collection period opens Identifies issues in the data structure and tables of the new collection cycle before they are an issue for the districts 14

15. CEPI Data Quality – MID collection Snapshot of data submissions taken with about one month left in the collection window Identifies anomalies in the current collection Each district’s authorized users are informed of the findings via e-mail with time to modify the data before the end of the collection window Identifies issues in the data structure and tables periodically throughout the collection period 15

16. CEPI Data Quality – POST collection Snapshot of data submissions taken immediately after the close of the collection Identifies anomalies in the current collection now completed Analysis is completed in about a week Each district’s authorized users are informed of the findings via e-mail Data cleansing period takes place allowing the authorized users to modify their data prior to it being used for reporting 16

17. CEPI Data Quality – What are we looking for? System edit violations or table integrity issues Data values that are anomalies  Values outside of the expected range, but that might not be ERRORS  Values that don’t match other data  Interactions with other data collections  Issues arising out of the whole of the collection  Comparisons to prior submissions 17

18. CEPI Data Quality – System Edits The system of validates each record as it is processed by the system Ensure required fields are submitted Ensure that the dependencies with other fields are followed Most of these system edits are also built into the data quality process Issues errors and warnings  Errors prevent the record from being saved  Warnings allow the record to be saved, but the data may need to be modified 18

19. CEPI Data Quality – System Edits There are limitations to what the system can validate  Cannot look at the submission as a whole  Cannot look at the prior year’s submission  Cannot have exceptions to the rules  Cannot be as flexible as the data quality process Several of the items in the Data Quality process have been turned into new system edits 19

20. SID DATA QUALITY 20 School Infrastructure Database

21. SID Data Quality – Basics Mostly looking for outliers Issues with Shared Space Entities Dual Enrollment data in high schools and only in high schools System Edit Checks 21

22. SID Data Quality – Scatter Plots Examine scatter plots of the raw number submitted and the "rate" per student reported 22

23. SID Data Quality – Scatter Plots Identify “outliers” based on different factors Too high of a number A building with 4500 incidents of bullying Too high of a rate A building with 300 students and 450 incidents of truancy Some incident types will flag any value reported as an outlier Homicides Drive-by shootings 23

24. SID Data Quality – Robbery Plot 24

25. SID Data Quality – Robbery Plot 25 These are the lines indicating the outliers

26. SID Data Quality – Robbery Plot 26 This line indicates the minimum we want to flag as an anomaly

27. SID Data Quality – Robbery Plot 27 The five circled points are what have been identified as outliers and feedback will be sent on them

28. REP DATA QUALITY 28 Report of Educational Personnel

29. REP Data Quality – Starting out Started looking at data using Excel and Access Focused on rules that could not be built into the Application Started with a dozen checks in EOY 2007 Grew to 25 checks in Fall of 2007 Continues to grow each collection Examples: Suffixes in First or Middle Name No Title IX Coordinator Submitted Too many classes taught by a single teacher 29

30. REP Data Quality – Name Issues Data Quality Checks built on name fields:  Titles in name fields o First Name of “Dr. Timothy” o Last name of “Smith, DDS”  Name changes  Incorrectly submitted Suffixes  First names incorporating “To the Estate of”  Names of “Test Data” and other artificial names used for testing purposes 30

31. REP Data Quality – Date Issues Data Quality Checks built on date fields:  Teachers that are too young  Staff members that are too old  Staff members that are hired too young  Enforcing the order of dates o Birth Date < Hire Date < Termination Date  Terminated records without a valid termination date  Credential Date issues 31

32. REP Data Quality – Title IX Issues Data Quality Checks built on Title IX Coordinator submissions:  No Title IX coordinator Submitted  Title IX coordinator submitted with a full FTE  Title IX coordinator submitted with a terminated status and no other staff member assigned to that position Have developed over time 32

33. REP Data Quality – Current State For Fall 2011: Over 300 Checks were run Districts were notified about 48 different issues 1381 messages were sent out 1058 different users of 540 districts received data quality feedback 33

34. REP Data Quality – Near Future Data Quality Checks are being added and improved Looking improving the following issues: Grade-Levels of Students submitted in MSDS Accounting Function Codes and their use in the FID Data contained in the Michigan Online Educator Certification System (MOECS) Teacher-Student Data Link (TSDL) related issues 34

35. EEM DATA QUALITY 35 Educational Entity Master

36. EEM Data Quality – Differences EEM is different from the other collections in that it does not have a window Data quality is ongoing and periodic Often checking for data that is not in the correct format A starting point for using our data profiling tools 36

37. EEM Data Quality – Sample Issues Issues between EEM and other applications  Grades for a student or teacher  Educational Settings  Lead Administrator issues System edits working Physical Addresses that do not exist Data profiling has allowed us to find issues in the contents of the data where they might not be in a consistent form 37

38. EEM Data Quality – Profiling Finds Fields that contain both the descriptive value and the code value in the same field  County records that contain both “Wayne” and “81” referring to the same thing Leading zeros or spaces in a text field  State entries of “_ _ _ _ MI”  Congressional Districts of “1” “01” and “001” Zip Code formatting  Zip+four containing the dash or not? Capitalization inconsistencies 38

39. CEPI DATA QUALITY 39 Questions and Answers