Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012

Topics Why identify critical markers of high quality data? Anticipated use 3 areas of focus Draft critical markers Where these might be discussed in the APR Reactions? 2 Early Childhood Outcomes Center

Looking for… Proposed markers that suggest a state’s EC outcomes systems may produce high quality data Use markers to: - systematically track over time - track internally (within states) - to produce a national picture 3 Early Childhood Outcomes Center

Anticipated Use Tracking state progress (based on APR info) Compare data from each state to a series of critical markers for summaries –X % of states met standard on each marker nationally –X% of states met standard on 7 out of 10 markers NOT a state by state report card Share detailed info with each state, upon request 4 Early Childhood Outcomes Center

3 Areas Completeness of data 5Early Childhood Outcomes Center

Completeness of data Missing data concepts –Are whole forms missing? (Have data from all kids expect to have data from) –Are the forms that you have complete? 6 Early Childhood Outcomes Center

3 Areas Completeness of data Accuracy of data 7 Early Childhood Outcomes Center

Accuracy of Data Do the data reflect the “true” performance of children in the program –Unknowable, so…. Are the patterns in data what you would expect to see if the data were accurate? Do they make sense? Or, are there red flags that raise more questions? 8 Early Childhood Outcomes Center

3 Areas Completeness of data Accuracy of data State efforts related to tracking quality of data 9 Early Childhood Outcomes Center

Criteria for Selection of Markers Important –necessary for or indicative of high quality data Accessible to ECO –Information to determine presence or absence of marker is available 10 Early Childhood Outcomes Center

Draft Markers: Completeness State calculates and publicly reports number of children missing outcomes data Percent of missing outcomes data is less than 5%. –Is this reasonable? Percent of missing data by proxy calculation is less than. –40% of exiters for Part C –20% of child count for Early Childhood Outcomes Center

Part C percent of exiters Early Childhood Outcomes Center

Draft Markers: Accuracy/Patterning State % in a is not overly high (GT 5%) State % in b is not overly low (LT 5%) or high (GT 50%) State % in c is not overly low (LT 5%) or high (GT 50%) State % in d is not overly low (LT 5%) or high (GT 50%) State % in e is not overly low (LT 5%) or high (GT 65%) 13Early Childhood OutcGTomes Center

Knowledge and Skills Part B 619 proportion of children that make no progress (progress category a) 14Early Childhood Outcomes Center

Knowledge and Skills Part B 619 proportion of children who improved functioning but not sufficiently to move nearer to functioning comparable to same age peers (progress category b) 15Early Childhood Outcomes Center

Knowledge and Skills Part B 619 proportion of children who improved functioning to a level nearer to same age peers but did not reach it (progress category c) 16Early Childhood Outcomes Center

Knowledge and Skills Part B 619 proportion of children who improved functioning to reach a level comparable to same aged peers (progress category d) 17Early Childhood Outcomes Center

Knowledge and Skills Part B 619 proportion of children who maintained functioning at a level comparable to same aged peers (progress category e) 18Early Childhood Outcomes Center

Percent of states meeting the Critical Markers for Early Childhood Outcomes Center Accuracy of data Completeness of data Both Part C Part B

Draft Markers: State Quality Review of Data 20 Early Childhood Outcomes Center

21 Early Childhood Outcomes Center Completeness/ Missing Data Accuracy/ Patterns Location in Suggested APR Template

22 Early Childhood Outcomes Center State Quality Review of Data Location in Suggested APR Template

Questions for you Are these markers important? –Related to high quality data –Things you value and might track or are already tracking them? –Would tracking them help you improve your system? Is there anything else you can think of as a good marker of high quality data 23 Early Childhood Outcomes Center

Summary – Draft Critical Markers State calculates and publicly reports number of children missing outcomes data Percent of missing outcomes data is less than 5% Percent of missing data by proxy calculation is less than X State % in a is not overly high (GT 5%) State % in b is not overly low (LT 5%) or high (GT 50%) State % in c is not overly low (LT 5%) or high (GT 50%) State % in d is not overly low (LT 5%) or high (GT 50%) State % in e is not overly low (LT 5%) or high (GT 65%) State conducts data quality checks State’s own analyses provide evidence of high quality data 24

Comments, Reactions, Questions? 25