Using data for program improvement Early Childhood Outcomes Center1

2 Using data for program improvement E vidence I nference A ction

Early Childhood Outcomes Center 3 Evidence Evidence refers to the numbers, such as “45% of children in category b” The numbers are not debatable

Early Childhood Outcomes Center 4 Inference How do you interpret the #s? What can you conclude from the #s? Does evidence mean good news? Bad news? News we can’t interpret? To reach an inference, sometimes we analyze data in other ways (ask for more evidence)

Early Childhood Outcomes Center 5 Inference Inference is debatable -- even reasonable people can reach different conclusions from the same set of numbers Stakeholder involvement can be helpful in making sense of the evidence

Early Childhood Outcomes Center 6 Action Given the inference from the numbers, what should be done? Recommendations or action steps Action can be debatable – and often is Another role for stakeholders

Early Childhood Outcomes Center 7 What can we infer? Poll results A: –Candidate I.M. Good 51%, Candidate R.U. Kidding 49% (+ or – 3%) Poll results B: –Candidate I.M. Good 56%, Candidate R.U. Kidding 44% (+ or – 3%)

Early Childhood Outcomes Center 8 Program improvement: Where and how –At the state level – TA, policy –At the regional or local level – supervision, guidance –Classroom level -- spend more time on certain aspects of the curriculum –Child level -- modify intervention

Early Childhood Outcomes Center 9 Key points Evidence refers to the numbers and the numbers by themselves are meaningless Inference is attached by those who read (interpret) the numbers You have the opportunity and obligation to attach meaning

10 E – I – A Jeopardy COSF users unaware of the need to answer the yes/no progress question 90% of exit COSFs in Program B missing a response to the yes/no progress question Revise COSF procedures to emphasize completion of yes/no progress question Conduct staff development on using the 7-point rating scale 75% of children in Program A received entry ratings of 2 COSF users misunderstand the definition of points on the 7-point scale Currently used tools are not accurately assessing children’s social emotional skills Invest resources in materials for assessing social- emotional skills 45% of children reported in category ‘e’ for statewide progress data, Outcome 1 $100 $200 $100 $300 $200 $300 $200 $100 $300

Early Childhood Outcomes Center 11 Continuous Program Improvement Plan (vision) Program characteristics Child and family outcomes Implemen t Check (Collect and analyze data) Reflect Are we where we want to be?

Early Childhood Outcomes Center 12 Tweaking the System Plan (vision) Program characteristics Child and family outcomes Implemen t Check (Collect and analyze data) Reflect Are we where we want to be? Is there a problem? Why is it happening? What should be done? Is it being done? Is it working?

Early Childhood Outcomes Center 13 Continuous means… ….the cycle never ends.

Early Childhood Outcomes Center 14 Outcome questions for program improvement, e.g. Who has good outcomes = Do outcomes vary by… Region of the state? Eligibility category? Services received? Age at entry to service? Type of services received? Family outcomes?

Early Childhood Outcomes Center 15 Examples of process questions Are ALL services high quality? Are ALL children and families receiving ALL the services they should in a timely manner? Are ALL families being supported in being involved in their child’s program? What are the barriers to high quality services?

Early Childhood Outcomes Center 16 Working Assumptions There are some high quality services and programs being provided across the state. There are some children who are not getting the highest quality services. If we can find ways to improve those services/programs, these children will experience better outcomes.

Early Childhood Outcomes Center 17 Examining COSF data at one time point One group - Frequency Distribution –Tables –Graphs Comparing Groups –Graphs –Averages