1. Child Outcome Data Broad Data Analysis

2. Broad Analysis: Child Outcomes Does our state’s data look different than the national data? Are our state child outcomes trends stable over time? Is the data trending upwards? Is the data trending downwards? Is our state performing more poorly in some outcomes than others? Are the child outcomes similar across programs? What about data quality? Can we be confident in our data?

3. Child Outcomes: State vs. National

5. National Vs. State Meaningful Differences

6. Child Outcomes: National vs. State FFY11 and State FFY12

7. Data Quality – Trends

8. Virginia Trends

9. Outcome 3: Use of Appropriate Behaviors to Meet Needs Outcome 2: Acquisition and Use of Knowledge and Skills

10. Virginia Trends

13. Child Outcome Results FFY11/FFY12 Meaningful Differences Calculation

14. Child Outcomes: National vs. State FFY11 and State FFY12

15. Child Outcomes: Local vs. State

21. Data Quality Elements Completeness of data number of children reported for the outcome/number who exited Virginia’s results: average= 65%; range for Local Systems = 17% - 100% Expected Patterns for Progress Categories Virginia’s state date is within these parameters for all three outcomes Child Outcomes State Trends Over Time As noted on previous slides, Virginia’s results do not show wide variations which would trigger concerns about data quality

22. Family Outcome Data Broad Data Analysis

23. Broad Analysis: Family Outcomes Does our state’s data look different than the national data? Are our state family outcomes trends stable over time? Is the data trending upwards? Is the data trending downwards? Is our state performing more poorly in some outcomes than others? Are the family outcomes similar across programs? What about data quality? Can we be confident in our data?

25. Family Outcomes: State Trends over Time

27. Family Outcomes: Local vs. State 2012-2013 4A: EI has helped the family know their rights

28. Family Outcomes: Local vs. State 2012-2013 4B: EI has helped the family communicate their children’s needs

29. Family Outcomes: Local vs. State 2012-2013 4C: EI has helped the family help their child develop and learn

30. Data Quality The data analysis for Virginia’ survey is quite extensive, using rigorous data analysis standards. Virginia’s response rate, like other states that use mailing as the means to disseminate the family survey, is lower than states who use other mechanisms.

31. Broad Infrastructure Analysis Broad Infrastructure Analysis GovernanceFiscal Quality Standards Professional Development Data Technical Assistance Monitoring and Accountability

32. Analysis Mechanisms Use of federal monitoring tools and procedures Communication with Local Systems through: Ongoing TA Regional Meetings Trainings System Manager Meetings Monitoring Results Record Reviews QMRs Individualized TA Local System Contract Deliverables

33. Infrastructure Analysis: Information Sources Local self-reporting Stakeholder input LSM surveys Training/meeting evaluations Monitoring/QMR Dispute Resolutions Observations through TA and other interactions

34. Broad Infrastructure Analysis Governance - Mixed Strengths Code of Virginia establishes infrastructure VICC State Interagency Agreement; interagency partnerships DBHDS local contract with Local Lead Agencies specifies LS infrastructure requirements Weaknesses Variable job roles and skill sets of LSMs Variable strength and specificity of LLA/provider contracts Variable support from LLA leadership

35. Broad Infrastructure Analysis Fiscal - Mixed Strengths Additional state funding Medicaid EI Services program Trainings on fiscal issues Strong working relationship with fiscal office at the state lead agency (DBHDS) Weaknesses Variable fiscal skill set of LSMs Code of Virginia does not require fiscal commitment at local level Variety of Local Lead Agencies Inconsistent reporting of fiscal data at local level Lack of fiscal data in state data system Perception/reality that there is not enough money

36. Broad Infrastructure Analysis Quality Standards – Not sure Practice Manual articulates expected practices, but these are not labeled quality practices Monitoring and Accountability – Strength Local self-monitoring and supervision for continuous improvement – variable Professional Development – Strength Technical Assistance – Strength Data – Weakness

37. Closing Thoughts Based on this broad review, did anything strike you as an area of focus for our Systemic Improvement Plan?