1. Val Bradley and Sarah Taub Human Services Research Institute
November 2002
National Core Indicators: How Does Orange County Compare? Phase IV ( ) Results

2. Project Beginnings NASDDDS and HSRI collaboration Launched in 1997
Seven field test states + steering committee
~60 candidate performance indicators
Development of data collection instruments

3. What has NCI Accomplished?
Nationally recognized set of performance and outcome indicators for developmental disabilities service systems
Benchmarks of performance
Trend data at the state & national level
Reliable data collection methods & tools

4. What are the Core Indicators?
Consumer Outcomes:
Satisfaction, choice, employment
Provider Agency/Workforce Stability:
Staff turnover
System Performance
Cost/Utilization
Access
Protection of Health and Safety
Incidents, Mortality, Restraints

5. Recent Developments
Currently 22 states plus Orange in Phase V of data collection
Mobilization of subcommittees to address specific issues
Governing structure will continue
Staff will continue to seek other sources of funding (e.g., Kennedy Foundation)
Next meeting will be in late July

6. Specific Recommendations
States want to continue to work on comparability of health and welfare indicators
Until then states will receive information on whether or not their particular trend line has changed (e.g., with respect to restraints)
Wellness items will be added to consumer survey (i.e., obesity, smoking, and exercise)

7. Specific Recommendations
Family support indicators have been added to indicator list
New states will be mentored by “veteran” states
Should pursue a “quality consortium” when new Real Choice grants are announced
Name change to National Core Indicators (NCI)

8. Participating States Phase VWA IN AZ UT NC IL IA OK VT WV KY PA MA DE RI CT MT WY NE HI
Orange
County AL SD SC ME

9. What Are State Doing With the Data?
Pennsylvania – External monitoring and quality improvement
South Carolina – Core of external monitoring
Wyoming – Annual reports
Massachusetts – Strategic planning
Maine and South Dakota – Provider profiles
North Carolina – Health indicators

10. National Core Indicators
Selected Results:
Orange County and National Benchmarks

11. Provider Survey Measures Staff Stability and Board Representation
38 agencies reported data
21% of agencies reporting provide both residential and day supports
5 NCI states collected provider survey data in Phase IV

12. Staff Turnover Rates FY1999-FY2001

13. Provider Survey: Staff Stability

14. Consumer and Family Representation on Agency Boards of Directors (FY2001)

15. Areas of Strength
Staff stability: Orange County’s turnover rate was the second lowest out of 5 NCI states reporting
Board representation: Orange County reported the highest percentage of consumers who were voting members on agency boards of directors

16. Adult Family Survey
Surveys of families with an adult family member living at home
881 surveys returned (out of about 2900)
Average age of respondent = 59
90% of respondents were parents
Most common supports received:
transportation (61%)
day services (61%)
financial (40%)
out-of-home respite (27%)
in-home support (21%)

17. Adult Family Survey

18. Adult Family Survey

19. Adult Family Survey

20. Adult Family Survey RCOC State Avg 88% 80% 93% 77% 58% 60%
(% shown = “yes” response)
RCOC
State Avg
Staff respect your choices and opinions
88%
80%
Staff are respectful and courteous
93%
Family has been informed of agency’s grievance process
77%
58%
Family satisfied with grievance resolution
60%

21. Adult Family Survey RCOC State Avg 61% 57% 54% 51% 84% 68% 46% 39%
(% shown = “yes” response)
RCOC
State Avg
Supports offered meet family’s needs
61%
57%
Help was provided in a crisis situation
54%
51%
Translators are available if necessary
84%
68%
Staff help connect family to natural supports
46%
39%

22. Adult Family Survey RCOC State Avg 62% 69% 33% 40% 32% 45% 41% 49%
(% shown = “yes” response)
RCOC
State Avg
Day provider involves family in important decisions
62%
69%
Family chooses support workers
33%
40%
Family knows how much $ is spent on family member
32%
45%
Family decides how the $ is spent
41%
49%

23. Areas of Strength Provision of informational material to families
Cultural competence (providing materials and assistance in native languages)
Respectful, knowledgeable staff
Process for filing and resolving grievances
Provision of needed services

24. Potential Areas for Improvement
Choice of support providers
Involvement of families in decision-making
Information about and control over budgets

25. Consumer Survey 456 surveys completed
81% were able to respond to Section I (compared with average 67% across other NCI states, this was the highest response rate out of all 16 states)
11% spoke primary language other than English (0.3% across all states)

26. Level of MR - National Sample

27. Place of Residence – National Sample

28. Consumer Survey Analysis
Four “scales” were created to combine sets of related items
Scales are considered to be reliable if alpha > .70
Service Coordination (.80)
Community Inclusion (.89)
Support-Related Choices (.92)
Personal Choices (.95)

29. Consumer Survey Analysis
Service Coordination Scale
Person knows service coordinator
Service coordinator helps people get what they need
Service coordinator asks people what is important to them

30. Consumer Survey – Service Coordination Scale Results
Orange County’s score = 0.83
90% know who their service coordinator is
80% said that the service coordinator helps them to get what they need
71% said that the service coordinator asked them what was important to them
Average for other 15 states = 0.81
No significant difference

31. Consumer Survey Analysis
Community Inclusion Scale
Goes shopping
Goes on errands or appointments
Plays sports or exercises
Goes out to eat
Attends religious services
Belongs to clubs or community organizations
Goes out for entertainment

32. Consumer Survey – Community Inclusion Scale Results
Orange County’s score = 0.81
Average for other 15 states = 0.78
Orange County scored significantly above average

33. Consumer Survey Analysis
Support Related Choices Scale
Chose job or day activity
Chooses support staff at home
Chooses support staff at job/day activity
Chose service coordinator
Chose residence

34. Consumer Survey – Supports Related Choices Scale Results
Orange County’s score = 0.65
Average for other 15 states = 0.61
Orange County scored significantly above average

35. Consumer Survey Analysis
Personal Choices Scale
Chose roommate
Chooses daily schedule
Chooses what to do in free time
Chooses what to buy with spending money

36. Consumer Survey – Personal Choices Scale Results
Orange County’s score = 0.80
Average for other 15 states = 0.75
Orange County scored significantly above average

37. Consumer Survey – Relationships
RCOC
State Avg
Proportion of people who have friends outside family and service system
75%
71%
Proportion of people who have a close friend
81%
80%
Proportion of people who can see friends when they want
79%
Proportion of people who can see families when they want
70%
76%

38. Areas of Strength Supporting participation in community activities
Supporting individuals to make decisions, offering choices

39. Potential Areas for Improvement
Aim for better than average scores on service coordination indicators (keeping in mind that caseloads are high in CA)
Increase focus on self-determination, opportunities for individuals (and families) to exercise control over their own supports (hiring staff, etc.)

40. What Do You Do With the Information?
Include at your web site
Prepare annual reports
Develop provider profiles
Use with sister agencies
Use in allocation decisions
Use to spot red flags

41. For More Information
Final Reports for Phase IV will be available on HSRI’s website by the end of the November: