National Core Indicators Overview for the State of Maine Sarah Taub & Giusi Chiri Human Services Research Institute January 30, 2003

Purpose and goals  What is NCI?  How will participation in NCI impact the Quality of Life Consumer Survey?  How will other NCI activities benefit the State of Maine?  What do the results look like?  How are states using the NCI data?

Project Beginnings  NASDDDS and HSRI collaboration  Launched in 1997  Seven field test states (plus steering committee)  ~60 candidate performance indicators  Development of data collection instruments

What has NCI Accomplished?  Nationally recognized set of performance and outcome indicators for developmental disabilities service systems  Reliable data collection methods & tools  Baseline and trend data at the state & national level  Benchmarks of performance

NCI Structure  Currently 22 states plus Orange County in Phase V (FY2003)  HSRI provides technical assistance under subcontract to NASDDDS  Subcommittees address specific issues  Meet with full steering committee annually  Funded primarily through state participation fees

Participating States Phase V

What are the Core Indicators?  Consumer Outcomes:  Employment (earnings, hours worked)  Community Inclusion  Choice and Decision-making (personal choices, support-related choices)  Self-determination (new)  Relationships  Satisfaction (with home and job)

What are the Core Indicators?  System Performance  Service Coordination  Family and Individual Participation (on boards of directors)  Utilization (types of services provided)  Financial Level of Effort  Cultural Competence (access to supports)  Access (relative to population)

What are the Core Indicators?  Health, Welfare, and Rights  Safety (mortality, injuries, crimes)  Health (routine exams)  Medications  Wellness (new)  Restraints  Respect/Rights

What are the Core Indicators?  Staff Stability and Competence  Staff Stability (turnover and vacancy rates, length of employment)  Staff Competence (new)

What are the Core Indicators?  Family Indicators  Information and Planning  Choice & Control  Access & Support Delivery  Community Connections  Family Involvement  Satisfaction  Family Outcomes

What are the data sources?  Consumer Survey  Family Survey  Adult Family Survey (at home, 18+)  Family Guardian Survey (out-of-home)  Children Family Survey (at home, <18)  Provider Survey  Staff Stability  Board Representation  System Data  Incidents  Mortality

What it means to be from… a new NCI state  Transition to new survey tool  Similar questions and structure  Some methodological differences but still comparable  Technical support re: training, administration, analysis  Risk adjustment  Adult Family Survey  53% response rate

What it means to be from… a new NCI state  Provider and system data  Common definitions  MIS improvements  General  Participation on subcommittees  Collaboration with other states, particularly New England region

National Core Indicators Selected Results

Provider Survey  Staff stability  Rising trend in staff turnover over past three years  Turnover is higher in residential settings vs. day settings  In FY2002, turnover rates ranged from 33% (Hawaii) to 54% (Indiana)

Staff Turnover Rates FY FY2001

Provider Survey: Staff Stability

Board Representation (FY2001)

Adult Family Survey  Surveys of families with an adult family member living at home  5567 total surveys (37% response rate)  Average age of respondent = 57  89% of respondents were parents  96% were primary caregivers

Adult Family Survey (% shown = “yes” response) Nat’l Avg Family receives information about services 50.7% Family helped develop service plan 72.9% State staff respect family’s choices and opinions 80.3% State staff are knowledgeable and effective 73.1%

Adult Family Survey (% shown = “yes” response) Nat’l Avg Supports offered meet family’s needs 57.2% Help was provided in a crisis situation 51.4% Translators are available if necessary 67.7% Staff help connect family to natural supports 39.2%

Adult Family Survey: Choice & Control (% shown = “yes” response) Nat’l Avg Family chooses support workers 39.6% Family has control over hiring & management of workers 36.4% Family wants control over hiring & management of workers 55.9% Family knows how much $$ is spent on behalf of person 31.9%

Consumer Survey  7917 surveys completed  67% of people interviewed were able to respond to Section I

Level of MR

Place of Residence Nursing Facility 3.7Other 4.3Apartment Program 8.5Specialized Facility 9.9Foster Care or Host Home 18.6Independent Home or Apt 24.7Group Home 29.1Parent or Relative Home

Scale Development  SCALES are composite measures that sum together a set of question responses to determine the level of some theoretical construct. EXAMPLE:  The “Community Inclusion” scale was created by adding up the answers to questions such as “Do you go shopping?”, “Do you always eat at home, or do you sometimes go out to eat?” and so forth.

Internal Consistency  A scale can be considered a reliable measure if its internal consistency is .70.  The statistic that assesses the scale’s reliability is called Cronbach’s alpha.

Comparisons Among States  The ANALYSIS of VARIANCE is a collection of techniques used to test for differences among more than two groups.  Post hoc (multiple comparison) tests provide information about which groups are different from each other.  Alternatively one can compare each state against the national average.

Consumer Survey Analysis  Four “scales” were created to combine sets of related items  All scales had alpha >.70  Service Coordination =.80  Community Inclusion =.89  Support-Related Choices =.92  Personal Choices =.95

Consumer Survey Analysis  Community Inclusion Scale  Goes shopping  Goes on errands or appointments  Plays sports or exercises  Goes out to eat  Attends religious services  Belongs to clubs or community organizations  Goes out for entertainment

N.E. Results vs. National Average Community Inclusion Scale VT (.84) CT (.79) RI (.77) Above average No difference Below average

Consumer Survey Analysis  Support Related Choices Scale  Chose job or day activity  Chooses support staff at home  Chooses support staff at job/day activity  Chose service coordinator  Chose residence

N.E. Results vs. National Average Support-Related Choices Scale VT (.84) CT (.79) RI (.77) Above average No difference Below average

Consumer Survey Analysis  Personal Choices Scale  Chose roommate  Chooses daily schedule  Chooses what to do in free time  Chooses what to buy with spending money

N.E. Results vs. National Average Personal Choices Scale VT (.84) CT (.79) RI (.77) Above average No difference Below average

Areas of Strength  92% of all respondents report that they have enough privacy  over 90% of respondents report that support staff treat them with respect  94% satisfied with home  96% satisfied with work/day program  Participation in community activities is generally high, ranging from 69% to 96%

Areas for Improvement  77% of all respondents report that service coordinators get them what they need, compared with 90% in FY2001  48% of respondents reported “sometimes” or “always” feeling lonely  only 52% of women had a GYN exam in the past year and 7% have never had one

What Can You Do with the Information?  Include on your web site  Prepare annual reports  Develop provider profiles  Use with sister agencies  Use in allocation decisions  Use to spot red flags

How Are NCI States Using Their Data?  Pennsylvania – Independent monitoring and quality improvement  South Carolina – Core component of external monitoring  Wyoming – Annual reports, CMS review  Massachusetts – Strategic planning  South Dakota – Provider profiles  North Carolina – Health indicators

For More Information  Final Reports for Phase IV (FY2002) are available on HSRI’s website:  Contact us with questions:  