1. Youth Outcome Questionnaire & Arkansas Indicators Compliance Rates and Satisfaction with Services Kazi Ahmed Paula Stone October 18, 2012

2. Objectives Analyze compliance rates for the Youth Outcome Questionnaire® (YO-Q®) administrations for SFY 2012 (July 1, 2011 – June 30, 2012). Note trends and improvements in compliance rates. Report on Parent and Youth Arkansas Indicators’ satisfaction questions. Summarize Arkansas Indicators Work Group concerns, analysis, and modifications of the indicators.

3. Youth Outcome Questionnaire (YO-Q ® ) Arkansas ACT 1593 of 2007 authorizes a measurement instrument and database for behavioral health outcomes. Rehabilitative Services for Persons with Mental Illness (RSPMI) providers serving Medicaid clients required to administer the Parent and/or Youth YO-Q®s at least once every 90 days. Compliance rates are calculated by determining how many clients are completing at least one Parent or Youth YO-Q® as compared with the number of clients receiving RSPMI services.

4. Data Source Reports generated by Hewlett Packard (HP) on September 21, 2012. Report based on RSPMI Medicaid claims for children 4-18 years and administered YO-Q® and Arkansas Indicators. YO-Q® and Arkansas Indicators data were extracted from the OQ® Measures Database.

5. Compliance Rate

6. Compliance Rate for children ages 4-18 Examined SFY 2012 OQ® data for the YO-Q® and RSPMI Claims data by quarters from July 2011 through June 2012. Data from 33 providers were used in the current analysis to calculate the compliance rates. ▫ Note: Providers with no matching claims data; providers with compliance rates over 100%; or providers not treating children (ages 4-18) were excluded from analysis.

7. 1 st Qtr. July-Sept. 2011

8. Chart 2: Compliance Rates for YO-Q® by Quarters of SFY 2012 Overall Compliance Rate

9. Compliance Improvement Compliance improved from 1 st quarter to 4 th quarter ▫ Average rate increased from 55% to 67%. ▫ 1 st quarter ranged from 9.2% to 100% compliance. ▫ 2 nd quarter ranged from 9.6% to 100% compliance. ▫ 3 rd quarter ranged from 20.4% to 100% compliance. ▫ 4 th quarter ranged from 23.7% to 100 % compliance. ▫ Seven providers increased rates 25% or more.  One increased from 34% to 100% compliance rate.  Another increased from 9% to 56% compliance rate. ▫ During 4 th quarter only seven providers had compliance below 50%.

10. Chart 3: Providers Ranked in the Top 25 th Percentile in 4 th Quarter SFY 2012 for YO-Q®

11. Chart 4: Providers ranked in the bottom 25 th percentile in the 4 th Quarter SFY 2012 for the YO-Q® administrations

12. Satisfaction with Services

13. Arkansas Indicators Arkansas Indicators measure quality and satisfaction with services. Every 90 days parents and youth are asked to complete a survey. Analysis was conducted on overall satisfaction rates on all providers completing a survey during the last quarter for SFY 2012 (April through June 2012).

14. Parent’s Satisfaction 49 providers completed Parent Arkansas Indicators during April-June 2012. 15,863 parents completed surveys on 14,498 clients—an average of 1.09 surveys per client. An average of 29% of satisfaction questions were unanswered. ▫ When parents complete the Arkansas Indicators, they are asked if this is their child’s first visit and are instructed to stop before answering the satisfaction questions.

15. An average of 29% of the satisfaction questionnaires were unanswered, included clients with new intakes Chart 5: Arkansas Indicators-Parent’s Satisfaction with Treatment 4 th Quarter SFY 2012

16. Youth’s Satisfaction 40 providers completed Youth Arkansas Indicators during April to June 2012. 6,209 surveys were completed by 5,776 youth-a rate of 1.07 per youth. An average of 34% of questions were unanswered. ▫ When youth are completing the Arkansas Indicators they are asked if this is their first visit and then instructed to stop before answering the satisfaction questions.

17. Chart 6: Arkansas Indicators Youth’s Satisfaction Questions for 4 th Quarter SFY 2012 An average of 34% of questions were unanswered including youth with intakes

18. Summary Compliance Rates for YO-Q®s have steadily increased during SFY 2012. The majority of providers have compliance rates over 50% (26 out of 33). Several providers have over 90% compliance rates. Both youth and parents indicate satisfaction with services: ▫ Range of 83% to 96% satisfaction for parents ▫ Range of 64% to 86% satisfaction for youth ▫ Overall satisfaction rate of 85% for parents ▫ Overall satisfaction rate of 81% for youth

19. Arkansas Indicators Recommended Changes

20. Arkansas Indicators Currently the Arkansas Indicators include 13 questions regarding the child/youth situation and seven satisfaction/barriers questions. Concerns were raised by RSPMI providers regarding the Arkansas Indicators including: ▫ Five “point in time” questions conflict with the 90 day administration schedule. ▫ Other states use a standardized instrument measuring community, home, and school functioning with different methods for administration and collection of data. ▫ Some questions included do not need to be asked every 90 days as the answer does not typically change during a treatment episode. ▫ DBHS administers a satisfaction survey for Community Mental Health Center (CMHC) clients.  The frequency of the Arkansas Indicators satisfaction survey is not ideal.  The satisfaction/barriers portion increases the length and administration time of the instrument.

21. Arkansas Indicators Work Group The Arkansas Indicators Work Group was convened in August 2012. The work group was tasked with reviewing the current Arkansas Indicators and proposing changes in content and method of collection. The work group was composed of Department of Human Services staff and Quality Assurance Officers from four RSPMI provider agencies.

22. Arkansas Indicators Purpose To collect: Demographic data to describe the population of children/ youth that are receiving behavioral health services in Arkansas Data points by which the Y-OQ® Reliable Change Index outcomes data can be sorted Measure child/youth and family functioning Track and report state level outcome data Collect consumer satisfaction data

23. Analysis of Arkansas Indicators The group examined the worker report from the Ohio Scales and the Arkansas Youth Information Form developed by Arkansas State University Office of Behavioral Health Research and Evaluation (ASU OBHRE). Information collected by providers to meet other requirements. The work group sorted the Arkansas Indicators questions into three categories:  Child or youth functioning/situation  Satisfaction  Barriers

24. Work Group Recommendations Arkansas Indicator questions should: □ Be in the same format and provide a list of choices for reporting the previous 90 days of client activity. □ Administered at the time of intake to gather data on the child/youths behaviors from the 90 days prior to treatment and administered every 90 days for the duration of treatment. The proposed Arkansas Indicators should continue to be collected in the same way, in conjunction with the Y-OQ®. The work group developed both a parent and youth version with the option of completing through parent or youth interview. Satisfaction and barriers questions were removed from the Arkansas Indicators and will be administered separately.

25. Questions?