Agenda History Performance Graphs in Oliver Provider Score Performance Measurements Questions Appendix Provider Score Metrics #3 (regularity) and #4 (frequency)

History September 2015: FIN, Providers, and POC hold a Performance Indicator summit based upon PCV Performance Indicators document HB 2264 is passed establishing performance-based contracts March 2016: Provider score feature is released 2013 2014 2015 2016 April 2016: Provider meeting to discuss performance measurements and provider score May 2015: Outcome summit at FIN to design PCV performance measures HB 1774 is passed to establish measures around safety, permanency and well-being January 2016: POC releases provider performance measure feature

Parent-Child Visitation Goals & Measures CUSTOMER EXPERIENCE Time between OPD and parent/child first visit Time between first visit and (ideally) reunification 1 Week 72 hrs GOALS 24 hrs RR STATES Requested Accepted Scheduled In Progress Resolved Original Placement Date RR entered into Oliver + assigned to Provider Original Request Date Provider Accepts | Rejects RR Provider Schedules 1st visit 1st visit occurs N th Visit occurs RR Resolved OLIVER EVENTS INSTRUMENTED Measures 1. Time to accept referral since requested 2. Acceptance within 24 hours goal 3. # of referrals accepted / rejected by provider 4. Average time to schedule first visit (72 hr goal) 5. Average time to first visit (1 week goal) Start Time: Accept / Reject 1, 2, 3 End Time: Accept / Reject Start Time: 1st Visit Scheduled End Time: 1st Visit Scheduled MEASURES 4 Start Time: 1st Visit occurred 5 End Time: 1st Visit occurred RR = Referral Request

Performance Graphs in Oliver

Time to Accept Referral Calculation Graph Measurement timing: Start: The Original Referral Request date Stop: Referral is Accepted

Number of Accepted Referrals Calculation Graph Measurement timing: This measures the number of Referrals accepted each calendar day

Number of Referrals Accepted in 24 Hours Calculation Graph This measures the number of referrals accepted within the 24 hour target. Measurement timing: Start: The Original Request Date on the Referral Stop: Referral state is changed to Accepted

Time to Schedule First Visit Since Request Date Calculation Graph Number of Days Measurement timing: Start: The Request Date on the Referral Stop: Referral state is changed to Scheduled for the first Parent-Child Visit that is attached to the Referral

Time to Schedule First Visit Calculation Graph This measures the number of days it takes to schedule a Parent-Child Visitation session. Measurement timing: Start: Referral state is changed to Accepted Stop: Referral state is changed to Scheduled

Time to First Parent-Child Visit Since Request Date Calculation Graph Number of Days Measurement timing: Start: Request Date on the Referral Stop: The first Parent-Child Visit Report date/time attached the Referral

Time to First Visit Calculation Graph Measurement timing: Start: Referral state is changed to Accepted Stop: The first Parent-Child Visit Report date/time

Provider Score

Providers Score This measurement is calculated by standardizing all two performance metrics we will review next into a standard form. Once the two metrics are standardized, the score is a sum of all the metrics. Note: Currently, there are two metrics. In the future, provider score will include four metrics. 1. 2.

Providers Score Consumer Credit Score Visitation Providers Score Score Range: 350 – 850 Calculated using 5 variables: Payment History (35%) Amounts Owed (30%) Length of Credit History (15%) New Credit (10%) Credit Mix (10%) Score Range: 0 – 500 Calculated using 4 variables: Time to Schedule (25%) Time to First Visit (25%) Visit Regularity (25%) Visit Frequency (25%)

Performance Metric 1 Average Duration of Visitation Scheduling The average duration of visitation scheduling is defined by the average number of days it takes for a provider to move the referral from accepted to scheduled status in Oliver. For each referral, we calculate the days from acceptance to the change to scheduled status.  We then calculate the monthly average number of days across all referrals in a given organization. Note: All Referral Types are included in this measurement

Performance Metric 2 Average Time Until First Visit The average time until first visit is defined as the average number of days before an initial visit takes place. For each referral, we calculate the days from acceptance to the first visit report.  We then calculate the monthly average number of days across all referrals in a given organization. Note: All Referral Types are included in this measurement

Provider Score Adjustments Needed The ability to account for: Referral type Case difficulty Incorrect Information Cases that switched providers Cases that are rejected by other providers Required time gaps from when the referral is accepted to when visits can start (ex. Parent in rehab for 2 more weeks) Potential Solutions Include: Weighted scoring approach Heating/Cooling scoring approach Olympic scoring approach

Questions How do performance measures account for case difficulty? At this time, case difficulty is not factored into the equations. This is something the team is looking into adding. Can we identify referral type? There isn’t a way in Oliver today to see referral type. We are looking at adding this functionality in the future. How are or will Performance Measures be used? Per legislation HB 1774 and HB 2264, Oliver is tracking provider’s performance, availability and geography. Measurements will be used to select provider(s) that will receive referrals.

Questions?

Thank You

Appendix

Providers Score Performance Metric #3 - Regularity Regularity is a measure of consistency for the day of week visit occur per referral across all service referrals in a given month. Frequency is determined by the number of child visits that occurred on the same day every week per referral creating a percentage. Example: a child is set to have 1 visit every week on Monday. The weekly visits attended were two on Monday, then one on a Wednesday and one on a Friday. The frequency percentage is 50% out of 100% since only two visits occurred on Monday. For an organization, the calculation is averaging all referrals regularity for the past 4 weeks. Example: an organization has 4 referrals. Each referral has a different regularity for the past 4 weeks. Referral 1 = 50% Referral 2 = 20% Referral 3 = 85% Referral 4 = 70% The organizations frequency for the past four weeks is 56% ((50%+20%+85%+70%)/4) Note: All Referral Types are included in this measurement

Providers Score Performance Metric #4 - Frequency Frequency is a measure of consistency for the volume of weekly visits across all service referrals in a given month. Frequency is determined by the number of child visits that occurred per referral creating a percentage. Example: a child is set to have 1 visit every week. The weekly visit was attended 3 of the 4 weeks. The frequency percentage is 75% out of 100%. For an organization, the calculation is averaging all referrals frequencies for the past 4 weeks. Example: an organization has 4 referrals. Each referral has a different frequency for the past 4 weeks. Referral 1 = 50% Referral 2 = 75% Referral 3 = 90% Referral 4 = 70% The organizations frequency for the past four weeks is 71% ((50%+75%+90%+70%)/4) Note: All Referral Types are included in this measurement