Performance Improvement Projects (PIPs): A Year in Review (FY16 -17)

Performance Improvement Projects (PIPs) Submissions Total Clinical Non-clinical PIPs Submitted Themes: Access to Care 20 7 13 Timeliness to Care 4 16 Quality of Care 22 15 Outcomes to Care 30 25 5 Status: Active and Ongoing 52 24 28 Completed 8 Concept Only 9

Access to Care PIPs Active or Completed PIPs 16 PIPs Concept Only, Not Yet Active 4 PIPs Nearly twice as many non- clinical PIPs 13 non-clinical PIPs 7 clinical PIPs

Access to Care PIPs Most clinical PIPs dealt with post hospitalization follow-up Most non-clinical PIPs addressed ACCESS Call Lines or 24/7 ACCESS

Timeliness of Care PIPs Active or Completed PIPs 17 PIPs Concept Only, Not Yet Active 3 PIPs Four times as many non- clinical PIPs 16 non-clinical PIPs 4 clinical PIPs

Timeliness of Care PIPs Most clinical PIPs dealt with improving psychiatric wait times Most non-clinical PIPs addressed improving timeliness to an appointment Timeliness to Appointment 4 clinical PIPs 8 non-clinical PIPs

Quality of Care PIPs Active or Completed PIPs 16 PIPs Concept Only, Not Yet Active 6 PIPs Twice as many clinical PIPs 15 clinical PIPs 7 non-clinical PIPs

Quality of Care PIPs Most clinical PIPs dealt with initial assessment or engagement Most non-clinical PIPs addressed integration of care

Outcomes of Care PIPs Large number of Concept Only, Not Yet Active 12 of the 25 PIPs rated as Concept Only were from this category Overwhelmingly clinical 25 clinical PIP submissions 5 non-clinical PIP submissions

Outcomes of Care PIPs Most clinical PIPs dealt with Recovery services Most non-clinical PIPS addressed implementation of a new service or team

Poll Results

Strengths of FY 16-17 PIPs Success of Clinical PIPs Defining the study population Identifying Interventions

Success of Clinical PIP Submissions More non-clinical PIPs (51) were submitted for review than clinical PIPs (41) 91% submission rate Non-Clinical PIPs 73% submission rate Clinical PIPs 78% of Clinical submissions were Active and Ongoing or Completed 69% of Non-Clinical submissions were Active and Ongoing or Completed

Active and Ongoing PIP strengths Defining the study population Identifying Interventions

CMS Protocol: Validating Performance Improvement Projects OMB Approval No. 0938-0786 EQR PROTOCOL 3: VALIDATING PERFORMANCE IMPROVEMENT PROJECTS (PIPs) A Mandatory Protocol for External Quality Reviews (EQR)

CMS Protocol: Validating Performance Improvement Projects 10 Protocol Steps: Select the study topic(s) Define the study question(s) Use a Representative and Generalizable study population Select the study variable(s) Use sound sampling techniques

Step 3: Use a Representative and Generalizable Study Population Each PIP must represent the entire Specialty Medi-Cal or DMC-ODS population to which the study indicators apply The Plan must decide whether to review data for the entire population or a sample of that population

Defining the Study Population – FY 16-17 PIPs No-Shows are higher for one segment of the MHP/DMC population In a Q4 FY’16-’17 PIP, the health plan analyzed data Determined that the Youth and Young Adult (YAYA) Division experienced high no-show rates for psychiatry services, as high as 36% in FY13-14, and 32% in FY14- 15. The health plan also reviewed literature which reflected no-shows as a sustained negative impact on treatment effectiveness. The health plan analyzed demographic variables of the YAYA targeted population as well.

Examples of Study Populations – FY 16-17 PIPs All clients with an inpatient hospitalization Medi-Cal eligible children and youth under the age of 18 after discharge from the Crisis Stabilization Unit. All adult clients “The target population included all adult mental health consumers regardless of age, race, gender, or language”

CMS Protocol: Validating Performance Improvement Projects 10 Protocol Steps: Select the study topic(s) Define the study question(s) Use a Representative and Generalizable study population Select the study variable(s) Use sound sampling techniques

Identifying Interventions – FY 16-17 PIPs Revise the referral and aftercare forms, streamline the referral process, and have regular meetings between Child Stabilization Unit (CSU) and Children and Family Services (CFS) staff. Support staff performing reminder calls, with two in advance and one the day of an appointment.

Identifying Interventions – FY 16-17 PIPs Assuring that the clinics were in possession of current contact/phone information, which is fairly dynamic. An appointment within three days given at discharge. This appointment is followed up by a telephone call to begin the engagement process.

Opportunities – FY 16-17 PIPs Topic Development Connecting Interventions to Indicators Project Planning and Implementation Strategies

Topic Development Defining a relevant issue that would benefit consumers, or that would improve consumer services. The purpose of the PIP is to assess and improve processes, and thereby outcomes, of care. Study Topic – this is the foundation of the PIP. Development of a PIP is creating a tool (study) that informs the MHP about program effectiveness. The focus of the PIP is to create an environment that encourages improvement – with the goal of improving services to consumers.

Topic Development Development of the study topic should be based on data collection and an analysis of the comprehensive aspects of consumer need, care and services (topics chosen should have as large a scope as possible and impact as many consumers as possible. Limiting the scope of a PIP limits its impact on improving the MHP’s services and consumer outcomes.) This should include research or a literature review that assists in supporting the topic choice. If national level information is used, the study topic discussion should also explain how the issue is pertinent to the local population.

Topic Development – Do We Have A PROBLEM??? Use DATA to identify a problem Determine possible reasons for the problem – providing more DATA Use MHP data and demographics to assist in topic development. Identify consumer barriers and develop PIPs w/ strategies to address these barriers. Establishing a baseline for the problem – more DATA

Assess Ask: Do we have a way to measure that? What data do we have that would show us if there is in fact an issue? Do we know if our issue is “out of whack” with other similarly sized counties?

Topic Development – Use Data to Identify a Problem – FY 16-17 PIPs The County was subject to a state audit, which found that psychotropic prescribing practices for foster care youth exceed state standards which had the potential to negatively impact the clinical outcomes of youth consumers. The baseline data of 17% for 30 day readmissions, is derived from admission records. Of a total of 566 admissions, 95 participants were readmitted. The readmission data is reported quarterly at CQI meetings. The current goal set by CQI for readmissions is 14%. Seven day readmission rates are also tracked and reported.

Assess Ask: Do we know if this is problem nationally? What are the national averages for this problem/issue/condition? Is there a norm we are not meeting? Is this a problem for our county?

Topic Development – Use Data to Identify a Problem – FY 16-17 PIPs A review of timeliness of access for Level 1 consumers (those who are discharging from acute care services) showed the MHP did not meet standards. MHP contractual requirements and industry practice standard calls for follow-up to be within five business or seven calendar days.

Assess If the MHP or DMC-ODS feels there is an issue, a baseline measurement must be established.

Topic Development MHPs should consider areas in which to develop PIPs on a continuous basis. If an issue that requires improvement has a potential impact on consumer outcomes, the MHP should consider how a PIP could be developed to improve the issue.

Continuous Quality Improvement How not to Assess: “I think we have an issue with X”. “Oh, ok, let’s make that a PIP.” How to begin to Assess: “I think we have an issue with Y.” “Why do you think there is an issue?” “Well, Dr. Jones says people are never here for their scheduled appointment.” OR “I keep seeing the same patients back in 10 days after discharge.” OR “Patients’ ANSA scores don’t seem to be improving”. Not yet - A PIP doesn’t start here.

Plan The MHP or DMC- ODS should gather input from stakeholders to explore solutions to the problem. What barriers exist? Can the MHP effect those barriers? How will the MHP/ DMC-ODS measure their effect? Not yet - A PIP hasn’t started yet.

Improve Identify interventions (Do not start a PIP with the intervention and then try to make it work) New treatment we want to try New training we want to try The MHP or DMC- ODS should decide what interventions they want to use to try to improve the issue after reviewing the causes and barriers. Then put those interventions in to place. A PIP has started.

Connecting Interventions to Indicators Develop measurable interventions – Appropriate interventions are key to a PIP’s ability to bring about improved mental health care outcomes. Any intervention should relate to causes/barriers identified through data analysis; should be implementable throughout the MHP system; and not be a “one-time” effort.

Connecting Interventions to Indicators Think about what next steps are necessary to impact the identified problem Identify interventions that will create the desired change. Develop at least one new intervention each year. Create interventions that can be shown to demonstrate a correlation between the issue identified and the interventions put in place.

Connecting Interventions to Indicators Interventions should not focus on procedures, but on consumers.

Interventions – FY16-17 PIPs Adjustments and new interventions as a result of identifying barriers were implemented July 1, 2016. The MHP has decided to continue the PIP for another year to assess the results of these Interventions. Interventions added or changed: 1) Intervention adjusted time for staff to call consumer within 6 days of access opening episode. 2) Intervention changed protocol that instead of consumer receiving a letter confirming an appointment and asking them to call, they will receive a letter confirming appointment and staff will call consumer.

Interventions – FY16-17 PIPs Strengths model training; Policy and procedure (making sure that all clients have an active MORS); and Improvement in the EHR. These interventions are both client focused and administrative and will most likely have clinical impact.

Interventions – FY16-17 PIPs Interventions included: Benzodiazepine contract between medication only clients and the MHP; Medication reconciliation with CURES (Controlled Substance Utilization Review and Evaluation System) database; Drug testing and coordination between primary care and emergency room physicians.

Project Planning and Implementation Strategies Understand and implement a sound method of data collection and analysis. The data collection plan should: Clearly identify all data to be collected. Identify data sources, and how/when the baseline and repeat indicator data will be collected. Identify who will collect data. Identify instruments to be used. Control for variability in clinician expertise and their ability to effectively measure the variable in question.

Project Planning and Implementation Strategies Data Analysis Plan Determine if qualitative or quantitative data (or both) are to be collected. Determine how often data will be collected (at least quarterly is recommended). Determine how to ensure that collected data are comparable to previously collected data.

Data Analysis Plan – FY 16-17 PIPs Clients hospitalized are tracked in an Excel spreadsheet on a SharePoint site so that all County staff involved in this project can view the population intended for intervention. The hospitalization data is pulled on a monthly basis from the Report 109 – Inpatient Tracking in Avatar. This is essentially the same information as held on the spreadsheet, but pulling from Avatar promotes consistency in the data retrieval. It also ensures that post discharge information is pulled for the proper clients. Based upon the names/dates in the Report 109, the presence or non‐presence of post‐discharge service delivery is checked manually in the Progress Notes of the medical record. This is labor intensive.

Data Analysis Plan – FY 16-17 PIPs Hospital Discharge data is collected on a daily basis with each admission and is summed monthly. This data has historically been collected by the SCMH Contracts Project Manager who is responsible for processing claims from providers. The Contracts Project Manager collects data from the EHR “Report 109 ‐ Inpatient Tracking.” The data in this particular report is pulled from the “Inpatient Tracking Non‐Episodic” screen in Avatar. Referrals are tracked ongoing. Clients who are already engaged in a case management program are not referred. The current program is expected to conduct a visit with their client while in the hospital and provide the 7‐day post‐discharge service. Results are summarized monthly for review with the PIP committee. Quarterly the results are presented in the form of charts. The committee reviews the information and identifies improvements needed.

Project Planning and Implementation Strategies Data Analysis Plan If manual data collection is the only choice, the PIP design must include: Appropriate staff who have appropriate clinical knowledge and skills, including conceptual, organizational, and documentation skills. Data collection instruments that provide for reliable and accurate data collection over the time of the study.

Project Planning and Implementation Strategies When periodic data is collected and analyzed, new or enhanced interventions should be considered, particularly when the data provides evidence that an intervention is not effective. Be sure that indicators are producing enough data to analyze.

Project Planning and Implementation Strategies Ensure that adequate data analysis occurs for all interventions, and that all variables involved in an initiative are analyzed. If a PIP implements multiple interventions, ensure that each are measured and all data from each intervention is analyzed.

Interventions – FY 16-17 PIPs

Project Planning and Implementation Strategies PIPs should involve consumer feedback as much as possible; consumers’ input can be valuable in determining the direction and effectiveness of interventions. This information should be included in data analysis.

Project Planning and Implementation Strategies Remember that effective analysis is not just presenting data, but providing narrative analysis of the data, with conclusions drawn as to the success of the PIP and planned follow-up activities are described.

Project Planning and Implementation Strategies When a change in performance occurs, determine whether the change is real, attributable to the interventions employed; or to an unrelated event, or random chance. Determine if the outcomes achieved are statistically significant. Real improvement results from a continuous cycle of measuring performance and ensuring that effective changes become part of the MHP’s regular organization operations. If real improvement has occurred, it should be able to achieve sustained improvement. Sustained improvement is demonstrated through repeated measurements over time.

Technical Assistance (TA) BHC offers TA through several venues, and throughout the year – we want you to be successful! Consult the PIP library on our website: www.caleqro.com Watch a video See a PowerPoint slideshow Read through other county PIPs Use the PIP Validation Tool to self-score your own PIPs Consult with your lead reviewer early and often Begin (or continue) working on your PIPs directly after your annual review Collect and analyze data AT LEAST quarterly, but monthly is better Submit a draft for feedback Set up a call to discuss ideas and/or findings

Thank you & Happy Holidays!