1. OSHPD WET 5-Year Plan: 2013 Needs Assessment & 2008-2013 Evaluation Initial Findings Resource Development Associates December 13, 2013 Kevin J. Wu, MPH Linda A. Hua, PhD

2. Topics 2  County-Reported Assessments  Educational Capacity  Program Evaluation  Forthcoming  Q&A

3. Highlights County-Reported Assessments 3

4. County-Reported Assessments: Overview 4  Purpose: To identify key needs in the public mental health workforce including:  Users of Public Mental Health System  Shortages  Hard-to-Fill, Hard-to-Retain  Declining Needs/Needs Met  Diversity Needs  Language Needs  State Administered WET Program Need/Participation  Analyses by:  Statewide  MHSA WET Region  County Size

5. County-Reported Assessments: Data Sources 5  OSHPD-led 2013 County-Reported Needs Assessment  MHSA Annual Updates  OSHPD-led Community Forums, Surveys, Key Informant Interviews  RDA-led County Needs Follow-Up Survey

6. County-Reported Assessments: Hard-to-Fill & Hard-to-Retain Positions 6

7. County-Reported Assessments: Workforce Shortages 7

8. County-Reported Assessments: Diversity Needs 8

9. County-Reported Assessments: MHSA WET Program Participation Rates by County Size 9 County Sizes Small (<200K people) Medium (200-800K people) Large (>800K people)

10. County-Reported Assessments: Key Findings – Public Mental Health Users  Users of the public mental health system are predominantly located in the southern part of the state, more likely to be of a minority race/ethnicity, and more likely to be adults. 10

11. County-Reported Assessments: Key Findings – Public Mental Health Workforce Needs  Counties’ identification of their workforce shortages is consistent with their hard-to-fill, hard-to- retain positions.  Psychiatrists are of highest need in the public mental health system, but educational graduation information regarding psychiatrists has not been available.  The Superior Region, small counties, and medium counties identified persons with bilingual capabilities as workforce shortages and hard-to-fill, hard- to-retain positions more often than any other MHSA region or large counties.  Most counties did not identify any declining workforce needs or needs met, except for the Southern Region counties.  Of those counties that identified declining workforce needs or needs met, non-licensed mental health staff members were most frequently noted.  Workforce race/ethnicity diversity needs identified by the counties coincide with the race/ethnicity compositions of the public mental health user populations across the state’s MHSA Regions. 11

12. County-Reported Assessments: Key Findings – Public Mental Health Workforce Needs  Workforce language diversity needs identified by the counties are reflective of the race/ethnicity compositions of the public mental health user populations across the state’s MHSA Regions.  The Southern Region (including Los Angeles County) has a workforce that is largely meeting the language needs of its Hispanic/Latino public mental health system user population.  Counties’ designated positions for consumers and/or family members are largely reserved for peer and administrative/clerical positions, not provider/professional positions, and do not come with set wages. 12

13. County-Reported Assessments: Key Findings – Statewide-Administered WET Programs  Statewide-WET programs are mostly being utilized by large counties.  The WET Stipend Program’s participation across the MHSA regions reflects the state’s distribution of graduates with mental health-related degrees or certificates.  The WET Psychiatric Residency Program is underutilized compared to the counties’ desire for psychiatrists.  The WET Mental Health Loan Assumption Program (MHLAP) is utilized by most counties, especially the large ones.  The WET Physician Assistant Program is being utilized in few counties throughout the state, but is consistent with the low rate of physician assistant graduates being produced across the state.  Los Angeles County has the highest number of graduates with mental health-related degrees or certificates, but it does not participate in the WET Psychiatric and Physician Assistant Residency Stipend Programs. 13

14. County-Reported Assessments: Limitations 14  Variability in consistency of County-Reported Assessments data  Burden of data collection and reporting on county agencies  Unclear if counties reported on behalf of contractors as well

15. Highlights Educational Capacity 15

16. Educational Capacity: Overview 16  Purpose: To identify the pipeline of future mental health providers

17. Educational Capacity: Data Sources 17  Integrated Postsecondary Education Data System (IPEDS)  California Postsecondary Education Commission (CPEC)  OSHPD-led Educational Institutions Survey

18. Educational Capacity: Graduates in All Mental Health Disciplines Projections 18 2014

19. Educational Capacity: Distribution of Mental Health Graduates by Discipline (1999-2009) 19 198,424 total graduates between 1999-2009

20. Educational Capacity: Distribution of Schools and Graduates 20 Educational InstitutionsGraduates

21. Educational Capacity: Limitations 21  IPEDS data has incomplete reporting  CPEC student pipeline data is truncated at 2009  Low response rates to RDA’s educational institutions survey

22. Highlights Program Evaluation 22

23. Program Evaluation: Approach 23  Evaluated program impact by actions and direct outcomes:  General Capacity  Cultural & Linguistic Competency  Formal Education Structure & Curricula  People with Lived Experience in the Workforce  Filling Gaps in the Five Regions  Data gathered via multiple methods:  Baseline: 2008 needs assessment  Progress: 2013 progress reports  Impact: 2013 needs assessment & RDA interviews and county survey

24. 24 Program Evaluation: General Capacity Hardest-to-fill/retain positions in 2008 # MHLAP Participants Relevant State- Administered WET Programs # Participants Total # 2009- 2013 Psychiatrist, General228Psychiatric Residency25253 Licensed Clinical Social Worker884Stipend1,838*2,722 Marriage and Family Therapist1,251Stipend4741,725 Licensed Supervising Clinician0--00 Psychiatrist, Child/Adolescent0Psychiatric ResidencyUnknown-- Registered Nurse0--00 CEO or Manager above Direct Supervisor 0--00 Psychiatric or Family Nurse Practitioner 1Stipend92 Licensed Clinical Psychologist0Stipend232 Analysts, tech support, quality assurance 0--00 Family Member Support Staff0WWTUnknown Clinical Nurse Specialist0--00 Psychiatrist, Geriatric0Psychiatric ResidencyUnknown Consumer Support Staff9WWTUnknown Positions not identified as hard to fill/retain in 2008 Physician Assistant0Song Brown1,382517 TOTAL SERVED:MHLAP: 2,363Stipend & Residency: 1,680Overall: 4,043 *Includes Stipend Program for Social Work participants from beginning of program in 2005.

25. Program Evaluation: General Capacity 25 State-Administered WET Programn Average Rating % Rating the Program Somewhat or Very Effective MHLAP263.4292% Social Worker Stipend Program212.6767% MFT Stipend Program202.5570% PMHNP Stipend Program172.2453% Psychiatric Residency Program191.6826% Clinical Psychologist Stipend Program171.5918% Song-Brown Residency for Physician’s Assistants 121.080% Overall--2.32-- Counties were asked to rate the effectiveness (impact) of each of the state- administered programs in meeting their WET needs Scale: 1 (not at all effective) - 4 (very effective)

26. Program Evaluation: Cultural & Linguistic Competency 26 Baseline 2008Progress 2013 Licensed Direct % Difference from target Licensed Direct % Difference from Target Caucasian/White 6,93854%-20.6%15,65051%-18.00% Hispanic/Latino 2,41719%+19.0%6,55821%+16.70% African American/ Black 1,0728%+9.0%2,5388%+9.30% Asian/Filipino/ Pacific Islander 1,34210%-3.4%3,25211%-4.00% American Indian/ Alaska Native 1101%+0.1%2611%-0.10% Multiple/Other 1,0768%-4.2%2,6378%-3.90% Total 12,955100%--30,896100%--

27. Program Evaluation: Cultural & Linguistic Competency 27 Non-English Language Number of public sector staff needed in 2008 Adjusted estimate of need Number served by statewide WET programs % of need potentially met by statewide WET programs Spanish6,092 12,671 1,708 13% Chinese 5131,067 109 10% Other Asian 97420,26 283 14% Other 221460 319 69% Total7,800 16,224 2,419 15% Linguistic Capacity of 2008 Workforce and WET Progress toward Targets

28. Program Evaluation: Cultural & Linguistic Competency 28 Program Impacts n Not at all effective Somewhat effective Very effective Don't know How effective have state-administered WET programs been in increasing the cultural and linguistic competency of the workforce in your county? 26 12%50%35%4% How effective have state-administered WET programs been in increasing the diversity of the workforce in your county so that the workforce is more representative of the population served in terms of ethnicity, cultural tradition, religion, LGBT identification, etc.? 26 27% 39% 19% 15%

29. Program Evaluation: Formal Education Structure & Curricula 29  Institutions reported many courses that highlighted  Use of evidence-based practices  Cultural competency across gender, race, religion, sexual orientation, etc.  Principles of wellness and recovery among adults and resiliency among youth  Several Regional Partnerships, in collaboration with local colleges and agencies, are developing PMH professional core competencies

30. Program Evaluation: People with Lived Experience 30 Working Well Together has developed the following:  Assessment tools for agencies to recognize where they are in terms of engaging consumers and family members in the workforce,  Curricula for training individuals who identify as consumers and family members, and trainings to preparing the workforce for employing consumers and family members  Toolkit for recruiting, hiring and retaining employees with lived experience within the public mental health workforce,  White paper on how to successfully employing people with lived experience within public mental health  Peer Certification standards and recommendations for a statewide plan

31. Program Evaluation: People with Lived Experience 31  Working Well Together Accomplishments:  62 Networking calls and Webinars combined RegionTA VisitsTrainingsParticipants Central 396159 Southern 4013343 Superior 29779 Bay Area 518104 Total15934685

32. 32  Insert matrix Program Evaluation: Regional Partnerships Evaluation 32 Regional Partner- ship Cultural compete ncy trainings Curric. focused efforts with colleges High school mental health career pathway Core competen cies project Move toward recovery focus in WET Programs targeting the under- served Explicit stigma reduction efforts First responder training and MH First Aid trainings Central XXXXXXXX Greater Bay Area X XXX X Los Angeles XX X Southern XXXXX Superior XX

33. Program Evaluation: Recommendations for Future Practice 33  Ensure that program offerings correspond to current workforce needs  Ensure that the pipeline to employment is considered so that programs can succeed in placing all graduates in the public mental health system workforce  Ensure a strategic and consistent approach to consumer and family member workforce development

34. Program Evaluation: Recommendations for Future Evaluation 34  Track participation consistently  Track LGBT and consumer/family member identification  Track progress of Regional Partnerships with more consistent tools  Monitor curricula with checklist

35. Program Evaluation: Limitations 35  Baseline Data Limitations:  Only 28 counties included in baseline report  Cannot disaggregate (e.g., by county, rural/urban/suburban, etc.)  Very limited in baseline information pertaining to two outcome areas: Formal Education Structure and Curricula and Filling Gaps in Five Regions  Progress Data Limitations:  Inconsistent reporting from programs  No consistent measures for looking at progress in Formal Education Structure and Curricula and Filling Gaps in Five Regions

36. Program Evaluation: Limitations 36  Impact Data Limitations:  Voluntary participation resulted in poor response rate  Baseline survey data were only available for 28 counties  Impact survey data were only available for 26 counties  Only 12 counties were represented in both baseline and impact county lists 16 counties for which no data were collected at all

37. Program Evaluation: Strengths of the Programs 37  The vast majority of the work that the statewide-administered WET programs have been conducting aligns with needs identified at the beginning of the process  There are significant program components in place to build the general capacity of the workforce  Programs have been very successful in recruiting and engaging people of color and people with non-English language proficiency  Curriculum changes have been noted that demonstrate alignment with MHSA principles  Trainings, TA visits and webinars focused on increasing the participation of people with lived experience have been offered to counties in all regions  The Regional Partnerships have engaged in work specific to the needs of their regions

38. Forthcoming 38

39. Forthcoming Work 39  Workforce Projections  Integrating controls to identify most important influences in supply trends  Acquiring and analyzing demand-side data to identify trends amongst consumers of public mental health services  Gap Analysis  Identify any anticipated shortages and surpluses over the next five years  Occupational type, prescribing authority, location, demographic needs (where data is available)

40. Workforce Projections: Data Sources 40  Supply: Current Workforce  National Provider Identification (NPI) Data  California Professional Boards  Demand for Public Mental Health Services  Client & Service Information (CSI)

41. Workforce Projections: Limitations 41  No perfect data source with exact counts of individuals currently in public mental health workforce  Limited number of variables  Time-intensive process of accessing Client Service Information (CSI) data

42. Strengths of Overall Project 42  Reviewed extensive amount of county-reported data  Utilizing multiple data sources to inform needs assessment, evaluation, and projections  OSHPD support in prioritizing best possible data sources

43. Thank you! 43  Questions?