1. Session # I6
Identifying a Path Toward Measuring Integrated Team-Based Core Competencies
C.R. Macchi, PhD, Clinical Associate Professor, Arizona State University
Patti Robinson, PhD, Director of Training and Program Evaluation, Mountainview Consulting Group
Rodger Kessler, PhD, Research Associate Professor, Arizona State University
Please insert the assigned session number (track letter, period number), i.e., A2a
Please insert the TITLE of your presentation.
List EACH PRESENTER who will ATTEND the CFHA Conference to make this presentation.
You may acknowledge other authors who are not attending the Conference in subsequent slides.
CFHA 18th Annual Conference
October 13-15, 2016  Charlotte, NC U.S.A.
Collaborative Family Healthcare Association 12th Annual Conference

2. Faculty Disclosure
C.R. Macchi and Rodger Kessler have NOT had any relevant financial relationships during the past 12 months. Patti Robinson is a consultant for Mountainview and received royalties as an author for one of the referenced titles during the past 12 months.
You must include ONE of the statements above for this session.
CFHA requires that your presentation be FREE FROM COMMERCIAL BIAS. Educational materials that are a part of a continuing education activity such as slides, abstracts and handouts CANNOT contain any advertising or product‐group message.
The content or format of a continuing education activity or its related materials must promote improvements or quality in health care and not a specific propriety business interest of a commercial interest. Presentations must give a balanced view of therapeutic options. Use of generic names will contribute to this impartiality. If the educational material or content includes trade names, where available trade names for products of multiple commercial entities should be used, not just trade names from a single commercial entity. Faculty must be responsible for the scientific integrity of their presentations. Any information regarding commercial products/services must be based on scientific (evidence‐based) methods generally accepted by the medical community.
Collaborative Family Healthcare Association 12th Annual Conference

3. Learning Objectives At the conclusion of this session, the participant will be able to:
Participants will review existing descriptions of behavioral health core competencies needed to effectively perform in primary care.
Participants will explore overlapping competencies required of each profession on a care team
Participants will investigate potential approaches needed to evaluate core competencies
Participants will consider components needed to develop valid, reliable measurement tools of core competencies.
Include the behavioral learning objectives you identified for this session
Collaborative Family Healthcare Association 12th Annual Conference

4. Bibliography / References
Miller, B., Gilchrist, E., Ross, K., Wong, S., Blount, S., Peek, C.J. (2016, February). Core competencies for behavioral health providers working in primary care. Prepared from the Colorado Consensus Conference.
Interprofessional Education Collaborative Expert Panel. (2011). Core competencies for interprofessional collaborative practice: Report of an expert panel. Retrieved from Washington, D.C.:
Kinman, C., Gilchrist, E., Payne-Murphy, J., & Miller, B. (2015). Provider- and practice-level competencies for integrated behavioral health in primary care: A literature review ( EF). Retrieved from Rockville, MD:
McDaniel, S. H., Grus, C. L., Cubic, B. A., Hunter, C. L., Kearney, L. K., Schuman, C. C., Johnson, S. B. (2014). Competencies for psychology practice in primary care. American Psychologist, 69(4), doi: /a
Miller, B., Gilchrist, E., Ross, K., Wong, S. G., Blount, A., & Peek, C. J. (2016). Core competencies for behavioral health providers working in primary care. Retrieved from
Robinson, P. J. & Reiter, J. T. (2015). Behavioral consultation and primary care: A guide to integrating services, 2nd Edition. NY: Springer.
Continuing education approval now requires that each presentation include five references within the last 5 years.
Please list at least FIVE (5) references for this presentation that are no older than 5 years.
Without these references, your session may NOT be approved for CE credit.
Collaborative Family Healthcare Association 12th Annual Conference

5. Learning Assessment A learning assessment is required for CE credit.
A question and answer period will be conducted at the end of this presentation.
Please incorporate audience interaction through a brief Question & Answer period during or at the conclusion of your presentation.
This component MUST be done in lieu of a written pre- or post-test based on your learning objectives to satisfy accreditation requirements.
Collaborative Family Healthcare Association 12th Annual Conference

6. Core Questions
What are the core domains, how do you develop your competencies?
Who assesses the findings and how are the competencies related to program outcomes?
How do you make it sustainable?
What are the next steps toward developing a valid, reliable measure of BH competence for each team member?
What are the central themes and items that could be used with multiple members of an integrated team?
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

7. Doctoral Training Approach to Evaluation
Pre-professional Internship Evaluation
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

8. Core Competencies & Skill Development
McDaniel, Campbell, Hepworth & Lorenz (2005)
Robinson & Reiter (2015)
McDaniel, Doherty & Hepworth (2014)

9. Tool Development Purpose of the tool
Provide a way for student interns and supervising raters to focus evaluation on key integrated, team-based practices in primary care settings
Student behavioral demonstrations of those practices
Student engagement with patients and other team members
Develop crosswalk of clusters/domains – knowledge, skills, and attitudes
Competencies for Psychology Practice in PC - Interorganizational Workgroup (McDaniel et al., ) - six competency clusters
Core competencies for integrated behavioral health and PC (SAMHSA, 2014)
Behavioral benchmarks for each competency
Provider-Level Competencies for Integrated Behavioral Health in PC (AHRQ, 2015)
Core Competencies for Behavioral Health Providers Working in Primary Care (Farley Health Policy Center, 2016)
Develop common clusters
Student performance standards for each domain reflect the skillset that is assumed to be
appropriate in the primary care environment
applicable to clinics of varied levels of integration
associated with improved care delivery
associated with increased levels of integrated behavioral healthcare delivery
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

10. Tool Development
Develop items that operationalize clusters based upon measureable behavioral anchors
Items of each evaluation related to the rater’s role and observations of BHCs clinical practice
Convergent items by domain
Discriminant items between domains
Align anchors based upon the three worlds – clinical, operational, & financial systems (Peek, 2008)
Develop evaluation domains that align performance between systems
Develop standard scale for rating each item
Facilitate developmental assessments of each domain
Identify student intern progress over the full internship course(s)
Make the full points for each domain meaningful and reflective of excellent, independent performance
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

11. Tool Development
Field leader reviews and comments and comparison with available rating tools
Rodger Kessler
DBH faculty
Compared tool with existing tools
TRI-SERVICE IBHC CORE COMPETENCY TOOL
Benchmark behaviors
2-point rating scale (pass/fail)
PRIMARY CARE BEHAVIORAL HEALTH PROVIDER ADHERENCE QUESTIONNAIRE (PPAQ)
5-point rating scale (never to always)
BEHAVIORAL HEALTH CONSULTANT CORE COMPETENCY SCALE
5-point rating scale (low to high)
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

12. Elements of the Tool Categorized core competencies
Three worlds – clinical, operational, financial
Behavioral anchors
Behavioral settings
Developmental scale

13. Core Competency Domains and Skills
Conceptual and professional development
Systems orientation
Primary care culture and adaptation
Professionalism
Practice-based learning
Biopsychosocial orientation
Quality improvement
Clinical skills/practice
Screening and assessment of BH issues related to medical conditions
Population-based approaches
Evidence-based interventions
Informatics and data
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

14. Core Competency Domains and Skills
Practice management
Documentation
Time management
Resource management
Compliance with clinic policies and protocols
Collaboration
Communication with providers and patients
Interprofessional relationships
Care coordination
Identifies and supports medical team shared goals
Support patient engagement on care team
Finances and utilization of resources – demonstrates an awareness and responsiveness to the financial implications of clinical and operational activities
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

15. Sample Items
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

16. Use of the Evaluation Tool
Multiple raters
On-site – site liaison and medical preceptor
Program faculty
Student intern self-assessment
Program faculty conducts continuous reviews
Weekly meetings
Mid-semester review comparing ratings
End-of-semester, comprehensive review comparing ratings
Program faculty meets with student intern to identify areas of progress and growth
Course grade is based upon
Student attainment of baseline skills
Demonstration of progress across core competency domains
Student meets thresholds for activity metrics

17. Companion Activity Metrics
Measuring student intern activities (targets)
Direct patient contact hours
% of total hours direct patient contact (40%)
Average time with each patient (less than 30 minutes)
% of patients seen who were administered approved, validated outcome measures with patients (65%)
Average number of outcome measures administered to each initial and follow-up patient visit (1 measure)
% of patients seen who were provided a self-management resource (25%)
Administrative hours
Training and education (e.g., IBH presentations, orientation)
Interprofessional interactions and team-building (e.g., huddles, warm handoffs, team meetings)
Patient care planning (e.g., case consultations, reviewing medical records, preparing for patient groups)
Data management (e.g., collecting/analyzing patient data, entering medical reports)

18. In-Field Retraining Approach to Evaluation
Behavioral Health Consultant Core Competency Tool
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

19. Development of PCBH BHC Tools
1996: Initial Primary Care Behavioral Health (PCBH) Core Competency Tool for Behavioral Health Consultants (BHCs) ((Refined 2002, 2007, 2015)
Other PCBH CC Tools: BHC Assistant CC Tool, BHC Mentor CC Tool, Clinic Leadership CC Tool
Additional Tools Related to BHC Tool
Referral Barriers Questionnaire 1
IPCMH Integration Tool 2
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

20. PCBH BHC Tool Development
Purpose of tools
Inform BHC training
Inform training of PCPs, RN, other staff
Prepare Clinic Leadership team to provide essential preparation and on-going evaluation and management
Evaluate job performance on an annual basis
Inform performance improvement planning
Influence graduate training (BH & PC)
Process for CC Tool development
Analysis of competencies associated with anticipated improvements to population health, patient and provider satisfaction and value of services, identifying and addressing barriers to achievement of outcomes
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

21. Core Competency Domains and Skills
Clinical
Consultation
Practice Management
Team-based Care
Documentation
Administrative
You can begin your own slides here. Feel free to use your own background on this and subsequent slides.
Collaborative Family Healthcare Association 12th Annual Conference

22. Training Venues BHC On-going monitoring and feedback
Phase 1 (foundation skills)
Group based
Didactic, modeling, guided rehearsal
Phase 2 (on-the-job skills)
In-clinic in process of delivering care
Simultaneous effort to train Leadership Team, PCPs, and Nursing Staff in PCBH Competencies
On-going monitoring and feedback
Fidelity metrics
PCBH program outcomes
PCP & BHC performance
BHC Mentor Training

23. ZZZ Clinic: PCP Satisfaction with IBHC Services (n = 11) (1=no apparent benefit, not helpful to 10 = extremely helpful)

24. ZZZ Clinic Integration Tool Results

25. Validation of Competent PCBH Program
Show Rate
Booked
Same-day
Routine
Ref to Spec
Group Appts
Under 18
18-60
60+
Top 10 DX
Provide comparison among clinics; distribute
Monthly to all PC providers, teams, management

26. Evolving PCBH
Re-vamping discipline specific training programs to train students for the needed workforce
Inter-disciplinary training venues
PCBH faculty in Family Practice residencies
University of Texas R – Deepu George, Leslie Allison
Central Washington Family Medicine – Bridget Beachy, David Bauman, Kirk Strosahl

27. Evolve PCBH Fidelity Training Mentors Supplementary CC Tools
Focused Acceptance and Commitment Therapy Competency Self Assessment (FACT-CAT)
Brief Intervention Competency Assessment Tool
(BI-CAT)
Fidelity
On-going measurement of fidelity to practice
Link of trained competencies, practiced with fidelity to optimal health care outcomes
Training Mentors
Mentor CC Tool

28. Assist PCBH Expansion m
To assist you in your pursuit of competence . . . m
PCBH Competency Tools*
Supplementary Competency Tools
Other PCBH Tools
*Robinson & Reiter, 2015;
See

29. Session Evaluation
Please complete and return the evaluation form before leaving this session. Thank you!
This should be the last slide of your presentation
Collaborative Family Healthcare Association 12th Annual Conference