1. PBHCI National Evaluation
RAND Corporation
&
National Council for Community Behavioral Healthcare

2. Welcome!
Audrey Burnam, PhD

3. The RAND Corporation
Nonprofit institution that helps improve policy and decisionmaking through research and analysis.
RAND is tasked with completing an independent and objective national-level evaluation of PBHCI
Funded jointly by SAMHSA and ASPE

4. National Council
Jeff Capobianco, MA

5. The National Council
The National Council for Community Behavioral Healthcare (National Council) has won a competitive grant funded jointly by SAMHSA and HRSA to establish the National Training and Technical Assistance Center for Primary and Behavioral Healthcare Integration.
The National Council is working closely with RAND and will provide technical assistance to grantees to insure they are able to successfully provide the data required for the evaluation.

6. …back to Audrey Burnam, PhD
RAND’s Tasks
…back to Audrey Burnam, PhD

7. Overview of RAND’s Tasks
Conduct a national evaluation of the PBHCI program that answers 3 questions:
Outcomes: Does integrating primary and behavioral health care lead to improvements in mental and physical health of persons with SMI and/or substance use disorders?
Process: Is it possible to integrate the services provided by primary care providers and community-based behavioral health agencies?
Models: Which models and/or respective model features of integrated primary and behavioral health care lead to better mental and physical health outcomes?
Contracted by ASPE and SAMHSA to…

8. Goals of This Webinar Part I: Achieve common understanding of:
The framework for the national evaluation
Data that support the national evaluation
Procedures for transmitting data to RAND
Part II:
Data collection procedures

9. PART I: Framework for the National Evaluation
Deborah Scharf, PhD

10. Structure-Process-Outcomes Evaluation Model
Independent Variables
Dependent Variables
STRUCTURE OF CARE
What services are available?
Core (required) components:
Screening/referral/treatment
Registry/tracking system
Care management
Prevention/wellness approach
Examples of optional components:
Co-location of nurse practitioners/PCPs
Identification of primary care supervising physician
Embedded nurse care managers
Use of EBPs
Use of SBIRT
Other
PROCESS OF CARE
To what degree are services implemented?
Examples:
Numbers/proportions of SMI patients screened/referred/treated & for what
Number of records/types of information tracked in registry
Number of SMI patients connected with care manager & for what
Number of individualized wellness plans developed and utilized
OUTCOMES OF CARE
Does it make a difference?
Improved overall health
status of clients:
Physical health outcomes
(e.g., BMI, blood pressure, blood glucose or HbA1c, lipid profile)
Mental health outcomes
Environment/Administrative Context
(e.g., size, auspice, location, health IT)

11. Data Sources (I) TRAC Quarterly reports Clinical registries
*Staff may be asked to participate in interviews or surveys
*Some sites may be invited to participate in other evaluation activities

12. Data Sources (II) 1. TRAC Entered directly into TRAC system
2. Quarterly Reports
Uploaded to PBHCI SharePoint site
3. Clinical Registries
a) Uploaded to PBHCI SharePoint site or
Transmitted directly from a data extraction system

13. TRAC: CMHS Transformation Accountability
Susie Lovejoy, MS

14. Transformation Accountability (TRAC)
In response to the 1993 Government Performance and Results Act (GPRA) requirements CMHS developed the TRAC system.
TRAC is the web-based system through which all SAMHSA CMHS grants report performance measurement data. 
Measures include: client level outcomes, infrastructure development indicators, measures of mental health promotion and prevention activities, and satisfaction with technical assistance services.

15. TRAC (II)
All of you (grantees) have been trained on the Infrastructure, Prevention, & Promotion (IPP) portion of TRAC.  
Grantees funded in Sept 2009 (cohort I) have been trained on the client-level outcomes measures for people receiving direct services through the grant. 
Grantees funded in 2010 (cohorts II and III) will be trained in February to collect and enter the remaining data into TRAC

16. TRAC (III)
Grantees must collect and report data on behavioral health outcomes for each person receiving services from the grant (NOMS).
Grantee staff interview consumers and enter data into TRAC at:
Entry into integrated care
Every 6 mos while enrolled
At discharge from the program 

17. TRAC (IV)
For PBHCI, grantees must also log physical healthcare indicators in TRAC
Required indicators: Height, weight, BP, glucose or A1C, cholesterol, triglycerides
Optional indicators: Waist circumference, Breath CO (for smoking status)

18. TRAC (V) Intake, 6 months, discharge: Height Weight Blood pressure
Waist circumference*
Breath CO*
Intake, annually, discharge:
Glucose or A1C
Triglycerides
Cholesterol
Successful 8h fast*
*Recommended
Note: 6 month indicators are collected quarterly but entered bi-annually

19. TRAC (IV) TRAC data can be downloaded
This can help you avoid double-entry
Collect and enter the physical healthcare data in the same time windows as you collect and enter the TRAC client-level interviews.
This TRAC change will be in place ~Feb 2011

20. PBHCI Secure SharePoint Site
Regan Main, BA

21. PBHCI Secure SharePoint Site
SharePoint is the primary method of transmitting quarterly reports and clinical registry data to SAMHSA and RAND

22. SharePoint (II) Use the SharePoint site because it is secure
Meets HIPAA data protection requirements
Each grant program has a password- protected folder
Uploading data is easy!

23. SharePoint (III)
Log-on to SharePoint using the link below:
*Watching this presentation from the archives? See separate file with additional SharePoint slides

24. Quarterly Reports
Nicole Eberhart, PhD

25. Quarterly Reports Required quarterly by every grantee
1 report per program
Uploaded to your folder in SharePoint
Quarterly reports are Word documents
RAND provides a template
Blank templates are on SharePoint

26. Quarterly Report: Labeling
VERY IMPORTANT!
Top of report:
Your grant number
Name of your site and program
Name of the Project Director
Name of the person to contact (if not the Project Director)
Label the report for the dates covered and the date submitted
Put the quarter number and year
Label electronic attachments with something that includes the abbreviated site name, Quarter #, and Year.
Example: ANN ARBOR CMHC 1st QTR 1/31/10

27. Quarterly Report: General Issues
Aim for 3-5 pages
Answer each question with regards to both
Core program features
screening/assessment
treatment and referral
registry/tracking
care management
prevention/early intervention/wellness
referral/follow up
Optional features, e.g.:
use of SBIRT and/or other evidence-based practices
primary care supervising physician
embedded nurse care manager, etc.

28. Quarterly Report: Content (I)
Describe your program’s accomplishments
Describe any changes in staffing since the last quarter
Clinical discipline
Role
Duties of grant-supported new staff
Changes in % effort
Attach resumes and any formal correspondence made between you and SAMHSA about staffing changes
Describe the involvement of consumers and families in your project

29. Quarterly Report: Content (II)
Barriers in implementing your program, solutions you generated, or your plans to resolve them
Delays in the program as you originally proposed it, changes in content or timing
Infrastructure activities, staff involved, and associated expenditures
Wellness-related education and programming, staff involved, associated expenditures

30. Quarterly Report: Content (III)
Progress with data collection (software, clinical registry, TRAC, qualitative data) and plans to monitor, analyze, and/or share the data
Specific program components supported by grant monies
Funding sources for key primary care services provided via your program, and sustainability beyond the official grant period

31. Quarterly Report: Content (IV)
Which patients are eligible to receive PBHCI services
All SMI patients? New SMI patients? SMI patients with physical health conditions? SMI patients receiving psychotropic medication?
Staff involvement in PBHCI group activities or with SAMHSA Project Officer
Group conference calls, grantee meetings, site visits and interaction with the Project Officer or Technical Assistance Provider(s).
Other sources of funding, and proportion of PBHCI consumers who are also participating other funded programs

32. Glossary
Assessment: Actions taken following a positive screen to provide a more comprehensive or in-depth picture of an individual patient’s specific physical health condition(s) and to determine the best service plan to address them
Early intervention: Interventions that are appropriate for, and specifically target, people displaying the early signs and symptoms of a physical health condition.
Prevention: Interventions that occur before the initial onset of a physical health condition to prevent the development of that condition.
Qualitative data collection methods: Interviews, focus groups, etc.
Screening: Refers to a preliminary procedure, such as a test or examination (e.g., blood pressure for hypertension, cholesterol for coronary artery disease), to detect the most characteristic sign(s) of a physical health condition that may require urgent attention or further investigation.
Wellness-related education and programming activities: Activities that promote wellness in mind, body, and spirit. They may include educational activities (e.g., nutrition education, physical activity education), physical activities (e.g., yoga classes, walking groups), and psychosocial activities (e.g., peer support programs, health-focused social functions).

33. Clinical Registries
Deborah Scharf, PhD

34. Clinical Registries Two purposes:
Improve clinical care
Record consumer-level clinical interactions
Data system for consumers with a specific type of disease diagnosed and treated at a practice*
Allows care team members to proactively manage consumers with chronic diseases*
Grantees must maintain a clinical registry
*Iowa Dept of Public Health, 2010

35. Benefits of a Registry*
Enable the provider to ensure that all their patients are getting proper care
Track progress of high-risk patients
Identify the need for follow-up services
Increase quality of care and improve patient outcomes
Coordinate care and identify gaps
Incorporate consensus guidelines for disease management
*Iowa Dept of Public Health, 2010

36. Evaluation Uses of a Clinical Registry
Facilitate data collection
Registries can and should include all key client-level data elements
Standardize client-level data collection
Data collection can be aligned with flow of integrated clinical care

37. Attempt To Balance Simplicity & Complexity for Serving Multiple Purposes
Overly Simplistic “Reasonable” Middle Ground Overly Complex
Meets minimal reporting expectations
Provides baseline data w/out longitudinal, clinically relevant follow-up information
Facilitates sites’ efforts to provide high-quality clinical care
Assists sites in meeting SAMHSA’s reporting requirements
Supports meaningful evaluation
Exceeds sites’ data collection resources/ capacities
Results in incomplete and/or unreliable data collection across sites

38. Clinical Registry Items Extracted for the Evaluation
Each consumer’s total contacts including these services:
Physical Health
Mental Health
Substance Use
Wellness
Screening / Assessment X
Referral
Treatment Planning
Medication Mgt.
Counseling
EBPs
List EBPs / Activities
Hospitalizations

39. Clinical Registry Items Extracted for the Evaluation (II)
Each consumer’s total contacts with:
Care managers
Primary care providers (MDs, LPNs, PAs)
Psychiatrists or psychiatric nurses
Counselors (LSW, psychologists, substance abuse counselors, others)
Peer specialists
Other specialists (dentist, nutritionist, others)
Wellness programs

40. Clinical Registry Items Extracted for the Evaluation (III)
IMPORTANT!
All data fields to be extracted from clinical registries should be whole numbers
i.e., 1, 2, 3, … 99
Even lists of EBPs should be coded as whole numbers
RAND will provide you with a list of EBPs and wellness activities associated codes
Example EBP: CBT = 1; MI = 2
Example wellness activity: Diabetes education = 1; Smoking cessation = 2

41. Registry: For Cohort I
Clinical registry data is taking the place of numeric data previously reported in quarterly reports
Data are now collected at the patient-level
Not in aggregate (i.e., not at the program-level)

42. If You Already Have a Clinical Registry…
Make sure it contains all necessary fields
Review these materials, pg 12 of your RFA, and talk to your contact at the Technical Assistance Center
Establish a plan for:
Extracting all necessary fields
Making all data fields numeric, whole numbers, and consistent with PBHCI requirements
Your registry will likely contain data that we do not collect for the evaluation

43. If You Do Not Have a Clinical Registry…
Look carefully at the data to be extracted from PBHCI registries
Work with the Technical Assistance Center to select a registry tool that will meet your clinical and research needs

44. When Choosing a Registry, Consider…
To what extent will using the registry improve the quality of your program’s integrated care?
To what extent is the organization of the registry aligned with the flow of integrated care at your site?
To what extent will the registry help you automate your data collection and reporting?

45. THANK YOU!