Telling the Story of SBHC Impact & Outcomes: Quality Measures & Data Collection Tools, Tips, & Strategies Sara Rigel, MPH Public Health-Seattle & King County WA School Based Health Alliance

Objectives Session participants will: Gain an understanding of different logical/results frameworks and indicators used to measure the success of school-based health centers.   Be able to name the five National Quality Initiative standardized performance measures for school-based health centers. Learn the challenges of data collection and how different SBHCs have built their processes and systems to measure their success and improve their quality over time.  Introduction of me. Who is in the room? Current SBHC providers/manager School staff Emerging 0-1 year experience in SBHC 1-4 5-10 10-20 20 + My goal is to help you think, if you haven’t already, about what you may want to measure, track and report on in your school based health center work. Learn about some of the efforts under way. Objectives. I am not an expert in evaluation You will hear from clinc staff about their efforts to measure and improve quality of their sbhcs

Why do we care about data, measurement and evaluation? Funding Sharing the story Justifying

Why measure outcomes? Measuring the effectiveness of an intervention Identifying effective practices Identifying practices that need improvement Providing value to existing and potential funders Getting clarity and consensus around the purpose of your program Measuring the effectiveness of an intervention. How do you know if a program was effective? If a program was not effective, would you want to know so that it could be improved? It is unlikely that we build capacity simply for the sake of building capacity, and because of this, we need to know whether or not our programs are effective, just as the nonprofits we serve need to know if their services are achieving the desired results. 2. Identifying effective practices. With the information you collect, you can determine which activities to continue and build upon. Some practices might be modified and replicated for other programs or initiatives based on your results. 3. Identifying practices that need improvement. Some activities may need to change in order to improve the effectiveness of your program. 4. Proving your value to existing and potential funders. The climate for funding social services and capacity building has undergone some radical transformations in the last few decades. Funders are keenly aware of the need to document the success of programs. Capacity building has lagged somewhat behind the social services field in having established evaluation standards. In 2003, the Alliance for Nonprofit Management reported that the capacity building field still lacked a framework for evaluation. However, this is changing, and it is important for capacity builders to stay ahead of the curve to be able to continue doing their work. 5. Getting clarity and consensus around the purpose of your program. Everyone in your organization, from board members to service staff to volunteers, should understand what is going on in your program and what it is intended to achieve. Outcome measurement helps to clarify your understanding of your program. Demonstrate the value of a SBHC

Goals, Outcomes, and Indicators, oh my! Goal: Broad statement of the ultimate aims of the program Outcome: the changes in the lives of individuals, families, organizations or community as a result of the program Indicator: the specific, measurable information collected to track whether an outcome has actually occurred. Goal — broad statement of the ultimate aims of a program ..Outcome — the changes in the lives of individuals, families, organizations, or the community as a result of the program ..Indicator — the specific, measurable information collected to track whether an outcome has actually occurred Outcome measurement will explore what your program provides, what its intended impacts are, and whether or not it achieves them. It will not prove that the changes that take place are a result of your program. People often get stuck in the outcome measurement process because of all the terminology. Is it an outcome, an accomplishment, an achievement, or a result? Is it a goal or an objective? Is it an indicator or a performance measure? Some people see goals and objectives as interchangeable; others see outcomes and objectives as the same. Don’t let this get in your way, but establish these definintions for yourself.

Logic Model A logic model is a graphic depiction (road map) that presents the shared relationships among the resources, activities, outputs, outcomes, and impact for your program. It depicts the relationship between your program’s activities and its intended effects. - Centers for Disease Control and Prevention

Performance measures- results based accountability “Trying Hard is Not Good Enough” –Mark Friedman Start with the END results and work backwards step by step Establish what the problem is before you decide what to do It’s all about results, not how you do it. How much did we do? How well did we do it? Is anyone better off? basic principle of R.B.A.: Start with the end result and work backwards step by step to the means. Establish what the problem is before you decide what to do Remember it’s all about results, not how you do it Think about the needs before the solutions.

Sorting Performance Measures: What is Measured in Each Quadrant How much did we do? How well did we do it? % Common measures e.g. client staff ratio, workload ratio, staff turnover rate, staff morale, % staff fully trained, % clients seen in their own language, unit cost # Clients /customers served % Activity-specific measures e.g. % timely, % clients completing activity, % correct and complete # Activities (by type of activity) Is anyone better off? % Skills / Knowledge e.g. parenting skills # This chart shows in detail the different types of measure we typically find in each quadrant, and the measures that go with the three basic categories of performance measurement: How much did we do? How well did we do it? Is anyone better off? In the upper left, How much did we do? Quadrant, we typically count customers and activities. In the upper right, How well did we do it? Quadrant, there are a set of common measures that apply to many different programs. And there is a set of activity specific measures. For each activity in the upper left, there is one or more measures that tell how well that particular activity was performed, usually having to do with timeliness or correctness. In the lower quadrants, Is anyone better off? We usually have # and % pairs of the same measure. And these measures usually have to do with one of these four dimensions of better-offness: Skills/knowledge, Attitude, Behavior and Circumstance. For each of these measures, we can use point in time measures or point to point improvement measures. % Attitude / Opinion e.g. toward drugs # % Behavior e.g. school attendance # % Circumstance e.g. working, in stable housing # 9

Data Sources Electronic health records School academic records School climate survey Healthy Youth Survey Patient satisfaction surveys Other?

Quality Improvement UDS, HEDIS, NCQA, PCMH CHIPRA, Meaningful Use Patient safety, infection rates SLR Why don’t we measure these things in SBH?

School based Health Alliance: National Quality Improvement Initiative 2014 development of standardized performance measures and 1st national cohort of clinics. Modeled on Institute for Healthcare Improvement Model for Improvement Framework

The Model for Improvement asks three questions — What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in improvement? — and then employs Plan-Do-Study-Act (PDSA) cycles for small, rapid-cycle tests of change. IHI uses the Model for Improvement in all of its improvement efforts.

Standardized performance measures for SBHCs Core Measures Annual well child visit Annual risk assessment BMI screening and nutrition/physical activity counseling Depression screening Chlamydia screening Stretch Measures Student disposition log (seat time saved) SBHC student user survey Annual well child visit Annual risk assessment BMI screening and nutrition/physical activity counseling Depression screening Chlamydia screening You decided that the best way to demonstrate the value add of SBHCs to the healthcare system is to work to ensure that children and adolescents have an annual (1) well-child visit, (2) risk assessment, and (3) BMI screening. You also decided that adolescents should be screened for (4) depression and (5) chlamydia We are thrilled with the measures you selected: They align with existing national child quality measurement frameworks (HEDIS, CHIPRA, Meaningful Use) – allowing SBHCs to compare themselves with one another and with other healthcare settings.

National Quality Measures

Call to Action Fall 2015 two sites were invited to join School Based Health Alliance National Quality Initiative (NQI) Collaborative Improvement and Innovation Network (CoIIN).

NQI MEASURES WELL-CHILD VISIT ANNUAL RISK-ASSESSMENT BMI DEPRESSION SCREENING & FOLLOW-UP CHLAMYDIA SCREENING

PDSA

Data collection: Expectation Reality

WCV

WCV

Flowsheets

Conversion to ICD-10. Staffing. Finding extractable data. Changes in data export. IT communication. Competing projects. Time.

Suggestions: Be patient. Takes time to get a good baseline. Be consistent. Charting and reporting. Develop (or maintain) good relationship with IT. Meet early and often. Be proud of our important work. (Make sure we can prove it).

Choosing Performance Measure Performance Measure #1: ANNUAL WELL CHILD VISIT Performance Measure #2: ANNUAL RISK ASSESSMENT Performance Measure #3: BODY MASS INDEX (BMI) SCREENING AND NUTRITION/PHYSICAL ACTIVITY COUNSELING Performance Measure #4: DEPRESSION SCREENING AND FOLLOW-UP Performance Measure #5: CHLAMYDIA SCREENING

+ CHLAMYDIA High rate of POSITIVE chlamydia Extremely high-risk student body Difficult for students at different sites to come in for testing

How can we get more students in for chlamydia screening??? Challenge How can we get more students in for chlamydia screening???

GO TO THE STUDENTS

Has this got you thinking?