1. Telling the Story of SBHC Impact & Outcomes:
Quality Measures & Data Collection Tools, Tips, & Strategies
Sara Rigel, MPH
Public Health-Seattle & King County
WA School Based Health Alliance

2. Objectives Session participants will:
Gain an understanding of different logical/results frameworks and indicators used to measure the success of school-based health centers.  
Be able to name the five National Quality Initiative standardized performance measures for school-based health centers.
Learn the challenges of data collection and how different SBHCs have built their processes and systems to measure their success and improve their quality over time. 
Introduction of me.
Who is in the room?
Current SBHC providers/manager
School staff
Emerging
0-1 year experience in SBHC
1-4
5-10
10-20
20 +
My goal is to help you think, if you haven’t already, about what you may want to measure, track and report on in your school based health center work.
Learn about some of the efforts under way.
Objectives.
I am not an expert in evaluation
You will hear from clinc staff about their efforts to measure and improve quality of their sbhcs

3. Why do we care about data, measurement and evaluation?
Funding
Sharing the story
Justifying

4. Why measure outcomes? Measuring the effectiveness of an intervention
Identifying effective practices
Identifying practices that need improvement
Providing value to existing and potential funders
Getting clarity and consensus around the purpose of your program
Measuring the effectiveness of an intervention. How do you know if a program was effective? If a program was not effective, would you want to know so that it could be improved? It is unlikely that we build capacity simply for the sake of building capacity, and because of this, we need to know whether or not our programs are effective, just as the nonprofits we serve need to know if their services are achieving the desired results.
2. Identifying effective practices. With the information you collect, you can determine which activities to continue and build upon. Some practices might be modified and replicated for other programs or initiatives based on your results.
3. Identifying practices that need improvement. Some activities may need to change in order to improve the effectiveness of your program.
4. Proving your value to existing and potential funders. The climate for funding social services and capacity building has undergone some radical transformations in the last few decades. Funders are keenly aware of the need to document the success of programs. Capacity building has lagged somewhat behind the social services field in having established evaluation standards. In 2003, the Alliance for Nonprofit Management reported that the capacity building field still lacked a framework for evaluation. However, this is changing, and it is important for capacity builders to stay ahead of the curve to be able to continue doing their work.
5. Getting clarity and consensus around the purpose of your program. Everyone in your organization, from board members to service staff to volunteers, should understand what is going on in your program and what it is intended to achieve. Outcome measurement helps to clarify your understanding of your program.
Demonstrate the value of a SBHC

5. Goals, Outcomes, and Indicators, oh my!
Goal: Broad statement of the ultimate aims of the program
Outcome: the changes in the lives of individuals, families, organizations or community as a result of the program
Indicator: the specific, measurable information collected to track whether an outcome has actually occurred.
Goal — broad statement of the ultimate aims of a program
..Outcome — the changes in the lives of individuals, families, organizations, or the community as a result of the program
..Indicator — the specific, measurable information collected to track whether an outcome has actually occurred
Outcome measurement will explore what your program provides, what its intended impacts are, and whether or not it achieves them. It will not prove that the changes that take place are a result of your program.
People often get stuck in the outcome measurement process because of all the terminology. Is it an outcome, an accomplishment, an achievement, or a result? Is it a goal or an objective? Is it an indicator or a performance measure? Some people see goals and objectives as interchangeable; others see outcomes and objectives as the same.
Don’t let this get in your way, but establish these definintions for yourself.

6. Logic Model
A logic model is a graphic depiction (road map) that presents the shared relationships among the resources, activities, outputs, outcomes, and impact for your program. It depicts the relationship between your program’s activities and its intended effects.
- Centers for Disease Control and Prevention

8. Performance measures- results based accountability
“Trying Hard is Not Good Enough” –Mark Friedman
Start with the END results and work backwards step by step
Establish what the problem is before you decide what to do
It’s all about results, not how you do it.
How much did we do? How well did we do it? Is anyone better off?
basic principle of R.B.A.:
Start with the end result and work backwards step by step to the means.
Establish what the problem is before you decide what to do
Remember it’s all about results, not how you do it
Think about the needs before the solutions.

9. Sorting Performance Measures: What is Measured in Each Quadrant
How much did we do?
How well did we do it?
% Common measures
e.g. client staff ratio, workload ratio, staff turnover rate, staff morale, % staff fully trained, % clients seen in their own language, unit cost
# Clients /customers served
% Activity-specific measures
e.g. % timely, % clients completing activity, % correct and complete
# Activities (by type of activity)
Is anyone better off?
% Skills / Knowledge e.g. parenting skills #
This chart shows in detail the different types of measure we typically find in each quadrant, and the measures that go with the three basic categories of performance measurement:
How much did we do?
How well did we do it?
Is anyone better off?
In the upper left, How much did we do? Quadrant, we typically count customers and activities.
In the upper right, How well did we do it? Quadrant, there are a set of common measures that apply to many different programs. And there is a set of activity specific measures. For each activity in the upper left, there is one or more measures that tell how well that particular activity was performed, usually having to do with timeliness or correctness.
In the lower quadrants, Is anyone better off? We usually have # and % pairs of the same measure. And these measures usually have to do with one of these four dimensions of better-offness: Skills/knowledge, Attitude, Behavior and Circumstance.
For each of these measures, we can use point in time measures or point to point improvement measures.
% Attitude / Opinion e.g. toward drugs #
% Behavior e.g. school attendance #
% Circumstance e.g. working, in stable housing # 9

10. Data Sources Electronic health records School academic records
School climate survey
Healthy Youth Survey
Patient satisfaction surveys
Other?

15. Quality Improvement UDS, HEDIS, NCQA, PCMH CHIPRA, Meaningful Use
Patient safety, infection rates
SLR
Why don’t we measure these things in SBH?

16. School based Health Alliance: National Quality Improvement Initiative
2014 development of standardized performance measures and 1st national cohort of clinics.
Modeled on Institute for Healthcare Improvement Model for Improvement Framework

17. The Model for Improvement asks three questions — What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in improvement? — and then employs Plan-Do-Study-Act (PDSA) cycles for small, rapid-cycle tests of change. IHI uses the Model for Improvement in all of its improvement efforts.

18. Standardized performance measures for SBHCs
Core Measures
Annual well child visit
Annual risk assessment
BMI screening and nutrition/physical activity counseling
Depression screening
Chlamydia screening
Stretch Measures
Student disposition log (seat time saved)
SBHC student user survey
Annual well child visit
Annual risk assessment
BMI screening and nutrition/physical activity counseling
Depression screening
Chlamydia screening
You decided that the best way to demonstrate the value add of SBHCs to the healthcare system is to work to ensure that children and adolescents have an annual (1) well-child visit, (2) risk assessment, and (3) BMI screening.
You also decided that adolescents should be screened for (4) depression and (5) chlamydia
We are thrilled with the measures you selected:
They align with existing national child quality measurement frameworks (HEDIS, CHIPRA, Meaningful Use) – allowing SBHCs to compare themselves with one another and with other healthcare settings.

19. National Quality Measures

20. Call to Action
Fall 2015 two sites were invited to join School Based Health Alliance National Quality Initiative (NQI) Collaborative Improvement and Innovation Network (CoIIN).

21. NQI MEASURES WELL-CHILD VISIT ANNUAL RISK-ASSESSMENT BMI
DEPRESSION SCREENING & FOLLOW-UP
CHLAMYDIA SCREENING

22. PDSA

24. Data collection:
Expectation
Reality

26. WCV

27. WCV

28. Flowsheets

29. Conversion to ICD-10.
Staffing.
Finding extractable data.
Changes in data export.
IT communication.
Competing projects.
Time.

30. Suggestions: Be patient. Takes time to get a good baseline.
Be consistent. Charting and reporting.
Develop (or maintain) good relationship with IT.
Meet early and often.
Be proud of our important work. (Make sure we can prove it).

35. Choosing Performance Measure
Performance Measure #1: ANNUAL WELL CHILD VISIT Performance Measure #2: ANNUAL RISK ASSESSMENT Performance Measure #3: BODY MASS INDEX (BMI) SCREENING AND NUTRITION/PHYSICAL ACTIVITY COUNSELING Performance Measure #4: DEPRESSION SCREENING AND FOLLOW-UP Performance Measure #5: CHLAMYDIA SCREENING

36. + CHLAMYDIA
High rate of POSITIVE chlamydia Extremely high-risk student body Difficult for students at different sites to come in for testing

37. How can we get more students in for chlamydia screening???
Challenge
How can we get more students in for chlamydia screening???

39. GO TO THE STUDENTS

42. Has this got you thinking?