Networks, and Outcome, and Tools! Oh My! Applying Social Network Analysis to Measure and Build Early Childhood Systems for Better Public Health in the Rockies Vail, Colorado; September 2015

Outline I.Social Network Analysis (SNA) as a Tool for Quality Improvement II.Short Exercise III.Examples From the Project a.Background b.State-level Results c.Communicating Results d.QI Examples IV.What We’ve Learned and What’s Next

Disclaimer Disclaimer: Points of view or opinions stated in this presentation are those of the authors and do not necessarily represent the official position or policies of the Colorado Department of Public Health and Environment, Colorado Department of Human Service or those of the U.S. Department of Health and Human Services, Health Resources and Services Administration.

I. Social Network Analysis (SNA) as a Tool for Quality Improvement

The “Network Way” of Working Graphic from NACCHO (MAPP website):

Working Across Boundaries is an Core Function for Organizations EMS Tribal Health Schools Dentists Law Enforcement Corrections Faith Instit. NGOs Labs HCP City Planners Transit Fire Civic Groups Employers Drug Treatment Elected Officials Mental Health CHCs Public Health Dept Parks and Rec Nursing Homes Neighborhood Orgs. Home Health Graphic from CDC/NACCHO (MAPP website):

Why Do We Need New Concepts & Tools? To provide an additional way to evaluate partnerships. Current Assumption = More is better. –More partners = successful collaboration (counting noses) Alternative Assumption = Less can be more. –Not based on how many partners you have, but how they are connected. YOU

Creating Network Ties Creating SubgroupsIncreasing Density Connecting the Network: No Isolates

Planning a Wedding…

Social Network Analysis Social Network Analysis (SNA) is a method to gather and analyze data to explain the degree to which network actors connect to one another and the structural makeup of collaborative relationships (Scott, 1991).

Elements of a Network Collects data on who is connected to whom How those connections vary and change Focus on patterns of relations Nodes (People, Orgs, Etc) Lines (Relationships)

II. Exercise: Your Personal Network

What Can You Do With Social Network Data?

With SNA, You Can Make Evidence-Based Decisions (as QI) 1. Evaluate how well your collaborative is working in terms of identifying essential partners and gauge their level of involvement, leveraging resources, and strategizing for how to improve the work of the collaborative. 2. Demonstrate to partners, stakeholders, evaluators, and funders how your collaborative is progressing over time and why working together is making tangible change. 3. Develop action steps based on the evidence (data).

SNA Can Reveal “Systems” Outcomes Systems outcomes are not the same as population or client outcomes; Rather, intermediary outcomes that reflect the way that organizations interact, share resources, and implement work (PROCESS Outcomes) Children & Family Outcomes Partnerships for Systems Building -Convening Stakeholders -Needs Assessments -Leveraging Resources Systems Outcomes Perceptions of: - Value, Trust, Authenticity - Goals, Outcomes, Process Measures of: - Coordination, Efficiency, Redundancy [Among Organizations] [Set in Community Context] How can I use this information to manage this system?

III. Examples From The Project

Project Background In partnership with CDPHE, CO Trust, and CDHS, the PARTNER team met with Early Childhood Councils (ECC) in MIECHV or Health Integration funded communities to assess system relationships for Quality Improvement. As part of the Quality Improvement (QI) effort, a Social Network Analysis was conducted to identify system relationships and develop an evidence base for systems building efforts. – PARTNER (Program to Analyze, Record, and Track Networks to Enhance Relationships, – Process Quality/Working Together Tool

How Communities Participated The ECC Coordinator from each community served as the Point of Contact to coordinate a stakeholder meeting and administer a survey to ECC members. The project consisted of the follow activities: – A one hour introductory webinar for ECC Coordinators (November) – A half-day onsite meetings with stakeholder groups (January/February) – Survey administration (April) – Debrief meetings with each site (April/May) – Each community received a Personalized Coalition Evaluation/Assessment and recommendations for actions steps to engage in systems building (July/August) – ECCs continue to participate in TA for translating data into action

Data Collection (PARTNER Survey) Feb/March 2013 PARTNER was used in this project to collect data and inform a Quality Improvement process. The survey combined the PARTNER survey with the Process Quality/Working Together survey already administered to the ECCs in the past.

What is PARTNER? PARTNER (Program to Analyze, Record, and Track Networks to Enhance Relationships) utilizes Social Network Analysis (SNA), a quantitative methodology that focuses on relationships between and among organizations, measuring and mapping relationships and flows between organizations. -- Housed in the School of Public Affairs at UC Denver--

Survey Questions DimensionMeasures Organizational Membership Organizational identification by name, type, and other organizational characteristics (e.g. size, mission of organization) Network Interaction Network patterns and positions identified by subgroups, key players, etc. Quality of Relationships Types and levels of communications among members Organizational Value to the Collaborative Power, involvement, resources Trust Reliability, shared belief in mission, opportunity for frank discussion Additional Questions Process Quality/Working Together

Who Participated? (Time 2)

Response Rate – Goal of 65%

State-Level Results

What is the value of the ECCs to Colorado’s Early Childhood System? 5,975 Early Childhood Partnerships in Colorado described by 394 respondents Of those, 3,755 Partnerships were attributed to ECC – 2,615 Partnerships Developed From ECC – 1,159 Strengthened by ECC In other words, 63% of all partnerships In EC System in Colorado attributed to the ECC – Each one of these partnerships has an accompanying story about what they are doing to strengthen the ECC, how the partnership benefits each organization, and how those partnerships are linked to improving the health and welfare of kids in Colorado.

5,975 Partnership in the Early Childhood System in Colorado Described in This Set of Data Number of Partnerships Described Across the State

Please describe how your relationship with each of these partners was developed. In all but 2 counties, at least 50% of all partnerships attributed to the ECC. At least 20% attributed directly to the work of the ECC. 2,615 Partnerships Developed From ECC 1,159 Strengthened by ECC 3,755 Partnerships Attributed to ECC

How many partnerships exist in Colorado (and in each county) in Different Domains?

What are all these partners doing together?

Communicating Results

QI Examples

QI Example 1: Reaching Home Visiting Goals  Home Visiting Committee set as a goal: ◦ To increase partnerships in the county  What they already know: ◦ A few organizations are active, but not a lot of knowledge about them  What PARTNER can help them know: ◦ Existing relationships? Nature of relationships? ◦ This information can help them find out more about these partnerships and potentially use them as best practices for increased involvement

Data About Partnerships Involving Home Visiting

Home Visiting Partnerships Relationship Scores Trust ScoresValue Scores

Translating the Data Into Practice Identify members who have partnerships around “home visiting” New/Unknown relationships? What types of partnerships are happening? – Challenges to developing these partnerships? – Best practices? – Programs/Resources that can be leveraged to develop more partnerships like these? – ECC can develop a mini-provider/partner network and convene them, or at least tap into their collective knowledge.

QI Example 2: Managing Relationships within the ECC The PARTNER survey asked respondents to rate others on trust, value, and authenticity Perceptions are the root of all relationships Understanding perceptions of trust, value, and authenticity provides managers with a tool to manage relationships

ECC Coordinator Indicated Lower Perceptions of Resource Contribution Data worksheet can provide more specifics Resources contributions can be identified either in report or from PARTNER tool

Translating the Data into Practice ECC Coordinators need members who can contribute resources! Coordinator can look at data (on Data tab) and see who was rated high/low Coordinator can then match up the resources that members say they can contribute to those that she needs

QI Example3: Profiling an ECC Member See Handout Some ECC Coordinators were interested in getting profiles of some of their members, based on the data Handout: One example of a member profile These can be used to build, manage, and strengthen systems through relationship management

IV. What’s Next (and What We’ve Learned)

Project Limitations Gap between having these data and the ability to really understand/apply the data – Network data are complex and “new” – Cultural/historical barriers to translating data into practice (in general) Survey may need to be revised Standardizing across sites is possible, but will take some work to implement – “Lists” of members – Systems measures need to reflect site differences; but standard measures can be developed to measure site specific goals

Translation to Practice is Challenging The PARTNER data’s primary purpose:  To give coordinators/communities data that can help them improve their processes; strengthen their systems work  It is NOT to compare one ECC to another, or measure how “well” each ECC is performing  We are not experienced in using data to make decisions, especially when it comes to building relationships among constituents  These types of data are atypical  We often depend on who we know, what we know about them and what we think we know about them

Key Take Away Points There is an evidence-based practice for managing networks & partnerships Social Network Analysis provides research-based tool Quality Improvement can be applied to networks & partnerships

Questions? Carsten Baumann Danielle M. Varda Sara Sprong PARTNER Team