Dial-in: Pass code: SPDG Directors’ Webinar Measuring Collaboration Presenters: Diana Autin, Executive Co-Director, Statewide Parent Advocacy Network; Director, NE-PACT/Region 1 Parent TA Center; Co-Director, NJ PTI Courtney Salzer, Co-Director, Parent Training and Information Center (PTIC) at WI FACETS, Co-Director of the Region 4 Parent Technical Assistance Center (PTAC) at WI FACETS Facilitators: Jennifer Coffey, OSEP Project Officer Carmen Sanchez, OSEP Project Officer John Lind, Interim Signetwork Coordinator 1

Roll Call

*6 / #6

5 Event Materials, Recording: t_calendar/events/1030

Directors’ Webinar Schedule 6 Mar 6 Organization Driver: Use of Data Apr 3 Organization Driver: Use of Data May 1 Family Engagement Jun 1 Organizational Driver: Facilitated Administration & Systems Jul 21 OSEP Project Directors’ Conf Sep 4 Leadership Driver Oct National Meeting – Will be finalized to 2-days

Introductions 7

The Power of Collaboration in Creating and Sustaining Systems Change Statewide Parent Advocacy Network 35 Halsey Street, Newark, NJ SPAN; Empowered Parents: Educated, Engaged, Effective! © 2014.

SPAN’S MISSION To empower families & inform & involve professionals & others interested in the healthy development & education of children, to enable them to become fully participating & contributing members of our communities & society.

Selected Key Partners

What is Systems Change Making positive changes by changing laws, policies, procedures, & attitudes that impact many children & families

MCHB Core Outcomes- Core Components of a System of Care Family-professional partnerships & satisfaction with services Ongoing, coordinated care within a medical home Adequate insurance to pay for needed services Early & continuous screening to identify special health care needs Community-based services organized for easy use Effective transition to adult systems of care & services

Integrated Systems Projects: Goals 1. Build/sustain leadership & structure needed for integrated system of services for CYSHCN 2. Improve participation of families, especially underserved families, in all aspects of individual child’s care & systems improvement 3. Improve access to care through medical homes; early & continuous screening to identify needs; community- based services; and adequate health insurance & financing 4. Increase focus on unique needs of YSHCN in transition

Collaboration Objective Establish & maintain Statewide Community of Care Consortium Core team Quarterly COCC meetings Work groups on each of the 6 core outcomes co-chaired by family & professional Implementation activities carried out by partners representing different constituencies Collaborative evaluation

Collaboration mechanisms Co-leadership at all levels Big Tent site housing all resources, meeting minutes, etc. Team development of all materials including proposal Team participation in development & delivery of all training Medical home quality improvement teams Medical home parent partners/leaders Healthcare financing workshops Child Care workshops Youth Transition workshops

Measuring Collaboration: “You treasure what you measure” Stakeholder Collaboration Survey Administered at beginning, middle & end of funding Measures stakeholder perception of extent of collaboration with all members of the Community of Care Consortium Focus groups with participants from stakeholder groups Discussions of barriers to collaboration, concerns with collaboration, at: Co-convener calls (SPAN Co-ED, AAP-NJ ED, NJ DOH Title V) Core team calls (SPAN, all state agencies, key partners) Quarterly COCC meetings

Family Leadership & Satisfaction-Inputs Family leadership development Family participation in Core team & all workgroups Family Resource Specialists (parent leaders) in all 21 SCHS CMUs, at Autism Clinical Enhancement Programs, & at FQHCs

Family Leadership & Satisfaction- Outcomes Parent partners on Medical Home QI teams increased skills to “make a difference” Statistically significant improvements in Medical Home Index-Pediatric Version Family Survey Parents receiving project services demonstrated higher levels on CSHCN survey than NJ average

Medical Home-Inputs Medical Home workgroup & Leadership Action Group County-wide medical home pilots with hospitals, FQHCs, 6-8 pediatric practices Targeted focus on immigrant families in Hudson, Union, Passaic Learning Collaboratives Trained parents on medical home teams FRSs mapping community resources

Medical Home-Outcomes Statistically significant improvements in “medical homeness” across all indicators on Medical Home Index-Pediatric across all regions Statistically significant improvements in Medical Home Index-Pediatric Version Family Survey

Adequate Insurance- Inputs Health Insurance Adequacy Workgroup Increase availability of and enhance family connection to resources to pay for needed care Organized parents to advocate for Autism and Other DD Health Insurance Mandate Engaging parent leaders to advocate for ACA implementation (exchanges, Medicaid expansion, Comprehensive Medicaid waiver)

Adequate Insurance- Outcomes Increased use of Autism & Other DD Health Insurance Mandate Increased utilization of healthcare financing resources by immigrant, LEP, other families of color NJ Governor accepted Medicaid expansion Maintained no co-pay in Medicaid under Comprehensive Medicaid waiver

Early & Continuous Screening- Inputs Early & Continuous Screening Workgroup Parent-led training for child care centers on early ID, screening & evaluation resources, services Parent leader story kicked off Help Me Grow Summit TA to Clinical Enhancement Programs on how to reach & partner with underserved families Parent-led training for FQHCs

Early & Continuous Screening- Outcomes Overall screening rate increased by 29%; overall screening rate for 9 month visit increased by 47% Use of AAP-recommended standardized screening tools increased dramatically Increase in provider referral to EI or specialist Increase in underserved family follow-up following ASD screening Participating providers indicated that the family story was a significant motivation for change

Community services organized for easy use- Inputs Community Services Workgroup Mini-contracts to community teams co-led by parent leaders Enhanced partnership with FQHCs, clinics, etc. Parent leader training to community providers such as child care to address needs of CYSHCN Parent leader outreach to schools on resources for children with challenging behavior through family- led Family Support Organizations

Community services organized for easy use- Outcomes Increased knowledge of community resources by practices Lack of community referral sources went from being rated as the #1 barrier to not being indicated as a barrier at all from participating practices Increased referrals to EI

Health Provider Feedback What do health providers say?  Over 90%:  Are better able to partner with parents  Are more knowledgeable about/connected to community resources for families  Feel more confident in coordinating health care services for CYSHCN “I always knew I needed to partner with my patients’ parents about their individual care, but it never occurred to me that parents could also help me improve my practice overall. Our parent partners are an incredible source of information and ideas. They know about the community resources in our area, and they know what parents need. They help us figure out what is going right, and what we need to improve. And they are an incredible resource for the other parents of children with special needs in our practice. I don’t know how we ever lived without parent partners!” Pediatrician, NJ Medical Home Practice

NJ Department of Health Title V Feedback What does NJ Department of Health say?  Over 90% of Special Child Health Services Case Managers say that Family Resource Specialists:  Help families partner with their child’s health, education, and other service providers  Help families more effectively navigate community services  Build parent confidence & competence in getting needed services for their child “Our collaboration-partnership is possible and effective because we have trust in each other, equality and a balance of power, a shared vision and commitment to the same goals; we highly value the contributions made by each agency; and we see the benefits to our respective agencies, but most importantly to the families and children we serve. “ Gloria Rodriguez, Assistant Commissioner, Family Health Services, NJ Department of Health and Senior Services

Family Feedback What do families say?  Over 90%:  Are better able to partner with their child’s provider and navigate community services  Feel more confident in getting needed services for their child “When my child was diagnosed, I cried in the doctor’s office parking lot for an hour…It took another parent to understand and guide me. By being paired with a parent who had “been there,” and who understood my family’s unique needs, it brought out the best in me at the worst time in my life. Through workshops and support, I learned specific leadership skills (advocacy, effective communication, organizational skills, coping with stress)…I received support from resourceful SPAN advocates and in return was able to navigate systems not only for myself and my own family for others as well. Now I participate on State Advisory Committees, do presentations for agencies about the issues facing families, and meet with state policymakers to advocate for the changes families need. Through SPAN’s support, I not only found my child’s voice, I found my own, and now I lend my voice to others not able to speak due to language barrier or disability.” NJ Latina Parent helped by SPAN

External evaluator’s quote: “The ultimate purpose of evaluation is to improve programs and help them achieve their goals. All the key stakeholders in this project placed a high value on collaboration and teamwork and our evaluation reflected this. By involving the stakeholders in all phases of the evaluation process (design, planning, implementing, interpreting results) we were able use their expertise to really focus on getting the info that they thought would be most useful and meaningful for implementing and improving the projects. The stakeholders were also very engaged in interpreting the evaluation findings and then using them to make quality improvements throughout the grant implementation periods. These improvements, in turn, resulted in successful outcomes. It’s a perfect example of how the process of evaluation can help programs achieve their goals.”

Sustaining the Work Because all partners felt integral to the process, the COCC has been continued: NJ DOH MCHB has incorporated COCC into MCH block grant NJ DOH MCHB has provided additional funding for COCC activities UCEDD secured funding to pay for parent-led training on screening & resources for FQHCs, is seeking funding to pay for parent-led training for Head Start agencies, & is collaborating with COCC on project focused on the adult side of transition (adult healthcare providers) Epilepsy Foundation secured funding for medical home Partners are covering meeting costs on rotating basis

SPAN’s integration of learning into ongoing work: PTI Grant application & implementation Convened stakeholders including state agencies across systems to identify strengths, needs, activities Created Stakeholder Board for PTI with face-to-face meetings & periodic workgroup conference calls focused on: Technical assistance/individual assistance to families Parent & professional development Resources/materials/website Work with stakeholders to implement their recommendations & report on actions taken Stakeholder Board has improved our work & our reach

Lessons learned: Collaboration Having the collaboration as a project goal, with measurement and evaluation attached: Kept us all at the table even when there were disagreements Made us build in mechanisms to support ongoing collaboration & deal with challenges Ensured that how collaboration was happening was on the agenda for every meeting “If everyone is moving forward together, then success takes care of itself.” - Henry Ford

Resources IDEA Partnership: Leading by Convening, connections/the-partnership-way.html connections/the-partnership-way.html Frey et al, Measuring collaboration among grant partners, /MeasuringCollaborationAmongGrantPartnersArticle.pdf /MeasuringCollaborationAmongGrantPartnersArticle.pdf Hogue’s Five Levels of Collaboration (1994) Interagency Partnerships: A Compendium of Measurement Instruments, James Bell Associates, n%20compendium%20final%2010_16_2012.pdf n%20compendium%20final%2010_16_2012.pdf Assessing your Collaboration,

SIG Network Webinar February 5, 2014 Presented by: Courtney Salzer Co-Director, Region 4 PTAC Co-Director, WI FACETS’ PTIC Measuring Collaboration: Is There A Recipe For Success? The contents of this document were developed under a grant from the US Department of Education, #H325R However, those contents do not necessarily represent the policy of the US Department of Education, and you should not assume endorsement by the Federal Government. Project Officer, David Emenheiser.

Courtney’s Disclaimers 1. I am NOT an evaluator – the information in this presentation is largely based on my experience and some of the existing working theories on collaboration. 2. I work at WI FACETS, which is one of over 100+ parent centers throughout the country. My experiences may not represent the experiences of every other parent center. 3. I do not believe in sugar coating things. I have worked on our SIG projects for almost 10 years. We have been through good times, bad times and just plain confusing times. 4. I have been told that my PowerPoints generally have too many words and not enough visuals, so please enjoy my clipart. I worked very hard on it.

Widely Used Collaboration Lifecycle Five Levels of Collaboration (Frey, et al., 2006) 1. Networking 2. Cooperation 3. Coordination 4. Coalition 5. Collaboration

The Dark Days First Date Making a commitment True Partnership The Lifecycle of our SIG/SPDG Collaboration

Lessons We’ve Learned Collaboration is a process, which ebbs and flows – this makes it somewhat challenging to measure, but not impossible! The term “Collaboration” means something different for everyone. Try to establish a common language. Effective measurement will be difficult without it. Measurement of collaboration should occur at several phases during its lifecycle. Each collaboration has vital signs that can be measured, a “recipe” of sorts.

What Goes Into “The Recipe”?

Ingredient 1: Systemic Vital Signs Communication Cooperation Shared Decision-Making Shared Vision & Goals Sufficient Resources Skilled Leadership Openness/Transparency Appropriate Cross- Section of Collaborators

Ingredient 2: Collaborator Vital Signs Are collaborators well-informed? Are collaborators willing to compromise? Do collaborators believe their group or constituents will benefit from the collaboration? Is everyone at the table? Are collaborators willing to work through conflict? Does respect exist between collaborators? Is there trust between collaborators?

1. “Leading By Convening. A Blueprint for Authentic Engagement” Source: IDEA Partnership Location: connections/the-partnership-way.htmlhttp:// connections/the-partnership-way.html 2. “Serving on Groups that Make Decisions. A Guide for Families” Source: WI FACETS Location: Two Helpful Resources for Building Collaboration

Evaluation is usually done, in part, for continuous improvement. In my experience, there is always much room for improvement. However, if all you focus on is improvement, you will in time kill the momentum of your collaboration. When your evaluation reveals successes or milestones, you must celebrate them – however small they may be. A Cautionary Note on Measuring Collaboration

Shifting Collaboration Paradigms Old Paradigm: Effective collaboration is a “nice by- product”. New Paradigm: Effective collaboration is an outcome. Old Paradigm: Measuring collaboration = are people talking to one another? New Paradigm: Measuring collaboration = measuring systemic + collaborator vital signs Old Paradigm: Successful projects are those with many outcomes and products New Paradigm: Successful projects are those with collaborative outcomes and products

QUESTIONS? Thank you!