Catulpa Community Support Services

 Use of an electronic data entry program to record demographic data and case notes to reflect service delivery  Specific data recorded and reported for each agency program to reflect funder requirements and our own internal standards for service delivery  Program driven annual parent satisfaction surveys

 Explore evaluation processes for all agency programs  Develop agency standards and guidelines for evaluation  Develop program specific logic models  Determine program and agency wide evaluation questions  Utilize evidence based outcome measures for evaluation  Determine if there is an evidence based outcome measure that could apply to multiple agency programs

 Executive Director and Board of Directors commitment to evaluation  Dedicated time for Quality Assurance lead  Support from the Ontario Centre of Excellence for Child and Youth Mental Health  Open, enthusiastic staff

 To enhance the support networks and care-giving capacity of the family/youth to develop or maintain a safe and healthy environment

 Resilience  Goal Attainment  Social Determinants of Health  Fidelity to Wrap Process  Participants experiences with Wrap Service and Wrap Evaluation

 Formation of the WrapAround Research Evaluation Network (WREN): Catulpa Community Support Services WrapAround Northumberland Shalem Mental Health Network

 To develop a flexible, innovative evaluation based on the fundamental principles of WrapAround.

 Resilience Confirm sustainable change for families/clients Ongoing consultation from Dr. Michael Ungar – Resilience Research Centre (Halifax) Contributions to research  Child & Youth Resilience Measure (CYRM)  Pathways to Resilience Youth Measure (PRYM)  Resilience Scale for Adults (RSA)

 Contributions to other Wrap programs  Informing a Catulpa Evaluation Framework

 Establish a shared evaluation framework across the 3 sites  Development of a strengths-based questioning for evaluation.  Development of a measure of family functioning based on the Social Determinants of Health

 WREN Evaluation 2012:  Across the 3 sites: 13 adults completed evaluations at T1 and T2 for Resilience, Functional Indicators 8 adults completed T1 and T2 for Goal Attainment Scaling A total of 33 WFI interviews were completed: 7 caregivers, 14 facilitators, 5 youth and 7 team members

 Based on responses to the Functional Indicators questionnaire (T1 to T2)  There was a significant increase in means for the following 5 questions: I enjoy working in the home My housing is secure and meets my personal/family needs I have at least one person to confide in I read adequately for my daily needs I write adequately for my daily needs

 Participants indicated strong agreement with the following statements: Overall I am satisfied with the services I received I felt I had someone to talk to when I was in trouble I had a say in how service was delivered to me I received services that were right for me I am now better able to cope when things go wrong

 “I knew nothing about Wrap previously. It is an amazing program and I would recommend it to anyone. It helped because people believed in me”. (Parent)  “WrapAround has improved a lot of aspects of my life by helping me find resources that are available, i.e. respite and other resources.” (Parent)  “The youth wouldn’t have returned home if not for the planning that occurred within WrapAround” (Team member)  “…the program has opened other avenues for the family and helped with independence.” (Team member)

 We have meaningful results and now have an effective, robust program evaluation framework with which we can move forward:  There is evidence that the use of the evaluation tools within our framework have “added value” for the children, youth and families served and also assists the Facilitators to be more effective in their implementation of the Wraparound process.

 There is evidence that the use of the Wraparound process can help to build resilience and help children, youth and adults get their needs met with respect to the social determinants of health.  There is evidence that the children, youth and families served are building connections to and developing a community of support and a positive sense of their future.

 The WREN collaboration was effective in developing and implementing a comprehensive program evaluation framework and in developing new evaluation tools  The added time invested to ensure the effective communication and coordination of our collaborative evaluation team (WREN) has resulted in many significant benefits and outcomes that individually we would not have achieved.

 Our sample size was relatively small and thus further evaluation is necessary to confirm our current results.  Further efforts are required to engage youth in the evaluation process

 Continue to conduct our evaluation and report data results within WREN  Devise multiple strategies to engage youth  Eliminate duplication of questions  Devise more creative and time efficient ways to collect data

 Flexible strategies for administering the tools that are responsive to children, youth and adults with varying levels of literacy, learning difficulties, developmental delays (etc).  Developing a range of strategies to report data back to participants

 Develop an Agency Logic Model  Formalize a consistent process for gathering client feedback  Determine program and agency wide evaluation questions  Utilize evidence based outcome measures for evaluation  Determine if there is an evidence based outcome measure that could apply to multiple agency programs  Confirm our terms of reference for an Ethics Review Committee