Measuring Hope for Children Living in Poverty: Engaging stakeholders in evaluation at City Kidz June 13, 2013 Rich Janzen & Liliana Araujo

Partners

Purpose To share the evaluation experience of City Kidz as an example of a collaborative evaluation that followed the principles of community based research. 1. Overview of community based research 2. Overview of the City Kidz evaluation 3. Mechanisms of engagement 4. Products of engagement

“The power of knowledge to impact change” - Since 1982 Centre for Community Based Research  30 years of social innovation in Canada  Over 350 community based research projects  Based on an entrepreneurial spirit in collaboration with many partners to initiate new projects

Hallmarks of Community Based Research Research that strives to be:  Community situated- begins with a research topic of practical relevance to the community and is carried out in community settings. (Indigenous tradition)  Participatory - community members and researchers equitably share control of the research agenda through active and reciprocal involvement in the research design, implementation and dissemination. (Southern tradition)  Action-oriented - the process and results are useful to community members in making positive social change and to promote social equity. (Northern tradition)

Functions of Community Based Research Knowledge production Knowledge mobilization Community mobilization

Definition “… a research approach that involves active participation of stakeholders, those whose lives are affected by the issue being studied, in all phases of research for the purpose of producing useful results to make positive changes” (Nelson, Ochocka, Griffin & Lord, 1998, p.12)  “Research with,” not “research on” people  Training and mentoring  Opportunity of meaningful involvement  Valuing experiential knowledge  Data for advocacy  Value-driven approach (Ochocka & Janzen, 2007)

A Growing Trend “Too often, important knowledge remains hidden in academia. Too often, governments develop policies without a full understanding of the big picture and without tracking the consequences of their policies. Too often, civil society organizations implement programs without adequate analysis of the underlying problem and careful consideration of how the program will play out. Solving the complex social, environmental and economic problems we face will require collaborative efforts that are radically inclusive of diverse perspectives and skills. Such collaborations become possible when faculty, staff, and students come to realize that people in community settings have knowledge, experience, and talents that complement their own.” - Fryer, 2012, University Affairs - emphasis added

Overview of City Kidz Evaluation

 World Vision and City Kidz history in Partners to End Child Poverty (PECP)  Desire to replicate to other communities  Purpose To assess the implementation processes of City Kidz’ core programs To assess the outcomes for program participants of City Kidz’ core programs To identify future directions for improving on and replicating City Kidz’ core programs in other sites across Canada Evaluation Background and Purpose

 How are core City Kidz programs presently being implemented?  How and to what extent have core City Kidz programs impacted the well-being of children in low-income communities of Hamilton?  What suggestions would help to improve and replicate core City Kidz programs? Main Research Questions

 Follows the principles of community-based research (community-situated, participatory, action-oriented)  Takes matters of faith seriously  Adopts World Vision’s model of child well-being  Program theory to reflect these program elements Evaluation Approach

Kinder Kidz Bus activities/food Theatre show (4x/Saturday) Biblical principles/prayer City Kidz Saturday Bus activities/food Theatre show (4x/Saturday) Biblical principles/prayer Junior Leadership Sat. volunteering & connecting Mid-week training (Kid Lead) Biblical principles/prayer Increased self concept and positive self worth Increased opportunity for, awareness of & participation in community activities Increased positive peer influence and relationships Increased trust in, credibility of and influence of adult mentors Increased prosocial behaviour Increased understanding of God’s love New relationships with adults who love unconditionally Increased leadership within City Kidz & community Healthier lifestyle boundaries More likely to make positive choices Weekly Home Visits -Personalized mentorship -Information about activities/events -Prayer -Encouragement to go to group activities -Support to family members Discerning God’s Lead Spiritual retreat Daily staff prayer Openness to miracles Miracle Sundays Organizational tithing 2013 Logic Model Increased belief that God created me Increased belief that God loves me Increased belief that God has a plan for my life Increased Hope Better able to dream of a future beyond the cycle of poverty Healthier Children Growth in physical, social, and mental capacity Enabled to confront challenges with resiliency and contribute to society with full potential Safer Children Better access to food, safe housing and stable employment Enabled to fully participate in society, now and in future More Educated Children Greater knowledge and skill development Enabled to achieve in life and employment preparation Healthier Neighbourhoods Across Hamilton Community- Level Increased cultural sensitivity and acceptance Healthier family dynamics More likely to imagine accomplishing great things Individual-Level Increased awareness of God- given potential Increased self- control and empowerment 400 Kidz 1,100 Kidz 150 Kidz 2,200 Kidz Group Activities Individual Activities Faith Outcomes Resiliency Outcomes Well-Being Outcomes Improved Well-Being for Children Living in Poverty – 100,000 Canadian kids by 2040 Spiritual Discipline Activities Increased social sensitivity and empathy Internal OutcomesRelationship Outcomes Action Outcomes

 Program tracking logs (quantitative tracking of program inputs and outputs)  Survey of participants City Kidz Saturdays: sample of present participants complete short survey interview with retrospective outcome, implementation, & future directions  Participant focus groups (3 groups with 8-12 sampled participants and/or parents about program implementation, outcomes and future directions)  Staff/volunteer interviews (2 focus groups with sampled staff and volunteers related to program implementation, outcomes and future directions)  Case studies (3 in-depth stories of program impact in the lives of purposively sampled present and past participants. Each story to consider interviews with the participant, one City Kidz staff/volunteer, and another support person in the participant’s life)  Key informant interviews (4 interviews to gain insight into outcomes and factors to consider when replicating core programs in other cities across Canada) Methods

Mechanisms of Engagement  Stakeholder steering group that guided each step of the evaluation  Training and supporting of “community researchers” to assist with data gathering  Methodology design that considered multiple stakeholder perspectives  Facilitation of active funder involvement (World Vision Canada) towards using evaluation findings to inform the replication of City Kidz nation-wide  Organizational feedback session to discuss evaluation findings

Products of Engagement Collaboratively developed products…  Comprehensive evaluation framework  Program logic model  Community researcher training manual  Qualitative and quantitative evaluation tools  User-friendly evaluation report that speaks to multiple audiences  Outcome survey based on program theory and informed by theory of hope and resiliency

Future Contact Rich Janzen Research Director, Centre for Community Based Research 73 King St. West Kitchener, Ontario x 233