1. Strengthening Families Initiative
Evaluation Report , Year 4 of the SF Initiative
First 5 Butte County Children and Families Commission
Prepared By: Social Entrepreneurs, Inc.

2. Introduction
The Strengthening Families Initiative (SF Initiative) is a Commission-funded effort with investments designed to ensure that all families are well-equipped to help their children reach their optimal development and full potential.
Expected long term outcomes include that:
Families access necessary support systems and community services
Children are raised in safe and supportive homes and communities
Families have the opportunity to come together and build networks of support
With four years of consistent data, we can now, for the first time, assert that SFI has successfully demonstrated evidence of those three long term outcomes.   

3. Five Protective Factors

4. Three Levers for Change
Research indicates that participants in the Strengthening Families National Network have found three key “levers for change” that are critical for fully realizing the promise of the Strengthening Families framework.

5. Questions to be Answered through Evaluation
Is the Strengthening Families Initiative changing systems to increase the Protective Factors?
Is the Strengthening Families Initiative Network increasing Protective Factors within the families that they serve?
Are families accessing help from the Strengthening Families Initiative Network partners and other community organizations, as well as improving self-help skills?
Is the Strengthening Families Initiative positively impacting the three levers for change?
How does formative evaluation help programs implement the Protective Factors?

6. Findings
SF Initiative impacted systems it funded and increased the Protective Factors for families served through Network Partners. However, extending that impact outside of funded partners has been challenging.
Initiative Network partners are increasing Protective Factors within the families that they serve.
Families continue to access help from the SF Initiative Network partners and other community organizations, and report improved self-help skills.

7. Findings Continued
The SF Initiative has positively impacted the three levers for change within the Network and the Commission but has been less able to impact the three levers countywide, with systems outside the Commission’s funding capacity.
Formative evaluation and increasingly, Network Meetings has helped Network partners implement the Protective Factors and Network partners routinely assist each other and programs outside of the Initiative to strengthen the Protective Factors with families they serve.

8. Conclusion
A number of things have proven effective in the past four years and should be acknowledged as strengths.
Systems are in place to effectively train staff using the online training system and more staff working with families with young children have completed Strengthening Families training than ever before in the county.
Tools such as the retrospective survey, BUILD Assessment, and program self-assessment have been useful in measuring impact and in identifying priorities to strengthen programs and systems.
Quarterly Network meetings foster collaboration and a peer learning environment.
Parent advisory teams are more evident and are used to evaluate program design and provide input into service delivery strategies.

9. Conclusion Continued
More culturally and linguistically appropriate resources and training are available than ever before.
ASQ’s, Parent Café’s and home visits have been successful in supporting parents, teaching them about child development and positive parenting while reducing isolation.
Parents have more opportunities to meet other parents and form networks of support.
More programs serving children in Butte County are aware of and implementing the Strengthening Families Framework.

10. Conclusion Continued
Opportunities exist to expand the Strengthening Families Initiative and make it sustainable countywide.
Partner with agencies that promote financial stability to help families learn, earn, save, invest and protect financial resources, which has been an ongoing concern for the majority of families served in the Initiative.
Encourage Network Partners to deepen their relationships with parent advisory groups by soliciting their input in staff hiring and training.
Engage leaders throughout the county to steer a Strengthening Families effort that goes beyond programs that receive First 5 funding.

11. Conclusion Continued
Link other Commission Initiatives to Strengthening Families in a more meaningful way as visually depicted in the Commission’s strategic plan.
Ensure that First 5 data systems can produce meaningful reports for Network partners to tell their story and seek funding from sources beyond First 5.

12. Evaluation Approach Processes Outcomes Site visits
Technical Assistance
Quarterly Network Meetings
Strengthening Families Program Self-Assessment
Training Evaluation
First 5ERS
Family Support Outcomes Protective Factors Survey
Ages & Stages Questionnaire

13. Changes in the 2014-15 Methodology
New Evaluation Components
Information from the CSCC pilot project
Self-assessment results
Changes in Emphasis of Evaluations Components
The report had more focus on the agency self-assessments and more focus on their progress in addressing areas in the evaluation report that were included as recommendations and didn’t report on the BUILD Assessment, which had been utilized three times previously.

14. Changes in the 2014-15 Methodology Cont.
Evaluation Components Consistent Between Reports
A comparison of SF participant demographics to the county as a whole
Retrospective Protective Factors Survey (PFS) results
Administration of the Ages & Stages Questionnaire (ASQ) by Network partners
SF Initiative Network meeting milestones

15. Limitations SF PFS surveys are not a true random sample
Lack of control group
SF Initiative Network Service Partner Variables
Survey administration
Generalization of Initiative results countywide based on program participants

16. Evolution of the Network

17. Self-Assessments and Milestones
The self-assessments were organized around seven program strategies. Questions within each strategy were answered using the following rating descriptions: strongly agree, agree, neither agree nor disagree, disagree, strongly disagree, or not applicable. The seven strategies are:
Facilitate Friendships and Mutual Support
Strengthen Parenting
Respond to Family Crises
Link Families to Services and Opportunities
Facilitate Children’s Social and Emotional Development
Observe and Respond to Early Warning Sign of Child Abuse or Neglect
Value and Support Parents

18. Self Assessment Results
Four programs re-administered the self-assessment in to identify changes between the initial assessment and the current assessment period:
Butte Baby Steps
Parents as Teachers – Thermalito
Valley Oak Children’s Services, and
Paradise Ridge
Initial assessment: a total of 25 respondents from the four agencies participated in the survey
2015: 43 respondents

19. Self Assessment Results Continued
Strategy
Description
Strategy 1
Facilitate Friendships and Mutual Support
Strategy 2
Strengthen Parenting
Strategy 3
Respond to Family Crises
Strategy 4
Link Families to Services and Opportunities
Strategy 5
Facilitate Children’s Social and Emotional Development
Strategy 6
Recognize and Respond to Early Warning Signs of Child Abuse or Neglect
Strategy 7
Values and Support Parents

20. Self Assessment Results Continued
Strategy
Description
Strategy 1
Facilitate Friendships and Mutual Support
Strategy 2
Strengthen Parenting
Strategy 3
Respond to Family Crises
Strategy 4
Link Families to Services and Opportunities
Strategy 5
Facilitate Children’s Social and Emotional Development
Strategy 6
Recognize and Respond to Early Warning Signs of Child Abuse or Neglect
Strategy 7
Values and Support Parents

21. Self Assessment Results Continued
Strategy
Description
Strategy 1
Facilitate Friendships and Mutual Support
Strategy 2
Strengthen Parenting
Strategy 3
Respond to Family Crises
Strategy 4
Link Families to Services and Opportunities
Strategy 5
Facilitate Children’s Social and Emotional Development
Strategy 6
Recognize and Respond to Early Warning Signs of Child Abuse or Neglect
Strategy 7
Values and Support Parents

22. Self Assessment Results Continued
Opportunities
Strategy 1: Facilitate Friendships and Mutual Support
Question 4b (n=34, 26.5% disagreement): The program provides opportunities for families to socialize and foster a sense of community through celebrations, graduations, and holidays.
Question 4d (n=42, 26.2% disagreement): The program provides opportunities for families to socialize and foster a sense of community through events celebrating cultural customs, potlucks, and other opportunities for parents to share and learn about each other’s home lives and cultural backgrounds.

23. Self Assessment Results Continued
Opportunities Continued
Strategy 2: Strengthen Parenting
Question 4a (n=41, 26.8% disagreement): Opportunities are created for parents to explore cultural/ethnic expectations and practices about parenting.
Strategy 3: Respond to Family Crises
Question 5 (n=33, 36.3% disagreement): Resources are made available to families in crisis, such as money from a small emergency fund, access to meals, or transportation.
Question 8 (n=34, 26.5% disagreement): If appropriate, staff mobilize other parents in the program to help out families in crisis.

24. Self Assessment Results Continued
Opportunities Continued
Strategy 4: Link Families to Services and Opportunities
There were no questions that had over 20 percent disagreement in strategy 4.
Strategy 5: Facilitate Children’s Social and Emotional Development
Question 2e (n=35, 25.7% disagreement): Staff receive training on sensory awareness and integration.

25. Self Assessment Results Continued
Opportunities Continued
Strategy 6: Recognize and Respond to Early Warning Signs of Child Abuse or Neglect
Question 1b (n=30, 33.3% disagreement): When parents enter the program they are informed of what constitutes abuse and neglect within the state.
Question 1c (n=31, 25.8% disagreement): When parents enter the program they are informed of the program’s protocols regarding child abuse and neglect.
Question 5 (n=34, 35.3% disagreement): All staff are trained on the impact of loss and trauma on children and how to respond appropriately.

26. Self Assessment Results Continued
Opportunities Continued
Strategy 7: Value and Support Parents
Question 19d (n=32, 43.7% disagreement): The program provides parents opportunities for input into staff hiring and training.

27. Parent Satisfaction Comments
What helped you the most in this program?
“The ability to ask for advice and also just someone who understands you and your struggles in life and helps you by explaining their own life experience.”
“That they learn new things. It helps me to better communicate with my children. We do new activities and my children are more attentive. I like it a lot.”
“That the teacher encouraged my daughter to eat better, to develop her mind. Learning new things that would help her in her development.”
“The additional resources and extra support (moral) I received from the program greatly reduced my stress.”
“Encouragement from others, support systems when in need, and positive reinforcement when making decisions!”

28. Participants SurveyedFY
1,528 families served from First 5ERS. ü
393 Surveys Completed to Measures Results. ü

29. Participants’ Time in the Program
Time Spent in Program
Percent: (N=114)
Percent: (N=231)
Percent: (N=250)
Percent: (N=363)
Less than 1 Month
0.9%
0.4%
22.8%
32.5%
Between 1 and 3 Months
26.3%
19.9%
15.6%
16.8%
Between 3 and 6 Months
11.4%
8.2%
13.6%
14.3%
Between 6 and 12 Months
15.8%
32.0%
23.6%
12.4%
Between 1 and 2 Years
39.5%
28.1%
13.2%
Over 2 Years
6.1%
11.3%
11.2%
7.2%

30. Survey Completion Method

31. Language PFS Surveys were Administered

32. Participant Gender

33. Ages of Children ServedQ1 Q2 Q3 Q4
Year Total
Ages 0 through 3
245
199
160
140
744
Ages 3 through 5
198
159
103
132
592
Age Unknown (ages 0 through 5)
180 3
183
Total Ages 0 through 5 (including ages unknown)
623
358
266
272
1,519

34. Child Protective Services Involvement

35. Family Income

36. Education Attainment

37. ASQ: Children Screened

38. ASQ: Developmental Concerns

39. ASQ: Developmental Concerns Continued
Most identified concerns
Medical problems
Concerns about behaviors
Talks like other kids his/her age

40. Protective Factors: Agreement
Today
Before

41. Protective Factors

42. Protective Factors

43. Protective Factors

44. Protective Factors

45. Protective Factors

46. Parent/Couples Cafés: Participants Served
First 5ERS
Participants Surveyed
A total of 14 surveys, in , were Couples Café Participants

47. Parent Cafés: Positive Changes
*All respondents who answered this question were part of Valley Oak Children’s Services’ Parent Cafés.

48. Parent/Couples Cafés: Satisfaction

49. Satisfaction Questions
Change was statistically significant for each question.
A t-test confirms that the change on Questions 1-7 was “significant at a p<.01 level.”

50. Ability to Meet Family Needs with Available Resources
“I know how to meet my family’s needs with the money and resources I have.”

51. Participant Choices for Reducing Family Stress
“I make choices about family schedules and activities that reduce family stress”

52. Respect from Staff

53. Suggestions for Improvement
Services
A desire for more home visiting, more frequent sessions, longer sessions and sessions conducted in the language of origin (Hmong) were all suggestions made for program improvement. As had been the case in previous years, participants also asked to not be limited to participation in TAPP to age 19 but to extend the age to 21.
Activities
Parents asked for more activities and events, noting how much they enjoy those that are offered.

54. Suggestions for Improvement Continued
A significant majority of participants indicated that they value and are satisfied with Network services.
Those participants that were part of SF Network programs that will not be funded in the future indicated their sadness and sense of loss that the programming would end.
Survey results across the four years of the evaluation have consistently demonstrated the impact that the funded programs have had on families, building the Protective Factors in those they have served.

55. Suggestions for Improvement Continued
As the Initiative continues, it is clear the foundation has been laid for continued positive results.
The challenge of the future is more systemic in nature.
At the systems level, the opportunity exists to engage other service providers and leaders in implementing the Strengthening Families Framework.
For families, the ongoing challenge has been to attain the resources they need to move out of crisis toward stability.
Exploring strategies that have been proven to help families achieve financial stability may be of value in addressing this challenge.

56. Recommendations
Tremendous gains have been made in implementing the Strengthening Families Framework and Initiative. For , the following recommendations could strengthen the Initiative, moving it to more of a countywide effort.
Strengthen Parent Partnerships: Continue to support existing and evaluate ways to develop new parent directed partnerships, including engaging parents in staff hiring and training activities
Support Fatherhood Programs: Continue to support father-friendly practices, father-centered services, and emerging fatherhood programs and coalitions.
Support County-wide Implementation: Evaluate and if feasible, convene a leadership group within the county to focus on policies and systems that promote Strengthening Families, potentially through the Systems Strengthening Initiative.
Engage Network Partners: Engage Network partners to train their peers on issues identified in the program self-assessments.

57. Commissioner Pause for Thought…