“If you don’t know where you are going, how are you gonna’ know when you get there?” Yogi Berra Common Elements of Strong Program Evaluation THEORY OF CHANGE : Cause/effect relationship of program LOGIC MODEL: Identifies key program components and their relationship to program outcomes PERFORMANCE MEASUREMENT & IMPACT EVALUATION: Differences between the two

If you deliver the intervention as planned, it will bring about a measurable change/outcome in the community/participants in relation to your identified community need. Community Need/Problem Statistics Specific Intervention Supports Cause/Effect Intended Outcome Theory of Change Main Elements

LOGIC MODEL INPUTS: What we invest ACTIVITIES: What we do OUTPUTS: Direct products from program activities EVIDENCE OF CHANGE/OUTCOMES Short-Term Outcomes – one year  Changes in knowledge, skills, attitudes Medium -Term Outcomes – three years  Changes in behavior or action resulting from new knowledge Long-Term Outcomes – ten years  Meaningful changes in condition

Birth & Beyond Evaluation Practices A home visitation program at 9 Family Resource Centers serving parents at risk of child abuse and neglect to prevent entry/re-entry into Child Protective Services  Common Outcomes  Sacramento County Commitment to CPS Evaluation  External Evaluator from the onset  Shared Database across all 9 sites  Monthly Data Review distributed program wide  Quarterly and Annual Reports  Program Improvement Based on Data & Outcomes

Birth & Beyond Lessons Learned  Integrated evidence-based practices into the model  Intervention must align with assessments  Analyze attrition of those served and ways to improve  Member training aligns with intervention  Since 2008, serving higher % of at-risk risk families  Data facilitates peer to peer learning  Track/monitor short-term outcomes to ensure meeting medium- term outcomes  Expanded services by adding one FRC (based on data)

Birth & Beyond Evaluation Challenge Impact Evaluation: strives for high rigor with scientifically-based research design comparing people exposed to an intervention (“experimental” group) to people not exposed (“control” or “comparison” group) How do you design an impact evaluation that does not deny services to parents at risk of child abuse & neglect?  Technical assistance from NORC  Work in partnership with Sacramento County Child Protective Services to en hance B&B evaluation conducting a quasi-experimental study  Select a matched sample of families who have been referred to CPS during the same time as Birth & Beyond families and did NOT receive Birth & Beyond services (“control” or “comparison” group)

Child Protective Services Pre-, During, and Post-Program

20 Year Child Death Review Team Report

PERFORMANCE MEASUREMENT & IMPACT EVALUATION Performance Measurement Provides a view of how program is functioning Does not “prove” that an intervention caused an outcome Strives for high quality data, striking a balance between rigor & staff time Focuses on shorter term changes, observed within a year Impact Evaluation Strives for high rigor; with scientifically-based research designs Seeks to prove causal relationship; program is the specific cause of improvements within the target population Evidence that intervention causes the outcomes Focuses on longer term changes as well as short term outcomes