Kathleen J. Phillips, Ed.D. Education Consultant 307 Moultrie St. San Francisco, CA (415) (T) Making Evaluation Work for You A Case Study of St. John’s Educational Threshold Center

St. John’s Educational Threshold Center (SJETC) After-school programs in academics, leadership, health and wellness, sports and recreation, and the arts Serves 1100 youth, ages 5-19 Low-income, mostly Latino, Asian American and African American population Serves the local elementary school and middle school and neighborhood kids Many stay involved for 5-8 years

SJETC Evaluation Strategy Criteria Enhance agency, staff and youth control over the evaluation process Provide comprehensive information Realistic Reflect agency values and mission Provide useful information for program planning and development Demonstrate program impact

SJETC Evaluation Strategies Focus groups and surveys collecting program- specific data Youth-led evaluation of specific aspects of SJETC programming Longitudinal survey data on all youth across all programs Review of relevant data gathered by public and private funders Review of relevant data collected by other community groups and researchers

Using the Evaluation Data 1. Assessing and understanding the impact of programs 2. Informing program planning 3. Supporting fundraising and public relations 4. Empowering everyone involved

1. Assessing and understanding the impact of programs Consistent documentation—not just stories Enables assessment of impact over time Helps us know what’s working and what’s not Provides a TRUE youth perspective

2. Informing program planning Informs mid-year program adjustments Helps identify and clarify needs Supports planning with collaborators Enriches agency-wide strategic planning

3. Supporting fundraising and public relations Allows us to tell our story on our terms Justifies our community and program needs Provides information and materials for general education and outreach

4. Empowering everyone involved Agency: Strengthened mission, own sense of legitimacy, and funders’ respect Youth: Took on a respected, powerful, and real role Staff: Gained understanding of, control over and appreciation for evaluation

Making Evaluation Work and Become a Source of Power Who needs to be involved? What needs to be in place?

Who needs to be involved? Organizational leader(s) All levels of the organization: youth, staff, administrators Someone who really understands evaluation— staff-members and/or consultants

What needs to be in place? An overall strategy, including plans for using evaluation findings in meaningful ways Dedicated funding and staff time A culture that supports youth empowerment (if incorporating youth-led evaluation) Skill in youth leadership training (if incorporating youth-led evaluation) Stipends and food whenever possible