Purposes of Evaluation Why evaluate? Accountability: Justify trust of stakeholders (funders, parents, citizens) by showing program results (Summative) Program Improvement: Demonstrate how things are working and where it needs to be improved (Formative)

Purposes of Evaluation Why evaluate? Strengthening Relationships: Evaluators/Leaders better understand audience by planning, listening to needs, experiences, outcomes; Youth better understand evaluators/ leaders/program by building trust and understanding Building Capacity: Program organization, leaders, and participants gain skills and insights by reflecting on program quality and impact

Purposes of Evaluation Why evaluate? Innovation: Program leaders and participants generate new ideas about or beyond the program through reflecting on it

Purposes of Evaluation When programs begin by… Strengthening Relationships…building trust and understanding…they initiate a listening process for… Improving Programs…learning from what we experience in programs…that leads to organizations and individuals… Building Capacity…growing deeper skills and broader perspectives needed for higher quality programs… Demonstrating Accountability (Outcomes)…achieving goals targeted by program designs, expected by stakeholders, and… Spawning Innovation…inspiring leaders and participants to go beyond current programs and results and create new opportunities

Planning Logic Begin with the end in mind… Inputs: curriculum, materials, equipment, expertise Activities: events, experiences Outputs: completed missions, projects, tools Outcomes: results of programming –Short-term: Knowledge-Attitudes-Skills-Attitudes –Mid-term: Practices, Behavior (Performance criteria) –Long-term: Educational goals and achievements; vocational and community involvement goals and achievements

Program Evaluation Steps Following through on scanning and planning… (Environmental Scanning and Needs Assessment) Planning: understanding the process and setting priorities for supporting and conducting short- and long-term evaluations Focusing: identifying purposes and types of evaluations and general questions or outcome indicators for a given program

Program Evaluation Steps Following through on planning… Designing: organizing the approaches and developing specific questions and indicators for a given program Selecting Methods: choosing appropriate and adequate tools and procedures to answer evaluation questions

Program Evaluation Steps Following through on planning… (Program Implementation) Collecting and Maintaining Data: systematically gathering, organizing, and securely storing data Analyzing and Interpreting: testing data and explaining results based on target questions to be answered Communicating Results: reporting and interpreting program quality and outcomes to a variety of interested people

Review on Evaluation Skills Evaluation Purposes –Product and process –People and Program Planning Logic –Where to begin…what to plan for –Steps to prepare…how to build quality Evaluation Steps –Sequencing and synchronizing –Intentional, not accidental

Key Qs on Evaluation Skills Training (input on E-Basics) What key concepts or skills are important for understanding and implementing evaluations of informal science programs? What strategies are most effective for teaching evaluation skills (and how do they sequence together)?

Key Qs on Evaluation Skills Training (input on E-Basics) What is the relationship between programming and evaluation and how should our learning of evaluation skills reflect that connection? What can we expect to accomplish in this webinar (relative to E-Basics) and what can we expect to accomplish in our state teams?

Key Qs on the Community of Practice How is this learning circle influencing your capacity as an individual and as a state team? What are individuals or your state teams doing between webinar meetings that strengthen your state science initiative? What resources or activities of the learning circle are/could be helpful in building and implementing your state plan?

Key Qs on the 4-H Sci E-LC Webinar (Conveners) How do you understand the purposes and process of the webinar after this session? Does the webinar focus fit with your state team’s needs and readiness? What actions do you recommend for the next session? How can each segment be improved? What do you plan to do with your state team before the next session?