Focusing Evaluation Getting ready

Focusing: Making Ready Stages of Evaluation Evaluability –Stakeholders –Standards Target Indicators –Outcome indicators –Process indicators Evaluation Management

Focusing: Making Ready for Process Stages of Evaluation: What’s ahead? –Planning process –Focusing: Identifying context, questions, targets –Designing study methods –Selecting study tools –Collecting data –Interpreting data –Using results

Focusing our own process Stages of Evaluation: Where are you? How often do you thinking about what to evaluate after the program is over? a) Pretty typical b) Somewhat typical c) Not typical d) Never wait until after e) What evaluation?

Focusing: Making Ready for People Stakeholders: Interests and Readiness –Sponsors (STEM careers; mentors from field) –Teachers (STEM achievement; collaboration) –Parents (STEM achievement; volunteerism) –Youth (STEM hands-on; participation) Not whether it’s evaluation-ready, but what evaluation it’s ready for.

Focusing: Making Ready for Program Standards: Investments and Readiness –Utility (credibility, relevance, collaboration) –Feasibility (management, viability) –Propriety (principled, facilitative) –Accuracy (sound and scientific) Not whether it’s evaluation-ready, but what evaluation it’s ready for.

Focusing: Ready Questions Evaluation Question: What do we want to know about a program? –Outcome questions: What does it do? –Process questions: How does it work?

Focusing: Making Ready Indicators Target Indicators How do we know a program is worth its salt? –Outcomes: KASA (Short-term); BA (Long-term) –Processes: Program Quality, Fidelity

Focusing: Making Ready Quality Target Indicators: Quality First –Program (Delivery) Processes In Preparation: Credibility, Consistency In Process: Engagement, Guided Inquiry

Focusing: Making Ready Outcomes Target Indicators: Now & Later –Program Outcomes for Individuals In Preparation: Credibility, Consistency In Process: Content (subject matter concepts); and Process (inquiry, creativity, problem solving) In Prospect: Current activities that contribute to future outcomes

Focusing our own goals Evaluation Targets: Where are you? How often do you check program quality or fidelity before collecting data on outcomes? a) Pretty typical b) Somewhat typical c) Not typical d) No need to check e) Fidelity to what?

Focusing: Making Ready Resources E valuation Management –Timeline –Budget –Human Capital/Deployment –Facilities/Optimization