Evaluation Basics Defining Evaluation Evaluation is an important strategy in achieving program quality and impact Evaluation means weighing the value of… actions that go into youth programs (processes) results of youth programs (outcomes)

Evaluation Basics Putting Evaluation in Perspective Evaluation brings together many streams or processes and is often most visible at the end of a program. But the truth is that evaluation, understanding the value of a program… Begins before a program ever begins Speaks to youth as much as to evaluators Relates to marketing and recruitment as much as to accountability and program improvement

Evaluation Basics Processes in Perspective Processes related to evaluation include… Evaluation logic, describing valued outcomes and strategies to reach them Best practices and conditions for quality programs Program development and organizational mission, framing program priorities Program evaluation process from planning through reporting results That’s a lot of ideas, but we will consider them one at a time!

Evaluation Basics Processes of Evaluation Evaluation answers questions about… actions, processes, or quality of programs Is the program running as designed? What elements make the most difference? results, outcomes, or impacts of programs How much difference did the program make? What audience benefits the most?

Evaluation Basics Evaluation Logic …and that quality and impact doesn’t happen by chance… Positive outcomes result from quality programs

Evaluation Basics Evaluation Logic For example, when a 4-H robotics club sets individual and club goals, Club leaders and youth ask… What do we need to know or do? (the impact question) What’s the best way to do it? (the quality question)

Evaluation Basics Evaluation Logic A 4-H robotics club might value outcomes related to… science and technology teamwork, goal-setting, problem solving serving their community

Evaluation Basics Processes of Evaluation Science and technology outcomes involved… building a robot programming a robot to perform Teamwork outcomes involved… working together in different roles Research outcomes included… investigating a topic developing a creative presentation using learning to serve the community

Evaluation Basics Processes of Evaluation Outcomes, in the short term, describe results of a program, including changes in… Knowledge—describe or explain an idea, object Attitudes—thinking or feeling about events Skills—abilities to do something Aspirations--plans to do something

Evaluation Basics Processes of Evaluation Outcomes beyond the short-term describe results of a program, including changes in… Applications—using knowledge or attitudes in new situations, to learn more, or teach others Practices—using skills, often with new attitudes, to perform at a higher level

Evaluation Basics Processes of Evaluation Outcomes for the long-term describe hoped-for changes in the quality of a community life… Social—relationships within the community Economic—work and business life Environmental—physical and natural setting Cultural—shared life and culture

Evaluation Basics Evaluation Logic If they put the resources and strategies, practices and conditions, short and long-term outcomes in order…. …they would create a Logic Model, or description of the logical sequence of actions and results… InputsOutputsShort-term Outcomes Mid-term Outcomes Long-term Outcomes Robot kit Skilled leader Shop to work Training Practice Skills in building, programming Problem solving Critical thinking Sci-tech savvy workers

Evaluation Basics Best Practices and Conditions The value of the club’s work is not just in the results, but in the process, or how they work, learn, and play together, including… hands-on learning through projects and trips youth leadership with adult support recreation and fun

Evaluation Basics Best Practices and Conditions Process Evaluation looks at best practices that define quality in youth programs such as… Hands-on learning through projects and trips -Ex: When is hands-on learning most effective? -Ex: What was learned/used from a field trip? Youth leadership with adult support -Ex: Did young leaders have adequate support? -Ex: How did leadership roles enhance performance in a competition?

Evaluation Basics Best Practices and Conditions Process Evaluation looks at best practices that define quality in youth programs such as… Teamwork in learning and working together -Ex: How do members fulfill/mesh roles? -Ex: (How) does the team work as one unit? Creative Presentation -Ex: Is delivery audible, clear, rehearsed? -Ex: Does team show enthusiasm, creativity?

Evaluation Basics Best Practices and Conditions Process Evaluation also looks at best practice conditions that promote quality programs, such as… Safety, physical and emotional Support from peers and adults Skill building opportunities Self-confidence-building Social norms promoting responsibility and service Social inclusion Structure balancing discipline and independence Synergy with family and community influences

Evaluation Basics Processes of Evaluation …more processes to follow…