EDEL8008 Module 1 Unit 1 Podcast 1 Evaluation Foundations

What is Programme Evaluation? The systemic study of something to determine its quality or value (Scriven, 1991). Used to make decisions on the programme

What can be evaluated? An evaluation can be done on: Policies Products A project, programme, or organisation Policies Products Processes or services Performing an evaluation involves examining processes as well as outcomes

What is a programme? A set of temporary activities that are conducted to reach an agreed upon objective, goal or benchmark (Encyclopedia of Evaluation) Sometimes the term programme is used interchangeably with project Examples: After-school programs Mentor program Technology program School building behavior program

Program Evaluation vs. Research Judges the worth or merit of a program Used to make decisions and recommend improvements Changes practice faster than research Research: Expands our general knowledge base Changes practice more slowly

The Evaluation Cast Project Director/Manager Project Staff Partners Provider Stakeholders Internal Evaluator External Evaluator

Project Director/Manager The leader He/she is in charge of overseeing the program Responsible for programme implementation and future success

Project Staff These are people whose main responsibility are to oversee the day-to-day aspects of program delivery. They work closely with the project director/manager

Providers A term used for those agencies, units, community-based organizations, schools, museums, etc. that work to provide services and/or activities in order to create the program

Stakeholders Individuals or groups who participate in the program and/or receive services For example: At risk students Teenage mothers Autistic students Teachers in a leadership programme

Internal Evaluator One who is currently employed with or has an affiliation with the agency or unit which is overseeing the implementation of the program Advantages Already trusted by stakeholders Familiar with the programme Disadvantages May not be as objective

External Evaluator One who has no affiliation with the agency or unit implementing the program Contracted to conduct the evaluation Advantages – may be more objective Disadvantages More difficulty gaining trust and access to data Stakeholders often see the evaluator as a threat

Data Collection: Documenting Activities: What happened? Documenting programme implementation: How did it work? Strengths and Barriers Documenting Activity Outputs: What changed? Documenting end outcomes: How are the participants different?

Formative vs. Summative Data Formative data is often provided along the way to help make current programme modifications Example Improving the hiring process as hiring is taking place Feedback from any training improves the next training session Weekly observations of program execution

Summative Data Often collected at the end of the project Often includes outcomes measured by more standardized assessments Example: Increase in student performance on regional mathematics examination Increase in student graduation rate

Data Triangulation Triangulation involves using multiple data sources in an investigation to produce understanding. Data triangulation reduces the risk that an evaluator will come to incorrect conclusions

Evaluation Reporting Should meet needs of the target audience Should be useful to the target audience Often includes Executive summary Summary of evaluation focus Evaluation plan Findings Conclusion and recommendations

Evaluation Foundations: Summary Defined evaluation Identified what can be evaluated Compared evaluation with research Identified who is involved in an evaluation Compared internal and external evaluators Identified types of data that can be collected Compared summative and formative data collection for an evaluation Described data triangulation Provided tips on reporting