A plain English Description of the components of evaluation Craig Love, Ph.D. Westat
An evaluation is a systematic way of determining whether the project activities are consistent with what is intended and is achieving the end results that was determined was needed to be accomplished.
Just share common methods and values Research intended to create generalizations, to create new knowledge Evaluation is intended to provide practitioners, funders, community members and other stakeholders information as to whether the intent of the program was met. Evaluation is amenable to community participation- ergo fits most Native American cultures.
Formative evaluation is a systematic collection of data that, in turn, is used to assess progress toward goals and to verify that the project is implemented as intended Used to adjust the program and is ongoing
Summative evaluation is an assessment of how well the program was implemented and how well it achieved its intended outcomes Usually completed at or near the end of the project For example, it can be used to determine whether the project is worthy of continuing after funding
Documentation of the initiating and implementation of the project. Included in both formative and summative evaluations Important to explanation of what is associated with the outcomes
Assessment of how well the outcome was achieved. Addresses questions- Did you meet your goals? Did you get what you wanted out of the project? Most effective when associated with process
An important aspect of the program planning and evaluation process Usually based on a survey of consumers, practitioners and other stakeholders Provides information on deficits- barriers to the desired level of performance. Is a good way to establish how to design the program to meet the identified needs in the community to be served.
We have determined that there is a need We have the following resources We are aware of the following barriers to be overcome We have a program designed to overcome barriers and/or create outputs that will help produce the outcomes needed
The program design begins with the needs/goals of the project The interventions (e.g. rewrite math curriculum) are intended to produce outputs (e.g. more interest on the part of students) that create the desired outcomes (e.g. improved math skills) and more Native American students pursue science and math careers (long- term outcome).
Identified problem- students in our college do not do well in math achievement tests (Need identified in achievement tests) Want to improve achievement scores by 20% within the next two years (goal- intended outcome) Barrier, the students are using a curriculum designed for big cities and white culture (Needs assessment by survey of students, review of text materials)
Intervention- modify curriculum so that the examples, issues etc. are appropriate for our population Expect to gain better response to students, help make the curriculum more relevant to students (output) Higher math achievement scores will be achieved (outcome) the students are better able to relate to the math text.
Logic model- Need to improve math scores- the curriculum found to be irrelevant to students- adjust curriculum to make it more relevant to the students- increase student attention and interest in math curriculum- increase scores on math achievement test Formative evaluation- review of revised text to determine wither the new text is indeed more relevant. Also determine whether students are more attentive and teachers approve of the new curriculum, identify unintended barriers
Process evaluation- identify whether the new curriculum is implemented, identify issues in implementation, creation of the modified curriculum, accepted by teachers, students seem more interested? Note attendance records, teacher comments, observe classes. Outcome evaluation- assess student perspective on new math curriculum, monitor math grades, assess standard scores before and after implementation of new curriculum
Questions or comments?