TYPES OF EVALUATION Types of evaluations ask different questions and focus on different purposes. This list is meant to be illustrative rather than exhaustive. Who does the evaluation? External evaluation- The evaluation is conducted by people outside the program in an effort to increase objectivity. Internal evaluation -Program staff conduct the evaluation. GENERAL CATEGORIES Process evaluationHow can the program be improved? To what extent have program goals been attained? Formative evaluations often rely mainly on qualitative research. Summative evaluationsShould the program be continued? If so, at what level? Summative evaluations often rely mainly on quantitative research
Components of all evaluations. (These are done before and after all evaluations) Evaluability assessment - What is the feasibility of various evaluation approaches and methods? This is done before every evaluation. Here you want to know, “Can this program actually be evaluated?” Meta-evaluationWas the evaluation well-done? Is it worth using? This should be done after every evaluation.. this is determined by the stake-holders Special category Needs assessmentWhat do clients need and how can those needs be met? This is often done before a program is started. It is sometimes done as part of a formative eval.And sometimes as part of a summative eval.
Types of Summative evaluation Outcomes evaluation- To what extent are desired client outcomes being attained? What are the effects of the program on clients? Summative and outcomes are very similar. Accreditation evaluation- `D oes the program meet minimum standards for accreditation or licensing? Cost/benefit analysis What is the relationship between program costs and program outcomes (benefits) expressed in dollars? Cost-effectiveness eval. What is the relationship between program costs and outcomes (where outcomes are not measured in dollars)? Criterion-referenced eval.-To what extent has a specific objective been attained at the desired level of attainment (the criterion)? Descriptive evaluation - What happens in the program? (No “why” questions or cause/effect analyses.). this is often a part of both formative and summative evaluations. Effectiveness evaluation- To what extent is the program effective in attaining its goals? Extensive evaluation- To what extent is this program able to deal with the total problem? How does the present level of services compare to the needed level of services? Goals-based eval.What are the actual effects of the program on clients (without regard to what staff say they want to accomplish)? Impact evaluation-What are the direct and indirect program effects? Longitudinal eval. - What happens to the program and to participants over time? Norn-referenced evaluation - H ow does this program population compare to some specific norm or reference group on selected variables? Comparisons are made with other programs, client groups or Social indicators. What routine social and economic data should be monitored to assess the impacts of this program?
Performance evaluationWhat are participants actually able to do as a result of participation in the program? Personnel evaluationHow effective are staff in carrying out their assigned tasks and in accomplishing their goals? Product evaluationWhat are the costs, benefits, and market for a specific product? Quality assuranceAre minimum and accepted standards of care being routinely and systematically provided to patients and clients? How can quality of care be monitored and demonstrated?
Process evaluations Process evaluation-What are the strengths and weaknesses of day-to-day operations? How can these processes be improved? Formative and process are often used interchangeably, although process evaluations are really a sub-group of formative. In process evaluations, you look at and describe the linkages and components of a program, but do not focus on outcomes. You look at how the linkages, procedures and processes fit together and what can be done to improve them. Formative eval. Occurs at the beginning of a program. Looks at how it is planned, whether the components are in place, congruent and functional. Decision-focused evalWhat information is needed to make a specific decision at a precise point in time? Descriptive evaluationWhat happens in the program? (No “why” questions or cause/effect analyses.) Efficiency evaluationCan inputs be reduced and still obtain the same level of output or can greater output be obtained with no increase in inputs? Effort evalauationWhat are the inputs into the program in terms of number of personnel, staff/client ratios, and other descriptors of levels of activity and effort in the program? A FORM OF DESCRIPTIVE EVAL. Longitudinal eval. What happens to the program and to participants over time? Quality assuranceAre minimum and accepted standards of care being routinely and systematically provided to patients and clients? How can quality of care be monitored and demonstrated? Linkage mapping and program flow – a sort of detailed diagram of the program linkages – among actiities,s ervices and clients.
OFTEN EVALUATIONS COMBINE FORMATIVE AND SUMMATIVE!!!!!!!! Utilization-focused evaluationWhat information is needed and wanted by decision makers, information users, and stakeholders that will actually be used for program improvement and to make decisions about the program? (Utilization-focused evaluation can include any of the other types BELOW AND IS USUALLY A COMBINATION OF PROCESS AND SUMMATIVE.) THE METHOD OF THE EVALUATION IS DETERMINED BY ITS PURPOSE AND THE QUESTIONS THE EVALUATION IS SUPPOSED TO ANSWER