Evaluation Research, aka Program Evaluation

Definitions Program Evaluation is not a “method” but an example of applied social research. From Rossi and Freeman: “Program Evaluation is the systematic application of scientific methods to assess the design, implementation, improvement or outcomes of a program.” Put more succinctly, program evaluation (or “evaluation research”) determines if programs “work.”

Dimensions of Program Evaluation Process Evaluation: Is the (ongoing) program being conducted as was intended? Outcome Evaluation: Did the (completed) program have the effect that was intended?

Stages of Program Evaluation Needs Assessment: Assesses nature of problem and what has to be “solved.” Program Theory: Formal description of the program, ie, how it addresses needs. Process Analysis: Evaluates if program theory is being implemented properly, eg, staffing and clientele outreach assessments. Impact Analysis: Examines effects of programs via causal analysis of some sort. Cost-Benefit Analysis: Determines economic efficiency of the program.

“Impact Analysis” and the Role of Experimentation Program Evaluation can in principle partake of any research method… However, the logic of program evaluation and its perception of “programs” as having causal effects on outcomes calls for an experimental orientation This need is evident even if the “experiment” isn’t laboratory based and even where survey or field- study data are collected, because for PE controlled comparisons are important.

Some examples of Program Evaluation Criminal justice (especially juvenile delinquency) programs Public health interventions Education, eg the U of C’s podcasting experiment