PPA 502 – Program Evaluation Lecture 8 – Managing Evaluation Projects Step by Step

Introduction  Definitions. –Evaluation project management: A process of considerations, decisions, and activities engaged in by the leadership of an evaluation project to facilitate its conduct. –Project director: evaluation management leader who is directly accountable to the project sponsor for successful completion of the evaluation. –Evaluation staff or evaluators: the individuals who conduct the evaluation, whose work is facilitated by evaluation leadership.

Introduction  Definitions. –Sponsor: the organization paying for the evaluation. –Client: the user of the evaluation products. –Stakeholders: multiple audiences who have an involvement or concern for the substance of the evaluation.  Core process. –Foster insight and creativity. –Apply focus and discipline to accomplish mandate on schedule and within resources.

Introduction  Areas essential to effective project management. –Clarifying the evaluation mandate. –Staffing and organizing for success. –Making assignments productive. –Monitoring interim progress. –Ensuring product quality and usefulness.

Evaluation Management Areas and Aims Area of Evaluation ManagementAim Clarifying the Evaluation Mandate Complete agreement between sponsor and evaluation management about the purpose, scope, resources, method, workplan, and schedule. Staffing and Organizing for Results Demonstration by staff, through past efforts, of the mix of qualifications needed to conduct the evaluation. Staff are organized into a structure that maximizes each member’s contribution and ensures control of evaluation expenditures and schedule. Making Assignments Productive Assignments that are product oriented, well defined, and agreed on by evaluation managers and the staff who undertake the effort. The sum of all assignments equals completion of all evaluation products. Each assignment is appropriate to the capabilities of the individuals/ groups undertaking it. Monitoring Interim progress Project monitoring information that accurately portrays and links technical progress on evaluation products, expenditures, and schedule status; a monitoring process that contributes to and sustains project momentum. Ensuring Product Quality and Usefulness Evaluation products given high ratings for usefulness. The evaluation products are used by policy and program management decision makers to accomplish the purpose of the evaluation.

Clarifying the Evaluation Mandate  Gain agreement on the evaluation mandate before or very early in the evaluation.  Check this agreement periodically during contacts with evaluation clients and sponsors.  Minimize the cost of maintaining agreement by integrating checks on the mandate with technical evaluation activities, such as client review of a draft data collection and analysis protocol.  Beware of tacit agreements.

Staffing and Organizing for Results  Use a staffing matrix and observations about interpersonal communication skills to facilitate initial selection of staff members.  Organize evaluation staff members into teams based on project tasks, evaluation methodology, or subject matter responsibilities.  In evaluations with two or more teams, form a synthesis group to foster coherent effort across the technical teams.

Making Assignments Productive  Be very clear about each assignment’s product, outcome, or end point as well as resources and expected completion date.  If possible, use well-chosen examples from similar projects to illustrate expected products.  Formalize assignment agreements.

Monitoring Interim Progress  Don’t confuse expected activity and expenditure levels with commensurate technical progress.  Time monitoring episodes to complement and not impede product development.  Ensure that project management staff are effective monitors.

Assuring Product Quality and Usefulness  Involve the sponsor, program, and other representatives of the evaluation audience in a four-step report development process.  Start early in the project with an outline, and then use a briefing to gain agreement about the content of the report.  Solidify the agreement on report content through follow-up briefings on draft products; Reserve polishing and editing resources until the technical content is finalized.