1. Evaluation Plan
Akm Alamgir, PhD
June 30, 2017

2. Integrated Planning and Evaluation Framework
What is the program/ intervention?
Who do the program serve to?
Are the stakeholders involved with the process?
Does the evaluation procedure based on theories and models?
Do the (SMART) objectives meet stakeholders expectations?
What aspect of the program will be measured, and to what extent?
Is the evaluation process embedded into the program planning?
What are the tools to measure the change?
Are the tools able to measure what is supposed to measure?
How is the quality of the data management?
What changes the program could make?
Could the program make the changes designed for?
What could be done different to have better outcome?

3. Management Evaluation Team
Who will manage and implement this evaluation?
What evaluation skills or approaches are needed to successfully conduct this evaluation?
Have you identified an external reviewer to provide feedback on the evaluation plan?
Individual
Title or Role
Responsibilities

4. Evaluation Plan
An evaluation plan sets out the proposed details of an evaluation - what will be evaluated, how and when.
The evaluation plan should include information about
what the evaluation is trying to do
what is to be evaluated,
the purposes of the evaluation
key evaluation questions
how it will be done
what data will be collected,
how and when data will be collected
how data will be analyzed
how and when results will be reported.

5. Evaluation Plan Steps • Writing a project description
• Stating an evaluation purpose
• Identifying evaluation stakeholders
• Choosing evaluation questions
• Selecting evaluation types
• Choosing evaluation tools
• Identifying evaluation sources

6. Writing a project description
Need: What need is your program designed to meet?
Context: What is the program’s context? That is, what contextual or cultural factors may affect its implementation or effectiveness?
Population Addressed: Who is included in the population for whom activities are intended?
Stage of Development: How long has the program been in place? Is it in the planning or implementation stage?
OR LOGIC MODEL
Resources/Inputs
Activities
Outputs
Outcomes
Initial
Subsequent
Short-Term/Intermediate
Long-Term

7. Stakeholder Assessment and Engagement Plan
Stakeholder Name
Stakeholder Category
Interest or Perspective
Role in the Evaluation
{May be an individual or a group}
{primary, secondary, tertiary}
{program participant, staff, etc.}
{planning team, external reviewer, etc.}

8. Description/Logic Model of {Insert Name of What You Are Evaluating}
Resources/Inputs
Activities
Outputs
Outcomes
Initial
Subsequent
Short-Term/Intermediate
Long-Term

9. Program Evaluation Dimensions
Reach and Relevance
Effectiveness
Efficiency
Impact and Sustainability
How important is the relevance or significance of the project regarding local and national requirements and priorities?
Is the Project responding to an identified community need?
Alignment between the stated aims of the project and the actual results.
Is the Project achieving its intended goals and objectives?
How big is the effectiveness or impact of the project compared to the objectives planned (Comparison: result – planning)?
How big is the efficiency or utilisation ratio of the resources used (Comparison: resources applied – results)?
How the resource of the project used to achieve the desired results.?
Long term ability and operational capacity of the project to continue delivering against its goals.
Are the positive effects or impacts sustainable/ scalable? How is the
sustainability or permanence of the intervention and its effects to
be assessed?
Does the development intervention contribute to reaching higher level development objectives (preferably, overall objective)?
Are these outcomes transferable to our agencies in the sector?

10. Evaluation Plan Template
Objectives
Key Evaluation Question
Questions
What questions do you need to ask to determine that your objectives have been met?
Information Required
What information do you require to answer each of these questions?
Data Source
How will you collect this information?
Better and more timely planning and delivery of community services and infrastructure in community X through partnership model.
Effectiveness
Sustainable & Scalable
What was the model?
Who was involved?
Was the diversity of the community represented?
What was delivered?
What was delivered differently because of the model?
Has capacity been built to make the model sustainable?
What lessons were learned?
Could this model be generalised to other areas?
Description of the model
List of partners over time
Significant organisations not involved in the partnership
Audit of services/ infrastructure delivered
Partners assessment of what happened differently because of the model
Partners assessment of the capacity that has been built and sustainability
Partners assessment of lessons
High level strategic opinion about the applicability of the findings to other areas
Project application
Bi-monthly reports
Stakeholder interviews
Achievement audit
Workshop on evaluation results
Increased level of community participation in general activities and governance.
Reach
Are more people from a range of backgrounds involved in general activities?
Are more people from a range of backgrounds involved in governance?
Number of participants - Age groups, CALD, indigenous etc
Enrolment forms
Pre and post population indicators
Achievement tool

11. Key Evaluation Question (KEQs)
What three to five major questions do you intend to answer through this evaluation?
Do the questions align with the Good Evaluation Questions Checklist?
Key Evaluation Questions should be developed by
considering  the type of evaluation being done,
its intended users,
its intended uses (purposes), and
the evaluative criteria being used.

12. Evaluation Question (Process)
How is the program being implemented?
How appropriate are the processes compared with quality standards?
Is the program being implemented correctly?
Are participants being reached as intended?
How satisfied are program clients? For which clients?
What has been done in an innovative way?

13. Evaluation Question (Outcome)
How well did the program work?
Did the program produce or contribute to the intended outcomes in the short, medium and long term?
For whom, in what ways and in what circumstances? What unintended outcomes (positive and negative) were produced?
To what extent can changes be attributed to the program?
What were the particular features of the program and context that made a difference?
What was the influence of other factors?

14. Evaluation Question (Economic)
What has been the ratio of costs to benefits?
What is the most cost-effective option?
Has the intervention been cost-effective (compared to alternatives)?
Is the program the best use of resources?

15. Evaluation Question (Appropriateness)
To what extent does the program address an identified need?
How well does the program align with government and agency priorities?
Does the program represent a legitimate role for government?

16. Evaluation Question (Effectiveness & Efficiency)
To what extent is the program achieving the intended outcomes, in the short, medium and long term?
To what extent is the program producing worthwhile results (outputs, outcomes) and/or meeting each of its objectives?
Efficiency:
Do the outcomes of the program represent value for money?
To what extent is the relationship between inputs and outputs timely, cost-effective and to expected standards?

17. Evaluation Design
What is the design for this evaluation? (e.g., experimental, pre-post with comparison group, time-series, case study, post-test only)
What is the rationale for using this design?

18. Program Evaluation Levels
Focus
Measurement Methods
Reaction
Perception of-
Knowledge
Satisfaction
Motivation
Usefulness
Rating scales
Surveys
Focus Groups
Structured in-depth interview
Learning
Acquisition of-
Skills
Attitude
Pre-post test
Essay questions
Case study analysis
Simulations / Role play
Transfer
Real life transfer of-
Problem solving
Record reviews and audits
Observations
Checklists
Critical incident report
Utilization
Real world outcome
Compliance reviews
Program institutionalization

19. What Constitutes “Success”? Data Collection Method
Evaluation Question
Criteria or Indicator
What Constitutes “Success”? 1. 2.
Evaluation Question
Data Collection Method
Source of Data 1. 2.

20. Data Analysis & Interpretation
What method(s) will you use to analyze your data (e.g., descriptive statistics, inferential statistics, qualitative analysis, such as content or thematic analysis)?
Provide example table shells, templates, or qualitative codebook that specifies the output for each type of analysis you plan to conduct.
Interpretation
Who will you involve in drawing, interpreting, and justifying conclusions? Does this group include program participants or others affected by the program?
What are your plans, including evaluation capacity building activities, to involve them in this process?
Analysis to Be Performed
Data to Be Analyzed
Person(s) Responsible
Due Date

21. Use and Communication Use
How will evaluation findings be used? By whom?
How does the timeline for reporting findings and potential recommendations align with key events for which you will need information from the evaluation (e.g., grant application, partner meeting)?
Who is responsible for creating and monitoring an action plan to guide the implementation of evaluation recommendations? What follow up is needed?
What lessons learned, including those about evaluation and evaluation capacity building, should be shared? How will they be documented?
KTE plan
Which evaluation stakeholders will you communicate with and for what purpose (e.g., update on status of evaluation, invite to meetings, share interim or final findings)?
What methods (e.g., in-person meetings, s, written reports, newsletter article, presentations) will you use to communicate with evaluation stakeholders?
Who is best suited to deliver the information (e.g., evaluator, program manager, coalition leader)?
Why are these methods appropriate for the specific evaluation stakeholder audience of interest?

22. Communication Plan Applicable (Yes/No) Purpose Format Messenger
Time/ dates
Notes
Include in decision making about evaluation design/activities
Inform about specific upcoming evaluation activities
Keep informed about progress of the evaluation
Present initial/interim findings
Present complete/final findings
Document the evaluation and its findings
Document implementation of actions taken because of the evaluation

23. Common Evaluation Tools
Client Experience Survey
Focus Groups
In-depth Interviews
Debriefing
Reflective diaries
Case studies
 Content Evaluation: Pre-Post
Other methods:
e.g., Art-based method

24. Thanks