1. Amy A. Germuth, Ph.D. EvalWorks.com Blog - EvalThoughts.com
Evaluation Post Mortems Dissect what went right or wrong and learn from it!
Amy A. Germuth, Ph.D.
EvalWorks.com
Blog - EvalThoughts.com

2. What’s a Post Mortem?
A formal analysis of the success and failures of the project to-date.
Findings are added to the general knowledge base for future use.

3. Post Mortems in Practice
Conducting post-mortems is a common practice in business.
Appears to be less common in evaluation.
Very little written about evaluations that don’t work, including why they don’t work.
Fear of reporting failures?

4. Post Mortems in Evaluation
What if we made post-mortems part of the evaluation process?
What would it look like?
Where would it fit?
Would it be beneficial?
If so, why and how?

5. Post Mortems: The Process
Two-step process:
Provide the persons involved a list of questions about the evaluation that they think about and respond to on their own.
Bring all persons together in order to share what they thought and discuss lessons learned.

6. Who to Involve May depend upon where you are in the evaluation cycle.
Ultimately want all stakeholders to weigh in.
Remember – about evaluation process – NOT evaluation findings.

7. General Questions to Consider
Are you/we proud of our finished deliverables (project work products)?
If yes, what's good about them?
If no, what's wrong with them?
What was the single most frustrating part of the evaluation?
How would you/we do things differently next time to avoid this frustration?

8. General Questions Cont.
What was the most gratifying or professionally satisfying part of the evaluation?
Which methods or processes worked particularly well?
Which methods or processes were difficult or frustrating to use?
If you could wave a magic wand and change anything about the evaluation, what would you change?

9. Evaluation-focused Questions
Did our stakeholders, senior managers, customers, and sponsor(s) participate effectively? If not, how could we improve their participation?
How accurate were our original estimates of the time, cost, and other resources required of the evaluation? What did we over- or under-estimate?
 Knowing what we know now, would we have chosen the same type of evaluation design as the one we used? If not, what could have pointed us to a design that would have been better suited for such a project?

10. Evaluation-focused Questions Cont.
Were our evaluation questions the best ones, or were there other questions we did not fully explore with stakeholders, through our evaluation, etc. that needed addressing?
 How would we rate the quality of the data we gathered and what could we have done to have collected more convincing data for formative and summative purposes?
 Did our presentation of results highlight the data so that stakeholders could make their own interpretations or understand the ones we made?
 What did we do to help stakeholders understand and use the evaluation findings?

11. Further Notes on Post Mortems
Separate from just following the program evaluation standards post-mortems have a very formal outcome, the identification of lessons learned.
Separate from a meta-evaluation as meta-evaluations are themselves evaluations and not designed to identify lessons learned, as much as to identify the value in the evaluation that was conducted.
Also, for meta-evaluation to be viewed as unbiased, they do need to be conducted by someone outside of the original evaluation, whereas post-mortems are specifically designed to engage the original evaluators.

12. Blog: EvalThoughts.com
For More Information
Amy A. Germuth, Ph.D.
EvalWorks, LLC
150 Solterra Way
Durham, NC 27705
(919)
Blog: EvalThoughts.com