1. MODULE 8: PROJECT TRACKING AND EVALUATION
REHABILITATION ENGINEERING RESEARCH CENTER ON TECHNOLOGY TRANSFER (T2 RERC) TRAINING MODULES ON TECHNOLOGY TRANSFER

2. Objective of the Module:
Participants will:
explain basic concepts and terms used in program evaluation/research;
define indicators of effectiveness and efficiency for a project of their interest
suggest methods to track the indicators;
identify how the results would be used for project monitoring and evaluation
Slides 13-15

3. BASIC CONCEPTS AND TERMS
Research and Evaluation
Merit and Worth
Management Orientation: Evaluation for Development
Effectiveness and Efficiency
Formative and Summative Evaluation
Outcome and Impact

5. Merit and Worth Merit refers to
intrinsic quality of person/ product/ system
its potential, or promise to be “valued” in its broader context.
Ex: Program inputs (resources, design, theory); processes (operations) and outcomes indicate its merit
Worth refers to
extrinsic value of person/ product/ system
its actual “value” in the broader context
Ex: program effects/ impacts on internal and external environments indicate its worth.

7. Fig 2. Evaluation and the Planning Cycle↓ ←
Context
Evaluation
Cycle
Planning Cycle ß ↑
What needs
to be done? [
Defining
objective]
Product Þ
Are
we there yet?
How to do it?
Recycle output or
[Defining Ü
Input
disseminate?
resources ]
Evaluation ↓ ↓
What is the ↑ ↑ ↓ ↓
optimal path ↑ ↑ ↓ → →
to get to → → ↑ ↓
goal? ↑ ↓
[process ↑ ↓
improvement ↑ ↓ ] ↑ ↑ ↓ Ý ↑ ↓
↑ ↑ → → → → → → →
Process → → → → → → → →
Evalu
ation

8. System characteristics:
INPUT ---- à P
ROCESS
---- à
OUTPUT ↑ ↓ ←
← ← ← ← ← ←
FEEDBACK
Effective: gets outcome at the desired
quality/quantity level
Efficient: Maximizes effectiveness, minimizes cost
System characteristics:
.Uses the “input” resources;
.“Processes” them [facilitates and manages the dynamics of their interaction];
.In function of a pre-defined objective [i.e., effort directed to achieving the corresponding output]
.Does “self-regulation”[i.e., checks/obtains feedback information about the product taking shape, constantly adjusts its processes so the desired quality of product is obtained].
Efficiency =Output/ Input
Slides 13-15

9. Formative vs. Summative Evaluation
Assesses quality of outcome during the process, while outcome is taking shape
Provides basis for improving/maintaining program actions
SUMMATIVE
Assesses quality of outcome at the end of process when outcome is in final stage
Provides basis for establishing program effectiveness & efficiency; [accountability]

10. Outcomes and Impacts Outcomes direct outputs or immediate effects
assess program’s performance- its effectiveness and efficiency; an indicator of merit
Impacts
long term effects; benefits to staff, participants, other stakeholders and community at large
assess program’s validity/relevance; indicator of worth

11. Framing program evaluation: three basic questions
Is the program effective? - has it achieved its goals?
Is it efficient? - did it optimize its methods, resources to achieve the goals?
Is it valid? - are its goals relevant to the context? Is it valued by its stakeholders and beneficiaries?

12. In Response: what to assess
Effective? --- Outcome quality; outcome quantity (for duration of program)
Efficient? -- Cost per outcome; Time per outcome
Valid? Worth it? -- internal and external impacts; long term effects on/ benefits to stakeholders and others involved
NOTE: outcome,cost/time [merit indicators] - assessed through tracking program performance. Impact [worth indicator]: - assessed through tracking effects beyond program period

13. Slides 13-15

16. Slides 13-15

19. Slides 13-15

22. Use of Information Monitoring project Performance
Improve intermediary outcomes [quality vs. timelines]
Improve process
re-distribute personnel
review protocols/timelines
Describing performance level -[effectiveness and efficiency]

23. Questions, Comments, Suggestions
Related Issues
Questions, Comments, Suggestions

24. Workshop Session Two
For your project, identify, or discuss applicability of:
Goals, Protocols, Action Plan (or similar framework)
Expected Outcomes - final [and intermediary, if applicable]
Outcome Tracking: (a) process and (b) use- improving outcomes (formative eval.) and finding effectiveness (summative eval.)
Tracking resource utilization (time/cost …): (a) process and (b) use for process improvement and finding efficiency (summative eval.)
Instrument and database alternatives