1. Lesson 7: Performance Macerata, 28th October
Andrea Gramillano, t33 srl

2. Verify the consistency of the analysis of needs
Obstacles (needs)
Potential for solutions
Ostacoli – colli di bottiglia

3. Agenda
Definition and meanings
Indicators
Multicriteria

4. Performance: problematic defintion.
OECD definition:
Commission Defintion:
The degree to which a development intervention or a development partner operates according to specific criteria/ standards/ guidelines or achieves results in accordance with
stated goals or plans.
The meaning of the word performance is not yet stable; it is therefore preferable to define it whenever it is used. Performance might mean that intended results were obtained at a reasonable cost, and/or that the beneficiaries are satisfied with them. Efficiency and performance are two similar notions, but the latter extends, more broadly, to include qualitative dimensions.

5. Performance: When it does take place
On going Evalaution Criteria
the project activities are delivered on time, the outputs respect the targets, the resources are duly absorbed, the procedures are done according to the rules.
Needs
Resources (inputs)
Output (implementation)
(Specific Objective)
Result

6. Performance questions:
The project:
is able to spend all the financial resources;
meets the procedural deadlines;
achieved the target in terms of physical realization.
And in the next period:
What is needed to increase the project performance?(human resources? political support? administrative enforcement?
Will the project be successfully completed?
Which might be future challenges
Performance evaluation is based on monitoring indicators (procedural, financial, physical, output indicators)

7. Operational life of a Project
Setting
Tender or Purchaise procedures
Start of Work / service
Implementation
Test, End and final payment

8. Procedural monitoring
Most public activities have to follow a more or less rigid schedule in which the different steps are mandated and the deadlines fixed (i.e.). Procedural monitoring usually provides information about how project pipelines are progressing (where and when calls for tenders have been published, contracts have been awarded, …).
Procedure
Status
Specifications ready
Call published
Contract awarded ……
Final payment
Expected
Actual P1
Open

9. Physical monitoring Example of physical monitoring: Micro N 142 91
Indicator (number of enterprises)
Unit of Measurement
Target
Achievement
Micro N
142 91
Small
133
111
Medium 39 21
Owner (women) 50 40
Owner (<30y) 26 3
Start-up 54

10. Financial monitoring Example of financial monitoring: Priority
Expected expenditures
Resources committed
Expenditures
Amount %
(a)
(b)
(b/a)
(c)
(c/a) P1
133.4
100.4
75.2
71.8
53.9

11. Multicriteria
Tool used to compare several interventions in relation to several criteria. Multicriteria analysis is used also in the ex ante evaluation for comparing proposals. It can also be used in the ex post evaluation of an intervention, to compare the relative success of the different components of the intervention. Finally, it can be used to compare separate but similar interventions, for classification purposes. Multicriteria analysis may involve weighting, reflecting the relative importance attributed to each of the criteria. It may result in the formulation of a single judgement or synthetic classification, or in different classifications reflecting the stakeholders' diverse points of view. In the latter case, it is called multicriteria-multijudgement analysis. (from EVALSED)

12. Step 2 scoring or ranking
Process
Step 1 define criteria
Step 2 scoring or ranking
Step 3 weighting
aggregating
Step 4
Before Step 1 : Definition of the projects or actions to be judged (here results/impacts?)

13. STEP 1: setting criterion
Financial Performance
Criterion 2:
Procedural
Criterion 3:
Physical realisation
Project 1
Project 2
Project 3
Project 4
Project n…

14. STEP 2: Score or rank for judgment
It is needed to find a way to appraise the project according to different aspects since we used different measuring units for different aspects of different process
We can opt for:
A) Scoring: by assigning a numeric value to different “interval” of performance . For example 3 for “above average” –1 for “on line with average” – 3 for “below average”
B) Ranking: we simply order the different projects according to their performance from the first to the last

15. STEP 2: Scoring Criterion 1: Criterion 2: Criterion 3: Project 1 12
Project 3 3
Project 4
Project n…

16. STEP 3: Establishment of weight
If some Criteria is more important than others it shall be given more importance. To do it we simple apply a multiplication factor > 1 (e.g. 1,5). Some criteria may have such importance that they have to be singled out. This is the case for criteria determined by a veto threshold (For example “Physical” if some project has 0 performance, it is excluded by the analysis).

17. STEP 3: Apply the weight Criterion 1: ( * 1,5) Criterion 2:
Project 1
1,5 1
Project 2 2
Project 3
4,5
Project 4 3
out
Project n…

18. STEP 4: Aggregate the score
Criterion 1:
( * 1,5)
Criterion 2:
Criterion 3:
Total (with weight)
Total (without weight)
Project 1
1(1,5) 1
3.5 3
Project 2 2
Project 3
3(4,5)
6,5 5
Project 4
out 6
Project n…

19. WORK OUT: SME INCUBATOR
Criterion 1:
Economic (average increase of turnover)
Criterion 2:
New Jobs
Criterion 3:
Satisfaction for quality service
Total
Project 1
Project 2
Project 3
Project 4

20. WORK OUT DATAS Quality levels Low Medium High Project 1 2 3 4 Economic
80%
70%
30%
Physical 50 10 30 5
Quality
High
Low
Medium
Quality levels
Low
Medium
High

21. Apply weight A Economic * 1,5 Physical Veto = N. of job < 10 B
* 2
Quality
Veto for “Low”

22. See you