1. Public Procurement Research Centre (PPRC) – www.pprc.eu
Non-intentional price preferences
Public Procurement Global Revolution VIII. June 12th 2017
Jan Telgen
Elisabetta Manunza
Public Procurement Research Centre
A cooperation between the University of Utrecht and the University of Twente
Public Procurement Research Centre (PPRC) –

2. Public Procurement Research Centre - www.pprc.eu
Our agenda for today
The setting: some background info
The (non)importance of weights
European Court of Justice: Dimarso
Non-intentional price preferences
Some empirical data
Conclusion
Public Procurement Research Centre -
June 12, 2017

3. The setting: the award phase of a tender

4. Public Procurement Research Centre - www.pprc.eu
Legal requirements
EU directives basically require:
Publication of award criteria
Publication of (relative) weights
But these two are only part of the award mechanism:
Public Procurement Research Centre -
June 12, 2017

5. Public Procurement Research Centre - www.pprc.eu
scoring
given
Bids
scores (scale 1-100)
criterion
weight A B C D
price
60%
3000
4000
2000
2500 50 10 80 60
quality
30%
good
fine
just OK OK
100 40
delivery time
10%
2 weeks
3 weeks
weighted scores 30 6 48 36 24 12 18 5 1
total score 59 37 65 55
scoring
Public Procurement Research Centre -
June 12, 2017

6. Weights: One example – 3 lessons

7. Weights: One example - 3 lessons
Tendering for vehicles
Award criteria and their weights:
Price 80%
Delivery time %
Which bid wins?
Public Procurement Research Centre -
June 12, 2017

8. Weights: One example - 3 lessons
Scoring rules:
Price points at $ 0, points at $
Delivery time points at 0 weeks, 0 points at 20 weeks
Per vehicle
Price
Delivery time
points
Total points
Bid 1 $
10 weeks 62 10 72
Bid 2 $
16 weeks 67 4 71
Public Procurement Research Centre -
June 12, 2017

9. Weights: One example - 3 lessons
Scoring rules:
Price points at $ 0, points at $
Delivery time points at 0 weeks, 0 points at 20 weeks
Per vehicle
Price
Delivery time
points
Total points
Bid 1 $
10 weeks 8 10 18
Bid 2 $
16 weeks 28 4 32
Public Procurement Research Centre -
June 12, 2017

10. Weights: One example - 3 lessons
Public Procurement Research Centre -
June 12, 2017

11. Weights: One example - 3 lessons
Lesson 1: Fraud. You can make every bid one by judiciously choosing the scoring method (afterwards)
Lesson 2: Weights do not matter. Even at 80% weight on price we can make price irrelevant – replace $ by $ (same happens with weights of 90, 95 or even 99% …)
Lesson 3: Stupid buyer. Bids 1 and 2 may be options for the same supplier. By not mentioning the scoring rule you may deprive yourself of a better bid!
Always mention/publish the scoring rule
Public Procurement Research Centre -
June 12, 2017

12. Dimarso

13. European Court of Justice
Conclusion ECLI:EU:C:2016:555 July 14, 2016
Article 53(2) of Directive 2004/18/EC of the European Parliament and of the Council of 31 March 2004 on the coordination of procedures for the award of public works contracts, public supply contracts and public service contracts, read in the light of the principle of equal treatment and of the consequent obligation of transparency, must be interpreted as meaning that, in the case of a public service contract to be awarded pursuant to the criterion of the most economically advantageous tender in the opinion of the contracting authority, that authority is NOT required to bring to the attention of potential tenderers, in the contract notice or the tender specifications relating to the contract at issue, the method of evaluation used by the contracting authority in order to specifically evaluate and rank the tenders.
Public Procurement Research Centre -
June 12, 2017

14. Dimarso considerations
26 The Court has accepted that it is possible for a contracting authority to determine, after expiry of the time limit for submitting tenders, weighting factors for the sub-criteria which correspond in essence to the criteria previously brought to the tenderers’ attention, provided that three conditions are met, namely that that subsequent determination (1) does not alter the criteria for the award of the contract set out in the tender specifications or contract notice; (2) does not contain elements which, if they had been known at the time the tenders were prepared, could have affected their preparation; and, (3) was not adopted on the basis of matters likely to give rise to discrimination against one of the tenderers
27      However, neither Article 53(2) of Directive 2004/18 nor any other provision thereof lays down an obligation on the contracting authority to bring to the attention of potential tenderers, by publication in the contract notice or in the tender specifications, the method of evaluation applied by the contracting authority in order to effectively evaluate and assess the tenders in the light of the award criteria of the contract and of their relative weighting established in advance in the documentation relating to the contract in question.
Public Procurement Research Centre -
June 12, 2017

15. Public Procurement Research Centre - www.pprc.eu
Dimarso in lay terms
Contracting authorities do not have to publish the scoring rules
Unless …….. it makes a difference
But it always makes a difference …….!
So: Dimarso is nonsense
Public Procurement Research Centre -
June 12, 2017

16. Non-intentional price preferences

17. Non-intentional price preferences
Frequent complaints about price being the distinguishing factor in MEAT tenders:
Intentional: price being given much weight (??)
Non-intentional:
Caused by scoring of quality – through choice of score graph:
‘Flat’
Adding lower boundaries
Caused by scoring of price - through choice of score graph:
Steep
Hyperbolic
Public Procurement Research Centre -
June 12, 2017

18. ‘Flat’ quality criteria
It is well known that flat scoring of quality reduces the impact of quality and hence increases the relative impact of price.
Examples:
Scoring on 1-10 scale with 6 being OK  all score 7-8
Maximum speed of a car on a km/h scale
Etc.
Public Procurement Research Centre -
June 12, 2017

19. Lower boundaries on quality
We really want to have good quality and avoid excessive price pressure.
So we set weights for price:quality at 40:60
And on top of that we require a minimum score of 7 (on a 0-10 scale) for quality.
Acceptable quality scores range from 7-10: a maximum difference of 3
At 60 % weight the maximum difference in the final score is 18 points
Price scores range from 0-100
At 40% weight this translates into 40 points in the final score
Which is more than twice the amount of points available for the quality score
Public Procurement Research Centre -
June 12, 2017

20. The situation and the remedy60
unacceptable
score 7 10
quality
Public Procurement Research Centre -
June 12, 2017

21. Hyperbolic score graph
Frequently used in practice
and in some authorities even mandatory …..!
Price score i = [max points for price] * [ best price / price i ]
(there are other issues with this formula that Elisabetta will talk about)
Public Procurement Research Centre -
June 12, 2017

22. Hyperbolic score graph
Best price
price
Public Procurement Research Centre -
June 12, 2017

23. Some empirical data

24. Public Procurement Research Centre - www.pprc.eu
Some empirical data
Tenders in the Netherlands - using MEAT Q1
Court cases: cases brought to a court
Winning or losing
Court case not necessarily involving award method
NL tenders using MEAT
random sample
court cases
n=184
n=41
scoring method not published 9%
20%
flat' quality criteria
47%
76%
lower boundaries on quality
12%
17%
hyperbolic score graph
29%
22%
relative scoring
67%
83%
Source: Robbert Hoogeveen, masterthesis Un. Twente, 2014
Public Procurement Research Centre -
June 12, 2017

25. Conclusion

26. Public Procurement Research Centre - www.pprc.eu
conclusion
Many issues in award phase of a tender
Known from Decision Sciences
But hardly adapted in Purchasing / Public Procurement
Many errors/mistakes/stupidities in practice ……
But most go unnoticed ……
Surprisingly little attention for award phase in EU directives
Even though most court cases are about this phase
Knowledge from decision theory not incorporated
And disregarded in legal rulings
Public Procurement Research Centre -
June 12, 2017

27. Prof. Elisabetta Manunza Prof. dr. Jan Telgen
Contact
Prof. Elisabetta Manunza
Universiteit Utrecht
Public Procurement Research Centre
Achter Sint Pieter 200
3512 HT Utrecht
Prof. dr. Jan Telgen
Universiteit Twente
Public Procurement Research Centre
Postbus 217
7500 AE Enschede
Public Procurement Research Centre -
June 12, 2017