1. ProZorro : Innovations in Digital Ex-Ante Monitoring Tools for Public Procurement in Ukraine
Eliza Niewiadomska

2. Contents of presentation
Ex-Ante monitoring in Ukraine - the challenges and opportunities
EBRD Technical Assistance Project
Solution
Benefits
Lessons learned and next steps
26 July, 2019

3. 1. The procurement monitoring challenge in Ukraine
Monitoring and audit of public procurement is necessary to ensure compliance with public procurement law and curb corruption and waste by deterrence and detection.
However, the high volume of procurement transactions in Ukraine made this impossible to do manually resulting in low detection rates or overreliance on retrospective audits.
26 July, 2019

4. 1. The monitoring opportunity in Ukraine
Development of the ProZorro e-Procurement system presented an opportunity to implement real time data driven monitoring and prevention through proactive intervention.
Enabled data processing algorithms in line with preselected risk management framework and risk assessment needs to be conceived.
Pioneer the use the Open Contracting Data Standard for monitoring and risk based auditing
26 July, 2019

5. 1. Ex-ante monitoring as a means to mitigate risks
Key principles for effective ex-ante procurement monitoring in Ukraine:
Monitoring body should be independent;
All monitoring activities should be open an accessible to the public when officially published;
Trigger for open monitoring case should be automated compliance and automated risk indicators;
26 July, 2019

6. 1. Ex-ante monitoring as a means to mitigate risks (cont.)
Key principles for effective ex-ante monitoring in Ukraine:
Monitoring decision should be evidence based, risk-based and data-driven;
Monitoring function should be introduced by the law;
Methodology of monitoring should be approved with the policy maker;
Methodology should consist of procedures selection rules and processes which should be evidence-based and data-driven;
26 July, 2019

7. 2. EBRD Technical Co-Operation Project Concept
EBRD developed a technical co-operation project with the Government of Ukraine to introduce:
Digital ex-ante monitoring to improve compliance, efficiency and effectiveness of public procurement;
Data-driven, evidence-based and risk-based approach to procurement monitoring;
Digitalized monitoring procedure in Ukraine’s e-procurement system ProZorro using the Open Contracting Data Standard (OCDS);
Risk mitigation in public procurement and optimize efficiency of the procedures.
26 July, 2019

8. 2. EBRD Technical Co-Operation Project Scope of Work
Development of:
The data-driven ex-ante monitoring methodology & automated risk indicators;
Automated procurement procedures with risk prioritisation concept for selection of high risk public procedures for ex-ante monitoring;
Automated risk indicators engine prototype and risk procedures prioritization engine developed on a proof of-concept basis.
26 July, 2019

9. 2. Legal and technical implementation of the ex-ante monitoring solution
January an amendment to the Public Procurement Law was adopted and ex-ante monitoring was introduced;
State Audit Service of Ukraine became responsible to conduct ex-ante audit for all public procurement procedures;
September 2018 official methodology of compliance indicators, risk indicators and risk-prone procedures prioritization was approved by the Ministry of Economy and Trade of Ukraine and State Audit Service.
26 July, 2019

10. 3. ProZorro ex-ante monitoring solution overview
26 July, 2019

11. 3. Procedures covered by the ex-ante monitoring solution
PROCUREMENT METHOD
PLANNING
NOTICE
BID PROPOSALS
AWARD
CONTRACT SIGNING
CONTRACT EXECUTION
Open tender above EU threshold 4 2 6 5
Open tender 8
Below threshold open tender
Negotiation 1
Direct purchasing (below threshold reporting)
TOTAL 16 17
26 July, 2019

12. 3. Legal and technical implementation of the ex-ante monitoring solution
October ProZorro launched new modules in pilot mode:
Indicators engine
Prioritization engine
Cabinet of State Audit Service
October 2018 official procurement monitoring started by the State Audit Service of Ukraine
26 July, 2019

13. 3. Operational performance of the system
Procedures risk indicators online calculation - every 10 minutes
In average times per day indicators are recalculated by the engine
times procedures are verified against indicators algorithms daily
In average indicators statuses are changed per day
Risky procedures online queueing - once an hour
Risky procedures online prioritisation - once an hour
26 July, 2019

14. 4. Ex-ante monitoring solution benefits
Enables real-time manipulation of the complete procurement dataset.
Provides the big picture in public procurement system for risk allocation.
Enables monitoring throughout the year, supporting a smoother process.
Presents real-time results for the entire procurement system as well as for the monitoring function when required.
Facilitates procurement risk analysis, transaction and controls efficiency assessment.
Keeps monitoring procurement transactions during its whole lifecycle and even while you sleep.

15. 4. Ex-ante monitoring benefits (cont.)
Developed for Ukraine - can be suitable for procurement sectors worldwide and is open source.
As an automated OCDS based monitoring solution, it enables smarter & intelligent procurement transaction monitoring at any stage of the procurement process for any purposes and by any interested authority.
For example, the solution can be effectively utilized by:
Official authorities for ex- ante public procurement monitoring.
Supreme Audit Institutions - to plan and execute audits.
Internal auditors - to evaluate and improve the effectiveness of risk management, control, and governance processes.
Civil societies and watchdogs involved in public procurement monitoring.

16. 4. Ex-ante monitoring benefits (cont.)
Official monitoring authorities are now able to find abnormal activity within the public procurement much quicker, more accurately and react appropriately.
Already positive signs that the solution will enable profiling and intelligent targeting of procurement activities that should be monitored at any or all stages of the procurement cycle.
The existence of the solution has had a positive impact on the perception of public procurement in Ukraine.
26 July, 2019

17. 5. Lessons learned so far
Successful implementation of a risk-based monitoring methodology is dependent on:
People: Government stakeholders are struggling with the implementation of the public procurement risk management system.
Lesson: Necessary to enhance risk management capacity of the officials responsible for the monitoring and introduce periodical risk assessment of the public procurement system.
26 July, 2019

18. 5. Lessons learned so far (cont)
Process: To build the uniformed automated risk-based monitoring approach to the procurement transactions in Prozorro system was challenging due to the risk specific nature of the project and the source of the data.
Lesson: The next phase should incorporate this experience into the methodology for the roll out.
26 July, 2019

19. 5. Lessons learned so far (cont)
Technologies: Data quality and availability is vitally important for any automated risk-based data analytical approach.
Lesson: The technologies should support up to the date monitoring processes and users experience to make it relevant (risk engine + auditors cabinet)
26 July, 2019

20. 5. What is next for automated ex-ante monitoring?
Evaluate pilot results and amend the solution with lessons learned if any.
Enhance risk management capacity of the officials responsible for the monitoring and introduce periodical risk assessment of the public procurement system in Ukraine. This will enable continuous development of the red flags and risk indicators.
Amend methodology with observed pilot suggestions.
Introduce the policy for automated monitoring solution periodic validation.
Assess indicators performance, implement indicators in a pipeline (more then 50 at the moment).

21. RANKS TRANSACTION
READS TRANSACTIONS
PREPARES DATA
ONLINE REFERENCING
AGGREGATES RISKS
ORDERS TRANSACTIONS
DELIVERS RESULTS