1. Quality assessment of EHR systems in Belgium
January 26, 2012

2. Quality Labelling and Certification
Distinguish “authority driven” and “market driven” QL & C
Authority driven
In principal “national”. European authority absent
Not only “state driven”
Cross border aspects through harmonisation of procedures and test content
Result: certificate OR label
Market driven
Industry driven OR “independent organisation”
EuroRec = independent organisation
I.H.E. and Continua Health Alliance = industry driven
In principle cross border
Result is a label or a quality mark / attestation
Vilnius, Lithuania
January 26, 2012

3. EHR-QTN Functional Diagram
Vilnius, Lithuania
January 26, 2012

4. Quality ‘Attestations’
Quality Assessment Procedure
Attestation granted
Third party assessment by a CAB being a public authority or an organisation granted power by a public authority either by law or by regulation.
Certificate
Third party assessment by a CAB on requirements issued by an organisation not empowered by law or by regulation.
Quality Label
Self-assessment with an external audit. Conformity assessment is done by the supplier and documented to a third party, being a public entity, a professional organisation or an industry federation.
No “attestation” but a Quality Mark on the product is allowed
Self-assessment by vendor who performed testing on his own products and affirms that they conforming to a given set of requirements.
Declaration of quality
Vilnius, Lithuania
January 26, 2012

5. Quality Labelling and Certification
Important to make a distinction between
Authority driven
Market driven
In principle “national”
Not mandatory “state driven”
Cross border aspects through harmonisation of test procedures and test content
Result: certificate OR label
Vilnius, Lithuania
January 26, 2012

6. Belgian Certification
Example of a National Regulatory Driven Certification
Not addressing in this presentation the EuroRec Cross Border Seal
Focusing on functionality, including data-exchange aspects
Interoperability is important but considered as a side-effect of certified application
Addressing “clinical” software for
General Practitioners
Private Physiotherapists
Homecare Nursing
Dentist
Logopaedists
> 10 years of experience in quality labelling and certification
Other domains, e.g. hospital software, limited to reporting and billing
Vilnius, Lithuania
January 26, 2012

7. When and how did it start?
Initial request from the market
Use of purely clinical systems since mid 80’s
Market Fragmentation in the 90’s:
> 50 GP systems: sometimes “kitchinette software”
Data exchange (lab data since late 80’s but – still - with different protocols)
Discontinuation of providers
User expectations are very disparate
Vilnius, Lithuania
January 26, 2012

8. When and how did it start?
Authorities willing to
Protect patient data
Protect users (healthcare professionals)
Have reliable and trustworthy systems on the market
Functional quality of the systems (all of them meeting essential user requirements…)
Improve “clinical availability” of patient data
Obtain “standardised” comparable data for management purposes (longer term objective)
Belgrade, Serbia
November 22, 2011

9. When and how did it start?
1997 Declaration of Minister of Health defining main functions of an EHR.
December 1997 Working Group Consensus Text “Functions of the Electronic Medical Record for the General Practitioner”
12 “specialised” physicians / users & academics
Main functions of the EHR
Expression / symbol of the relation between patient and healthcare professional (GP)
Care related functions: memory, continuity, synthesis, prevention and risk management, coordination & communication, quality of care monitoring
Evaluation of clinical practice
Decision support, research activities, epidemiology, education
Medico – legal
Practice Management
Belgrade, Serbia
November 22, 2011

10. When and how did it start?
1998 – 1999: ProRec-BE project
Mission: translating the document of the Ministry in concrete “Technical Implementation Requirements”
Joint working party of main vendors and user representation.
Result: 333 criteria listed with indication: optional / mandatory. ProRec-BE project 1999.jpg
One of the main sources for the EuroRec “fine grained statements”
Vilnius, Lithuania
January 26, 2012

11. Kings Decree 78,
KB/DR78 ( ) about “health professions”
Law adding EHR principles to the decree in 1999 (art. 45bis)
§1 His Majesty may define minimal criteria to be met by software applications managing the electronic medical and electronic nursing records in order to be homologated by the Minister of Health.
§2 The criteria to be met may address the following – not limited – aspects:
Vilnius, Lithuania
January 26,2012

12. The functional processes
The medical and nursing datasets used internally and their exchangeability
Structure of the record
Codes and coding systems
Clinical decision support (help in diagnosis and prescription)
The patient data to be sharable
The patient identification
The billing and reporting for health insurance
Vilnius, Lithuania
January 26, 2012

13. King’s Decree 78,
§3 The criteria will be issued by His Majesty after decision by the Government and based on an advice of a multi- disciplinary group nominated by Royal Decree
§4 The workgroup sub §3 has at least one representative of the Ministries of Health, of Social Affairs, of Justice and of Economic Affairs as well as representatives of the different professions.
§5 The Minister is entitled to homologate Electronic Medical and Nursing Records based on the advise of the multi- disciplinary workgroup.
Vilnius, Lithuania
January 26, 2012

14. King’s Decree 78, 1999-01-25 Article 45ter about “incentives”
§1 Health professionals recording patient data and transferring anonymous data to the Ministry and the National Health Institute may be granted a subvention.
§2 His Majesty issues, after decision by the Government, the requirements to be met by the healthcare professionals (also regarding the data) in order to obtain that subvention.
Belgrade, Serbia
November 22, 2011

15. EMDMI Vilnius, Lithuania December 26, 2012
Not for profit organisation, created Jan. 2000, with
Representatives of Ministry of Health and of Social Security
Domain experts and Academics
Role
Validation of the technical implementation criteria
Prioritisation & Classifying the criteria
Mandatory within 1 year
Mandatory over 2 – 3 years
Optional but will become mandatory (soon)
Optional but important on longer term
Optional / Commercial differentiation of the products
Nominating members of a “multi-disciplinary group”
Prioritised Criteria.jpg
Vilnius, Lithuania
December 26, 2012

16. Sessions GP Software Session 2002 (18 April) Session 2003 (October)
Limited set of 37 criteria to be tested
Double test procedure:
Functional test by a user
Technical test (database, versioning…)
Administrative compliance validation (maintenance contracts,…)
21 applications (from >50), 17 succeeded
Session 2003 (October)
135 criteria mandatory
Important focus on
Coding and the use of a “standard” national drug database + ATC
Structuring the EHR
Vilnius, Lithuania
January 26, 2012

17. Sessions GP Software Session 2004 (November) => 2005
Sumehr (Summary EHR)
Export only
Structuring the EHR
Postponed to March 2005
Session 2006 (September)
Sumehr
Import (non structured)
Coding
Session 2007 => stucked
Change of responsible authority in 2008
Vilnius, Lithuania
January 26, 2012

18. Test session 2010-2011 eHealth platform Preliminary activities Focus
Created by law of 21 august 2008
Mission : healthcare “backbone” and IT services to the health community
Also responsible for quality assessment of healthcare software
Preliminary activities
Updating existing functional and interoperability criteria (03-04/2009)
Selection of new and updated criteria to be effectively tested in (04-09/2009)
Meeting with suppliers => 587 questions
Focus
Sumehr export and import as document (= patient summary)
Prevention management (breast ca, colon ca, cervix ca)
Access management, encryption and versioning of data
GP Software Migration (export and recuperation of complete practice databases)
Vilnius, Lithuania
January 26, 2012

19. Test Session 2010-2011 Tender for CAB (Conformity Assessment Body)
Two lots: scenario development and effective testing of the applications
Redaction of a tender (09-11/2009)
Launching tender and selection (12-05/2010)
Setting up context for quality assessment
Redaction of scenarios and documentation (04-06/2010)
Definition of test population and test patients (05-07/2010)
Validation of documentation, scenarios and use cases (07-08/2010)
Distribution of documentation, Q&A (08-09/2010)
Analysis of maintenance contracts (01-02/2011)
First session
Test period (10/2010 – 01/2011): 17 applications
Analysis of the results (02/2011)
Presenting results to National Commission (03/2011)
Documenting decisions & feedback suppliers (03-04/2011)
Vilnius, Lithuania
January 26, 2012

20. Session 2010-2011 - Final Second Session GP Migration Format Test
2 applications only two criteria to be tested
11 applications: full retest
5 still failed for a limited number of criteria (=> 3rd test)
GP Migration Format Test
Purpose: recuperation of the complete practice database when changing application by decision of user or because of failure of supplier
Goal:
Reconstruct the patient files “as recorded”, rebuilding history of data entry
New data should be a trustworthy copy (in juridical sense)
17 tested, 6 failed
Retest successful for all
Certificates were granted to 17 end 2011
Vilnius, Lithuania
January 26, 2012

21. Session Budget
Directly related to quality labelling and certification
Budget spent by health authorities €
Total cost for involved suppliers related to the tests €
First session per supplier: €
Second session per supplier: 2.500€
Other costs for suppliers
Total budget (product development) : €
Budget per supplier between € and €
Data entry test population
Upgrading of the application
Pretesting
Attending meetings
Test session(s)
Incentives
800€ per user
Estimated users (some are refusing subvention)
Total incentives budget: €
Vilnius, Lithuania
January 26, 2012

22. Why are the parties doing these efforts?
The authorities want:
Quality
Reliable and trustworthy systems
Comparable and more and more interoperable systems
Their projects to be implemented and available to the users
The users want:
Good functionality and practice management support
A health added value (prevention, decision support…)
Sharable patient data
The suppliers want:
A fair competition, cooperating with authorities and not competing with authorities
Market fragmentation reduction is NOT a goal, but may be an effect
Validated application, not having to prove all the time compliance
Systems evolving in the same direction
Happy users
Belgrade, Serbia
November 22, 2011

23. Conclusions
Involvement of ALL stakeholders is essential (= also the industry).
Health authorities should take the lead.
Testers and Certifiers should be different organisations.
Quality has a cost.
Take profit from existing experience.
Take your time… there is no rule that applications need to be certified every year.
Belgrade, Serbia
November 22, 2011