1. Alain PRAT, Technical adviser, QSM/EMP/HSS WHO - Geneva
Assessment of Medicines Regulatory Authorities (MRAs)
Technical Briefing Seminar on Essentials medicines Policies WHO, Geneva, Switzerland November 2010
Alain PRAT, Technical adviser, QSM/EMP/HSS
WHO - Geneva

2. Plan of the session The assessment process The process approach
The content of the assessment tool
Some figures and outcomes

3. Why an assessment tool ? In line with one of the strategic objectives
To strengthen National Regulatory Authority's capacities
To assess in order to to identify strengths and weaknesses
To make recommendations on identified gaps for improvement
To propose/suggest supporting activities to satisfy the identified needs
To use assessment results as a tool for convincing decision- makers to obtain more support;

4. Main steps of the assesssment / 1
Expression of a need
Internal / External
Scope of the assessment
Objectives and expected outcomes
Assessment team
Qualification, experience, availability
Minimun 2
Staff from the organization assessed
Preparation works
Request baseline information
Study of available information
Validation of the scope covered
Preparation of the assessment plan
Validation of the plan with the institution

5. Main steps of the assessment / 2
Opening session
Presentation of assessment team, objectives, methodology
Presentation of the authority
Conducting the visit
Follow planned activities,
Collection of evidence
Closing session
Presentation of the main findings and related recommendations
Presentation of the institutional plan
Closing remarks
Follow up
Provide for the draft report, collect the comments and finalize
Initiate/consider supportive actions

6. Mapping the regulatory actors
Related actors / Institutions
Regulatory functions
MoH
Manufacturers, distribution channel licensing
NRA
Registration
Inspectorate
Regulatory Inspections
NQCL
Quality control laboratory
National Vigilance center
Safety monitoring of marketed products
Institutional Review Boards
Clinical trial
Drug Information Center
Control of drug promotion

7. Mapping the processes / Historical findings based on 21 assessments
Marketing authorization
Licencing
Regulatory Inspection
Quality Control
Pharm.Vigilance 3 2 6 1 4
Number of MRA = 21

8. Assessment methodology
Not based on impressions, feelings or any subjective considerations
Based on objective evidence
Evidence collected through interviews should, whenever possible, be confirmed by more objective means
Possible deficiencies or gaps should be thoroughly investigated and validated
Consensus should be reached at the end with auditees 

9. Documented evidence Interviewing personnel
Reviewing manuals, guidelines developped
Studying records
Reading reports
Scanning files or applications
Analyzing data, indicators
Observing activities performed and facilities where they are performed
Examining conditions during these activities

10. Documented evidence
Investigations not only to check the presence or the absence of a document (law)
But to pursue to find the evidence on the implementation
And ideally the evidence that it provides the desire results…..
Examples :
Law has been issued and Regulation is missing
Law and regulation are published but no guidance is provided to applicants
Administratives procedures are established but no records are demonstrating its implementation
Administratives decisions are published but without any legal framework

11. Mapping the processes / 1
Ressources are input for the process
Results are the outputs of the process
Outputs from the process N
are inputs for the process N+1
Outcomes are very often difficult to assess
Structures/
Inputs
Outputs
Processes
Outcomes

12. Mapping the processes / 2
Manage vision, mission, strategies, policies
Manage the ressources needed : human and others
Manage the relation with patient and customers
Manage the production of products and/or services
Manage supporting activities
Manage continuous improvement

13. MRA is a process in itself
Control Drug
Promotion
Quality Control
Product
Information
Marketing
authorization
Regulatory
Inspection
Pharmaco
vigilance
Licensing
Import
Control
Few business processes

14. Recent improvement for the tool
Chapter on clinical trial developed
Chapter on vigilance aspects developed
New chapter on Narcotics
New chapter on International cooperation
Quality management system for NRAs
No more open question
All chapters reviewed and design consolidated

15. Design of the assessment tool
Same format for each modules / functions
Legal basis, framework
Guideline and Documentation
Organisation and structure
Planning and internal procedures
Human and other Ressources
Records and others outputs
Availability of these information

16. Chapters of the tool

17. Institutional structure / 1
Legal basis
Governance structure
Organization in place
Quality management system
Funding
Management of human resources

18. Institutional structure / 2
Independence and impartiality
Transparency and confidentiality
Management of committees and external expertise
Infrastructure and equipment
Monitoring and accountability
IT Management

19. Marketing Authorization Planing and monitoring
Equipments and facilities maintained
Qualified personal
Pertinent guideline
SOP implemented
Adequate legal framework
Records
Reception
Evaluation
Pre-Market Inspection
Tests
Comittee
Planing and monitoring
Application
Decision
MA + Ann.
Assessment process

20. Examples of key findings and gaps (Inspections) / 1
Regulation
No regulations for compliance with good manufacturing practices (GMP) and good distribution practices (GDP)
Guideline
The available GMP guidelines are not updated to comply with WHO guidelines.
There are no Good Distribution, Storage or Wholesaling Practice Guidelines
Written official GMP guidelines do not exist at the moment except the unfinished draft
There is no official National GMP guideline and the WHO GMP guidelines have not been officially adopted, although they are being used

21. Examples of key findings and gaps (Inspections) / 2
Human resources
Lack of competency in GMP auditing
Limited human resources in GMP and GDP activities
Some of the staff have received limited but not adequate training in GMP inspection. They also lack experience.
There is no legal officer in the department.
The number and skills of inspectors is not adequate, considering the number of manufacturers to be inspected.
Independence/Impartiality
No code of conduct for inspectors
No provision or guidelines regarding conflict of interest of inspectors exist

22. Examples of key findings and gaps (Inspections) / 3
QMS and SOP
No periodicity defined for performing inspection and inadequate duration of inspection
No quality management system for inspectorate and in particular no documented procedure for documentation control, no internal periodic review or auditing
Documented procedure for planning, preparation, inspection and follow up not finalized
No procedure for qualification as inspector and inspection team leader
ProcedureS for follow up of deficiencies identified during inspections do not exist.
Written checklist and plans for inspection of manufacturers and distribution channels are not available.

23. Examples of key findings and gaps (Inspections) / 4
Equipment / IT
Limited access to the internet
Inadequate logistical support i.e. communication tools on the field, vehicles and computers
Organisation
Poor communication with the regional inspectors
No coordination or collaboration or exchange of information on inspection activities
No mechanism to demonstrate that all inspectors in the country follow the same procedure
Records and outcomes
No consolidated list of inspection activities performed

24. Usefulness of indicators
Marketing Authorization
Number of application received, Number of MA delivered .
Time to acknoledge receipt of application ;
Time to deliver a MA ;
Time/delay to publish MA with related Product Information.
Inspection
Number of inspection
Time between two inspections ;
Time between initial inspection and follow up inspections (in case of non compliance)

25. List of institutions to be visited and Personnel to be met / 1
Ministry of Health / Ministry of industry,
Representatives of the Regulatory authority and any other organizations involved in the regulatory functions,
Staff of Regulatory authority or organization,
Representatives of the Industry Association of manufacturers, distributors, importers and exporters,
Representatives of the Professional Association of general practitioners, nurses and pharmacist,
Professionals councils (medical practitioners council, pharmacists council)

26. List of institutions to be visited and Personnel to be met / 2
Representatives of Consumers' associations,
Journalists.
Non-governmental associations
Procurement agencies, National medicines stores
Health research organizations
Chairmen or representatives of Advisory committee
Chairmen or representative of IRB / IEC
Representative of university academician

27. Main source of documented evidence / 1
Regulatory authority
Act, Law, Decree or circular establishing the Regulatory authority
Corporate, strategic and business plan of the NRA
Mission, vision, objectives and indicators of the NRA, i
Quality manual,
List of Internal procedure
List of internal forms and templates
List of the fees applicable for licensing, registration or authorization
Organigram/organization charts
Code of conduct/code of ethics
List of staff with their qualification
List of external experts
Annual report, self-assessment report

28. Main source of documented evidence / 2
Regulatory functions (Registration, Licensing, Inspection, Clinical Trail, Pharmaco-vigilance, Market control, Import Control, Quality Control Laboratory, control of drug promotion
Act, Law, Decree or circular establishing legal provisions for each regulatory functions
Guidance published on this domain
Internal procedure
List of equipment
Job descriptions
Decision, Authorization and their annexes

29. Issuance of Recommandations
Based on the various life cycles of the activities needed such as :
To convince politicians
To change laws or decrees
To develop a guidance with consultative process
To reorganise and reshape the structure (centralized/decentralized activities)
To implement QMS, to develop procedures and planning
To manage HR
To change records
To implement communication strategy

30. Guidance to use this tool
Regulatory assessment process
Main steps of a Regulatory assessment
Different categories of assessment in the time
Duration
General advice about assessment
For each regulatory function
Brief narrative description of the function
List of questions
Quantitative indicators of activities, of performances
Documented evidence to be reviewed (plan, procedures, reports, job descriptions, etc)
References to be consulted, based on WHO bibliography
Annexes
Templates

31. MRA assessments worldwide
44 Assessments performed on 40 Regulatory systems (with the involvement of HQ)
AFRO - 21 COUNTRIES / 24 ASSESSMENTS
EURO - 2 COUNTRIES / 2 ASSESSMENTS
EMRO - 4 COUNTRIES / 5 ASSESSMENTS
SEARO - 4 COUNTRIES / 4 ASSESSMENTS
WPRO - 7 COUNTRIES / 7 ASSESSMENTS
PAHO - 2 COUNTRIES / 2 ASSESSMENTS
WHO Regional assessments
????
Self-assessments
???

32. MRA assessments in AFRO region
2008
2007
2006
2004
2003
2003 No

33. AFRO : Outcomes / Few examples
Publication of the main drug law

34. AFRO : Outcomes / Few examples
Various kinds of MRAs

35. AFRO : Outcomes / Few examples
Committees within the marketing authorization processes

36. AFRO : Outcomes / Few examples
GMP requirements

37. Thanks for your attention