1. BEMA III questionnaire – draft proposals

2. BEMA III programme development BEMA programme includes several elements: ‘Questionnaire’ Methodology Assessors Database Reports

3. BEMA III proposals for questionnaire BEMA SG has begun development of the BEMA III programme, starting with the ‘questionnaire’. Current BEMA questionnaire sections: Organisation Assessment Pharmacovigilance Inspection (interfaces) 19 KPIs and 71 questions/SPIs Issues raised by assessors and agencies: Duplication of questions Redundancies Unclear questions PV questions too detailed Does not reflect current issues

4. BEMA III proposals for questionnaire BEMA SG subgroup has reviewed the questionnaire and is proposing a new structure which: Addresses the issues raised by agencies and assessors Simplifies the structure Introduces a logical sequence to the KPIs Addresses HMA strategic issues (in bold) Proposes some new SPIs (in italics) Reduces the number of KPIs (reduced from 19 to 13 at this stage) and SPIs (reduced from 71 to 38 at this stage)

5. BEMA III proposals for questionnaire Proposed new set of KPIs: Strategy and planning Leadership and culture Stakeholders Quality management Risk management Crisis management Human resources management Operations management Information management Interfaces Scientific decision-making Impact/effectiveness of scientific work Pharmacovigilance

6. BEMA III proposals for questionnaire Proposed new set of KPIs includes… Assessment Pharmacovigilance Inspection …and covers quality assurance of scientific work: Recruiting staff with the appropriate expertise Training, both regulatory and scientific Providing the necessary resources Allocating on the basis of competence and risk Ensuring quality standards and consistency Ensuring integrated opinion and formal approvals …as well as impact of scientific advice and authorisations

7. BEMA III proposals for questionnaire This proposal is still at an early stage and further development will include: Revisions based on HMA comments Development of guidance Methods to improve consistency of ratings Report structures to allow good practices to be described in sufficient detail Review of questionnaire and guidance by agencies and BEMA assessors

8. BEMA III proposals for questionnaire HMA is asked to approve, in principle, the draft list of KPIs and SPIs for the BEMA III questionnaire. HMA is asked to re-confirm the scope of the benchmarking programme in the context of the current status of the JAP programme for inspectorates and the legislative requirement for pharmacovigilance audits.

9. BEMA III proposals for questionnaire