1. GMP Inspections – A Global Perspective Auditing of Computerised System Suppliers
IPCMF & ISPE Conference
Global Pharma Networks Tom Farmer
25th June 2004

2. Overview of Presentation
Reasons to Audit Computerised System Suppliers
Outline Procedure for Audit
Case Studies/Examples
Overview of Audit Repository Centre (ARC)
Typical Issues/Observations
Summary and Conclusions

3. Reasons to Audit Suppliers
Business Reasons
Risk Management
Good Practice
Cost/Schedule/Quality Benefits
Regulatory
Requirements and Expectations

4. Business Reasons Risk Assessment
Regulatory Impact
Data Integrity
Security
Product Quality
Business Risk
Risk To manufacturing/process equipment
Corporate Reputation
Safety Concerns
Patient Safety
Environmental Safety
Supplier Audit is not part of Risk Assessment as such, but an integral part of the overall Project Management process which includes Risk Management.

5. Business Reasons (Cont.)
Good Practice
Communication/Develop Relationships
Helps identify misunderstandings and risks
Set or clarify expectations and intentions with regard to documentation & other deliverables
Identify potential gaps in procedures (eg: change control, configuration management) at an early stage, so they can be addressed with minimal impact.
Cost/Schedule/Quality Benefits
Reduce Rework
On-Time Delivery
Aim for “Right First Time”

6. GMP Regulations EU Vol 4 Annex 11 21 CFR Part 210, 211 (Drugs)
21 CFR Part 820 (Medical Devices)
21 CFR Part 11 (Electronic Records & Signatures)
ICH Q7A (Active Pharmaceutical Ingredients)

7. EU Regulations ICH EC Guide to GMP for Medicinal Products
Vol 4, Annex 11
“5. The software is a critical component of a computerised system. The user of such software should take all reasonable steps to ensure that it has been produced in accordance with a system of Quality Assurance.” (emphasis supplied)
ICH
Q7A (API Manufacturing)
Not explicit regarding requirement for supplier audits (ie: Supplier audits/assessments not stated as mandatory)

8. FDA Regulations
Do not explicitly mandate computerised system supplier audits, with possible exception for Medical Devices (21 CFR 820)
But note that for Medical Devices, computer system supplier could be providing components that are part of the finished product.

9. FDA Regulations (Cont.)
21 CFR (a)
“Each manufacturer shall establish and maintain procedures to ensure that all purchased or otherwise received product and services conform to specified requirements. (a) Evaluation of suppliers, contractors, and consultants. Each manufacturer shall establish and maintain the requirements, including quality requirements, that must be met by suppliers, contractors, and consultants.
Each manufacturer shall:
(1) Evaluate and select potential suppliers, contractors, and consultants on the basis of their ability to meet specified requirements, including quality requirements. The evaluation shall be documented. (emphasis supplied)
(2) Define the type and extent of control to be exercised over the product, services, suppliers, contractors, and consultants, based on the evaluation results.
(3) Establish and maintain records of acceptable suppliers, contractors, and consultants. “

10. FDA Warning Letter – Oct 2003
“4. Your firm failed to evaluate and select potential suppliers on the basis of their ability to meet specified requirements, including quality requirements with all evaluations documented as required by 21 CFR (a)(1). You failed to document supplier audits and your audit procedure (PUR-0200) fails to describe supplier audit procedures [FDA 483, Item #7].”

11. FDA Regulations 21 CFR Part 11
Again, not explicit on supplier audits, but implied by the Validation and Training requirements
Need to look to FDA guidance documents

12. FDA Guidelines
For validation, Scope and Application guideline specifically references
FDA’s guidance for industry and FDA staff General Principles of Software Validation (CDRH - Medical Device guidance)
GAMP 4 Guide
Guidance for Industry - Part 11, Electronic Records; Electronic Signatures — Scope and Application
“We recommend that you base your approach on a justified and documented risk assessment and a determination of the potential of the system to affect product quality and safety, and record integrity.”
Convergence in approach to computerised systems across the different regulatory groups (CDER, CBER, CDRH)

13. General Principles of Software Validation; Final Guidance for Industry and FDA Staff (CDRH - Jan 2002)
6.3. VALIDATION OF OFF-THE-SHELF SOFTWARE AND AUTOMATED EQUIPMENT
Where possible and depending upon the device risk involved, the device manufacturer should consider auditing the vendor’s design and development methodologies (empasis supplied) used in the construction of the OTS software and should assess the development and validation documentation generated for the OTS software. Such audits can be conducted by the device manufacturer or by a qualified third party. The audit should demonstrate that the vendor’s procedures for and results of the verification and validation activities performed the OTS software are appropriate and sufficient for the safety and effectiveness requirements of the medical device to be produced using that software.

14. GAMP Guideline Section 7.1 - Determining Validation Strategy
“Suppliers should be formally assessed as part of the process of selecting a supplier and planning for validation. The decision whether to perform a Supplier Audit should be documented and based on a Risk Assessment and categorisation of the system components.”
Also highlights role of end-user in assisting and educating suppliers
Appendix M2 – Audit Guideline and Checklist

15. Other Industry Guidelines
PIC/S : GOOD PRACTICES FOR COMPUTERISED SYSTEMS IN REGULATED “GXP” ENVIRONMENTS (Aug 03)
The assurance of the reliability of a Supplier’s software products is attributable to the quality of the software engineering processes followed during development. This should include design, coding, verification testing, integration, and change control features of the development life cycle, (including after sales support). In order for customers to have confidence in the reliability of the products, they should evaluate the quality methodology of the supplier for the design, construction, supply and maintenance of the software.(emphasis supplied) A formal, extensive review of the history of the Supply Company and the software package may be an option to consider where an additional degree of assurance of the reliability of the software is needed. This should be documented in a Supplier Audit Report.

16. Other Industry Guidelines (Cont.)
PDA TR 32 “Auditing of Suppliers providing Computer Products and Services for Regulated Pharmaceutical Operations”
Includes a very detailed procedure and checklist for supplier audits
Basis of audit procedure for Audit Repository Centre (ARC) shared audits

17. Summary – Reasons to Audit Suppliers
If Predicate Rules do not explicity mandate Supplier Audits, why conduct them?
Business Benefits
Risk Management
Good Practice
Cost/Schedule/Quality
Regulatory expectations
FDA Guidelines & Requirements for System Validation
Implication of Industry Guidelines (GAMP, PIC/S, PDA etc)
Audits are not mandatory but are considered ‘good practice’, and it is for the regulated user to determine any auditing needs, scope and standards. Recommend that the need for supplier audit/assessment be linked to Risk Assessments (and GAMP Categorisation)

18. Summary – Reasons to Audit Suppliers
Organisations are expected to demonstrate control of the processes and systems that affect data integrity, product quality, and patient safety
Quality cannot be inspected or tested into the finished product – it needs to be designed and built in
For software, one method to help achieve this is to follow a formal software development lifecycle (SDLC).
Audit of suppliers helps ensure that a SDLC is in place and is followed.

19. Approach to Supplier Audits

20. Approach to Supplier Audits
Pharma Industry has not yet formally embraced a single standard method for supplier audit, in spite of regulatory expectation to evaluate suppliers as part of the technical acquisition process.
Possible sources
GAMP 4 & VPCS Best Practice Guide
PDA TR32
PIC/S
ISO Guidelines for auditing quality systems Part 1: Auditing
Other industry guidelines
Have procedure/SOP in place, and ensure personnel are trained accordingly.

21. Approach to Supplier Audits (Cont.)

23. Approach to Audits - Initiation
Determine Need for Audit
Based on Risk Assessment
System Categorisation
System Scale and Complexity
Define appropriate Audit Type
Document justification for Audit (or otherwise)
As part of Risk Assessment and/or within Validation Master Plan

24. Risk Assessment (GAMP – App M3)

25. GAMP Software Categorisations

26. GAMP – GLP Good Practice Guide

27. Audit types Postal Audits / Assessments System Audit / Detailed Audit
Surveillance Audit / Monitoring Audit
Joint Audits
Shared Audits

28. Approach to Audits – Planning
End User inputs
Risk Assessment & System Categorisation
Supplier Audit Procedure and Standard Checklist
Validation Master Plan
System Specifications/User Requirements/Project Brief
Supplier Inputs (If available)
Supplier Profile
Organisation Chart
Product/Service Details
Development Methodology
Proposal/Quotation

29. Approach to Audits – Planning (Cont.)
Preparation
Audit Agenda
Audit Specific Checklist
Contact Supplier
Agree date and timescales with Supplier
Copy Supplier with agenda and checklist
Confirm resource availability with Supplier

30. Approach to Audits - Conduct
Typical Agenda
Introductions
Company Overview Presentation (including Quality System Overview/Workflow Methodology)
Office/Facility Tour
Inspection of selected Audit Areas
Include QMS and Procedures, Project Documentation, Maintenance etc
Review Findings/Observations
Present Findings to Supplier

31. Approach to Audits – Checklist/Audit Areas
GAMP
Company Overview
Organisation and Quality Management
Planning and Product/Project Management
Specifications
Implementation
Testing
Completion and Release
Support/Maintenance
Supporting Processes and Activities
PDA TR 32
Quality System
Project Management
Methodology
Testing
Configuration Management
Manufacturing
Document and Records Management
Security
Training and Education
Maintenance

32. Comment on use of checklists
Need to be careful with use of standard checklists
Key is in preparation for audit
Tailor audit criteria and checklist based on supplier product and/or services
Try to ensure that audit criteria is suitably descriptive within checklist
When documenting audit execution
Try to avoid simple yes/no type responses
Include comments as appropriate to elaborate on and explain answers and observations
Include objective evidence, by unambiguous reference, or attachment

33. Approach to Audits – Observations
Communicate observations back to Supplier for response
Consider categorisation of observations (eg: Critical, Major, Minor, Comment only)
Helps to set priorities for actions
Helps justification of decision
Highlight positives also – ie: where supplier meets or exceeds industry best practices

34. Approach to Audits – Report
Audit Report to be issued for approval
Include completed “checklist”, not just observations/findings
Include positive observations also
Formal Response on observations required from Supplier

35. Approach to Audits – Follow Up
Need to ensure close out of observations by supplier
Remember, past performance is a good indicator of future performance (but not a guarantee)
Continuous monitoring is important, particularly for larger projects and customised systems
Use audit as an opportunity to establish open dialog and collaboration with your suppliers

36. Case Studies / Examples

37. Case Study/Example # 1
“Operations” identify need for new automated system
“Projects Group” prepare equipment specification
Projects Group select Vendor/Supplier
Main Criteria Used
Cost (Cheapest!)
Technology
At FAT stage, Projects Group request input from Quality Unit / Validation

38. Case Study/Example # 1 (Cont.)
Assessment/Audit planned & executed
Findings:
No Quality Systems/Procedures in place at supplier
Poor Design Documentation (No Functional Specification, etc)
Poor Change management/configuration management
Planned FAT Testing of limited benefit from Validation perspective – additional testing required.

39. Case Study/Example # 1 (Cont.)
Actions
DQ developed and executed
Develop additional detailed test procedures for FAT, Site testing & qualification
Impact
Project Schedule Impact (Project Delayed)
Cost Impact (Internal costs as well as Supplier)

40. Case Study / Example #2
Project Group include Quality Unit /Validation at project definition phase.
Postal Assessment of all prospective suppliers, followed by conference call with each supplier (ie: Quality issues addressed as part of bid analysis, as well as technical and commercial issues)
Prefered Supplier agreed by all parties (Operations, Projects, Quality)

41. Case Study / Example #2 (cont.)
Detailed systems audit conducted in selected suppliers premises at start of project
Limited/minor issues only raised at audit
On-going monitoring of supplier during project, up to system delivery to site
On time, within budget – no surprises

42. Case Study / Example #3 Replacement of Laboratory Analysers
Categorised as GAMP Category 3 (Standard Software), but systems considered GxP critical
Supplier Audit indicated by company SOP
Issues:
Multiple Suppliers (> 6)
Local Supplier location not the same as System Development location.

43. Case Study / Example #3 (Cont.)
Following consultation with end-user, decided on Postal Audit (GAMP VPCS Good Practice Guide used as template).
Review of responses followed up by conference call with supplier to clarify written replies and request additional information
Reserved right to request “face-to-face” detailed audit, if deemed necessary
Main Benefits - Cost Saving.
Also, get involvement/feedback from system development group, as well as local distribution and support groups.

44. Case Study / Example #4 Manufacturing Control System
Categorised as GAMP 4/5, and system considered GMP Critical
Challenges
Scale of project
Multiple locations involved
Quality Unit an integral part of project process.
Aim to instill a culture of “Right First Time”

45. Case Study / Example #4 (Cont.)
Supplier Audit conducted as part of selection process
Follow-up surveillance audits/reviews during implementation (monthly) to supplement suppliers internal quality group.
Benefits
Identified issues at an earlier stage, and helped minimise impact
Reduced Rework/Retesting on site
Helped ensure overall schedule targets met

46. Shared Audits

47. ARC (Audit Repository Centre)
Centralised repository for audit reports, available to subscribing end-user companies. Not strictly limited to computerised system suppliers.
Statistics (as at June 2004)
Audits Available – 28
Audits Scheduled - 13
Audits under consideration – 18
Links

48. ARC (Cont)
Probably more appropriate as part of corporate system selection, may not be fully suitable for local (smaller) system implementations.
Access to ARC report does not absolve end-user from responsibility – need to analyse audit results and observations, and make decision on supplier acceptability.
For project based systems, will still require on-going monitoring of supplier/implementer

49. Typical Issues / Observations

50. Considerations – Client side
Timing of Audit/Assessment
Scope / Intent of Audit
Supplier Preparation (for Audit)
Follow up on Audit Observations/Findings
Overall business benefits of Quality (and Operations) input at the design and procurement stages

51. Typical Issues – Supplier Side
Training
In-House Quality Procedures
Regulatory Issues/GAMP/21 CFR Part 11
Technical Training
Change Control
Typical focus on cost and schedule impact, lacking definition of re-test requirements and traceability
Configuration Management
Uncertainty as to requirements, Procedures and Baselines not clearly defined

52. Summary and Conclusions

53. Summary
Supplier Audit does not solve all problems, and is not a standalone process, but is a integral part of the overall project management process to help ensure sucessful system implementation
Use Risk Assessment to determine need for Audit/Assessment
Focus on Business Benefits, as well as Regulatory Needs
Improve quality of delivered systems
Schedule Impact/Delivery to Market
Cost Reduction (Minimise Rework)
Teamwork & Partnership
Enhance co-operation between Quality, Projects & Operations
Develop relationship and Improve communication with supplier

54. Questions? Contact: Tom Farmer Mobile: 087-299 5454