1. Validating one-shots with a multi shot hand-gun
Robert Snijder
Astellas, Leiderdorp

2. The issue
Today guidelines are mainly about system validation and give limited to no information about the validation of a one-shot program.
Our daily work is the creation of tables, listing or figures or the creation of derived datasets. In many cases there is just one program connected to one or a few pieces of output.
This presentation will focus on the “translation” of system validation guidelines to our daily practice 2

3. The multi shots
General Principles of Software Validation; Final Guidance for Industry and FDA Staff
GAMP (Good Automated Manufacturing Practice)
21 CFR part 11
ACDM/PSI Computerized systems validation in clinical research

4. About 21 CFR Part 11 Hand-written signature 100’s years of law
The question that 21 CFR part 11 intent to answer:
A law for ‘computerized’ signatures
Consistency across EU member states
US law
Makes data legally reputable
The issue of 21 CFR part 11:
Written by lawyers, for lawyers
No specific technology discussed
21 CFR part 11 is about source data

5. Introduction of Astellas EU validation process
Validation process at Astellas EU is based on the principles known as Software Development Life Cycle (SDLC)
More specific: the methodology used is based on the ACDM/PSI Computerised systems validation in clinical research and then the part of “validation methodologies for one-off programs”

6. Basic idea (1/4) Validation answers three vital questions:
What is the program supposed to do?
What does the program actually do?
Are they the same thing
To answer these questions one needs complete knowledge of
Requirements
Data
Programs

7. Basic idea (2/4) Cycle Validation is more then testing alone
Get a life…
Cycle

8. Basic idea (3/4) The specific tasks;
It specifies for each phase;
The specific tasks;
Methods and procedures for each task;
Task acceptance criteria;
Criteria for defining and documenting outputs in terms that will allow evaluation of their conformance to input requirements;
Inputs for each task;
Outputs from each task;
Roles, resources, and responsibilities for each task;
Documentation of user needs.

9. Basic idea (4/4)
If nothing is documented then nothing is done!!

10. Life cycle Maintenance Change request Requirements Design UAT
Implementation

11. Study team Including stats
Or waterfall
Change
control
Requirements
Statistician
Clinical
programmers
Design &
planning
Version control
Code
development
& review
Acceptance
testing
Study team Including stats
Validation
administrator
Maintenance
Retirement

12. What do the guidelines say
SOFTWARE LIFE CYCLE
Software validation takes place within the environment of an established software life cycle. The software life cycle contains software engineering tasks and documentation necessary to support the software validation effort. In addition, the software life cycle contains specific verification and validation tasks that are appropriate for the intended use of the software.

13. Change control
Documented decision that starts or restarts the life cycle
In “our” case this could be:
The intention to start a study
Outcome of a study team meeting (they want additional tables)
An with a request to change a table

14. What do the guidelines say
REQUIREMENTS
A documented software requirements specification provides a baseline for both validation and verification. The software validation process cannot be completed without an established software requirements specification (Ref: 21 CFR 820.3(z) and (aa) and (f) and (g)).
Astellas EU  For most of our programs, the Statistical Analysis Plan (SAP) in combination with a Tables Manual (TM) contain the requirements. Where this is not the case, the requirements of the program are documented separately. The SAP and TM are created by a statistician.

15. Requirements The system intended function
A contract between the user and the developer
NOT HOW
A lot of documentation already exist like:
Analysis plan
SOP
Table layout standards
Standard tables

16. Requirements Test 20 X Programming 10 X Design 5 X Requirements 1 X
If the error is found and fixed in this phase
Additional cost
Delivery
200 X

17. What do the guidelines say
PLANING
The software validation process is defined and controlled through the use of a plan. The software validation plan defines "what" is to be accomplished through the software validation effort. Software validation plans are a significant quality system tool. Software validation plans specify areas such as scope, approach, resources, schedules and the types and extent of activities, tasks, and work items.

18. Planning and design documents
For every program it can be indicated who will create it and who will review it
Documentation of the review method that will be used(for example: double programming, code review etc.)
Documentation of the need for design documents
If a design is not created the reason why is not documented
Be aware of the environment you use
Assure that the environment is well described and documented
Environment include
OC version
Platform
Directory structure
Read and write rights of directory structure
Protection against accidental changes

19. Design
HOW is the question

21. Difference between requirements and design
The baby alarm example
Requirements for Baby alarm could be as simple as:
It should warn me when the baby is crying, it should be silent when the baby is asleep.
It should work wireless
The design could be:

22. Difference between Requirements or Design

23. Implementation
In Astellas-EU the implementation is the actual programming and the QC

24. Implementation - Coding
Things to consider in programming:
Naming conventions
Modular code (ie macro’s)
Programming conventions
Indention and capitalization
Programming efficiency
Error prevention (for example by: simple, easy to understand programming tat is used by all programmers)
Clarifying notes and clear headers
Defensive programming

25. Implementation QC QC should be according validation plan
Testing should be based upon requirements and design
Internal QC performed by independent programmer
There are in general two methods
Black box (only test the output of the program)
White box (check the source code or in -between steps as well)
Document errors/findings
Make use of checklist
Use (signed off) checklists
Make use of SAS itself as a QC tool (especially proc freq and log files)
QCer creates log of errors/findings

26. What does the guidelines say
INDEPENDENCE OF REVIEW
Validation activities should be conducted using the basic quality assurance precept of "independence of review." Self-validation is extremely difficult. When possible, an independent evaluation is always better, especially for higher risk applications. Some firms contract out for a third-party independent verification and validation, but this solution may not always be feasible. Another approach is to assign internal staff members that are not involved in a particular design or its implementation, but who have sufficient knowledge to evaluate the project and conduct the verification and validation activities. Smaller firms may need to be creative in how tasks are organized and assigned in order to maintain internal independence of review.

27. What do the guidelines say
DEFECT PREVENTION
Software quality assurance needs to focus on preventing the introduction of defects into the software development process and not on trying to "test quality into" the software code after it is written. Software testing is very limited in its ability to surface all latent defects in software code. For example, the complexity of most software prevents it from being exhaustively tested. Software testing is a necessary activity. However, in most cases software testing by itself is not sufficient to establish confidence that the software is fit for its intended use. In order to establish that confidence, software developers should use a mixture of methods and techniques to prevent software errors and to detect software errors that do occur. The "best mix" of methods depends on many factors including the development environment, application, size of project, language, and risk.

29. User acceptance testing
End users test if their requirements are met
End users are for example statisticians, study managers, medical writers etc.
Documented ‘end of review’ (UAT sign off)
Programs are now production environment approved
Version number update

31. Installation
What has changed in going from the test environment to the production environment?
After internal QC the output will be distributed to users. Is this part of testing or part of Installation/acceptance?
Archiving of documentation
Protect programs for changes

32. Installation Procedures Assure backup/recovery
Arrange proper security of final programs: assures that programs/logs/output cannot be modified anymore
QA audits by validation administration
checks completeness of documentation, signatures
Checks date/time stamps
In case of new requests or serious errors: new program will be created (following SDLC from the start)

33. What does the guideline say
SOFTWARE VALIDATION AFTER A CHANGE
Due to the complexity of software, a seemingly small local change may have a significant global system impact. When any change (even a small change) is made to the software, the validation status of the software needs to be re- established. Whenever software is changed, a validation analysis should be conducted not just for validation of the individual change, but also to determine the extent and impact of that change on the entire software system. Based on this analysis, the software developer should then conduct an appropriate level of software regression testing to show that unchanged but vulnerable portions of the system have not been adversely affected. Design controls and appropriate regression testing provide the confidence that the software is validated after a software change.

34. Retirement In case projects are ended the programs will be retired.
Retirement should assure that the programs are stored sufficiently long.
Platforms, SAS version used, should remain available

36. What do the guidelines say
TIME AND EFFORT
To build a case that the software is validated requires time and effort. Preparation for software validation should begin early, i.e., during design and development planning and design input. The final conclusion that the software is validated should be based on evidence collected from planned efforts conducted throughout the software lifecycle
Astellas experience learns that time and effort spend in the early phase of the life cycle prevents time loss at a later stage. As a consequence: since we use the SDLC methodology we MUCH are quicker then before (with higher quality).

37. Conclusion
Although there is no specific guideline for one shot programs, still many of the existing guidance can be used
At start intensive validation looks time consuming but at the end saves time
Validation plans and requirement documents can save a lot of discussion afterwards
Although the principles of validation can be very simple the implementation can become very complex. 37