1. How to Write Good Test Cases The who, what, why, and how of testing your clinical applications. Exceptional Presented By: Henry Sabia Senior Account Manager 877.765.0100 x 408 hsabia@sts-healthcare.com sts-healthcare.com Good morning/afternoon, Please use the phone for audio. 614-715-3660 #300038

2. VALIDATION Why do I have to write test cases?

3. What is validation? It’s purpose is to ensure the computerized system specifications, requirements, and functionality guidelines. Validation ensures all components of an information system function as intended in your environment! Specifications Requirements Functionality

4. Validation Plan Does the system achieve the business goals? Does the system satisfy the end user’s requirements? Does the integrated system behave as per the design? Do individual software modules behave as per the design? A model of dynamic computerized system testing and validation

5. Components of a Computerized System Computerized System Software Software Version System Build System Parameters Interface Data HardwareServers Connected Devices Policies & Procedures Regulatory Guidelines Not just the components, but also the HOW you use them!

6. When should I validate? When software programs are implemented or changed: Application Upgrades and Fixes Interfaces Operating System Changes When changes are made to: User defined tables or dictionaries Peripheral devices

7. What needs to be validated? Process Critical Control Points A function, or area in a process or procedure, that failure, or loss of control, may have an adverse effect on the quality of the finished product and may result in health risk. Order Entry Order Processing Resulting Product Entry Unit Processing Transfusion Storage Interface Transactions Label Printing Report Printing Etc.

8. Why do I have to validate my LIS software?

9. Good Business Practice Quality Management MonitoringObjectivesTestingAnalysisProcessesImprovementProceduresEffectivenessAuditControlMeasures Testing is one piece of a Quality Management System Test Case Development and Deployment affects: Objectives Testing Analysis Effectiveness Audit Measures Monitoring Without a quality management process or system, the clinical application is an at risk system.

10. Regulation hurdles! All regulatory bodies have requirements associated with validation of clinical computer systems when installed or modified. On top of the resource crunch, there are more and more regulations to contend with, each with different requirements.

11. Cost Implications 1 10 100 The rule explains how failure to take notice of one cost escalates the loss in terms of dollars. There are many costs of non-quality such as: (1) prevention, (2) appraisal, (3) internal failure, and (4) external failure. Of these types of costs, prevention cost should probably take priority because it is much less costly to prevent a defect than to correct one. 1-10-100 Rule of Total Quality Management $1 Prevention = $100 Failure Cost

13. If the product has a FDA 510(k) Clearance, do I really have to validate? The 510k processes: Documents the vendor’s software development processes. FDA reviews submitted documents FDA does not test the software 510k does NOT equal Validation!

14. Do I have to validate if SCC Soft Quality Control has validated the software? SCC Soft confirms basic functionality. You validate site parameters, maintenance, and operational workflow based on your SOPs. You need to validate local installation, peripherals, and communications.

15. Why Validate SoftLab? The SCC Soft system is flexible, every site is different! It operates within an environment associated with multiple variables. The computerized system must be validated in the environment where it will be used - with all system variables included in the validation.

16. The How of Validation 1.Know your Validation Policy 2.Perform a System Assessment 3.Develop your Risk Assessment 4.Create the Validation Plan 5.Scenario Development 6.Write Test Cases 7.Review and Approve Test Cases 7 Steps to Validation

17. Validation Plan A Validation Plan is simply an outline of what will be validated Specific to each validation Reason for the validation - Upgrade Scope – Full system, chemistry calculations High risk control points identified Identifies the approval authority and reviewers Reason for validationScope of validationRisk AssessmentTest MethodDocumentationTraining PlanReview and Approval Email Henry for more information!

18. Perform a System Assessment SCC SoftBank examples: Inventory/Delivery Patient & Unit Retype Patient Entry Blood History Order Processing Product Change Transfusion Break the system into functional/task oriented units Break down each functional unit to determine all critical control points Determine result variables for the critical control points DeliveryAttributesCMVN

19. Perform a System Assessment… Create a document of the assessment information – a spreadsheet is great! This working document is useful for creating scenarios and test cases You can also use it to confirm that all conditions are thoroughly tested

20. Know your Validation Policy Broad policy governing all validation activities – Organizational Policy Use as a reference for Test Case development Document Format requirements Document Content requirements Definitions Objectives Screen captures

21. Develop your Risk Assessment Answers the question of “how much testing” is needed by measuring the severity of clinical and business risk associated with a specific process or task Each computer function has one or more “control points” where data is entered and/or accepted Measured analysis of the risk associated with a failure for each control point Points deemed to have the highest risk should have the highest level of validation. Evaluate the control points identified in your system assessment. Assign a numeric weight to each control point. The greater the risk of the activity to patient outcome, the greater the risk value assigned

22. Using the Risk Assessment Used to determine the level of validation Examples of high risk tasks that may require more rigorous validation include: New or re-designed features Calculations Auto-file implementation Electronic Crossmatch Helps you guarantee that you thoroughly tested the critical points of failure within your system Use to highlight areas of greater risk for more rigorous testing

23. Scenario Development Now you know why, let’s talk about what your test cases might look like.

24. Anatomy of a Test Case Description Test Script FunctionsRelative Data Review Criteria ObjectivesConditions Tested Outcomes Data/Screen CapturePass/Fail Data

25. Test Description Enter the Valid Result for Crossmatch. Say what you are testing and how you will test it Include the process you are testing Include a description of your basic workflow Example: Validate Ordering and Resulting Type & Screen and Crossmatch on a Patient. Enter an order for a Type and Screen, and Result the ABO/RH and Antibody Screen with Valid Results. Not only will a clear description assist with documentations and inspections, it will also help you review the test case for future use.

26. Test Script StepFunctionAction 1RegisterRegister a patient 2Order EntryEnter an order for a type and screen [TSC] and a red cell product for patient. 3Patient – Result PatientProcess type and screen order entering reaction results for an O Negative blood type and a negative antibody screen. 4Inventory DeliveryEnter a unit of O Negative red cells. 5Results – Unit WorksheetPerform product testing for unit. Enter blood type results for O Negative. 6Product SelectAllocate Unit to order. 7Patient – Result UnitProcess order. Enter compatible results for the immediate spin crossmatch. 8Product IssueIssue unit for order. The script lists each step and actions along the workflow.

27. Review/Success Criteria Detail out what outcomes are needed for each step to be successful StepActionReview Criteria 1Register a patientA patient is registered 2Enter an order for a type and screen [TSC] and a red cell product for patient. An order for a TSC is created An order for red cells is created 3Process type and screen order entering reaction results for an O Negative blood type and a negative antibody screen. The order is resulted as O Negative Antibody screen is Negative 4Enter a unit of O Negative red cells.The O-Neg unit is Delivered and Unresulted 5Perform product testing for unit. Enter blood type results for O Negative. Unit is retyped as O-Negative 6Allocate Unit to order.Unit status is Selected 7Process order. Enter compatible results for the immediate spin crossmatch. Crossmatch is resulted as Negative 8Issue unit for order.Unit is issued and status is Issued

28. The STS Version Test Description Test Script Review Criteria

29. Test Case Approval Test Cases should be reviewed and approved before they are used for validation. The spreadsheet will be useful to the approval authority to verify: Control points validated Risk level of each control point Level of validation of each control point. Once created, test cases can be stored for future use.

30. Test Outcomes To prove completed testing, we recommend you document and collect the following: Outcome Data ScriptCriteria Relative Data Screen Capture Pass/Fail Status

31. Documentation Testing Outcome Report (Proof of testing) Pass/Fail Report (Include signatures) Test Case Index (Cross-reference to locate specific task validation) Inspections / Assessments Validation Policy System & Risk Assessments Validation Plan Documentation serves many functions. Consider all aspects of documentation when generating your final documents.

32. Proof of Testing Report Example Test Description Test Script Review Criteria Captured Data Comprehensive testing data in an easy to follow report.

33. Pass/Fail Report Example Test Description Reviewers Information Signatures A summary report with signature spaces for pre-live approvals

34. Testing Index Examples Testing Condition Test Cases that included that condition An index makes referencing past test cases easy!

35. Value of Good Validation System System Errors Patient Risk Data Validity Data Integrity System Stability Peripherals A thorough validation… Reduces the likelihood of system errors resulting in harm to patients Mitigates damages if an error does result in patient harm Assures that patient information and billing will be correct. A good validation should result in a smoother live event for the entire enterprise!

36. Value of Good Validation Great Documentation impresses inspectors and assessors….. So your Inspection, Assessment or Survey is a breeze…. And…… Y OU L OOK G REAT !!

37. KEEPCALMANDAUTOMATEONSTS SOFTWARE TESTING SOLUTIONS