1. A consolidated review of multiple analyses using JMP Clinical
Can Statistical monitoring really improve data integrity?
A consolidated review of multiple analyses using JMP Clinical
Chris Wells
Abstract
Method
Results
What is ‘Data Integrity’?
It is said that it is a fundamental component of information security but what is the meaning of this? Broadly, it refers to the accuracy and consistency of data stored in a database over its entire lifecycle but it may have widely different meanings depending on the specific context.
Within the context of a clinical trial, it refers to data being accurate, complete and unique to the associated ‘living’ patient.
Data Integrity is needed to protect the well-being of study participants and maintain the integrity of the final analysis results, so that patients, regulators and all interested parties can have belief in the data
What is the opposite of ‘Data Integrity’?
The opposite of data integrity is data corruption, which is a form of data loss. Data that is recorded incorrectly results in a loss of data for that patient or falsified data results in bias of the trial results.
We have many methods for reviewing the impact of ‘missing data’ but whilst methods exist to review the impact of data corruption, they are not readily utilised within the clinical trial setting, therefore it is important to be vigilant and apply statistical monitoring of the data in order to keep data corruption/loss to a minimum.
To review the overall findings from 20 clinical studies using JMP Clinical and report on the level of findings and how they contributed to enhancing the overall data integrity.
Firstly the demographic distributions are reviewed to graphically understand the data snapshot.
Next, in the Data Quality and Fraud tab of the JMP Clinical Tool, there are 9 further tests that can be run.
All of the 10 tests were run, with the addition of further looking at patients who have no Adverse Events reported plus further graphical representations of multivariate inliers/outliers
Based on 20 studies, with median number of patients = 549 (min:max (22;3080) and median number of sites = 99 (min;max 4; 353), we highlighted 32 pairs of possible duplicate patients of whom 18 warranted further investigation.
1 study was not included in further analysis due to low number of patients and lack of data/findings
The following results were found in a small number of sites within each study.
14/19 studies displayed evidence of possible carryover or calibration issues, 10/19 studies displayed evidence of possible estimation of a minor parameter
9/19 studies displayed evidence of rounding error, whilst 12/19 appeared to show duplication of results across 3 or more parameters between 2 or more patients
All studies had data issues flagged that would have been queried but 12 studies were identified as having unusual data points that may not normally be flagged by clinical data management, including 6 studies where patterns were observed in the multivariate space.
7/19 studies (3394 patients) were reviewed and it was found that 499 patients (14.9%) had not had adverse events recorded. This could be due to several reasons including lag time at site or length of time in study, as well as other reasons that are worthy of investigation
Challenges
Conclusion
Used Version 5.0 JMP Clinical, on a Citrix Server. Difficulty in applying Holidays/Events in a standard way due to current computing environment.
SV dataset not used in-house (Statistical Programmers derive a visit dataset, therefore SV datasets are often of poor quality)
No training courses available initially so difficulty understanding available options within the tool and complex understanding of interpretation of outputs.
We are still working on the ideal model for implementation of the task. Also we are eagerly awaiting IT to finish their validation of version 6.
There are still challenges in interpreting output and feeding back to study teams in a way that issues can be addressed.
This exercise has enabled a valuable insight into the condition of our clinical data. Overall it appears that our data is of high quality but it has enabled us to identify priority areas for investment of time/training or process changes to further enhance the already high data integrity that currently exists.
Objective