Querying Data Anomalies: What ,Why & How? Anne Hoehn Data Manager

What Are Data Anomalies? Inconsistent or inaccurate data fields within a form: Or across one (or more) forms: Discrepancies are a part of clinical trials – catching them is the key! Data anomalies, or discrepancies, are inconsistent or inaccurate data fields within a form or across forms As hard as we try to avoid them, data discrepancies are inherent part of any clinical trial Need to find reliable ways of identifying discrepancies - Queries let us identify discrepancies remotely and inform sites of issues

What Are Data Queries? Identify data discrepancies Supplement on-screen, or front end, checks Skip logic Ranges Detect discrepancies after data entered Use more complicated programming Data queries, or edit checks, identify data discrepancies Queries supplement on-screen, or front end, checks Skip logic – Screen shot, example Ranges – Screen shot, example Unlike on-screen checks, queries detect discrepancies after the data have been entered Since queries are generated through the back end of the EDC system, they can use more complicated programming Complicated: Uses multiple fields and/or forms, etc…..

What Are Data Queries? Data queries vs. on-screen checks On-screen checks may be cleaner but aren’t always practical Use data queries when: Skip logic would be overly complicated Query utilizes two or more forms Use on-screen checks when: Data must be entered correctly the first time Participant self-report On-screen, front end, in line (consistency checks) may be cleaner but aren’t always practical Front-end cross checks can be bulky and complicated Can increase form load and save time if multiple forms are pulled in or if many fields are affected If one form is updated, “downstream” forms are not automatically updated. Site needs to open and re-save “downstream” forms (in which case a query should be implemented to make sure re-save occurs) Excessive on-screen checks may overwhelm the user Use data queries when: Skip logic would be overly complicated (e.g., multiple “parents”) Query utilizes two or more forms inconsistencies are expected (e.g., self-report*, when data must match source document) Use on-screen checks when: Data must be entered correctly the first time (e.g., responses affect enrollment criteria) Participant self-report: when visits few and far between. Information is unreliable when asking someone to recall data from many weeks or many months prior

Why Implement Data Queries? Reporting accurate results is crucial but accurate results are only possible with accurate data Tracking missing data isn’t always enough Catch discrepancies Track changes to discrepant data Data queries have other functions Reminders to complete additional forms Reporting accurate results is crucial but accurate results are only possible with accurate data Data validity and integrity are crucial issues in clinical trials Tracking missing value and missing form exceptions isn’t always enough Queries catch data inconsistencies, not just missing data Queries track changes to discrepant data Queries help tell the full story of data changes Important to know why data were changed Data queries have other functions Reminders to complete of additional forms Explain “triggesr” and “visit schedule” Queries implemented when programming on visit schedule would be too complicated Participant reported attending an HIV care visit in the AUS form; however, a corresponding AUH form has not been submitted. Please complete the AUH form.

How to Implement Data Queries Identifying data queries Creating data checks Distributing queries Tracking data queries Resolving data queries

Identifying Data Queries Involve the Data Manager, Protocol Monitor, Statistician and other key staff Thoroughly review the Protocol, Manual of Procedures and other study documents Critically review all CRFs Look for similar or related questions Consider places the coordinator may get confused Involve the Data Manager, Protocol Monitor, Statistician and other key staff involved in implementing the protocol Each position brings unique perspective Look for similar or related questions (e.g., participant is not of childbearing potential but pregnancy test completed) -remember during development that collecting more data is not always better Confusing: Though we try to keep questions simple & straight forward, there are always areas of potential confusion May help to have “fresh eyes” review workflow and CRFs

Standard Queries Assessments completed with defined window Visits completed in sequential order AEs reported in protocol-specified timeframe Eligible pre-screen enrolled Pre-populated enrollment criteria not updated Con meds and AEs are not ‘ongoing’ after termination Once standard queries are developed, they can often be implemented across multiple protocols with minimal updates Skip over if too long (will be in handout)

Activity: Identifying Data Queries Brief Symptom Inventory

Activity: Identifying Data Queries Retinal Vein Occlusion (RVA) diagnosis

Activity: Identifying Data Queries Retinal Vein Occlusion (RVA) diagnosis

Writing Data Checks Key components: Participant identifier Relevant “key fields” (e.g., visit number, medication name, AE description) Discrepant fields Other useful information e.g., if a participant randomized more than 30 days after enrollment, provide the “lag” time Error code Important to think about the end user Include everything the site needs to understand/resolve the query But it’s just as important to remove superfluous information that could confuse the user or muddle the information provided

Writing Data Checks Error codes are important Provide first instructions to the site Clearly and concisely state actions required by the site Is a study deviation/violation required? Are additional documents requested? According to the study MOP, IOP must be measured using the same method at each visit. More than one tonometery method has been used for this participant. Review and reconcile. If the information submitted is correct, submit a Study Departure (SDP) form in EDC. The incorrect "Validation date code" has been recorded. To receive the correct code, send a copy of the source document to STUDY@emmes.com. (Note that all PHI must be removed from the source document prior to sending.) -The clearer the code, the more easily the site can interpret the information If requesting a study deviation/violation, state the source (e.g., protocol, MOP) Also let sites know they’re expected If requesting additional documents, provide the e-mail address

Distributing Data Queries Posted within EDC Distributed via paper PDF EDC: Less paperwork but more programming to create a query management system Paper PDF: A little more straightforward Both achieve the desired effect of remote identification and review

Resolving Data Discrepancies Provide training Clear instructions in the EDC manual Live trainings Recorded trainings (e.g., webinars) Follow up Queries can be confusing! Expect a lot of questions when query process is implemented and when new queries are developed The queries have been created…now what? Provide training Provide clear instructions in the EDC manual Remember that training is ongoing Consider both live training (e.g., national training) and recorded webinar We’ve found that sites like having a quick, easy reference (5 minutes or less) – reduces calls to us Consider taping a webinar so sites can revisit the instructions Follow up Queries can be confusing! Expect a lot of questions when query process is implemented and when new queries are developed example

Resolving Data Discrepancies Resolving “incorrect as submitted” data Most data entry errors can be resolved by the site Be careful: Make sure sites understand data should change only when entered incorrectly Most data entry errors can be resolved by the site As long as you provide clear error codes so sites understand the problematic data and expectations to correct it Correct data Be careful: Make sure sites understand data should change only when entered incorrectly Use clear words like “review” instead of “update”

Resolving Data Discrepancies Not all discrepancies are data entry errors Communicate when data can’t be changed Resolving “correct as submitted” data via EDC Resolving “correct as submitted” data Some discrepancies can’t be resolved Decide which inconsistencies are acceptable Participant self-report data often can’t be reconciled Maintain good documentation (emails, minutes, etc.) Keep a decision log so similar issues can be resolved in the same manner Determine whether a study deviation/violation is required Not all discrepancies are data entry errors Involve DM, PM, PS, Stat, PI in decision-making process

Resolving Data Discrepancies Resolving “correct as submitted” data via PDF When using paper trail, need to hard code removal into database

Resolving Data Discrepancies Send reminders Monthly data quality email Send reminders Utilize automated reminders if possible But follow up when necessary (e.g., delinquent sites, compicated data quality issues) Consider a monthly email with outstanding queries Include a due date Notify sites when queries have been developed Inform sites when new queries have been implemented or when queries have been modified Review new queries or changes on weekly site calls It may be helpful to provide examples (either on site call or in email)

Tracking Data Queries Data query status Resolution Issued Exception requested (e.g., “verified”) Exception granted Exception denied Resolved (due to data change) Resolution Supporting documentation Consider keeping a decision log Identified – Issued or Not Issued Issued – Resolved (data change) or Verified Verified – Granted or Denied (request inappropriate, insufficient information)

Data Query Limitations Queries are not generated automatically If using EDC, expect a 1 to 3 day lag time On-screen checks may still be the best practice Queries don’t eliminate need for manual checks Review of free text fields for PHI May not be able to import extremely complex programming Recognize limitations of data queries On-screen checks will still be the best practice for certain situations (repeat) May not be able to import extremely complicated checks into EDC

Conclusions Discrepancies are a part of clinical trials – catching them is the key! Data cleaning is ongoing process Leverage online technology Keep on top of query review Catch issues early to prevent reoccurrences Continue site dialogue Try to be as efficient as possible and keep data as clean as possible – most efficient way is often through remote monitoring. Also prevent issues from occurring or re-occurring. Continue site dialogue – make sure they understand queries and are working to resolve them in timely manner