ASPIRE Study Procedures and Data Elements, Sources, Uses, and Issues Thomas Delate, PhD, MS Clinical Pharmacy Research Scientist

Learning Objectives ASPIRE Class 3: Study Procedures and Data Elements, Sources, Uses, and Issues Describe basics of a study procedure Illustrate data elements and sources Characterize methods for identifying study patients/subjects, exposures, and outcomes Appraise data limitations and means to overcome limitations Identify factors needed for and to calculate a sample size A study’s procedures section describes how a study is to be conducted. Mostly about data collection and analysis. This includes both exposures and outcomes. I will point out the basics of this process. There are myriad types of data and ways to collect data. From using existing administrative sources to directly collecting information from study patients/participants. Each data collection methodology has limitations that you will need to appreciate and identify ways to address. An IRB is going to want to know an estimate of how many patients/participants will be included and justification for this amount.

Study Procedures: Basic Contents Objectives & the hypotheses underlying them Study design Inclusion & exclusion criteria Sample size estimation Data collection & sources Data analysis These are the minimum contents of a protocol’s procedures section Some investigators prefer to include information on how the collected data will be manipulated. For example, creating quartiles of LDL values to be used as a categorical rather than continuous variable.

Study Procedures: A “How to” Conduct Your Study Cookbook - Be specific and clear about what you will be doing At a minimum, will need to describe: How patients will be identified - by whom and with which data source(s) If necessary - how, where, and by whom patients recruited What, how, and when data will be collected How data will be analyzed My preference is for detailed a step-by-step recipe on how the study is to be conducted. Other investigators prefer to be less specific to maintain an ability to be agile with their methodology. Either way, anticipate “worse-case” scenarios and what you plan to do if one occurs

You Will Need Data For Sample Size/Power Estimation Patient/Participant Identification Apply inclusion/exclusion criteria Recruitment (where applicable) Group Assignment (independent variable) Other Exposure Measurement Patient/Participant descriptions Assess/Control for potential biases and confounders Outcome Measurement (dependent variable) Perform statistical analysis

Types of Data Primary Data – Collected Specifically for Research Purposes Surveys Patient/Caregiver Healthcare Provider/Payor Key Informant Reports Diary Focus Group Interview These data can be qualitative (e.g., describing patient perspectives) or quantitative (e.g., counts of inhaler usage).

Types of Data (cont.) Secondary Data Collection - Initially Collected for Non-Research Purposes Medical Charts Electronic Medical Record Databases Billing (Claims) Databases Registries Government Databases Medical Record Abstraction Internal Administrative Record Electronic Database Electronic Medical Record (EMR) Pharmacy Dispensing Inpatient/Outpatient Claim External Billing Database Purchase from a vendor (e.g., IMS) Registry Disease Management Reportable Disease (e.g., Cancer) Government Database Birth/Death Certificate Population-Based Information (e.g., Medical Expenditure Panel Survey)

Primary vs. Secondary Data Primary Data Can assess both qualitative and quantitative issues Researcher controls how and what data are collected Time-consuming and expensive to collect Potential for incomplete/biased information Secondary Data Broader data than a researcher can collect on her/his own Contains longitudinal information Required information may not be available Requires training/ability and specialized technology to access I

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Identification of Potential Study Patients Primary data Provider recommendation In-clinic recruitment Secondary data Random or Targeted chart review Administrative data query (e-screening) Coronary angiography status from procedures (e.g., claims) or registry database Use ICD-9/ICD-10, Current Procedural Terminology (CPT), or Diagnosis-Related Group (DRG) codes Age, drug benefit, and membership status from membership database Vitamin D measurement information from laboratory database Obtain patient identifiers (e.g., medical record number (MRN))

Secondary Data Collection Sources Electronic Medical Record Pharmacy Claims Registries Describe some data that are available in each example: Dispensing data: return to stock, determine adherence EMR: encounters, lab values Claims: hospitalizations, ER visits, SNF stays Registries: Patients with cancer, diabetes

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Data for Group Assignment Primary data Phlebotomy Provider notification of laboratory values Secondary data Use patient identifiers and chart review for laboratory values Vitamin D information from electronic query of laboratory database Use patient identifiers and Logical Observation Identifiers Names and Codes (LOINC), CPT, or internal laboratory codes Determine quartiles of Vitamin D levels 1st to 25th percentile vs. 75th to 100th percentile

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Exposure Data What are critical exposures to capture? Primary data Are exposures occurring before and/or after group assignment? Primary data Provider/Patient/Caregiver supplied Secondary data Use patient identifiers and chart review for diagnoses, procedures, medications use, and sociodemographic information Use patient identifiers and diagnoses, procedures, medications use, and sociodemographic information for electronic queries of administrative databases Use ICD-9/ICD-10, CPT, DRG, Healthcare Common Procedure Coding System (HCPCS) codes Geocoded Census data

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Outcome Data Primary data Secondary data Provider/Caregiver supplied Use patient identifiers and chart review for mortality and cause of death (COD) information Generally unavailable for patients who terminated membership prior to dying Use patient identifiers to obtain mortality information electronic queries of membership database Use patient identifiers (e.g., SSN, name & address) to obtain mortality and COD information from external source(s) (e.g., SS Administration, state health department, death-record.com)

Data Limitations – Confounding and Bias A systematic error in the design, conduct, or analysis of a study Can be differential (affects only one group) or systematic (affects all groups) Apprehensions about these are common with non-randomized studies Methodologies are available to temper concerns of their impact on a study’s findings

Data Limitations – Selection Bias Can occur with non-randomized assignment to groups Suspected in prospective and retrospective cohort studies when an intervention (e.g., novel pharmacy service) is being investigated Can confound the relationship between group assignment and study outcome For example, sicker/healthier patients may be more likely to receive a medication When randomization is not practical, patient matching can be used to address Propensity score based on likelihood of receiving medication modeled with logistic regression using factors related to intervention Matching of eligible intervention and control patients on propensity score and/or other relevant factors (e.g., age, sex, diagnoses, lab values, seasonality)

Data Limitations – Measurement Bias Can occur when exposure/outcome variable is measured/reported inaccurately For example, suspected with using non-validated questions in a survey, diagnosis codes with low positive predictive ability, non-validated data sources (e.g., laboratory values, infusions, deaths) Can result in the misclassification of study patients on an exposure/outcome For example, mixing up the codes for 25-hydroxyvitamin D & 1,25-dihydroxyvitamin D when extracting laboratory values electronically Attention to detail in the study procedures can prevent this bias Review codes to used carefully, review programming code for errors, chart a review a sample of patients to confirm exposure/outcome, use only validated codes & datasets

Data Limitations – Confounding Can occur when an extraneous variable correlates with both the dependent variable and the independent variable Can result in a mistaken estimate of an exposure’s effect on the risk of disease When randomization is not practical, can address confounding by Specification Exclude patients who exhibit/possess confounding factor (e.g., smokers) Matching Match patients on the confounding factor (e.g., match smokers to smokers) Multivariate regression analysis Adjustment of the effect of the confounding factors Stratification Perform separate analyses on patients who do and do not exhibit/possess confounding factor

Sample Size and Power Sample Size - The number of patients needed to detect a statistically significant difference, if one exists, between groups Power - The probability to detect a statistically significant difference, if one exists, between groups with the sample size available These are used to assess if there are sufficient numbers of patients obtainable to conduct feasibly a study so as to provide conclusive answers to the study questions Conversely will help avoid over enrollment, which can be costly and unethical

Sample Size Estimation Determine the type of response variable you will have Generally, response variables fall into one of three categories: Dichotomous (e.g., yes/no, survive/die) Event rates (%) compared between the intervention and control groups Continuous (e.g., blood pressure, $, cholesterol level) Estimated mean (or medians) in the intervention group compared to the control group Time to failure (e.g., time to death) Hazard ratio is determined between the intervention and control groups Can also assess non-inferiority or other measures of equivalence

Sample Size: Key Concepts Hypothesis Testing Based on the hypothesis you developed with study objectives H0: No difference in event rates between groups (null hypothesis) Null hypothesis assumed to be true until proven otherwise Your goal in conducting the study is to accept or reject the null No difference between groups, you will accept the H0 Difference between groups, you will reject the H0 and accept the alternative (Ha)

Sample Size Estimation Factors included in sample size: Alpha Usually α = 0.05 Beta β=0.20 (1- β = power) Expected Difference Proportional or mean Standard deviation (continuous outcomes only) A priori vs. post-hoc

On-line Sample Size Calculators For Studies: http://www.stat.ubc.ca/~rollin/stats/ssize/ http://hedwig.mgh.harvard.edu/sample_size/size.html For Surveys: http://www.surveysystem.com/sscalc.htm Try this out for your study