ASPIRE Class 4 Study Procedures and Data Elements, Sources, Uses, and Issues Thomas Delate, PhD, MS Clinical Pharmacy Research Scientist

Learning Objectives ASPIRE Class 3: Study Procedures and Data Elements, Sources, Uses, and Issues Describe basics of a study procedure Illustrate data elements and sources Characterize methods for identifying study patients/subjects, exposures, and outcomes Appraise data limitations and means to overcome limitations Identify factors needed for and to calculate a sample size

Elements of a Research Protocol Background Population Study Design Objectives Procedures Analytical Plan

Study Procedures: General Overview Cook book - a “how to” Be very specific and clear How patients will be identified, by who, at what time point, and through what data source What, how, and when data will be collected Anticipate “worse-case” scenarios and what you plan to do if one occurs

Study Procedures: Contents Objectives & their hypotheses Study design Inclusion & exclusion criteria Sample size estimation Data collection & sources Data manipulation Data analysis

What You Need Data For Sample size/Power estimation Study subject/patient identification Apply inclusion/exclusion criteria Recruitment (where applicable) Group Assignment (independent variable) Exposure(s) measurement Describe subjects/patients Assess/Control for potential biases and confounders Outcome(s) measurement (dependent variable) Perform statistical analysis

Types of Data Primary Data – Collected Specifically for Research Purposes Surveys Patient/Caregiver Healthcare Provider/Payor Patient/Caregiver/Provider Reports Diary Focus Group Interview I

Types of Data (cont.) Secondary Data Collection - Initially Collected for Non-Research Purposes Medical Charts Internal Administrative Record Electronic Databases External/Commercial Billing Databases Registries Government Databases Medical Record Abstraction Internal Administrative Record Electronic Database Electronic Medical Record (EMR) Pharmacy Dispensing Inpatient/Outpatient Claim External Billing Database Purchase from a vendor (e.g., IMS) Registry Disease Management Reportable Disease (e.g., Cancer) Government Database Birth/Death Certificate Population-Based Information (e.g., Medical Expenditure Panel Survey)

Primary vs. Secondary Data Primary Data Can assess both qualitative and quantitative issues Researcher controls how and what data are collected Time-consuming and expensive to collect Potential for incomplete/biased information Secondary Data Potentially more and higher-quality data than an individual researcher can collect on his/her own More amenable to assessing change(s) over time Requires permission, training/ability, specialized computer/software to access Can be overwhelming to manipulate large datasets I

Secondary Data Collection Sources Electronic Medical Record Pharmacy Claims Registries Describe some data that are available in each example

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Identification of Potential Study Patients Primary data Provider recommendation In-clinic recruitment Secondary data Random or Targeted chart review Administrative data query (aka e-screening) Coronary angiography status from procedures (e.g., claims) or registry database Use International Classification of Diseases, Ninth Revision (ICD-9), Current Procedural Terminology (CPT), or Diagnosis-Related Group (DRG) codes Age, drug benefit, and membership status from membership database Vitamin D measurement information from laboratory database Obtain patient identifiers (e.g., medical record number (MRN))

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Data for Group Assignment Primary data Phlebotomy Provider notification of laboratory values Secondary data Use patient identifiers and chart review for laboratory values Vitamin D information from electronic query of laboratory database Use patient identifiers and Logical Observation Identifiers Names and Codes (LOINC), CPT, or internal laboratory codes Determine quartiles of Vitamin D levels 1st to 25th percentile vs. 75th to 100th percentile

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Exposure Data What are critical exposures to capture? Primary data Are exposures occurring before and/or after group assignment? Primary data Provider/Patient/Caregiver supplied Secondary data Use patient identifiers and chart review for diagnoses, procedures, medications use, and sociodemographic information Use patient identifiers and diagnoses, procedures, medications use, and sociodemographic information for electronic queries of administrative databases Use ICD-9, CPT, DRG, Healthcare Common Procedure Coding System (HCPCS) codes Geocoded Census data

Data Needs for Prospective Cohort Study Study Objective: To determine if endogenous 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D levels are associated with all-cause and cardiovascular mortality Patient Identification Inclusion 1) Age >=18 years 2) Undergoing coronary angiography 3) Vitamin D level drawn at time of procedure 4) 180-day membership prior to procedure 5) Prescription drug benefit Group Assignment Outcomes: All-Cause Mortality Cardiovascular Mortality Study Outcomes Lower Vit D Quartiles Follow-up What is good about prospective cohort study is that you can ensure that all patients have required labs drawn, in this instance, Vit D levels before follow-up Cohort studies evaluate ‘associations’ but can do little to assess ‘causality’, Causality can really only be assessed with experimental studies. Also, cohort studies are useful if you are determining risks where randomization may not be ethical. Exposures: Demographics Potential Confounders Upper Vit D Quartiles Pre-Period

Outcome Data Primary data Secondary data Provider/Caregiver supplied Use patient identifiers and chart review for mortality and cause of death (COD) information Generally unavailable for patients who terminated membership prior to dying Use patient identifiers to obtain mortality information electronic queries of membership database Use patient identifiers (e.g., SSN, name & address) to obtain mortality and COD information from external source(s) (e.g., SS Administration, state health department, death-record.com)

Data Limitations – Confounding and Bias A systematic error in the design, conduct, or analysis of a study Can be differential (affects only one group) or non-differential (affects all groups) Apprehensions about these are common with non-randomized studies Methodologies are available to temper concerns of their impact on a study’s findings

Data Limitations – Selection Bias Can occur with non-randomized assignment to groups Suspected in prospective and retrospective cohort studies when an intervention (e.g., novel pharmacy service) is being investigated Can confound the relationship between group assignment and study outcome For example, sicker/healthier patients may be more likely to receive the intervention When randomization is not practical, patient matching can be used to address Propensity score based on likelihood of receiving intervention modeled with logistic regression using factors related to intervention Matching of eligible intervention and control patients on propensity score and other relevant factors (e.g., age, sex, diagnoses, lab values, seasonality)

Data Limitations – Measurement Bias Can occur when exposure/outcome variable is measured/reported inaccurately For example, suspected with using non-validated questions in a survey, diagnosis codes with low positive predictive ability, non-validated data sources (e.g., laboratory values, infusions, deaths) Can result in the misclassification of study patients on an exposure/outcome For example, mixing up the codes for 25-hydroxyvitamin D & 1,25-dihydroxyvitamin D when extracting laboratory values electronically Attention to detail in the study procedures can prevent this bias Review codes to used carefully, review programming code for errors, chart a review a sample of patients to confirm exposure/outcome, use only validated codes & datasets

Data Limitations – Confounding Can occur when an extraneous variable correlates with both the dependent variable and the independent variable Can result in a mistaken estimate of an exposure’s effect on the risk of disease When randomization is not practical, can address confounding by Specification Exclude patients who exhibit/possess confounding factor (e.g., smokers) Matching Match patients on the confounding factor (e.g., match smokers to smokers) Multivariate regression analysis Adjustment of the effect of the confounding factors Stratification Perform separate analyses on patients who do and do not exhibit/possess confounding factor

Sample Size and Power Sample Size - The number of patients needed to detect a statistically significant difference, if one exists, between groups Power - The probability to detect a statistically significant difference, if one exists, between groups with the sample size available These are used to assess if there are sufficient numbers of patients obtainable to conduct feasibly a study so as to provide conclusive answers to the study questions Can help avoid over enrollment, which can be costly and unethical

Sample Size Estimation Determine the type of response variable you will have Generally, response variables fall into one of three categories: Dichotomous (e.g., yes/no, survive/die) Event rates (%) compared between the intervention and control groups Continuous (e.g., blood pressure, $, cholesterol level) Estimated mean (or medians) in the intervention group compared to the control group Time to failure (e.g., time to death) Hazard ratio is determined between the intervention and control groups

Sample Size: Key Concepts Hypothesis Testing Based on the hypothesis you developed with study objectives H0: No difference in event rates between groups (null hypothesis) Null hypothesis assumed to be true until proven otherwise Your goal in conducting the study is to accept or reject the null No difference between groups, you will accept the H0 Difference between groups, you will reject the H0 and accept the alternative (Ha)

Population Truth No real difference (H0 is true) Real differnce (Ha is true) Study Findings Accept H0 Correct (1-α) No agreement Type II Error Reject H0 Type I error (1-β) ‘power’ You don’t know if you have an error when you conduct your study. You are attempting to minimize your chance of Type 1 and ii errors by setting up specific parameters by which you can then determine the sample size required.

Sample Size Estimation Factors included in sample size: Alpha Usually α = 0.05 Beta β=0.20 (1- β = power) Expected Difference Proportional or mean Standard deviation (continuous outcomes only) A priori vs. post-hoc

On-line Sample Size Calculators For Studies: http://www.stat.ubc.ca/~rollin/stats/ssize/ http://hedwig.mgh.harvard.edu/sample_size/size.html For Surveys: http://www.surveysystem.com/sscalc.htm Try this out for your study

Class 4 Assignment Draft study methods 2: List data needed to identify and describe your patients and measure outcomes List the source of each data element Identify the elements needed to calculate the sample size/power for your primary outcome Try using one of the on-line calculators for your study Please come prepared with the above items September 15th, 2:30-5:00 p.m. @ Kaiser Permanente Central Support Services