ASPIRE Workshop 4: Study Procedures and Data Elements, Sources, Uses, and Issues Thomas Delate, PhD, MS

Outline Example study data needs Review source(s) for data elements Identify the elements needed to calculate the sample size/power for 1o outcome Sample size calculation example Review sources of and options for addressing bias

Example Study Review Study Question: Among patients with DM eligible for statin therapy, does an intervention involving a letter, a pre-ordered statin prescription, and pharmacist counseling increase statin initiation compared to no intervention? 1o Outcome: Compare statin-start rate (i.e., purchase of a statin RX within 3 months after mailing date) between groups 2o Outcomes: Compare statin persistence rate (i.e., statin purchase 1 year after mailing date +/- 45 days) between groups Compare abnormal CK (>600) or ALT (>200) rate (i.e., at least one abnormal lab result within 6 months after mailing date) between groups Study Design: Quasi-experimental

Example Study Review Inclusion Criteria: KPCO member Age 40-80 years Diabetes diagnosis Total Cholesterol > 135 mg/dL No statin purchase in past year Exclusion Criteria: ALT > 60 or SCr > 2.0 in past year CK > 2x ULN in past 2 years Fibrate or cyclosporine Rx purchase w/in past 90 days

Data Sources Report of patients from the electronic health record (EHR) who have diabetes Membership in EHR (date of birth, membership status) Laboratory results from EHR (TC, ALT, SCr, CK) Pharmacy dispensings from EHR (statins, fibrates, cyclosporine)

Study Sample Size How many letters need to be sent to assess whether the intervention works? Need to know: Estimated % of statin starts without intervention Estimated % of statin starts with intervention (difference or ratio = effect size) Desired power (1-β) Significance level (α)

Study Sample Size Estimated % of statin starts without intervention: 10% Estimated % of statin starts with intervention: 20% Desired power (1-β): 80% Significance level (α): 0.05

Study Sample Size Elements http://www.select-statistics.co.uk/sample-size-calculator-two-proportions

What happens to the required sample size as the group difference increases? The required sample size increases The required sample size decreases There is no association between the required sample size and the reference proportion It is data dependent on whether the sample size increases or decreases Answer: B

What happens to the required sample size as the group difference increases? The required sample size increases The required sample size decreases There is no association between the required sample size and the reference proportion It is data dependent on whether the sample size increases or decreases Answer: B

What is the relationship between effect size and required sample size, with power and alpha remaining constant? The larger the effect size, the larger the required sample size The larger the effect size, the smaller the required sample size There is no association between the effect size and sample size It is data dependent whether the sample size increases or decreases Answer: B August 26, 2018 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.

What is the relationship between effect size and required sample size, with power and alpha remaining constant? The larger the effect size, the larger the required sample size The larger the effect size, the smaller the required sample size There is no association between the effect size and sample size It is data dependent whether the sample size increases or decreases Answer: B August 26, 2018 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.

Addressing Potential Study Bias With a quasi-experimental (i.e., non-randomized) study design, groups may be non-equivalent Sicker/healthier -or- more motivated/less motivated patients are over represented in one group (selection bias) Factors affecting both the exposure and outcome are present (confounding) Exposures/outcomes are not measured accurately (misclassification) How can we tell if groups are non-equivalent and if so, how do we address non-equivalence?

Addressing Potential Selection Bias Determine if patients who received the intervention were the similar to those who did not Calculate comorbidity/disease burden scores from historical patients and compare these values between clinics Compare patient characteristics between days of the week If not similar 1) Use clinics where patients are similar 2) Match patients on characteristic(s) 3) Stratified analysis Example: Patients residing in lower down-town may be sicker than those in suburban Highlands Ranch

Addressing Potential Confounding Assess (based on literature, provider input, own knowledge) what factors could affect the 1o outcome and compare between groups during analysis phase For example Distance from pharmacy Age Insulin use Use these factors in multivariate regression(s)

Addressing Potential Misclassification Can outcomes, characteristics, potential confounders be measured and are they accurate? Medication purchase from a non-network pharmacy Lab result not present Lab measurement technique changed so values are reported differently over time Need valid and reliable data sources for both groups

http://www.ASPIREKPCO.WEEBLY.COM August 26, 2018 | © 2011 Kaiser Foundation Health Plan, Inc. For internal use only.