Center Specific Outcomes Reporting Does a 3 year OS metric reflect additional value? Douglas Rizzo, MD MS October 20, 2016

Overview Why do we do center outcomes reporting? Methods Reporting Results Results for 2016 Limitations Engaging the HCT community Future Work

How do we maintain engagement of the HCT community? CIBMTR Center Outcomes Forum

What is the purpose? Engage the relevant stakeholders in meaningful discourse about the process and with each other regarding uses and expectations Transparency and accountability Acquire meaningful input on statistical methodology, risk adjustment methodology, relevant data collection, meaningful display of results, appropriate use and avoiding misuse, adaptation to future trends in quality reporting.

Key Question 4: Are there new measures of quality not currently reported publicly by the CIBMTR which should be included in future iterations of the center-specific survival analysis (with risk adjustment) on behalf of the HCT community?

KQ4: New quality measures? Develop and test a 3 year risk adjusted overall survival measure – pilot for center use Work in progress: pilot testing underway for 2015 Report (HCT 2009-2013) Assess feasibility of collecting and reporting Patient Reported Outcomes QOL pilot analysis in final manuscript phase Exploring tools by which CIBMTR and centers can collect these data EMR or PROMIS (NIH)

Center Outcomes Report 2016

Center Outcomes Report Final study population - 2016 Centers must have >90% overall f/u at 1 year 2 centers closed or became inactive 179 US centers; 23,004 patients first allo HCT Primary outcome: One year survival Overall: 68.9% (72.5% REL, 66.3% UNR) Center outcomes report 2016 includes 3 full years of data: Unrelated and Related HCT 2012 – 2014 Multivariate analysis adjusts for ‘risk factors’

Limitations - 2016 Only outcome is 1 year survival Only one outcome, only one year Balances HCT center control, transplant approach/type of regimen, preferred long-term outcome desired by patient/society Is not sufficiently ‘real-time’ 2016 report, includes HCT 2012 – 2014 Report issued annually May not ‘sufficiently’ adjust for risk factors associated with income/ SES Balance challenges and benefits of data collection

Significant Risk Factors Patient: Race of recipient Recipient Age* Recipient CMV status Year of HCT Karnofsky/Lansky perf. Score* HCT-CI Disease (con’t) Disease and stage* NHL subtype Disease sensitivity (NHL and HL only) Time from dx to tx (ALL and AML not in CR1/PIF only) Transplant: Donor type/graft type and HLA Donor Age Donor/recipient sex match Prior autoHCT Conditioning regimen intensity

Preliminary Results 3 year OS outcome

3 year OS modeling Uses data available for the 2013 Center specific analysis – HCT years 2009-2011

Center Outcomes Report Final study population - 2013 Centers must have >90% overall f/u at 1 year One center excluded in 2013 for incomplete reporting of allogeneic HCT Most centers have ≥ 99% follow-up @1 y 168 centers; 19,958 patients first allo HCT Primary outcome: One year survival Overall: 65.7% (71% REL, 62% UNR) Center outcomes report 2013 includes 3 full years of data: Unrelated and Related HCT 2009 – 2011

3 year OS modeling Uses data available for the 2013 Center specific analysis – HCT years 2009-2011 When restricted to completeness of 80% follow-up at 3 years Loss of 5 centers

Significant Risk Factors – 3 y OS Patient: Race of recipient Recipient Age Recipient CMV status Year of HCT Karnofsky/Lansky perf. Score HCT-CI Disease (con’t) Disease and stage NHL subtype Disease sensitivity (NHL and HL only) Time from dx to tx (ALL and AML not in CR1/PIF only) Transplant: Donor type/graft type and HLA Donor Age Donor/recipient sex match Prior autoHCT Conditioning regimen intensity

OS at 3 year by 1 year

Comparison by Z scores

Center Performance 1 vs 3 years All centers regardless of follow-up 151 (89%) no change 5 (3%) improve 13 (8%) decline Table of p1yr by p3yr p1yr(1 yr model) p3yr(3 yr model) Frequency -1 1 Total 19 3 22 8 124 2 134 5 13 27 132 10 169

Center Performance 1 vs 3 years Centers with at least 80% follow-up 5 centers eliminated 146 (89%) no change 5 (3%) improve 13 (8%) decline Table of p1yr by p3yr p1yr(1 yr model) p3yr(3 yr model) Frequency -1 1 Total 18 3 21 8 120 2 130 5 13 26 128 10 164

Preliminary conclusions 3 year follow-up represents a challenge for some centers Factors associated with 1 year and 3 year OS are similar Center performance for 1y OS and 3 y OS are similar for 90% of centers

What are the limitations?

Limitations – 3y OS metric Does not address ‘value’ ($) (beyond outcome) Significant delay between years of HCT and the analysis Centers must follow patients for at least 3 years after the HCT Currently, inadequate follow-up at centers Patients do not always have access to the HCT center in later years after HCT

Discussion: Would the addition of a risk adjusted 3 y OS metric add value? To whom? Are there suggestions for what would make this more valuable?

Potential Benefits Could stimulate better virtual tools to enhance long term patient care/access to transplant center

Learn more at: http://www.cibmtr.org/Meetings/Materials/CSOAForum/Pages/index.aspx http://www.cibmtr.org/ReferenceCenter/SlidesReports/USStats/Documents/CIBMTR_HCT_Center_Survival_Report_Methodology.pdf

Limitations - 2016 Some Adult and pediatric programs at centers are combined Autologous HCT are NOT included Full representation of transplanted patients essential Conveying complex data to the non-statistician Misunderstandings & misrepresentation Unintended consequences Not intended to directly compare centers May inappropriately affect patient selection for HCT May stifle investigational approaches Remains unclear whether public reporting of outcomes stimulates improvements in outcomes

Where do we go from here?

Where are we headed next ? FACT/JACIE Standard B4.7.5 - June 2015: Allogeneic requirements The clinical program should achieve ..within or above the expected range when compared to national or international outcome data (CIBMTR, BSBMT, SBST) If not met ….. Corrective Action Plan Future Standards… 2018 ? Expectations ?

Extra slides

What about missing data? Incomplete follow-up compromises outcomes Require minimum of 90% (or more) completeness of follow-up at the center Censored data logistic regression model Special case: Incomplete follow-up but complete reporting of those deceased (easier) Incomplete data puts centers at risk! Missing data for covariates If adequate number, model “missing” as a category Small numbers of missing data can be imputed