1. Center Specific Outcomes Reporting
Does a 3 year OS metric reflect additional value?
Douglas Rizzo, MD MS
October 20, 2016

2. Overview Why do we do center outcomes reporting? Methods
Reporting Results
Results for 2016
Limitations
Engaging the HCT community
Future Work

3. How do we maintain engagement of the HCT community?
CIBMTR Center Outcomes Forum

4. What is the purpose?
Engage the relevant stakeholders in meaningful discourse about the process and with each other regarding uses and expectations
Transparency and accountability
Acquire meaningful input on statistical methodology, risk adjustment methodology, relevant data collection, meaningful display of results, appropriate use and avoiding misuse, adaptation to future trends in quality reporting.

5. Key Question 4:
Are there new measures of quality not currently reported publicly by the CIBMTR which should be included in future iterations of the center-specific survival analysis (with risk adjustment) on behalf of the HCT community?

6. KQ4: New quality measures?
Develop and test a 3 year risk adjusted overall survival measure – pilot for center use
Work in progress: pilot testing underway for 2015 Report (HCT )
Assess feasibility of collecting and reporting Patient Reported Outcomes
QOL pilot analysis in final manuscript phase
Exploring tools by which CIBMTR and centers can collect these data
EMR or PROMIS (NIH)

7. Center Outcomes Report
2016

8. Center Outcomes Report Final study population - 2016
Centers must have >90% overall f/u at 1 year
2 centers closed or became inactive
179 US centers; 23,004 patients first allo HCT
Primary outcome: One year survival
Overall: 68.9% (72.5% REL, 66.3% UNR)
Center outcomes report 2016 includes 3 full years of data:
Unrelated and Related HCT 2012 – 2014
Multivariate analysis adjusts for ‘risk factors’

9. Limitations - 2016 Only outcome is 1 year survival
Only one outcome, only one year
Balances HCT center control, transplant approach/type of regimen, preferred long-term outcome desired by patient/society
Is not sufficiently ‘real-time’
2016 report, includes HCT 2012 – 2014
Report issued annually
May not ‘sufficiently’ adjust for risk factors associated with income/ SES
Balance challenges and benefits of data collection

10. Significant Risk Factors
Patient:
Race of recipient
Recipient Age*
Recipient CMV status
Year of HCT
Karnofsky/Lansky perf. Score*
HCT-CI
Disease (con’t)
Disease and stage*
NHL subtype
Disease sensitivity (NHL and HL only)
Time from dx to tx (ALL and AML not in CR1/PIF only)
Transplant:
Donor type/graft type and HLA
Donor Age
Donor/recipient sex match
Prior autoHCT
Conditioning regimen intensity

11. Preliminary Results
3 year OS outcome

12. 3 year OS modeling
Uses data available for the 2013 Center specific analysis – HCT years

13. Center Outcomes Report Final study population - 2013
Centers must have >90% overall f/u at 1 year
One center excluded in 2013 for incomplete reporting of allogeneic HCT
Most centers have ≥ 99% y
168 centers; 19,958 patients first allo HCT
Primary outcome: One year survival
Overall: 65.7% (71% REL, 62% UNR)
Center outcomes report 2013 includes 3 full years of data:
Unrelated and Related HCT 2009 – 2011

14. 3 year OS modeling
Uses data available for the 2013 Center specific analysis – HCT years
When restricted to completeness of 80% follow-up at 3 years
Loss of 5 centers

15. Significant Risk Factors – 3 y OS
Patient:
Race of recipient
Recipient Age
Recipient CMV status
Year of HCT
Karnofsky/Lansky perf. Score
HCT-CI
Disease (con’t)
Disease and stage
NHL subtype
Disease sensitivity (NHL and HL only)
Time from dx to tx (ALL and AML not in CR1/PIF only)
Transplant:
Donor type/graft type and HLA
Donor Age
Donor/recipient sex match
Prior autoHCT
Conditioning regimen intensity

16. OS at 3 year by 1 year

17. Comparison by Z scores

18. Center Performance 1 vs 3 years
All centers regardless of follow-up
151 (89%) no change
5 (3%) improve
13 (8%) decline
Table of p1yr by p3yr
p1yr(1 yr model)
p3yr(3 yr model)
Frequency -1 1
Total 19 3 22 8
124 2
134 5 13 27
132 10
169

19. Center Performance 1 vs 3 years
Centers with at least 80% follow-up
5 centers eliminated
146 (89%) no change
5 (3%) improve
13 (8%) decline
Table of p1yr by p3yr
p1yr(1 yr model)
p3yr(3 yr model)
Frequency -1 1
Total 18 3 21 8
120 2
130 5 13 26
128 10
164

20. Preliminary conclusions
3 year follow-up represents a challenge for some centers
Factors associated with 1 year and 3 year OS are similar
Center performance for 1y OS and 3 y OS are similar for 90% of centers

21. What are the limitations?

22. Limitations – 3y OS metric
Does not address ‘value’ ($) (beyond outcome)
Significant delay between years of HCT and the analysis
Centers must follow patients for at least 3 years after the HCT
Currently, inadequate follow-up at centers
Patients do not always have access to the HCT center in later years after HCT

23. Discussion:
Would the addition of a risk adjusted 3 y OS metric add value?
To whom?
Are there suggestions for what would make this more valuable?

24. Potential Benefits
Could stimulate better virtual tools to enhance long term patient care/access to transplant center

25. Learn more at:

26. Limitations
Some Adult and pediatric programs at centers are combined
Autologous HCT are NOT included
Full representation of transplanted patients essential
Conveying complex data to the non-statistician
Misunderstandings & misrepresentation
Unintended consequences
Not intended to directly compare centers
May inappropriately affect patient selection for HCT
May stifle investigational approaches
Remains unclear whether public reporting of outcomes stimulates improvements in outcomes

27. Where do we go from here?

28. Where are we headed next ? FACT/JACIE
Standard B June 2015:
Allogeneic requirements
The clinical program should achieve ..within or above the expected range when compared to national or international outcome data (CIBMTR, BSBMT, SBST)
If not met ….. Corrective Action Plan
Future Standards… 2018
? Expectations ?

30. Extra slides

31. What about missing data?
Incomplete follow-up compromises outcomes
Require minimum of 90% (or more) completeness of follow-up at the center
Censored data logistic regression model
Special case: Incomplete follow-up but complete reporting of those deceased (easier)
Incomplete data puts centers at risk!
Missing data for covariates
If adequate number, model “missing” as a category
Small numbers of missing data can be imputed