1. Clinical Event Classification: Strategies and Practices
Mark A Hlatky, MD Stanford University School of Medicine September 27, 2018

2. Outline Is clinical event classification necessary?
Experience in WHI study
Possible implications
What are best practices when it is done?
Some take-aways from this meeting

3. Why classify events? What does a CEC add?
Independent, blinded adjudication of events increases credibility of study results
Standardized data collection and definitions of key outcome events
Satisfying regulatory requirements for new drug approval
Cost, time, and complexity to RCTs
Should test value added by CEC process

4. Comparison of CEC and claims
Pragmatic clinical trials may use existing data collection processes to simplify studies
The Women’s Health Initiative (WHI), a large NIH RCT, linked the trial database to CMS claims
Opportunity to evaluate the accuracy of hospital claims data in assessing CV outcomes: MI, CABG, and PCI
Circ Cardiovasc Qual Outcomes 2014;7:

5. Methods Women ≥65 yo at WHI enrollment
WHI MI = best available adjudication (central > local)
CMS MI = ICD.9 code of 410.x1 (acute MI, initial admission)
Censored at end WHI follow-up, loss of Medicare Part A, or HMO enrollment
Training set = not in hormone RCTs
Application to hormone RCTs
Circ Cardiovasc Qual Outcomes 2014;7:

6. Definition of MI WHI process Participant reported a possible event
Records requested and reviewed by CEC
MI Dx by ECG, biomarkers, symptoms
MI categorized as “resulting from a procedure”
CMS Process
1º = principal reason for admission,
2º = conditions also present
ICD9 Codes: 410 = acute MI, 411 = other acute CHD, 412 = old MI, = angina, 414 = other chronic CHD
Circ Cardiovasc Qual Outcomes 2014;7:

7. CMS MI vs. WHI MI in Training Set
37,397 women ≥65 yo w CMS data
1,345 WHI MI cases (172 procedure related)
1,195 CMS MI cases (1º dx)
306 more with acute MI as 2º dx
__ WHI MI_ Kappa
Yes No
CMS (1º) Yes
No ,771
CMS (1º, 2º) Yes ,
No ,613
Circ Cardiovasc Qual Outcomes 2014;7:157-16

8. Reasons for CMS Yes/WHI No
281 with CMS MI = 1º MI dx, no WHI MI
No event reported to WHI (49%)
Inadequate documentation—no records (5%) or key data missing (9%)
Adjudicated, but
No CHD outcomes (16%)
Another CVD outcome (23%)
CHD deaths w/o MI (2%)
Circ Cardiovasc Qual Outcomes 2014;7:157-16

9. Reasons for CMS No/WHI Yes
431 with WHI MI, no CMS MI (1º diagnosis)
No CMS admission <30 days (15%)
CMS 2º dx of MI (32%)
WHI MI procedure related (13%)
Other CVD 1º dx (39%)
Non-CVD 1º dx (15%)
Circ Cardiovasc Qual Outcomes 2014;7:157-16

10. Intention to treat comparison

11. Inights from comparing CEC with Claims
Claims data identified events not reported by WHI participants
Agreement between claims and CEC data was very good for MI, and excellent for PCI and CABG (kappa 0.88 to 0.91)
Some disagreements due to:
No or incomplete records for adjudication
Differences in diagnostic criteria applied by CEC adjudicators vs local physicians
MIs due to procedures or non-cardiac illness were often missed in claims

12. Some reflections on best practices
CEC is more valuable when study intervention is applied during hospitalization for acute illness or procedures
More difficult to identify MI in this setting
CEC more important for drugs in development than for comparative effectiveness studies of well-established drugs
Regulatory requirements to meet

13. More reflections on best practices
Application of clear, standardized diagnostic criteria for events decreases variation
Interobserver variation is more often due to different dx criteria than to variable application of standardized dx criteria
Standardizing CEC processes, however, could impede innovation
Could test importance of key processes (e.g. meetings vs independent evaluations)
CEC might be applied to a random sample to validate process (e.g. claims) for all events