Concerns About Adverse Event Reporting in Randomized Clinical Trials Yusuf Yazıcı, MD NYU Hospital for Joint Diseases

Overview Completeness of safety reporting is inadequate Methodological concerns Reporting selective results Errors in reporting 1- Time to event 2- SIR 3- Adequate sample size 4- False impression of safety Ioannidis J, Lau J. JAMA 2001

Time to AE Problems in reporting AE in RCT Time to event  Clue to causality  When to expect AE in clinical care Instantaneous risk might be different for drugs but the cumulative risk will be similar “Average risk” does not reflect the true risk faced by patients, especially when decisions are being made at the beginning of treatment

Time to AE reporting in RCT All RCT of COX-2 inhibitors and TNF inhibitors  91% industry sponsored  1/3 gave time to AE in the report either as a table, text or Kaplan-Meier curve  No better reporting for SAE 8/17 RA TNF trials reporting malignancy used SEER database

Time to AE and SAE

SEER database Based on annual incidence in population, which is expected to be evenly distributed If there is some skewing as to time of malignancies in the treatment arm, this would make comparisons based on an annual incidence inaccurate If timing of a malignancy is not evenly distributed then we shouldn’t be using a yearly rate to compare incidence rates  SEER=12 expected lymphomas in RA patients over 12 months, 1 per month  TNF=12 seen in RCT, oConclusion is that there is no difference

SEER database (2) However  TNF lymphoma o8 in the first 4 months = 2/month o4 in the last 8 months = 0.5/month Does not carry the same message for the practicing physician Variable period comparison needs to be used  2,3,4,6 monthly rates Excluding events in the first days  Not done with other AE  No strong evidence base for doing so

TNF and lymphomas Brown et al, A&R 2002 TNF use and lymphoma development, as reported to FDA 26 lymphomas  14 occurred within 2 months of use NHL  2 monthly incidence in normal population 3/100,000  10/16 occurred in 2 months  2 monthly NHL occurrence in etanercept users=10/100,000 oNearly 4 times Yazici H, A&R 2003

TNF inhibitors and lymphoma An increased frequency of lymphomas, tuberculosis and demyelinating CNS disease have been associated with TNF – α antagonist use. In the case of lymphoma development,  using annual incidence risks in the general population as comparator is inappropriate  development of such lymphomas was within the few months after drug initiation in more than 50% of these patients (1) The expected random occurrence of lymphomas in the general population would make the real risk look smaller (2). (1) Brown SL et al. Arthritis Rheum 2002; (2) Yazici Y, Yazici H. Arthritis Rheum 2004 Suppl.

Demyelinating CNS disease Mohan et al. TNF-α antagonist use and demyelinating CNS disease.  9 cases among 77,152 users of etanercept during a 19 month period.  They noted that this frequency was not different than that found in the general population of 4-6/ per annum. Half of the cases within 4 months of etanercept use,  4 monthly incidence risk of etanercept use of 5.8/100, monthly risk in the general population on the other hand would be 2.0/100,000, if the upper limit, 6.0/100,000, quoted for the general population is used.

Coccidioidomycosis and TNF Bergstrom et al. from Tucson, USA, reported (A&R 2004) about the increased risk of coccidioidomycosis in patients treated with TNF-α antagonists. They documented 13 cases of coccidioidomycosis associated with TNF therapy. Again 7/13 cases developed within 2 months of starting TNF therapy.

TNF and CNS, TB The annual incidence risks Because more than half of these cases occur during first months of therapy, quarterly or monthly incidence risks in the normal population should be used for comparison. Increased incidence risks for TNF–α antagonist related adverse events would necessitate a re-look at the screening protocols and patient selection criteria used for these therapies. Finally in the case of lymphomas this approach may also point to a decrease of incidence risk later in disease, consistent with less lymphoma incidence in RA as inflammation is better controlled * * Baecklund E et al. Arthritis Rheum 2003; 48:

Infection and Neoplasia with Anti-TNF Agents

Neoplasia Median: 19 weeks

Time to Neoplasia Courtesy of H Yazici, “Based on” T Bongartz et al JAMA 2006

Infections

Claims database 20 months follow up period Hospital records reviewed Compared TNF based treatment vs. MTX (or other DMARD-based) treatment 187/217 identified was reviewed  TNF=65 in 2393 exposed persons (2.7%)  DMARD=58 in 2933 exposed persons (2.0%)  Number needed to harm=143  Median time from exposure to medication to infection was <30 days for both groups  93% occurred in 90 days Multivariable adjustment ~2 times in TNF vs MTX

Increased early risk In the first 6 months  32 in TNF  19 in DMARD Adjusted hazard ratio of infection in the first 6 months after start of treatment  TNF compared to MTX controls was 4.2 (95% CI ) Similar to JAMA paper

“Statistics are like swimwear – what they reveal is suggestive but what they conceal is vital.” Ashish Mahajan, Lancet 2007

Patient years Commonly used in RCT: “patient-years” to define the time frame of AE incidence. 6 manuscripts were found in the survey. Relatively rare idiosyncratic drug reactions usually occur early in the treatment course and in only a few individuals. Apart from the few with AEs, remaining patients who are prescribed the drug will never get these reactions however long they use the drug. Unduly inflate the denominator of the related incidence ratio; potential of under-representing AE.  Late onset AEs are also apt to be missed when we use patient-years. In short, only “…when an event is (or is believed to be) likely to occur at any stage during continuous treatment with a drug then an event rate with a time component (rate per person year, etc) has a true meaning.” Paterson KR. BMJ. 1995; 310: 1470

Patient years (2) Not clear whether more than one event per patient goes into the numerator. If more than one AE/patient is included in a numerator, statistics done with that incidence ratio will be erroneous. An AE can repeat itself, like skin rashes in TNF-α antagonist use in any one patient. This leads to over-representation of the said individual in tests of significance.

“Efficacy and safety” RCT Trend of increasing number of RCT called “efficacy and safety” trials. All RCT of TNF inhibitors in RA, PsA and AS. Only original reports  “Efficacy”, “safety” or “efficacy and safety”  Power for safety and efficacy  Type II error Power  Alpha  Beta  Expected treatment effects in both arms Yazici Y, Adler NM, Yazici H. ACR 2007

“Efficacy”, “safety” and “efficacy and safety”

Powering for safety analysis 24/34 (71%) called “efficacy and safety” trials 5 called such in the title 2/24 so called “efficacy and safety” trials were powered to look at safety Of the remaining 22, only 3 (14%) included any consideration of type II error as a reason for lack of noted harm Of the total 34, only 5 discussed type II error 9/34 (28%) did not give any power/sample size calculation for efficacy

Before and after 2004

RCTs

“Efficacy and safety” (continued) Designation RCT as “safety” as well as “efficacy” has the potential to give false impressions that lack of AE increase is evidence of overall safety of the medication tested Except 1, none of these trials were powered to look at safety In addition, RCT are highly selective for lack of major comorbidities  So even if they were powered to look at safety, results need to be interpreted with caution  Phase 4 trials RCT of course report on AE but these efficacy trials cannot provide any conclusive evidence about safety and these shortcomings should be reported, and required by journals and reviewers

Conclusions Time to adverse effect oCausation oWhat to expect for the clinician Patient years oFalsely give sense of experience oRare adverse effects hard to analyze SEER RCT provide information about efficacy and not safety. They report on AE that happen during a defined time in a small, selected population “A conclusion is the place where you got tired of thinking” —Arthur Block