1. 1 Randomized Clinical Trials in Rheumatology: Reporting Safety and Outcome Measures Yusuf Yazıcı, MD Assistant Professor of Medicine, NYU School of Medicine Director, Seligman Center for Advanced Therapeutics NYU Hospital for Joint Diseases

2. 2 Overview RCT trials are the leading source of data in determining treatment choices Good source of efficacy data Completeness of safety reporting is inadequate –Methodological concerns –Reporting selective results –Errors in reporting Focus on: –Time to event –Standardized incidence ratios (SIR) –Radiographic data reporting –Adequate sample size –False impression of safety –Disease activity assessment measures Ioannidis J, Lau J. JAMA 2001;285:437–443.

3. Time to adverse event 3

4. 4 Problem in reporting AEs in RCT Time to event –Clue to causality –When to expect AEs in clinical care Instantaneous risk might be different for drugs, but the cumulative risk will be similar ‘Average risk’ does not reflect the true risk faced by patients, especially when decisions are being made at the beginning of treatment

5. 5 Risk of adverse effects related to drug exposure Collet JP, Bolvin JF. In: Pharmacoepidemiology, 3 rd Edn, Strom BL ed. 2000.

6. 6 TNF inhibitors and lymphomas An increased frequency of lymphomas, tuberculosis and demyelinating CNS disease has been associated TNF inhibitor use SEER database Based on annual incidence in the population, which is expected to be evenly distributed If there is some skewing as to time of malignancies in the treatment arm, this would make comparisons based on an annual incidence inaccurate 1.Brown SL, et al. Arthritis Rheum 2002;3151–3158. 2.Yazici Y, Yazici H. Arthritis Rheum 2004; 50: S302

7. 7 TNF inhibitors and lymphomas TNF inhibitor use and lymphoma development, as reported to the Federal Drug Administration (FDA) 1 Twenty-six lymphomas –Fourteen occurred within 2 months of use Non-Hodgkin’s lymphoma (NHL) –Two-monthly incidence in normal population 3/100,000 –Ten out of 16 occurred in 2 months 2 –Two-monthly NHL occurrence in etanercept users =10/100,000 Nearly 3-4 times 1. Brown SL, et al. Arthritis Rheum 2002;46:3151–3158; 2. Yazici H. Arthritis Rheum 2003;48:2389.

8. 8 Demyelinating central nervous system disease Tumor necrosis factor-α antagonist use and demyelinating CNS disease –Nine cases among 77,152 users of etanercept during a 19-month period –They noted that this frequency was not different from that found in the general population – 4–6/100,000 per annum Half of the cases occurred within 4 months of etanercept use –Four-monthly incidence risk of etanercept use of 5.8/100,000 Four-monthly risk in the general population on the other hand would be 2.0/100,000, if the upper limit, 6.0/100,000, quoted for the general population is used Mohan N, et al. Arthritis Rheum 2001;44:2862–2869.

9. 9 Coccidioidomycosis and TNF inhibitors Risk of coccidioidomycosis in patients treated with TNF-α antagonists Thirteen cases of coccidioidomycosis associated with TNF-α therapy were reported Again seven out of 13 cases developed within 2 months of starting TNF-α therapy Bergstrom L, et al. Arthritis Rheum 2004;50:1959–1966.

10. 10 Summary of malignancies in randomized controlled trials Median: 19 weeks Bongartz T, et al. JAMA 2006;295:2275–2285.

11. 11 Time to neoplasia 0 2 4 6 8 10 12 14 0–1213–2425–3637–4849–6061–7272+ Weeks Number of neoplasms Based on data from Bongartz T, et al. JAMA 2006;295:2275–2285, courtesy of Dr H Yazici

12. 12 Infections Curtis JR, et al. Arthritis Rheum 2007;56:1125–1133.

13. 13 Infections Hospital medical records with claims database reviewed –Median follow up of 17 months Tumor necrosis factor-based treatment compared with MTX (or other DMARD-based) treatment Reviewed 187 out of 217 suspected bacterial infections –TNF-α antagonist =65 in 2393 exposed persons (2.7%) –DMARD =58 in 2933 exposed persons (2.0%) –Number needed to harm =143 –Median time from exposure to medication to infection was <30 days for both groups –93% occurred in 90 days Multivariable adjustment approximately two times in TNF-α versus MTX Curtis JR, et al. Arthritis Rheum 2007; 56:1125–1133.

14. 14 Increased early risk In the first 6 months –Thirty-two infections in TNF- α cohort –Nineteen infections in DMARD cohort Adjusted hazard ratio of infection in the first 6 months after start of treatment –TNF- α compared with MTX controls was 4.2 (95% confidence intervals [CI] 2.0-88.8) Similar to JAMA paper Curtis JR, et al. Arthritis Rheum 2007; 56:1125–1133.

15. 15 Surveillance, epidemiology and end-results database (SEER) If timing of a malignancy is not evenly distributed, we should not be using a yearly rate to compare incidence rates –SEER = 12 expected lymphomas in RA patients over 12 months, 1 per month –TNF- α = 12 seen in RCT Conclusion is that there is no difference TNF- α lymphoma – Eight in the first 4 months = 2/month – Four in the last 8 months = 0.5/month In the case of lymphoma development –Using annual incidence risk in the general population as a comparator is inappropriate –Development of such lymphomas was within the few months after drug initiation in more than 50% of these patients 1 The expected random occurrence of lymphomas in the general population would make the real risk look smaller 2

16. 16 Tumor necrosis factor, time to AE As more than half of these cases occur during the first months of therapy, quarterly or monthly (rather than annual) incidence risks in the normal population should be used for comparison Increased incidence risks for TNF-α antagonist-related adverse events (AEs) would necessitate a reexamination of the screening protocols and patient selection criteria used for these therapies Does not carry the same message for the practicing physician Variable period comparison needs to be used –2-, 3-, 4-, 6-monthly rates Excluding events in the first 60–180 days –Not done with other AEs –No strong evidence base for doing so

17. 17 Time to adverse event reporting in randomized controlled trials All RCT of cyclo-oxygenase-2 (COX-2) inhibitors and TNF-α inhibitors –Industry-sponsored RCTs (91%) –One-third gave time to AE in the report either as a table, text or Kaplan-Meier curve –No better reporting for serious adverse event (SAE) Eight out of 17 RA–TNF-α trials reporting malignancy used the Surveillance, Epidemiology and End Results (SEER) database Yazici Y, Yazici H. Ann Rheum Dis 2007;66:124–127.

18. 18 Time to adverse event and serious adverse event COX-2 (26 studies)TNF-α (44 studies)Total (70 studies) Number of patients enrolled Mean1541251- Median602112- Range67–807620–1049- Duration of RCT Mean number of weeks1243- Median624- Range4–1562–52- n (%) AEs (n and % reported)19 (73)36 (82)55 (79) Table given for AEs21 (81)34 (77)55 (79) Time to AEs reported (including SAEs)6 (23)17 (39)23 (33) Time to SAEs reported3 (12)16 (36)19 (27) Time reported for  50% of SAEs 3 (12)9 (20)12 (17) Time reported for >50% of SAEs000 Time reported for all SAEs07 (16)7 (10) SIR based on annual SEER figures08 (18)8 (11) Patient-years as time frame2 (8)4 (9)6 (9) Yazici Y, Yazici H. Ann Rheum Dis 2007;66:124–127. COX-2=cyclo-oxygenase-2; TNF- α =tumor necrosis factor-α; RCT=randomized controlled trial; AE=adverse event; SAE=serious/severe adverse event; SIR=standardized incidence ratio; SEER=Surveillance, Epidemiology and End-Results.

19. Patient years 19

20. 20 Patient years Commonly used in RCT: ‘patient–years’ to define the time frame of AE incidence What is the problem? –Relatively rare idiosyncratic drug reactions usually occur early in the treatment course and in only a few individuals –Apart from the few with AEs, remaining patients who are prescribed the drug will never get these reactions however long they use the drug Paterson KR. BMJ 1995;310:1470.

21. Patient years 21 AE 20% chance in the first month, and not seen after 100 patients followed for one year 80% of patients after one year will have no AE 20/100 patient years One more year, no one gets AE 20/200 patient years =10/100 patient years Nothing has changed in the absolute risk when it is started in a new patient.

22. 22 Patient years (2) Unduly inflate the denominator of the related incidence ratio; potential of under-representing AE –Late-onset AEs are also likely to be missed when we use patient–years In short, only “…when an event is (or is believed to be) likely to occur at any stage during continuous treatment with a drug then an event rate with a time component (rate per person year, etc) has a true meaning” Not clear whether more than one event per patient goes into the numerator Paterson KR. BMJ 1995;310:1470.

23. 23 Patient years (3) If more than one AE/patient is included in a numerator, statistics done with that incidence ratio will be erroneous An AE can repeat itself, like skin rashes in TNF-α antagonist use in any one patient This leads to over-representation of the said individual in tests of significance Paterson KR. BMJ 1995;310:1470.

24. 24 “Statistics are like swimwear – what they reveal is suggestive but what they conceal is vital.” –Ashish Mahajan, Lancet 2007

25. Reporting radiographic outcomes in RA RCTs 25

26. Conventional wisdom… Schett et al. Arthritis Rheum 2008

27. Conventional wisdom… “…studies of anti-TNF therapy plus MTX, compared with the effect with MTX alone, have shown that although MTX is relatively effective at relieving clinical symptoms, it has little or no effect on underlying radiological progression.” Emery P, McInnes IB, van Vollenhoven R, Kraan MC. Rheumatology (Oxford) 2008

28. 28 Etanercept versus methotrexate Klareskog L, et al. Lancet 2004;363:675–681. ACR=American College of Rheumatology; MTX=methotrexate.

29. 29 Adalimumab versus methotrexate Breedveld FC, et al. Arthritis Rheum 2006;54:26–37. § p<0.001 vs ADA alone and p=0.022 vs MTX alone; † p<0.001 vs ADA alone and p=0.002 vs MTX alone; #p=0.043 vs ADA alone; *p<0.001 vs ADA alone and vs MTX alone; ACR=American College of Rheumatology; ADA=adalimumab; MTX=methotrexate.

30. 30 PREMIER X-ray Breedveld FC, et al. Arthritis Rheum 2006;54:26–37. MTX=methotrexate.

31. 31 Demographic and study characteristics of four methotrexate-naïve randomized controlled trials ERATEMPOIFX early RAPREMIER Patients, n424682641799 TNF arm, n207223NA274 MTX arm, n217228282257 Combination arm, nNA231359268 Double blind duration12 months Mean TJC at baseline3133 31 Mean SJC at baseline24232221 Primary efficacy outcome ACR-N (6 months) ACR-N (12 months) ACR 50 (12 months) Primary radiographic outcome Sharp (12 months) Yazici Y, Yazici H. Clin Exp Rheumatol 2008;26:449–452.

32. 32 12-month efficacy and radiographic outcomes in four methotrexate-naïve randomized controlled studies StudyACR 20ACR 50ACR 70ACR NSharpNNT ERA ETA72NR* 1.0015 ERA MTX65NR* 1.59- TEMPO combination85694318.3–0.5412 TEMPO ETA75482414.70.52- TEMPO MTX76431912.22.8- IFX combination62.445.632.538.90.413 IFX MTX53.632.121.226.43.7- PREMIER combination736246NR1.310 PREMIER ADA544126NR3.0- PREMIER MTX634628NR5.7- Yazici Y, Yazici H. Clin Exp Rheumatol 2008;26:449–452. ACR=American College of Rheumatology, NNT=number needed to treat; ERA=Early RA; ETA=etanercept; NR=not reported; NR*=values not given, only depicted in graphic form; MTX=methotrexate; TNF=tumor necrosis factor inhibitor; TEMPO=Trial of Etanercept and MTX with radiographic Patient Outcomes; IFX=infliximab; ADA=adalimumab.

33. 33 1 1.59 0.52 0.4 3.7 1.3 3 5.7 2.8 0 1 2 3 4 5 6 ERA ETAERA MTXTEMPO ETA+MTX TEMPO ETA TEMPO MTX ASPIRE IFX+MTX ASPIRE MTX PREMIER ADA+MTX PREMIER ADA PREMIER MTX Radiographic outcomes in four methotrexate- naïve randomized controlled trials –0.54 Yazici Y, Yazici H. Clin Exp Rheumatol 2008;26:449–452. Sharp scale Sharp score

34. 34 Radiographic outcomes in four methotrexate- naïve randomized controlled trials Yazici Y, Yazici H. Clin Exp Rheumatol 2008;26:449–452. Sharp score 11.59 –0.540.520.4 3.7 1.3 3 5.7 2.8 0 50 100 150 200 250 300 350 400 450 Sharp scale ERA ETAERA MTXTEMPO ETA+MTX TEMPO ETA TEMPO MTX ASPIRE IFX+MTX ASPIRE MTX PREMIER ADA+MTX PREMIER ADA PREMIER MTX

35. 1.3 (PREMIER combo) vs 5.7 (PREMIER MTX alone) 1.3/448=0.003 5.7/448=0.01 0.01-0.003=0.007 Similar to saying pain score of 49.8 (combo) vs 49.1 (alone) is clinically significant and should lead to treatment change/preference

36. 36 Abatacept: limited inhibition of radiographic progression – mean change in total Sharp scores (Genant modification) 50% inhibition 1.2 0 1 2 3 4 6120 Months Mean change from baseline Abatacept 10 mg/kg + MTX (n=391) Placebo + MTX (n=195) 2.3 Kremer J, et al. Ann Intern Med 2006;144:865–876; Genant H, et al. Ann Rheum Dis 2005;64(Suppl III):56. Abstract OP001. *p<0.05 vs placebo + MTX; † p<0.01 vs placebo + MTX. MTX=methotraxate; JSN=joint space narrowing. * Mean change from baseline at Month 12 Treatment Erosion scoreJSN scoreTotal Sharp score Placebo + MTX1.141.182.32 Abatacept + MTX0.63*0.58 † 1.21*

37. 37 A new way of assessing radiographic outcomes: cumulative probability plots Landewe R, van der Heijde D. Arthritis Rheum 2004;50:699–706. Unchanged Decreasing score Increasing score –10 0 10 20 30 40 0.00.10.20.30.40.50.60.70.80.91.0 Cumulative probability Change in total Sharp score

38. 38 Probability curves

39. 39 Goekoop-Ruiterman YP, et al. Arthritis Rheum 2005;52:3381–90. Leflunomide SSA MTX 25 mg MTX 15 mg MTX + IFX Gold MTX + CSA + pred AZA + pred MTX + SSA + HCQ MTX + SSA MTX 25 mg MTX 15 mg MTX + SSA + HCQ + pred MTX + IFX MTX + CsA + pred Leflunomide Gold MTX + IFX MTX + CsA + pred MTX 25 mg + SSA + pred MTX 7.5 mg/wk + SSA + pred 60  7.5 mg/day Leflunomide Gold AZA + pred SSA MTX + IFX 10 mg/kg MTX 25 mg + IFX 3 mg/kg Leflunomide MTX + CSA + pred Gold AZA + pred n=49 n=12 n=25 n=120 Group 1 Sequential Mono Group 2 Step-up Combo Group 3 Initial Combo Group 4 IFX + MTX n=126 n=121 n=133n=128 BeSt study: design

40. 40 BeSt study Goekoop-Ruiterman YP, et al. Ann Intern Med 2007;146:406–15.

41. 41 TICORA: intensive versus routine control of disease activity in rheumatoid arthritis Objective: –18-month study to determine whether closely monitored step-up therapy with non-biologic DMARDs would result in significantly better outcomes than routine care Study population: –110 patients aged 18–75 years with RA of 2.4) Outcomes: –Primary: % of patients achieving a DAS score of <2.4 –Secondary: % of patients achieving remission (DAS score <1.6) % of patients achieving American College of Rheumatology– 20, 50 and 70 responses Grigor C, et al. Lancet 2004;364:263–269. TICORA=Tight Control of Rheumatoid Arthritis.

42. 42 TICORA: intensive versus routine control of disease activity in rheumatoid arthritis (2) Intensive care group: –Monthly review of disease activity and measurement of DAS –Intra-articular corticosteroid injection of swollen joints as required –Intramuscular corticosteroid injection given as ‘bridge therapy’ during first 3 months of a new DMARD –Structured escalation of therapy if DAS >2.4 after 3 months of a new DMARD: SSZ  MTX + SSZ + HCQ   MTX (up to 25 mg/wk);  SSZ (up to 5 g/d)   prednisolone 7.5 mg/d  switch to MTX + CSA  switch to leflunomide Routine care group: –Review every 3 months (with no measure of DAS) –Management at the discretion of attending rheumatologist (i.e. DMARD therapy; intra-articular, intramuscular and/or oral corticosteroids) Grigor C, et al. Lancet 2004;364:263–269. TICORA=Tight Control of Rheumatoid Arthritis.

43. 43 Tight control for rheumatoid arthritis: TICORA study

44. 44 Grigor, et al. Lancet 2004;364:263–269. *Mantel-Haenszel procedure used. EULAR=European Union League Against Rheumatism; ACR=American College of Rheumatology. Number of patients responding at 18-month assessment Intensive group (n=55) Routine group (n=55) Odds ratio (95% CI) p*p* EULAR good response 45 (82%)24 (44%)5.8 (2.4–13.9)<0.0001 EULAR remission 36 (65%)9 (16%)9.7 (3.9–23.9)<0.0001 ACR 20 response 50 (91%)35 (64%)5.7 (1.9–16.7)<0.0001 ACR 50 response 46 (84%)22 (40%)6.1 (2.5–14.9)<0.0001 ACR 70 response 39 (71%)10 (18%)11 (4.5–27)<0.0001 Tight control for rheumatoid arthritis: TICORA study (2)

45. TICORA 2 96 RA patients, with <1 yr of disease No previous DMARDs (except HCQ) DAS28 every month, treatment adjusted –IA as needed Primary outcome was DAS28 –Blinded assessment 85% had erosions at baseline DAS28 6.9 45 Saunders SA et al. Arthritis Rheum 2008;58:1310-17

46. TICORA 2 46

47. TICORA 2 47

48. TICORA 2 48

49. ‘Efficacy versus safety versus efficacy and safety trials’ 49

50. 50 ‘Efficacy and safety’ randomized controlled trials Trend of increasing number of RCT called ‘efficacy and safety’ trials All RCT of TNF- α inhibitors in RA, psoriatic arthritis (PsA) and ankylosing spondylitis (AS) Only original reports –‘Efficacy’, ‘safety’ or ‘efficacy and safety’ –Power for safety and efficacy –Type II error Power –Alpha –Beta –Expected treatment effects in both arms Yazici Y, et al. Rheumatology 2008;47:1054–1057.

51. 51 Powering for safety analysis Twenty-four out of 34 (71%) so-called ‘efficacy and safety’ trials –Five called such in the title Of these 24 so-called ‘efficacy and safety’ trials, only 1 gave safety as a primary or secondary end point Only one study, labeled safety gave it as an endpoint No power calculations in 22/24 to look at AE in “efficacy and safety” studies –Only 3/22 had “type II error” discussion

52. 52 ‘Efficacy’, ‘safety’ and ‘efficacy and safety’ TNF- α RCT 26% 3% 56% 15% Efficacy - 9 Safety - 1 Efficacy and Safety - 19 Efficacy and Safety in title - 5 TNF- α =tumor necrosis factor inhibitor-α; RCT=randomized controlled trial.

53. 53 Powering for safety analysis (2) 11 gave statistics for SAE –2/11 had power calculations –2/11 showed statistically significant differences –Of the remaining 9, only 2 discussed type II error 24 articles gave partial statistics for AE 18/24 (75%) no type II discussion 6 out of 34 (18%) did not give any power or sample size calculation for efficacy 19/28 (68%) with power information gave all 3 components

54. 54 Before and after 2004 8 16 5 4 0 5 10 15 20 25 Before 2004After 2004 Safety/safety and efficacyEfficacy

55. 55 ‘Efficacy and safety’ Designating RCT as ‘safety’ as well as ‘efficacy’ has the potential to give false impressions that lack of AE increase is evidence of overall safety of the medication tested Only one of these trials was powered to look at safety In addition, RCT are highly selective for lack of major comorbidities –So even if they were powered to look at safety, results need to be interpreted with caution –Phase IV trials Randomized controlled trials report on AEs, but these efficacy trials cannot provide any conclusive evidence about safety; these shortcomings should be reported and required by journals and reviewers

56. Disease activity measurement in RA 56

57. Pincus et al. Arthritis Rheum 2003

58. Patient–reported outcomes Strand et al, Rheumatology 2004

59. 59 Conclusions Time to adverse effect Causation What to expect for the clinician Patient years Falsely give sense of experience Rare adverse effects hard to analyze SEER Radiographic data not as important as clinical response Randomized controlled trials provide information about efficacy and not safety. They report on AEs that happen during a defined time in a small, selected population

60. 60 “We become confident in our educated guesswork to the point where it is easy to confuse personal opinion with evidence, or personal ignorance with scientific uncertainty” –David Naylor, MD, PhD (1954–)

61. Esra & Leyla