1. Methods for Identifying Treatment-Emergent Symptomatic Adverse Events from the Patient Perspective with Application to the PRO-CTCAE
Gina L. Mazza, Ethan Basch, Lauren J. Rogak, and Amylou C. Dueck
Associate Professor of Biostatistics
Mayo Clinic, Scottsdale, AZ
Joint Statistical Meetings
August 1, 2019
Denver, CO

2. Common Terminology Criteria for Adverse Events (CTCAE v5)

3. Hortobagyi GN, et al. N Engl J Med. 2016; 375(18):1738-1748.

4. Attribution (or relatedness): 1 = Unrelated 2 = Unlikely 3 = Possible
4 = Probable
5 = Definite
Sharma P, et al. J Clin Oncol. 2019; 37(19):

5. How accurate is attribution?
9 double-blind placebo- controlled randomized trials
Relatedness overestimated
High proportion of “possibly” related
Placebo arms: significant rate of related AEs
Fatigue, nausea, vomiting, diarrhea, constipation, and neuropathy overly reported as related
Le-Rademacher J, et al. Ann Oncol. 2017; 28(6):

6. PRO-CTCAE™ Item Library
124 patient-reported items representing 78 symptomatic adverse events (e.g., dysphagia, nausea, sensory neuropathy)
Items assess frequency, severity, interference with daily activities, presence, amount
Covered adverse events are drawn from the US National Cancer Institute's “Common Terminology Criteria for Adverse Events” (CTCAE)
Item generation: Basch E, et al. J Natl Cancer Inst. 2014; 106(9).
Cognitive testing: Hay JL, et al. Qual Life Res. 2014; 23(1):
Validity, reliability, & responsiveness: Dueck AC, et al. JAMA Oncol. 2015; 1(8):

7. Adverse Event Grading: Clinician-Reported versus Patient-Reported
Musculoskeletal and connective tissue disorders
Adverse Event
Grade 1 2 3 4 5
Arthralgia
Mild pain
Moderate pain; limiting instrumental ADL
Severe pain; limiting self care ADL -
Please think back over the past 7 days:
How OFTEN did you have ACHING JOINTS (SUCH AS ELBOWS, KNEES, SHOULDERS)?
Never / Rarely / Occasionally / Frequently / Almost constantly
What was the SEVERITY of your ACHING JOINTS (SUCH AS ELBOWS, KNEES, SHOULDERS) at their WORST?
None / Mild / Moderate / Severe / Very severe
How much did ACHING JOINTS (SUCH AS ELBOWS, KNEES, SHOULDERS) INTERFERE with your usual or daily activities:
Not at all / A little bit / Somewhat / Quite a bit / Very much

9. Also available in:
Chinese (Simplified & Traditional)
Czech
Danish
Dutch
French (Canada; Belgium, France, Switzerland)
German
Greek
Hungarian
Italian
Japanese
Korean
Malay
Polish
Portuguese (Portugal & Brazil)
Romanian
Russian
Spanish
Turkish

10. Summary measures – max score & max baseline-adjusted score
PTID W0 W1 W2 W3
Max score
post-baseline
Max baseline-adjusted score
001 1 3
002 4
003 2
004

11. Summary measures – max score & max baseline-adjusted score
PTID W0 W1 W2 W3
Max score
post-baseline
Max baseline-adjusted score
001 1 3
002 4
003 2
004
If the max score is worst than baseline, then use max score.

12. Summary measures – max score & max baseline-adjusted score
PTID W0 W1 W2 W3
Max score
post-baseline
Max baseline-adjusted score
001 1 3
002 4
003 2
004
If the max score is the same or better than baseline, then use 0.

13. COMET-2: Prospective Double Blind Placebo Controlled Trial
Cabozantinib: 60 mg PO qd Placebo mitoxantrone: q3w IV Placebo prednisone: PO bid
Castration-resistant Prostate Cancer with bone metastasis and pain despite narcotic use
Prior treatment with docetaxel and either abiraterone or enzalutamide
Loss of clinical benefit
Randomization (1:1)
Mitoxantrone: 12 mg/m2 q3w IV Prednisone: 5 mg PO bid Placebo cabozantinib: PO qd
N=119
PRO-CTCAE: 12 symptomatic AEs (21 items) by automated telephone (at-home reporting), automated daily reminders followed by telephone data collection by data coordinator after 72 hours
Baseline
Every 3 weeks during treatment
End of treatment

14. COMET-2 analysis details
119 patients enrolled, 117 started treatment
114 patients completed PRO-CTCAE at baseline (112 started treatment)
112 patients completed PRO-CTCAE post-baseline (all started treatment)
107 patients completed PRO-CTCAE at baseline + at least one post- baseline (all started treatment)
Used as the analysis population
Zeros imputed for electronically skipped items
End-of-treatment questionnaire omitted from analysis (N=68)

15. Basch EM, et al. Eur Urol. 2018. pii: S0302-2838(18)30932-1.

16. Decreased appetite (S) 41 ( 77%) 47 ( 87%) 0.21 9 ( 17%) 11 ( 20%)
 BASELINE RATES
Score >0
Score ≥3
PRO-CTCAE Item
Cabozantinib
N ( %)
Mitoxantrone-pred
Fisher P
Constipation (S)
43 ( 81%)
42 ( 78%)
0.81
15 ( 28%)
7 ( 13%)
0.06
Decreased appetite (S)
41 ( 77%)
47 ( 87%)
0.21
9 ( 17%)
11 ( 20%)
0.80
Decreased appetite (I)
30 ( 57%)
35 ( 65%)
0.43
8 ( 15%)
1.00
Diarrhea (F)
24 ( 45%)
22 ( 41%)
0.70
3 ( 6%)
2 ( 4%)
0.68
Fatigue (S)
52 ( 98%)
54 ( 100%)
0.50
29 ( 55%)
29 ( 54%)
Fatigue (I)
51 ( 96%)
52 ( 96%)
28 ( 53%)
30 ( 56%)
0.85
Insomnia (S)
39 ( 74%)
43 ( 80%)
5 ( 9%)
0.27
Insomnia (I)
32 ( 60%)
37 ( 69%)
0.42
12 ( 23%)
0.47
Mouth or throat sores (S)
0.20 --
Nausea (F)
35 ( 66%)
39 ( 72%)
0.53
0.79
Nausea (S)
33 ( 62%)
0.31
4 ( 8%)
6 ( 11%)
0.74
Numbness/tingling in hands/feet (S)
33 ( 61%)
0.22
4 ( 7%)
Numbness/tingling in hands/feet (I)
19 ( 35%)
0.33
0.72
Pain (F)
53 ( 100%)
45 ( 85%)
49 ( 91%)
0.39
Pain (S)
36 ( 68%)
36 ( 67%)
Pain (I)
31 ( 57%)
Rash (P)
Shortness of breath (S)
25 ( 47%)
0.08
0.32
Shortness of breath (I)
19 ( 36%)
0.03
Vomiting (F)
16 ( 30%)
Vomiting (S)
14 ( 26%)
0.40

17. Without and with baseline adjustment
# of AEs with significant between-arm differences (0 vs ≥1):
By investigator report (CTCAE): 0
By patient report (PRO-CTCAE without adjustment for baseline): 4
By patient report (PRO-CTCAE with adjustment for baseline): 7
Dueck AC, et al. To appear in
JAMA Onc.

18. Large difference (impacts less severe scores more)
High baseline:
Low baseline:
Small difference

19. Max baseline-adjusted vs. max change-from-baseline score
With adjustment ≥1
With adjustment ≥2
With adjustment ≥3
PRO-CTCAE Item
Cabo N
Mito N
Cabo N ( %)
Mito N ( %)
Fisher P
Constipation (S) 53 54
25 ( 47%)
16 ( 30%)
0.08
22 ( 42%)
12 ( 22%)
0.04
14 ( 26%)
7 ( 13%)
0.09
Decreased appetite (S) 52
34 ( 65%)
18 ( 33%)
0.002
32 ( 62%)
17 ( 31%)
0.003
20 ( 38%)
8 ( 15%)
0.008
Diarrhea (F)
42 ( 81%)
26 ( 48%)
<0.001
35 ( 67%)
23 ( 44%)
6 ( 11%)
Fatigue (S)
24 ( 45%)
0.17
19 ( 36%)
0.30
Insomnia (S)
22 ( 41%)
0.84
13 ( 25%)
20 ( 37%)
0.21
1.00
Mouth or throat sores (S)
33 ( 63%)
19 ( 35%)
0.006
17 ( 33%)
10 ( 19%)
0.12
6 ( 12%)
1 ( 2%)
0.06
Nausea (F)
31 ( 60%)
Numbess/tingling in hands/feet (S)
28 ( 54%)
0.049
11 ( 20%)
0.02
12 ( 23%)
4 ( 7%)
0.03
Pain (F)
Rash (P)
0.65 --
Shortness of breath (S) 50
29 ( 58%)
21 ( 39%)
20 ( 40%)
0.31
7 ( 14%)
Vomiting (F)
9 ( 17%)
0.52
Change-from-baseline ≥1
Change-from-baseline ≥2
Change-from-baseline ≥3
0.29
3 ( 6%)
2 ( 4%)
0.68
19 ( 37%)
0.01
27 ( 52%)
0.004
4 ( 8%)
0.44
0 ( 0%)
0.49
0.07
18 ( 35%)
0.005
11 ( 22%)
0.19
15 ( 29%)
5 ( 9%)
0.20
IDENTICAL!

20. Max baseline-adjusted vs. max change-from-baseline score
With adjustment ≥1
With adjustment ≥2
With adjustment ≥3
PRO-CTCAE Item
Cabo N
Mito N
Cabo N ( %)
Mito N ( %)
Fisher P
Constipation (S) 53 54
25 ( 47%)
16 ( 30%)
0.08
22 ( 42%)
12 ( 22%)
0.04
14 ( 26%)
7 ( 13%)
0.09
Decreased appetite (S) 52
34 ( 65%)
18 ( 33%)
0.002
32 ( 62%)
17 ( 31%)
0.003
20 ( 38%)
8 ( 15%)
0.008
Diarrhea (F)
42 ( 81%)
26 ( 48%)
<0.001
35 ( 67%)
23 ( 44%)
6 ( 11%)
Fatigue (S)
24 ( 45%)
0.17
19 ( 36%)
0.30
Insomnia (S)
22 ( 41%)
0.84
13 ( 25%)
20 ( 37%)
0.21
1.00
Mouth or throat sores (S)
33 ( 63%)
19 ( 35%)
0.006
17 ( 33%)
10 ( 19%)
0.12
6 ( 12%)
1 ( 2%)
0.06
Nausea (F)
31 ( 60%)
Numbess/tingling in hands/feet (S)
28 ( 54%)
0.049
11 ( 20%)
0.02
12 ( 23%)
4 ( 7%)
0.03
Pain (F)
Rash (P)
0.65 --
Shortness of breath (S) 50
29 ( 58%)
21 ( 39%)
20 ( 40%)
0.31
7 ( 14%)
Vomiting (F)
9 ( 17%)
0.52
Change-from-baseline ≥1
Change-from-baseline ≥2
Change-from-baseline ≥3
0.29
3 ( 6%)
2 ( 4%)
0.68
19 ( 37%)
0.01
27 ( 52%)
0.004
4 ( 8%)
0.44
0 ( 0%)
0.49
0.07
18 ( 35%)
0.005
11 ( 22%)
0.19
15 ( 29%)
5 ( 9%)
0.20
Difficult or impossible to achieve in patients with pre-existing symptoms at baseline

21. Does max score or max baseline-adjusted score (0 vs ≥1) better differentiate between arms?
Difference in area under the ROC curves significant for:
Decreased appetite severity: ΔAUC=0.12, 95% CI , p=0.008
Shortness of breath severity: ΔAUC=0.09, 95% CI , p=0.04
Shortness of breath interference: ΔAUC=0.09, 95% CI , p=0.03
Median ΔAUC across the 21 items = 0.04 (range: to 0.12)
Significant overall effect: ΔAUC = 0.03, 95% CI , p=0.004

22. Is the baseline adjustment method ready for primetime?
Replicated in three other trials to date
See Gounder MM et al. (N Engl J Med. 2018; 379[25]: ) for published application
However, open questions remain
Impact of reduced variability
Ceiling effect – what if patients all start at scores of 3 or 4?
Intended to identify harms (so may not work well for improving symptoms)
Interpretability, best language for labeling tables/graphics
Baseline subtraction changed to baseline adjusted
Worsened from baseline?
Highest or worse?

23. Concluding thoughts
Baseline adjustment appears to better identify treatment harms than maximum score
Recommend longitudinal depictions to supplement tabular summary baseline adjustment method
Advanced analytics (e.g., model-based analysis) are more sensitive for detecting differences between arms, but are less “user friendly”
The future:
NCI Tolerability MoonshotSM Consortium
Standardized analysis and reporting

24. Thanks!
@BiostatGirl SAS & R code forthcoming (