1. The Mean Doesn’t Mean As Much Anymore
The University of Pennsylvania
Annual Conference on Statistical Issues in Clinical Trials - Targeted Therapies
29 April 2009
The Mean Doesn’t Mean As Much Anymore
Stephen J. Ruberg
in conjunction with
Lei Chen, Yanping Wang, Doug Haney
Eli Lilly & Company
Company Confidential
Copyright © 2000 Eli Lilly and Company

2. Disclosure I am a full time employee of Eli Lilly
I own stock in Eli Lilly
I will be using examples involving 2 Eli Lilly compounds
The examples represent ongoing analysis and interpretation by Eli Lilly and represent off-label information
Company Confidential
Copyright © 2000 Eli Lilly and Company

3. Problem Statement
“Doctors are men who prescribe medicines of which they know little, to cure diseases of which they know less, in human beings of whom they know nothing.”
Voltaire (1694 – 1778)
French writer and philosopher
Company Confidential
Copyright © 2000 Eli Lilly and Company

4. Average drug efficacy is low
Problem Statement
Average drug efficacy is low
Therapeutic Area Effective Rate (%)
25%
On average only about 50% of patients respond to prescribed drugs
Company Confidential
Copyright © 2000 Eli Lilly and Company
Spear et al. TRENDS in Molecular
Medicine Vol. 7 No. 5 May 2001

5. Average Effects Active Drug vs. Placebo
Problem Statement
Average Effects Active Drug vs. Placebo * * * * * *
*p<0.001
Graph need update
Company Confidential
Copyright © 2000 Eli Lilly and Company

6. The Individual and Group Profile
Problem Statement
The Individual and Group Profile
Company Confidential
Copyright © 2000 Eli Lilly and Company

7. Problem Statement Conclusion
It is not enough to show the mean effect of a new treatment is statistically significantly better than control.
Patients, physicians, payers want (are demanding) more.
Company Confidential
Copyright © 2000 Eli Lilly and Company

8. Dimensions of Tailored Therapeutics
GOAL: Improve individual patient outcomes and health outcome predictability through tailoring drug, dose, timing of treatment, and relevant information.
The Continuum
One size fits all
Targeted Therapy
Tailoring
(e.g. oncology products comprising drug and companion diagnostic)
Perspectives
Prospective
Retrospective
Company Confidential
Copyright © 2000 Eli Lilly and Company

9. Prospective Tailoring
Define target population on a molecular basis (e.g. gene, biomarker)
Engineer molecules to target such specific populations (and companion diagnostics as needed)
Many oncology examples
Drug metabolism examples
Not so much in other areas (psychiatry)
Company Confidential
Copyright © 2000 Eli Lilly and Company

10. Retrospective Tailoring
Sub-group analyses and data mining
Examples of non-biomarkers
Marriage and smoking cessation
Insurance and emergency room
Child abuse and depression
Obesity is affected by those around you
Alimta and non-squamous histology
Company Confidential
Copyright © 2000 Eli Lilly and Company

11. Tailoring to the Whole Patient
Diabetes Illustration
Patient Factors
Vascular Comp.
Co-Morbidities
Hi Triglyceride
Hi LDL-C
Personal History
Obesity
HTN
Demographics
Genetics
Diet / Exercise
Compliance
Positive Benefit-Risk
Negative Benefit-Risk
Pre-Diabetes
Type II – Exer/Wgt
Type II – 1 Oral
Type II – 2 Oral
Type II – 2 Oral + Ins
Type II – 2 Oral + Glp
Type I
Each box represents a phenotype
The calculus of benefit risk may change for each phenotype
Disease Parameters
Source: Paul, S. Tailoring Therapies for Better Patient Outcomes: Drug Development Meets Evidence-Based Medicine. IOM 37th Annual Meeting presentation – Oct 8, 2007.
Company Confidential
Copyright © 2000 Eli Lilly and Company

12. Data Mining – Classification Trees, …
Tailored Therapeutics Analysis (From Sub-group Analysis To Variable Selection)
Traditional Approach
Proposed Approach
Define Responders / Non-responders
Efficacy: Y1, Y2
Safety: S1, S2, S3
Efficacy Model
Y = f (TRT, xi)
Possible Predictors (100’s)
Baseline, Early Response, PK,
Genomic, Environmental? Social?
Assess well-known sub-groups
Age, Gender, Race, Baseline
Sub-group Analysis (one at a time)
Y = f (TRT, xi) + Age + TRT*Age
Data Mining – Classification Trees, …
heterogeneity test
Company Confidential
Copyright © 2000 Eli Lilly and Company

13. A decision tree model consists of
Tailored Therapeutics Analysis (From Sub-group Analysis To Variable Selection)
A decision tree model consists of
a set of rules for
dividing a large heterogeneous population into
smaller, more homogeneous groups
with respect to
a particular target variable (e.g., adverse event).
Very useful for finding complex interactions.
Company Confidential Copyright © 2000 Eli Lilly and Company

14. Retrospective Tailoring Examples
First Example
Identify baseline information that helps us decide who should get a treatment
Tailoring on phenotypic/clinical measures
Second Example
For those who get a drug, how do we decide quickly whether they are on the right drug or not
Tailoring on timing of treatment
Company Confidential
Copyright © 2000 Eli Lilly and Company

15. Example 1 Disease outcome can be assessed as a dichotomous response
Many covariates analyzed one at a time
Stepwise logistic regression to select multiple covariates in one functional prediction
Exploratory analysis of 60+ potential covariates/predictors
Other studies/analyses needed to confirm
Company Confidential
Copyright © 2000 Eli Lilly and Company

16. Example 1 - Objective
What marker(s) can be used to predict the largest population of patients that are most responsive to Treatment?
If the belief is such that Treatment works best in the highest risk patients, what marker(s) are the best predictors of high risk?
What are the simplest marker(s)?
Easiest to measure, least expensive, available
Measurable / responsive over time
Could a ‘complex’ marker be made simpler thru a new diagnostic?
What is the sub-group size associated with marker(s)?
Company Confidential
Copyright © 2000 Eli Lilly and Company

17. Outcome Based on Placebo Data
825 Pbo Patients
YES NO
Variable X22 > A
P: 90/476=0.19
P: 163/349=0.47
YES NO
Variable X37 > B
P: 99/250=0.40
P: 51/69=0.74
YES NO
Variable X4 < C
P: 80/223=0.36
P: 18/22=0.82
Company Confidential
Copyright © 2000 Eli Lilly and Company

18. Treatment vs. Pbo in Subgroups Based on CART
825 Pbo Patients
823 Treatment Patients
YES NO
P: 90/476=0.19
T: 86/468=0.18
P-value=0.87
RR=0.05
Variable X22 > A
P: 163/349=0.47
T: 118/355=0.33
P-value<0.0001
RR=0.30
YES NO
P: 99/250=0.40
T: 79/259=0.31
P-value=0.03
RR=0.23
Variable X37 > B
P: 51/69=0.74
T: 29/75=0.39
P-value<0.0001
RR=0.47
YES NO
Variable X4 < C
P: 80/223=0.36
T: 69/236=0.29
P-value=0.14
RR=0.19
P: 18/22=0.82
T: 8/20=0.40
P-value=0.01
RR=0.51
Company Confidential
Copyright © 2000 Eli Lilly and Company

19. Example 2 - Schizophrenia
Help practicing physicians decide what to do in treating schizophrenics
Inadequate Response
Minimum Number of Weeks to Wait
Maximum Number of Weeks to Wait
Initial Antipsychotic
Little or no response 3 6
Partial response 4 10
Second Antipsychotic 5 11
Adapted from Expert Consensus Panel for Optimizing Pharmacologic Treatment of Psychotic Disorders. J Clin Psychiatry 2003;64 (suppl 12): 2-97.
Company Confidential
Copyright © 2000 Eli Lilly and Company

20. Early Response Assessment
GOAL:
Identify what amount of change …
in which of the fewest symptoms/measures …
at the earliest time in treatment …
predicts both responders and non-responders.
Has to be “implementable” for the typical clinician on a routine basis (i.e. not a research tool as part of research studies)
Company Confidential
Copyright © 2000 Eli Lilly and Company

21. Example 2 – Zyprexa & Atypicals
Predicting Efficacy Responders
Response = 30% reduction in PANSS Total Symptom Score at 8 weeks
Predictors are the change in individual symptom ratings from PANSS at week 1 and week 2 of treatment
30 individual symptoms = 60 predictors
Integrated data from 6 studies (1494 patients)
Moderately to severely ill patients
All patients on active atypical antipsychotics
Company Confidential
Copyright © 2000 Eli Lilly and Company
PANSS = Positive and Negative Symptom Scale

22. Generic Classification Tree
Example 2 – Zyprexa & Atypicals
Generic Classification Tree
R: %
NR: %
N number
PPV
NPV
Mixed
Miscls
Symptom Criteria #1 NO
YES
R: %
NR: %
N number
R: %
NR: %
N number
Symptom Criteria #2N
Symptom Criteria #2Y NO
YES NO
YES
R: Mis%
NR: NPV% N
R: %
NR: %
N mixed
R: %
NR: %
N mixed
R: PPV%
NR: Mis% N
Company Confidential
Copyright © 2000 Eli Lilly and Company

23. Initial Findings Early Response CART – 2 Week Assessment
Example 2 – Zyprexa & Atypicals
Initial Findings Early Response CART – 2 Week Assessment
PPV 79%
NPV 67%
Mixed 0%
Miscls 31%
R: 43%
NR: 57%
N 1494
At least 2 unit drop in Item Unusual Thought Content? NO
YES
R: 33%
NR: 67%
N 1205
R: 79%
NR: 21%
N 289
Company Confidential
Copyright © 2000 Eli Lilly and Company

24. Initial Findings Early Response CART – 2 Week Assessment
Example 2 – Zyprexa & Atypicals
Initial Findings Early Response CART – 2 Week Assessment
PPV 76%
NPV 67%
Mixed 0%
Miscls 31%
R: 43%
NR: 57%
N 1494
At least 2 unit drop in Delusions? NO
YES
R: 33%
NR: 67%
N 1178
R: 76%
NR: 24%
N 316
Company Confidential
Copyright © 2000 Eli Lilly and Company

25. Final Model Early Response CART – 2 Week Assessment
NR: 57%
N 1494
At least 2 unit drop in at least 2 psychotic items? NO
YES
R: 28%
NR: 72%
N 1049
R: 79%
NR: 21%
N 445
At least 2 unit drop in excitement?
PPV 79%
NPV 75%
Mixed 8%
Miscls 24% NO
YES
R: 25%
NR: 75%
N 929
R: 53%
NR: 47%
N 120
Psychotic items = Unusual Thought Content, Delusions, Hallucinatory Behavior, Conceptual Disorganization, Suspiciousness
Company Confidential
Copyright © 2000 Eli Lilly and Company

26. Proportion of mix- response Proportion of misclassification
Example 2 – Zyprexa & Atypicals
Model Evaluation
Yes
P(response)
=PPV
No/No
P(non-response)
=NPV
No/Yes
Proportion of mix- response
Proportion of misclassification
6 pooled studies
79%
75% 8%
24%
Study A
70%
77% 7%
25%
Study B
76%
72%
29%
Study C*
63% 5%
*acute illness study
Company Confidential
Copyright © 2000 Eli Lilly and Company

27. Average Total Symptom Scores Over 8 Weeks of Study
NO/NO
NO/YES
YES
Company Confidential
Copyright © 2000 Eli Lilly and Company

28. Conclusions Company Confidential
Copyright © 2000 Eli Lilly and Company

29. BusinessWeek Medical Guesswork
29 May 2006
Medical Guesswork
From heart surgery to prostate care, the medical industry knows little about which treatments really work
“What's required is a revolution called ‘evidence-based medicine,’ says Eddy, a heart surgeon turned mathematician and health-care economist.
“The human brain, Eddy explains, needs help to make sense of patients who have combinations of diseases, and of the complex probabilities involved in each.”
Company Confidential
Copyright © 2000 Eli Lilly and Company

30. Conclusions (1) Tailor to the whole patient
There is prospective and retrospective tailoring approaches
Physicians like decision trees
Understandable and implementable
Move from sub-group analysis mindset to variable selection mindset
CART is a useful omnibus tool
Company Confidential
Copyright © 2000 Eli Lilly and Company

31. Conclusions (2)
More and more, there is less and less interest in the overall mean response in a broad population of patients.
There is a shift to greater interest in smaller, more responsive populations.
The key questions emerging seem to be:
Company Confidential
Copyright © 2000 Eli Lilly and Company

32. Conclusions (3)
“What is the largest population that has a very high probability of showing a clinically meaningful benefit?”
A really large benefit in a really small population may be useful but will have less medical or public health impact.
The exceptions are rare diseases.
Company Confidential
Copyright © 2000 Eli Lilly and Company

33. Conclusions (4)
“What measurable/observable characteristics define that population?”
What are the easiest and cheapest characteristics to measure?
They may not be genetic or biochemical?
It doesn’t have to be perfect, just better than what we do now.
Company Confidential
Copyright © 2000 Eli Lilly and Company

34. Conclusion (5)
Pharmaceutical research will continue to refine our understanding of who is likely to respond to drugs.
Personalized medicine as a general rule has a long way to go, and it may never be achieved in some disease states.
Tailored medicine is happening today and refinements in treatment paradigms are being studied at the present time.
This area of medicine is ripe with statistical problems, and much more research is needed.
Company Confidential
Copyright © 2000 Eli Lilly and Company

35. Thank you. Merci. Вы. 謝謝。 Danke. Grazie. Obrigado. ありがとう。
      
Obrigado.
ありがとう。
Thank you.
Dank u.
Σας ευχαριστούμε.
너를 감사하십시요.
Gracias.
Company Confidential
Copyright © 2000 Eli Lilly and Company