1. Biostatistical Aspects of Rational Clinical Trial Designs Elizabeth G. Hill, PhD Associate Professor of Biostatistics Biostatistics Shared Resource Hollings Cancer Center Medical University of South Carolina

2. Types of Research Studies in Cancer  Basic Science  Translational  Clinical Exploratory/Pilot/Correlative Phase I Phase II Phase III Other: e.g. prevention, survivorship  Epidemiological

3. Phases of Drug Development  Phase I Dose finding Usually designed to find the highest safe dose. 12-30 patients  Phase II Preliminary efficacy and safety Generally not ‘head to head’ comparison 20-80 patients  Phase III Definitive comparative trial against the standard of care Usually hundreds or thousands of patients

4. Clinical Trials: the beginning  Write a clinical trial protocol  Usually 70-180 pages  Not like writing a grant  Every detail spelled out: no page limit!  There are standard templates that can/should be used.

5. Imagine….  You are principal investigator (PI) of a clinical trial  In the middle of the trial, you change careers  You are now an astronaut and fly to the moon  Meanwhile, a new patient is enrolled.  The new PI needs to know: How should the patient be assigned to a dose? How should dose modifications occur? What measurements should be taken and when? What are the definition of the primary and 2ndary outcomes? Who and how are the data to be reviewed for safety and efficacy?

6. Statistical design and development of clinical trials  Statistical considerations permeate the design and analytic plan  Requires interaction with your statistician call early! before you have “fixed” the design bad: “i have almost finished writing the protocol, and then i will send to you to insert a statistical plan”  Really, we are here to make your life easier

7. Where do I find this statistician?  Academic cancer centers have biostatistics cores or biostatistics shared resources  It is the role of these biostatisticians to help design clinical trials  Find them!  Other places: University settings usually have biostatistics departments or divisions Pharma will have biostatisticians on site or have biostatistical consultants available  If your institution does NOT have biostats support, tell them they MUST HAVE IT!!!

8. Design of Clinical Trials : Striking a Balance  Answer the question (correctly) Control risk of errors in conclusions  Minimize potential harm and maximize potential benefit Limit number of participants treated at sub- therapeutic doses Limit number of participants treated with ineffective therapy or exposed to toxicity  Maximize feasibility Make it simple enough to carry out Writing a detailed protocol can help avoid unforeseen feasibility issues

9. Statistical Considerations: 5 part process I. Stating research aims II. Determining your outcome measures III. Choosing the experimental design IV. The analytic plan V. Sample size justification

10. Motivating Example  Phase II trial of induction gemcitabine/oxaliplatin/cetuximab (GOC) followed by intensity modulated radiotherapy (IMRT) with capecitabine to improve resectability in borderline and frankly unresectable pancreatic cancer  Principal Investigator: Nestor Esnaola  Single arm study  Treatment plan: Patients are treated with GOC for six 14-day cycles. If resectable, taken to surgery If not, radiochemotherapy (IMRT + capecitabine (RCT)) and restaged

11. I. Stating research aims  Authors devised a protocol, beginning with research aims  Aims should be concrete and include measurable outcomes  Bad examples: To evaluate the effect of GOC on cancer. To see if GOC + RCT improves cancer outcomes To determine the safety profile of GOC + RCT  What is wrong with these aims? what does “effect” mean? what kind of cancer, in what patients? “Improves” compared to what? what is the outcome of interest? what does a “safety profile” mean?  Think about how you are going to determine if this treatment approach works or not

12. I. Stating research aims  Better examples: To evaluate the 6 month progression-free survival of GOC + RCT in patients with locally advanced, unresectable or borderline resectable, non- metastatic adenocarcinoma of the pancreas when treated with neoadjuvant GOC with or without RCT followed by definitive surgery. To determine the tolerance of this regimen, defined as the proportion of patients who follow the treatment plan.  Keywords for primary outcome: determine, estimate, evaluate, describe efficacy, safety

13. Devising your aims  Generally, there is ONE primary aim and your study is designed to address the primary aim  Common (generic) aims per phase: Phase I: primary aim is finding the “recommended” dose Phase II: primary aim is determining if there is sufficient efficacy Phase III: primary aim is to determine which of two (or more) treatment combinations yields the longest overall survival  Secondary aims: important, but do not drive the design  pharmacokinetics  pharmacodynamic (e.g., methylation)  response  safety  change in gene expression

14. Aims and Hypotheses  Aims are often accompanied by hypotheses.  Stating the hypothesis to be tested can be a useful guide for the analytic plan:  “The 6 month PFS will be at least 70%”

15. II. Determining your outcome measures  The outcome measure will depend on the parameter of interest  Examples of possible parameters of interest in phase II: response rate Median progression-free survival 6 month progression-free survival rate  Synonyms: outcome, endpoints  Aim ≠ endpoint  What is an endpoint or outcome? patient-level measure of “effect” of interest measured on each patient in the study it is QUANTIFIABLE

16. Parameter of interest vs. outcome Parameter of interestOutcome Response rate: proportion of patients with CR or PR Response (CR or PR) Median overall survivalTime from enrollment to death (or last follow-up) 6 month overall survivalTime from enrollment to death (or last follow-up) Mean change in quality of lifeDifference in quality of life scores from baseline to follow-up

17. II. Determining your outcome measures  Example: Parameter of interest is the 6 month progression- free survival rate Progression is objectively defined: the tumor has not increased by 20% or more comparing the baseline to the 6 month tumor measurement. Each patient is determined to either be or not be “progression-free” at 6 months. BINARY endpoint in this example

18. The following are NOT endpoints  These are estimates of parameters: response rate median survival AE rate safety profile  These describe the time course of the study in some way ( don’t let the term ‘endpoint’ confuse you): length of time of treatment time until patient goes off-study length of study

19. Determining clinical outcomes: RECIST criteria  Definitions of response, stable disease and progression are not quite as ‘simple’ as they may seem in solid tumors  RECIST: Response Evaluation Criteria in Solid Tumors.  Version 1 is from 2000, Version 1.1 published in 2008.  Key features: Definitions of minimum size of measureable lesions Instructions on how many lesions to follow Use of unidimensional measures for overall tumor burden  See Eisenhauer et al., Eur J of Cancer (2009), 45, 228-247.

20. Definitions (briefly*)  Complete Response: disappearance of all target lesions.  Partial Response: at least a 30% decrease in the sum of the diameters of the target lesions, taking as a reference the baseline sum of diameters  Progressive Disease: At least a 20% increase in the sum of diameters of target lesions. Increase must constitute at least 5mm absolute increase.  Stable Disease: Shrinkage<30% or increase<20%. * based on target lesions and ignoring lymph node criteria for simplicity

21. III. Choosing the experimental design  Based on the aims and the outcome, a design can be identified.  Other considerations patient population accrual limitations previous experience with the treatment of interest in this or other populations results from earlier phase studies

22. III. Choosing the experimental design  There are common approaches within each phase of drug development  However, there are often many options and seemingly small details that can make big differences.  Two common ‘philosophies” Frequentist Bayesian  Buzzword: “adaptive”

23. Phase I trial goals  Classic Phase I trials: Find the highest dose that is deemed safe: the Maximum Tolerated Dose (MTD) DLT = dose limiting toxicity Goal is to find the highest dose that has a DLT rate of x% or less (usually ranges from 20% to 40%)  Newer Phase I trials: Find the dose that is considered to be safe and have optimal biologic/immunologic effect (OBD). Goal is to optimize “biomarker” response within safety constraints.

24. New paradigm: Targeted Therapy How do targeted therapies change the early phase drug development paradigm?  Not all targeted therapies have toxicity Toxicity may not occur at all Toxicity may not increase with dose  Targeted therapies may not reach the target of interest  Implications for study design: Previous assumptions may not hold Does efficacy increase with dose? Endpoint may no longer be appropriate Should we be looking for the MTD? What good is phase I if the agent does not hit the target?

25. Phase II  Provide preliminary information on whether a treatment is efficacious  Provide preliminary data about the relationship between dose and efficacy.  Often controlled but They are small: generally cannot find large differences in treatment effects Their endpoints are “short-term”  Phase II endpoint: response  Phase III endpoint: overall survival  Often unblinded

26. GOC + RCT trial  “This is a single arm phase II trial to evaluate the 6 month PFS rate in patients with borderline and frankly unresectable pancreatic cancer.”  The goals: determine if the 6 month PFS rate is significantly better than 50%.  Study design: A single arm, single stage study was designed The null hypothesis is that the 6 month PFS rate is 50% The alternative hypothesis is that the 6 month PFS rate is 70%  Alternatives: Randomized phase II design Early stopping for futility (e.g. Simon two-stage).

27. Phase III  If agent (or combination) succeeds in phase II, the next logical step is phase III.  Usually designed by large companies or cooperative groups (e.g. ECOG, CALG-B).  Comparative trial Two or more arms Standard outcome is overall survival (with rare exception in cancer) Goal: show a significant improvement in survival  Generally very large and expensive  Must be strong evidence in phase II to conclude that Phase III study will succeed

28. Phase III  Very large undertaking Multicenter  Infrastructure  IRB and scientific approvals at each site Talks with FDA and other regulators Establishment of DSMB specifically for trials  The statistical design is relatively simple compared to the practical issues of running a Phase III trial.  Practical issues will often drive the design

29. IV. Analytic Plan  Do you want to compare?  Do you want to estimate?  Do you want to test a hypothesis?  These questions, in regards to your stated aims, will determine your analytic plan  Recall primary aim: To determine the 6 month progression-free survival rate.  Recall primary endpoint: 6 month progression- free survival indicator.

30. IV. Analytic Plan  The analytic plan for the primary outcome usually involves two things: estimating a parameter of interest testing that the parameter is different than in another setting (e.g., different treatment)  Estimation: a point estimate and some measure of precision  Example: “The 6 month PFS rate will be estimated with its confidence interval.” this provides us with an estimate of the proportion of patients who are progression-free at 6 months. it also provides us with a measure of precision about the estimate

31. The 95% confidence interval  an interval that contains the true value of the parameter of interest 95% of the time.  “we are 95% confident that the true 6 mo. PFS rate lies in this interval”  Example: below shows examples where observed rate is 0.50. 95% confidence interval width depends on the sample size  Depending on the sample size, we have greater or less precision in our estimate

32. IV. The analytic plan  Hypothesis testing: Determining if the treatment is worthy of further study.  Recall our hypotheses: The 6 month PFS rate of patients in this study will be at least 70%.  What is a sufficiently LOW 6 month PFS rate that we are not interested in further pursuit?  Based on the study team’s experience, a 6 month PFS rate of 50% is too low to warrant further study of this treatment approach.

33. IV. Analytic Plan  We perform a hypothesis test: Ho: p = 0.50 (null) Ha: p = 0.70 (alternative)  This test is performed using an exact binomial procedure.  The result is a p-value that provides “evidence” to either reject or fail to reject the null hypothesis  If this were a randomized phase II study: the test is performed in each arm the arms are not directly compared to one another (that is a different test)

34. Recall the p-value  p-value: the probability of observing a result as or more extreme than we saw in our study if the null hypothesis is true.  Small p-value: evidence that the null is not true (“significant result”)  Large p-value: not sufficient evidence to reject the null (“not signficant”)  Threshold for significance? we usually think of 0.05, but in phase II, often use 0.10.

35. IV. Analytic Plan  Depends on the design and the goals  Example is a Phase II trial single arm approach to analysis compare to historical 6 month PFS rate (e.g., 0.50)  Phase I studies often the analysis plan is descriptive rare to see hypothesis testing (for primary aim)  Phase III studies head to head comparison of groups Hazard ratio compares event rates per group Time to event methods are required:  Log rank test  Cox regression

36. V. Sample size justification  Two basic approaches power (most common) precision  Recall: Limit number of participants treated at sub-therapeutic doses Limit number of participants treated with ineffective therapy or exposed to toxicity  But, also we need to enroll enough patients to achieve our aims  Balancing act: Too few patients: you cannot answer the question Too many patients: you have wasted resources and potentially exposed patients to an ineffective treatment unnecessarily  Most commonly motivate sample size by a hypothesis testing approach

37. Refresher of alpha, beta and power   Type I error Type II error  = probability of Type I error (level of significance)  = probability of Type II error 1-  = Power Accept H o Reject H o H o is True H o is NOT True

38. “Plug and Chug”  With power of 80% and one-sided alpha of 0.05, and Ho and Ha, a one-stage design was selected. A single stage design was chosen. To achieve a power of 80% with a one-sided alpha assuming a null 6 month PFS of 50% and an alternative rate of 70%, 39 patients need to be enrolled. If 25 or more of the patients are progression-free at their 6-month visit, then we will reject the null hypothesis at the 5% level and conclude that the treatment approach is worthy of further study.

39. “Plug and Chug” with interim look  With power of 80% and one-sided alpha of 0.05, and Ho and Ha, a Simon two-stage design could have been adopted.  The sample size per arm will be 15 patients or 43 patients (depending on early stopping) The Simon’s two stage design used is defined as follows. Our null hypothesis is that the 6 month PFS rate is 50% and our alternative hypothesis is that it is 70%. At the first stage we will enroll 15 patients. We will close accrual to an arm if < 8 patients are PF at 6 months. If 9 or more are progression-free at 6 months, then the study will remain open for an additional 28 patients. The treatment approach will be considered promising if at least 27 patients are progression-free in 43 patients. This study has power of 80% and a one-sided alpha of 5%.

40. V. Sample size justification  Hypothesis testing is not always the way to go  Sometimes estimation is sufficient (but not always! it is not an ‘escape route’)  In that case, sample size can be justified by precision  Example: with 39 patients, we will be able to estimate the 6 month PFS rate with a 90% confidence interval with half-width no greater than 0.16.  Difficult part: is 0.16 half-width sufficiently precise? how to rationalize that?

41. Sample size is generally chosen based on 1. budget 2. expected accrual 3. the clinical effect size of interest 4. type I and type II errors 5. 3 and 4 6. all of the above

42. Feedback loop  The process is actually not completely linear as stated  Examples: Design issues may cause you to change your outcome or restate your aim Accrual limitations may cause you to change the design  “Dynamic process”

43. Additional aims (correlatives, etc.)  VERY important aims!  Not discussed here due to space/time.  Same principles apply for stating aims, determining outcomes, writing analytic plan  Usually power/sample size is less of a concern for secondary aims  “correlative” does not mean you can be vague! these need to be well-conceived often on biopsy tissue, pre post design will you really learn anything?

44. Early Stopping Rules and Interim Analyses  As a general rule, consider incorporating early stopping rules  Why? Ethics and resources  Lots of reasons for stopping  Example: phase II designs Early stopping for safety (one or more arms) Early stopping for futility Early stopping for harm

45. Implications of early/interim looks  Early looks can/will affect Type I error Type II error  Consequences? They need to be “built” into study design and power calculations  Misconception: The DSMB will stop the study early if needed for safety or harm so there is no need to account for early looks.  Having ‘independent’ review does not mean that the interim looks should not be built in to design.

46. Data Safety Monitoring Board/Committee  DSMB or DSMC  Standard in phase III trials.  Independent body of experts, usually clinical researchers + 1 or more statisticians + an ethicist.  They periodically review ongoing trial results  Have access to unblinded treatment assignments if necessary.  Often assumed they will do more than they should (e.g. redesign the study in midstream)

47. Take-home points  Talk to your statistician early and often when you want to design a study  Write clear aims and define clear endpoints  Let the statistician help you with the design, analysis plan and power calculation: that is our job.  How to learn more? Visit your institution’s cancer protocol review committee Try a workshop (check out AACR workshop schedule)

48. Some good text books on trials  General Trials: Clinical Trials: A Methodologic Perspective (Piantadosi) Clinical Trials (Meinert)  Specific to Cancer: Classic:  Clinical Trials in Oncology (Green, Crowley, Benedetti and Smith) Recently published  Principles of Anti-Cancer Drug Development (Hidalgo, Eckhardt, Garrett-Mayer, Clendenin)  Oncology Clinical Trials: Successful Design, Conduct, and Analysis (Kelly, Halabi, Schilsky)