Fundamentals of Biostatistics Lecture 2 1.Clinical Trials 2.Validity/Reliability 3.Assessing Evidence

Randomized Clinical Trials (RCT)  Randomization first used by RA Fisher in agriculture expts in 1920s  First clinical trial using randomization in 1931 by Anderson on use of sanocrysin on TB patients. Also, first trial using blinding.  Placebo first used in RCT in 1938 in cold vaccine trial.

Randomized Clinical Trials (RCT) Fundamental Point: A properly planned clinical trial is a powerful expt’l technique for assessing effectiveness of a drug or intervention. Defn: A clinical trial is a prospective study comparing the effect of intervention(s) against a control in humans.

Randomized Clinical Trials (RCT) Phases of clinical trials: Phase I: Determine tolerance of a compound in humans (i.e., how large a dose can be given until unacceptable toxicity?). Phase II: Evaluation of biologic activity or effect, and estimate rate of adverse events. Phase III: Definitive comparative trial. Designed to determine effectiveness and its role in clinical practice.

Randomized Clinical Trials (RCT) Terminology Efficacy: How well an intervention works in an ideal setting. Effectiveness: How well an intervention works in actual practice. Phase IV: Evaluation of long-term saftey of an intervention believed to be effective in phase III trials.

Randomized Clinical Trials (RCT) Why are clinical trials needed? Because evaluating the effectiveness of a treatment using uncontrolled observations is very difficult, since other factors affecting treatment outcome may not be balanced in treatment groups.

Randomized Clinical Trials (RCT) Advantages of RCT:  Groups more comparable b/c confounding variables are balanced  Ability to detect small effects  Most stat tests based on assumption of random allocation of pts to trt groups (validity)

Randomized Clinical Trials (RCT) Disadvantages of RCT:  Expensive and time consuming  Subject pool may not be representative  Effective treatment may be withheld  Expose pts to dangerous drugs

Randomized Clinical Trials (RCT) What is the question?  Each clinical trial must have a primary question.  The primary, as well as secondary questions, must be carefully selected, clearly defined, clinically relevant, and stated in advance.  This includes the choice of response variable (ie., true clinical endpoint or surrogate endpoint).

Randomized Clinical Trials (RCT) Study population  The study population is a subset of the population with the condition or characteristics of interest defined by the eligibility criteria.  This population should be defined in advance, stating unambiguous inclusion (eligibility) criteria.  The impact that the inclusion criteria will have on study design, ability to generalize, and participant recruitment must be considered.

Randomized Clinical Trials (RCT) Randomized control studies  comparative studies with an intervention group and a control group  the assignment of the participant to a group is determined by the formal process of randomization.

Randomized Clinical Trials (RCT) Randomized control studies...  Sound scientific investigation almost always demands that a control group be used against which the new intervention can be compared.  Randomization is the preferred way of assigning participants to control and intervention groups.

Randomized Clinical Trials (RCT) Randomization  Randomization tends to produce study groups that are: 1.comparable with respect to known and unknown risk factors 2.Removes investigator bias in the allocation and treatment of patients 3.Guarantees statistical tests will have valid significance levels.

Randomized Clinical Trials (RCT) Randomization...  Simple randomization is easiest to understand and use, but randomization can also be blocked, stratified, adaptive, etc.  Randomization is best accomplished by an independent central statistical unit.

Randomized Clinical Trials (RCT) Blinding (Masking)  Ideally, a clinical trial should use a double- blind design to avoid potential problems with bias during data collection and assessment.  If using a double-blind design is not feasible, a single-blind design and other measures to reduce bias should be used.

Randomized Clinical Trials (RCT) Baseline Assessment  Relevant baseline data should be measured in all study participants before the start of intervention.  These baseline measurements can be used to determine eligibility (if obtained prior to assignment to treatment group).  Can be also used to determine if the randomization produced identical groups (if not, can be used as covariates for adjustment of imbalance).

Randomized Clinical Trials (RCT) Data Analysis  Excluding randomized participants or observed outcomes from analysis and sub-grouping on the basis of outcome or response variables (including non- adherence) can lead to biased results of unknown magnitude or direction.  Including all randomized participants in the analysis, in the group they were assigned, is the intent-to-treat principle.  The intent-to-treat analysis is viewed as the most valid approach (least susceptible to bias).

Randomized Clinical Trials (RCT) Intent-to-Treat Principle Once randomized Always analyzed!

Randomized Clinical Trials (RCT) Covariate Adjustment  While randomization eliminates bias, it does not guarantee comparable baseline characteristics of patients in different treatment groups in a particular trial.  Baseline balance is not a requirement for obtaining valid variables.

Randomized Clinical Trials (RCT) Covariate Adjustment...  Imbalance will matter only if characteristic is related to patient outcome, ie., it is prognostic.  When randomization leads to chance baseline imbalance, estimates of treatment effects will be biased when using unadjusted analysis.

Validity and Reliability Validity Represents the degree to which a measurement represents a true value. Reliability A measure of the reproducibility of a result or observation. ie., How closely do repeated measurements on the same object agree?

Validity and Reliability  Errors can be caused by a lack of either validity or reliability  Validity and reliability are related. If a measure is unreliable it is not capable of producing valid results.  Better reliability is necessary, but not sufficient for validity.

Threats to Study Validity Bias Selection Information Confounding Regression to the Mean

Threats to Study Validity Selection Bias Distortion of effect estimate because of (i) manner subjects selected: biased sampling, (ii) selective losses: loss to follow-up and non- responses. (Case-control studies are particularly susceptible)

Threats to Study Validity Information Bias Distortion of effect estimate when measurement of exposure or disease is systematically inaccurate. (i) misclassification bias - incorrect classification of exposure or disease, (ii) recall bias: accuracy of self-reported data varies across comparison groups.

Threats to Study Validity Other common types of bias: (i) Investigator/Patient bias (ii) Measurement error Many forms of bias can be eliminated or reduced by using randomization and blinding (preferably double- blinding).

Threats to Study Validity Confounding Type of bias occurring when effect of exposure mixed-up with one or more extraneous variables. can create appearance of exposure-disease relationship, when none exists. can hide true nature of exposure-disease relationship.

Threats to Study Validity Confounding... due to presence of key relationships between extraneous variable(s) and both exposure and disease. confounding variables are risk factors for the disease, or a correlate of a causal factor. confounding variables are associated with exposure of interest.

Threats to Study Validity Confounding... Addressing the Problem: At the design stage we can use: – Randomization – Matching At the analysis stage we can use: – Stratified analysis (Mantel-Haenszel Methods) – logistic regression

Threats to Study Validity Regression to the Mean (RTM) tendency of extreme observations by chance to move closer to the mean when repeated What is the danger of ignoring RTM? Causality may erroneously be inferred

Threats to Study Validity Regression to the Mean (RTM)... Examples: Patients having higher than average cholesterol levels at initial screening, have lower levels on repeat. Acute pain patients seek help when symptoms severe, and any change is likely to be improvement => useless treatment may appear effective.

Threats to Study Validity Regression to the Mean (RTM)... RTM is a function of: Correlation between pre- and post-treatment values. How extreme the pre-treatment values are.

Threats to Study Validity Regression to the Mean (RTM)... RTM implies that if we select subjects because they appear abnormal on some test, AND do nothing to them, they will seem to improve when retested, thus treatment effects become confounded with RTM.

Threats to Study Validity Regression to the Mean (RTM)... Four ways to minimize RTM: 1. Increasing reliability of screening test 2. Testing each subject twice, and requiring all tests be extreme before entry into study. 3. Adjust for the correlation of pre/post measures 4. Use a randomized placebo-control trial

Types of Reliability 1.Interobserver – agreement among observers (Kappa, Intraclass correlation) 2. Test-Retest – stability, does the same measure give the same results repeatedly? 3. Parallel form – Two parallel test forms with different items are correlated. 4. Split-half- Split individual measure into two random parts.

Ten Steps for Evaluating Evidence 1.Be skeptical 2.Don’t rely on biological plausibility 3.Reliable info requires comparison 4.Ensure cause precedes the effect 5.Post-trial questions maybe unreliable 6.Pre-trial question should be specific and clinically relevant

Ten Steps for Evaluating Evidence... 7.Discovering small effects requires randomization 8.Be wary of conflicts of interest 9.Non-specific exposure effects can be important 10.Unblinded examiners may introduce bias.