Appraising Evidence About Prognosis Updated for the third edition of the Users' Guides to the Medical Literature.

Patient Five A’s of EBM Ask Act Acquire Apply Appraise EBM, evidence-based medicine. This slide returns to the evidence cycle first explained in the Education Guide “An Approach to Evidence-Based Medicine.” This Education Guide focuses on the “Appraise” step in the evidence cycle, as we appraise evidence about prognosis. Appraise

Outline Introduction Users’ Guides to a study of prognosis Summary Objectives Definitions of key terms Design considerations Users’ Guides to a study of prognosis How serious is the risk of bias? What are the results? How can I apply the results to patient care? Summary 3

Objectives Be able to Define prognosis and prognostic factors Evidence literacy Recognize threats to claims that an attribute predicts future outcomes Understand measures of outcome over time: survival curves Evidence numeracy 4

Prognosis Clinicians help patients in 3 broad ways: diagnosing or ruling out medical and health-related problems, administering treatment that does more good than harm, and giving them an indication of what the future is likely to hold

Prognosis Definitions Prognosis: Prospect of survival and/or recovery from a disease as anticipated from the usual course of that disease or indicated by special features of the case Prognostic factors: Patient or participant characteristics that confer increased or decreased risk of a positive or adverse outcome from a disease Prognostic study: A study that enrolls patients at a point in time and follows them forward to determine frequency and timing of subsequent events

Prognosis Issues Possibilities (qualitative) Which outcomes could happen? Probabilities (quantitative) How likely are they to happen? Periods (temporal) Over what time period?

Prognosis Uses Predictive Prescriptive Adjusted analysis What the future is likely to hold Prescriptive To select treatment that does more good than harm, anticipate future state with treatment Adjusted analysis To deal with prognostic imbalance in a study comparing 2 management studies

Studies of Prognosis Study design consideration: Strength of association Factor Outcome Association

Studies of Prognosis Study design consideration: Strength of association ? Factor Outcome Causation

Studies of Prognosis Experience target outcome Patients at risk of experiencing target event Prognostic factor Time Do not experience target outcome Cohort and case-control designs are used to explore the association between prognostic factors and outcome. If controls are used, they are patients with different prognostic factors.

Studies of Diagnosis Target condition present Patients suspected of having target condition Criterion- standard test Diagnostic test The diagnostic design is presented here for comparison with the prognostic design on the previous slide. The rules of evidence for judging prognosis studies are similar to those of diagnostic studies. In a prognostic study, time is the criterion-standard test. Target condition absent

Outline Introduction Users’ Guides to a study of prognosis Summary Objectives Definitions of key terms Design considerations Users’ Guides to a study of prognosis How serious is the risk of bias? What are the results? How can I apply the results to patient care? Summary 13

Users’ Guides: Risk of Bias How serious is the risk of bias? Was the sample of patients representative? Were the patients classified into prognostically homogenous groups? Was follow-up sufficiently complete? Were outcome criteria objective and unbiased?

Risk of Bias Was the sample of patients representative? Ideal Population Sample Frame Sample Start Study Complete Study

Risk of Bias Was the sample of patients representative? Expectation People with a particular condition will, on average, experience similar outcomes as a population of similar people Requirements Ability to define and observe experiences in populations over time Enough information to match people to populations

Risk of Bias Was the sample of patients representative? Matching people to populations Demographics Age, gender, socioeconomic status, etc Diseases Severity, subtype Disorders Other illnesses or relevant conditions

Risk of Bias Was the sample of patients representative? Threats Referral bias Failure to clearly define study patients Lack of objective criteria for defining demographics, diseases, or disorders

Referral Bias Occurs when characteristics of patients differ between one setting (such as primary care) and another setting that includes only referred patients (such as secondary or tertiary care) Example: risk of recurrent childhood seizures 1%-5% in family practice 3%-76% in neurology clinics

Risk of Bias Was the sample of patients representative? Remedies to risk of bias Report of explicit or implicit filters passed before entering study Clear description of which patients were included and which were excluded from study

Risk of Bias Were the patients classified into prognostically homogenous groups? Expectation Outcome for the group should be applicable to each member of the group Requirement Patients should be at similar point in disease process

Risk of Bias Were the patients classified into prognostically homogenous groups? Threats Different demographics Different diseases Stage Severity Different disorders Comorbidities that may define subgroups with different prognoses

Risk of Bias Were the patients classified into prognostically homogenous groups? Protections against risk of bias Define and track any subgroups Ensure same demographic, disease (stage, severity), and disorders in each prognostic group Use clinical common sense Have investigators missed important subgroups with different prognoses?

Risk of Bias Was follow-up sufficiently complete? Threats Significant losses to follow-up Increased likelihood that those lost have significantly different outcomes

Risk of Bias Was follow-up sufficiently complete? Protections against risk of bias Simple sensitivity analysis Recalculate risk based on best-case scenario where all losses were free of adverse outcome Recalculate risk based on worst-case scenario where all losses suffered the adverse outcome Compare these recalculations to gauge the potential impact of losses

Risk of Bias Were study outcome criteria objective and unbiased? As subjectivity of outcome determination increases, importance of blinding to prognostic factors increases Impact of Observation Bias Objective Subjective Nature of Outcome of interest

Users’ Guides: Importance What are the results? How likely are the outcomes over time? How precise are the estimates of likelihood?

What Are the Results? How likely are the outcomes over time? Measures that relate events to time Survival rate Percent surviving at a given time Median survival Time at which 50% still surviving Survival curve Percent of original sample who have not yet had outcome of interest Where events are discrete and time of event is precisely known

Life Tables Common starting point of life table ................ .................... ............. ............ ..... ..... .................... .................... Start of Study Patient Enters Study

Survival Curves Left, Survival after myocardial infarction. Right, Results of hip replacement surgery: percentage of patients who survived without needing a new procedure (revision) after their initial hip replacement. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 1988;2(8607):349-360.

What Are the Results? How precise are the estimates of likelihood? Confidence intervals Range within which it is likely that true point estimate lies Precision drops as time from exposure increases Losses to follow-up and patients enrolled later in recruitment period

Survival Curve Precision This figure shows the risk for an event by age group in children with neuroblastoma. Survival curves are more precise in the earlier follow-up periods, indicated in the figure by narrower confidence bands around the lefthand parts of the curve. Wood LA, Coupland RW, North SA, Palmer MC. Outcome of advanced stage low grade follicular lymphomas in a population-based retrospective cohort. Cancer. 1999;85(6):1361-1368.

Users’ Guides: Applicability How can I apply the results to patient care? Were the study patients and their management similar to those in my practice? Was follow-up sufficiently long? Can I use the results in the management of patients in my practice?

How Can I Apply the Results? Were the study patients and their management similar to those in my practice? Threats Uneven application of therapies to different subgroups Uneven applications of therapies over time

How Can I Apply the Results? Was follow-up sufficiently long? Threats Study follow-up too short to detect all important outcomes

How Can I Apply the Results? Can I use the results in the management of patients in my practice? Does the effect of the prognostic factor cross a decision threshold? Reassure Observe Intervene

Outline Introduction Users’ Guides to a study of prognosis Summary Objectives Definitions of key terms Design considerations Users’ Guides to a study of prognosis How serious is the risk of bias? What are the results? How can I apply the results to patient care? Summary 37

Summary The design goal of a study of prognosis is to avoid systematic overestimation or underestimation of the likelihood of outcome events in the patients under study—to make the population representative Fore more advanced information on studies of prognosis, see How to Use an Article About Genetic Association

Original slides created by Robert Hayward, MD, Centre for Health Evidence Updated by Gordon Guyatt, MD, Kate Pezalla, MA, and Annette Flanagin, RN, MA

Terms of Use: Users Guides to the Medical Literature Education Guides PowerPoint Usage Guidelines JAMAevidence users may display, download, or print out PowerPoint slides and images associated with the site for personal and educational use only. Educational use refers to classroom teaching, lectures, presentations, rounds, and other instructional activities, such as displaying, linking to, downloading, printing, and making and distributing multiple copies of said isolated materials in both print and electronic format. Users will only display, distribute, or otherwise make such PowerPoint slides and images from the applicable JAMAevidence materials available to students or other persons attending in-person presentations, lectures, rounds, or other similar instructional activities presented or given by User. Commercial use of the PowerPoint slides and images are not permitted under this agreement. Users may modify the content of downloaded PowerPoint slides only for educational (non-commercial) use; however, the source and attribution may not be modified. Users may not otherwise copy, print, transmit, rent, lend, sell, or modify any images from JAMAevidence or modify or remove any proprietary notices contained therein, or create derivative works based on materials therefrom. They also may not disseminate any portion of the applicable JAMAevidence site subscribed to hereunder through electronic means except as outlined above, including mail lists or electronic bulletin boards.