Validation / citations

Validation u Expert review of model structure u Expert review of basic code implementation u Reproduce original inputs u Correctly respond to changes in parameters u Consistent with other analyses u Consistent across versions

Documentation specifications Validation requires adequate documentation to… uDescribe clinical model uDescribe programming algorithms uDescribe inputs (sources / values / data quality) uDescribe validation strategy

Citations u Basic rationale and structure -- Matchar et al., Annals of Internal Medicine (1997) u Bootstrapped sensitivity analysis -- Parmigiani et al., Medical Decision Making (1997) u Application to acute stroke trial -- Samsa et al., Journal of Clinical Epidemiology (in press)

Outline u Rationale for modeling u Stroke model described u Applying the SPM to a randomized trial (*) u Extensions

Applying the SPM to an RCT Basic idea: After the short follow-up period typical of an RCT, we are likely to observe that the intervention induces small shifts in the distribution of morality / disability (measured at the conclusion of follow-up). uModeling can be used to account for the effects of these shifts on long-term outcomes.

Disability measure u Disability could be measured by the Rankin Scale (RS), Barthel Index (BI), or a number of other instruments. u For concreteness, we use the RS with 5 categories. The analysis easily extends to other scales and/or cut-points.

Risk (hazard) ratios for disability on subsequent outcomes* RS Utility h(IS) h(MI) h(DT) Cost *As derived by an expert panel, supported by published and unpublished literature

Rationale u The utilities for RS of 0-1 and 5 were obtained from the PORT’s patient survey. Other values obtained by interpolation Consistent with unpublished RCT data u Increased hazard of death results from sequelae of inactivity such as aspiration pneumonia, etc. u Daily costs greatly increase when disability level implies high likelihood of institutionalization.

SPM results by Rankin* RSSurvivalQALY Cost , , , , ,911 *Beginning 6 months after IS, for patients with mean age 70 years

Thought experiment -- typical RCT results (6 month follow-up) Intervention Placebo QALY Cost 27,000 24,000 That is, little difference in QALY in absolute terms. For cost, we assume equivalent utilization plus the cost of the drug.

Typical RCT results (cont’d) RS Intervention Placebo 0-131% 25% 221% 15% 311% 10% 46% 10% 511% 15% Died20% 25%

Typical results (cont’d) The previous results describe a “break-through drug”, but the ICER based upon 6 months of follow-up is only (27,000-24,000) / ( ) = 150,000 $/QALY.

Combining short- and long-term outcomes For each patient, we assign long-term outcomes (cost and QALYs) based upon RS at 6 months. For example, total costs then become 6-month costs (observed) + expected long- term costs (simulated).

Long-term QALYs by group u Intervention: (.31)(6.07) + (.21)(4.70) + (.11)(3.37) + (.06)(2.13) + (.11)(1.09) + (.20)(0) = 3.49 QALY u Placebo: (.25)(6.07) + (.15)(4.70) + (.10)(3.37) + (.10)(2.13) + (.15)(1.09) + (.25)(0) = 2.94 QALY –Identical multipliers -- groups only differ in proportion of patients in each RS category –Same weighted average calculations for cost

Comprehensive CEA u Intervention QALY = = 3.74 u Intervention cost = 27, ,818 = 194,818 u Placebo QALY = = 3.17 u Placebo cost = 24, ,275 = 200,275 u ICER = (194, ,275)/( ) = -9,573 $/QALY

Conclusion By considering its long-term effects, the intervention moves from “not cost effective” to “cost saving,” even if the price of the drug is increased substantially. This result holds across a wide range of sensitivity analyses.

Outline u Rationale for modeling u Stroke model described u Applying the SPM to a randomized trial u Extensions (*)

Extensions u International applications u Planning trials

International applications u Basic idea: Use the same SPM but change the inputs. u In theory, every component of the input data (e.g., natural history, effect of covariates, effect of interventions, QOL, cost) could differ by nation. u In practice, the biggest differences will involve utilization patterns (and thus cost).

Comment The size of the task depends upon the difficulty in estimating the relevant parameters. uThis, in turn, depends what we are willing to assume versus what will require additional data collection. uOnce new estimates have been derived, inserting them into the SPM is straightforward.

International costs Costs are affected by uDifferent utilization patterns uDifferent unit prices uDifferent payment mechanisms / perspectives (affecting which components of cost to include in the calculation)

Recommendation u Begin with US data on utilization and unit prices. u Then use expert judgement (supported by the literature as available) to posit any changes in utilization patterns. u Attach country-specific unit prices. u Limit the analysis to the cost categories and time periods of interest to decision-makers in that country.

Planning trials u The SPM can help to determine the (minimum) clinically important difference in short-term disability. u This, in turn, would be the basis for the trial’s sample size calculations.

Observation u Many stroke trials have been under- powered. u That is, the size of the effect which is significant from the perspective of public health / CEA is much smaller than that suggested by clinical intuition (Samsa et al, Journal of Clinical Epidemiology, in press).

Observation Many RCTs of acute stroke treatments have had other sources of statistical inefficiency - - for example, pertaining to the choice of target population, the choice of measure, and so forth.

Summation u Because most benefits of stroke treatments accrue in the long-term, modeling is necessary. u The SPM is a well-validated model which can be helpful in both the planning and analysis of trials. u Planned extensions include international practice patterns and user-friendly software.