The Duke Stroke Policy Model (SPM) MI IS TIA ASY DTH HS Bleed

Developers u David Matchar, MD -- principal investigator u Greg Samsa, PhD -- project director, statistician u Giovanni Parmigiani, PhD -- statistician, software developer u Joe Lipscomb, PhD -- health economist u Greg Hagerty, MS -- software developer

Outline u Rationale for modeling (*) u SPM described u Applying the SPM to a randomized trial u Extensions

Rationale for modeling u Why model? u Arguments for modeling u Arguments against modeling u Discussion u Conclusions u Application to stroke

Why model? (cont’d) “To me, decision analysis is just the systematic articulation of common sense: Any decent doctor reflects on alternatives, is aware of uncertainties, modifies judgements on the basis of accumulated evidence, balances risks of various kinds,...”

Why model? (cont’d) “ considers the potential consequences of his or her diagnoses and treatments, and synthesizes all of this in making a reasoned decision that he or she decrees right for the patient…” (cont’d)

Why model? “… All that decision analysis is asking the doctor to do is to do this a lot more systematically and in such a way that others can see what is going on and can contribute to the decision process.” -- Howard Raiffa, 1980

Advantages of modeling u Clarifies decision-making u Simplifies decision-making u Provides comprehensive framework u Allows best data to be applied u Extrapolates short-term observations into long-term u Encourages “what if” analyses

Disadvantages of modeling u Ignores subjective nuances of patient- level decision-making u Problem may be incorrectly specified u Inputs may be incorrect / imprecise u Usual outputs are difficult to interpret or irrelevant to decision-makers

Individual decision- making is subjective u For individual decision-making, primary benefit of modeling is clarification. u As normative process, decision-making works better for groups. –Most applications involve group-, rather than individual-level, decisions (e.g., CEA, purchasing decisions, guidelines).

An aside u Interactive software (possibly including models) shows great potential to help decision-makers (e.g., patients, physicians, pharmacy benefits managers) clarify and make better decisions. –We are developing prototype for a “user-friendly version” of the SPM.

Some models are mis-specified u A good model will simplify without over-simplifying. u Poor models exist, but this need not imply that modeling itself is bad. u We need more explicit standards under which models are developed, presented, and assessed.

An observation u The fundamental problem with many of the poor models in circulation is that they assume the answer they are purporting to prove (often, that a treatment which is trivially effective or even ineffective is nevertheless cost-effective). u Users are understandably wary.

Model inputs may be incorrect/ imprecise u This problem is often most acute for utilities and costs, and least acute for natural history and efficacy. –We need more and better data on cost and quality of life. u The less certain the parameter, the greater the need for sensitivity analysis.

An aside u In practice, the conclusions of a model / CEA are never stronger than the strength of the evidence regarding efficacy. u If the evidence about efficacy is weak, then modeling / CEA should not be performed.

Usual outputs are difficult to interpret u In academic circles, results are presented as ICERs using the societal perspective. –Present this as a base case for purposes of publication / benchmarking. –Also present multiple outcomes from multiple perspectives (vary cost categories, vary time periods, present survival, event- free survival, QALY, …).

General conclusions u Modeling is of great potential benefit and indeed is sometimes the only reasonable way to proceed. However, models must be held to a high standard of proof. u Although the standard reference model cannot be ignored, modeling should be done flexibly, with the needs of the end user in mind.

Application to acute stroke treatment u RCTs follow patients in the short- term, but the large majority of benefits accrue in the long-term. u Simple heuristics will not suffice to adequately trade off complex risks, benefits, and costs. u Modeling allows a large body of evidence to be efficiently synthesized.

Outline u Rationale for modeling u Stroke model described (*) u Applying the SPM to a randomized trial u Extensions

SPM described u History / background u Types of analysis u Structure u Validation / citations

SPM history / background

SPM development (cont’d) u First version developed in 1993 by Stroke PORT u Goals of stroke PORT: –Summarize epidemiology of stroke –Describe best stroke prevention practices –Describe current practices, and test methods for improving practice

SPM development SPM was used: uTo summarize epidemiology of stroke u To support CEA uAs a basic organizing structure for the PORT

SPM versions u Original C++ code (uses waiting time distributions, research tool, difficult to extend) u New S+ code (uses waiting time distributions, highly structured code used as development tool, inefficient) u New C++/Decision-Maker code (uses Markov-based cycles, intervention language, better interface, extendable)

New C++ version u Decision-Maker used to specify natural history and effect of interventions in a decision tree format u Efficient C++ code used as simulation engine u Expandable into a web-based tool