Using Value of Information to Prioritize Future Research: A Case Study Gillian D. Sanders Ph.D. Duke Evidence-Based Practice Center Duke University

Duke EPC Collaborators n n Evan Myers M.D., M.P.H n n Laura Havrilesky M.D. n n David Matchar M.D. n n Benjamin Powers M.D. n n Matthew Crowley M.D. n n Ravi Dhurjati Ph.D. n n Gregory Samsa Ph.D. n n Amanda McBroom Ph.D. n n Michael Musty B.A. n n Rebecca Gray Ph.D.

Overview of Project n n Investigating role of value of information analysis (VOI) in helping prioritize research gaps identified as part of EPC reviews l l Systematic review of priority setting methods l l Survey of existing priority setting methods l l Two case studies of evidence reports with models Comparative effectiveness of ACEI/ARBs in patients with ischemic heart disease Comparative effectiveness of management strategies for uterine fibroids

Background n n Uterine fibroids are the most common, non-cancerous tumors in women of childbearing age and the second most common reason women of childbearing age undergo surgery n n Despite the prevalence and possible complications of uterine fibroids, the currently available evidence-based literature on their natural history and optimal management is insufficient n n DECIDE project to develop research priorities for comparative effectiveness research for fibroids l l Grew out of evidence gaps identified in systematic reviews conducted by EPCs l l Used modified delphi/ nominal group methodology n n Current project investigating use of modeling in the uterine fibroids evidence gap prioritization process

Methods n n Updating model developed for original Duke Evidence Report on Fibroids n n VOI for subset of priority questions identified through DeCIDE project n n Comparison of VOI results to results of DeCIDE project n n Presentation and discussion of VOI results with subset of stakeholders who participated in DeCIDE project l l Value of value of information? (VOVOI!) l l Optimal timing relative to qualitative process

Questions n n Is VOI feasible for a condition like fibroids where the evidence gaps are so large? n n Can VOI complement other priority-setting methods? n n If so, what is best timing of VOI? n n Model focuses on subset of research gaps identified in DeCIDE project l l Further model development needed l l Results interesting, but preliminary l l Focus is on whether this process could be helpful, not these specific results

Model Schematic

Model n n Interventions: l l Myomectomy (surgical removal of fibroid) l l Uterine artery embolization (UAE—angiographic occlusion of uterine blood supply) l l MRI-guided focused ultrasound (FUS—ultrasonic energy applied directly to fibroid) n n All 3 preserve uterus l l Varying amount, quality of data on recurrence FUS—shortest recovery time, highest recurrence l l Limited data on pregnancy outcomes with UAE, FUS (both discouraged in women seeking pregnancy)

Model n n Patient population l l Fibroids more common in African-American women At younger ages, more severe disease l l Age and racial distribution taken from patient registry of women undergoing UAE Mean age 40, 48% African-American l l Assumed 25% planned pregnancy Based on pre-procedure data from registry Approximately 5% reported pregnancy in follow-up Age distribution modeled separately based on desire for pregnancy

Model n Age- and race-specific probabilities of fertility, miscarriage, preterm delivery l Older women less likely to get pregnant, higher risk of miscarriage or preterm birth l African-American women higher risk for preterm birth n This version of model assumes no difference in pregnancy outcomes between treatments l Key area of uncertainty l Left out for purposes of simplifying analysis/presentation for stakeholders

Model n Specific treatments/outcomes after recurrent symptoms not modeled l Simplification l Subjects stayed in “retreatment” state until end of simulation l Utility of “retreatment” varied widely n Model run for 3 years or until patient reached age 45 l Spontaneous pregnancy highly unlikely after 45 l Evidence for interaction between declining ovarian function and treatment efficacy Necessary next step, but out of scope

Model Output n n Cumulative probability of events l l Assumes 25% of patients attempting pregnancy 2500 women actively trying to get pregnant within 6 months of procedure required to pregnancies, live births within 3 years Sample size requirements to determine differences in reproductive outcomes between procedures daunting

Graph shows the proportion of simulations (n=10,000) that a given strategy was preferred at different thresholds of cost- effectiveness. Graph shows the proportion of simulations (n=10,000) that a given strategy was preferred at different thresholds of cost- effectiveness. FUS has shorter length of stay, faster return to work, lower short- term complications than other procedures  lower costs FUS has shorter length of stay, faster return to work, lower short- term complications than other procedures  lower costs

PARAMETEREVPPI Utility after retreatment$1053 Time to return to work$1047 Time before trying to get pregnant$1046 Relative recurrence rates$1043 Time between recurrence and retreatment $1038 Cost of complicated cases$1037 Length of stay (all)$1030 Cost of uncomplicated cases$9.50 Individual Parameters Highest VOI related to recurrence—both probability and impact on QOL Highest VOI related to recurrence—both probability and impact on QOL Peri-procedural factors with high uncertainty or high potential contribution to differences (cost of complications, time to return to work) also high VOI Peri-procedural factors with high uncertainty or high potential contribution to differences (cost of complications, time to return to work) also high VOI

Different Patient Populations Relatively small differences when run separately for white and African- American patients Main known differences between groups are younger age, higher risk of adverse pregnancy outcomes in black women Differences likely greater if age-specific treatment effects or treatment effects on reproductive outcomes included WhiteAfrican-American

Different Patient Populations In this version of model, pregnancy parameters are fixed Pregnancy is a competing risk with recurrence, and assume QOL if pregnant same as improved symptoms Fewer recurrences, higher overall QOL Again, potential importance of age-dependent recurrence risk, differential effects of treatment on pregnancy Desires pregnancyDoes not desire pregnancy

(VERY) Cautious Interpretations of VOI Results n n Highest priority topics by VOI related to durability of treatment and effect on QOL l l Aligns with #1 priority from DeCIDE qualitative process l l Unlikely to change with additional model refinement n n Importance of impact of treatments on pregnancy outcomes population-dependent l l Pregnancy outcomes important, but small proportion of overall patient population l l Sample size issues in addressing this research gap

Feedback from Stakeholders n All found results provocative, interesting, helpful in thinking about research priorities n Some found VOI results most helpful after qualitative exercise l Qualitative process narrowed options Easier to understand VOI when only a few parameters varied n Others preferred before or parallel to qualitative process l Would have helped focus discussion Potential way to resolve persistent differences between stakeholders

General Lessons/Next Steps n n Even with pre-existing model as basis, more work than anticipated l l Would add significantly to resources needed if done as part of CER Some of this may be attributable to relatively poor state of evidence and relative complexity of modeling outcomes (symptoms AND pregnancy) for this clinical area n n Stakeholders generally very positive about use of quantitative analysis l l One major challenge is need to educate on both VOI approach in general AND content-specific application Could consider shared resource on methods

General Lessons/Next Steps n n Optimal timing still not clear l l Need to develop better method for testing n n Fibroids model/VOI promising l l Adding uncertainty about relative treatment effects on reproductive outcomes l l Adding age-specific effects on ovarian function and interaction with treatment effects on symptoms l l More specific modeling of retreatment options l l Move decision point earlier in process to evaluate nonsurgical treatment (medication, watchful waiting)