Global Optimization of Complex Healthcare Systems Zelda B. Zabinsky Industrial and Systems Engineering University of Washington Seattle, WA April 18, 2017 Institute for Disease Modeling (IDM) Symposium

Global Optimization of Complex Healthcare Systems Often need numerical simulations to describe a complex healthcare system: Natural progression, spread and management of disease New technologies becoming available Allocation of scarce resources Global optimization (or simulation-optimization): Problem may be non-convex, black-box, discontinuous, involve integer and real-valued decision variables Uncertainties may be present Multiple objectives Global optimization methods are needed when the problem is ill-structured, non-convex, and possibly discontinuous, and will be discussed briefly. A new global optimization algorithm (Probabilistic Branch and Bound) has been applied to two problems in health care, and preliminary results will be presented. The first problem is allocation of a new technology, and involves a simulation and multiple objectives. The second problem is a Markov decision process with constraints, and the algorithm can provide a type of sensitivity analysis.

Several Random Search Algorithms for Global Optimization Sequential random search algorithms: Simulated annealing Cross-entropy Population-based: Genetic algorithms (cross-over, mutation) Particle swarm (position, velocity) Interacting particle algorithm with hit-and-run [Zabinsky, Stochastic Adaptive Search in Global Optimization, 2003] [Zabinsky, Stochastic Search Methods for Global Optimization, a book chapter in the Wiley Encyclopedia of Operations Research and Management Science, 2011] [Zabinsky, Random Search Algorithms for Simulation Optimization, a book chapter in the Handbook on Simulation Optimization, 2015]

Improving Hit and Run (IHR) IHR: choose a random direction and a random point f(x1,x2) x2 x1 [Zabinsky and Smith, Hit-and-Run Methods, a book chapter in Springer’s Encyclopedia of Operations Research & Management Science, 2013]

Optimization: What Do We Really Want? Do we really just want the optimum? What about sensitivity? Do we want to approximate the entire surface? Or just a region of interest? Multiple objectives? Role of objective function and constraints? How to account for uncertainty? Ex – do we have recourse variables or do we have to make a decision before “uncertainty is revealed”

Probabilistic Branch and Bound (PBnB) Approximate a set of “good” solutions (quantile, level set approximation) Handles black-box, noisy functions, including both integer and real-valued variables Single or multiple objective functions Provide probability bounds assuring the quality of solution set [Huang and Zabinsky, “Adaptive Probabilistic Branch and Bound with Confidence Intervals for Level Set Approximation,” Proceedings of the 2013 Winter Simulation Conference, 2013. ] [Huang and Zabinsky, “Multiple Objective Probabilistic Branch and Bound for Pareto Optimal Approximation,” Proceedings of the 2014 Winter Simulation Conference, 2014. ]

Approximated Level Set (Best 10%) Three test functions with contours: (A) Rosenbrock (B) Sinusoidal function (centered) (C) Sinusoidal function (shifted) Bold contour is target level set pruned maintained undecided

Parameter Estimation Given m=10,000 data points that have been randomly generated from the density with undisclosed parameters Determine the parameters by solving a maximum likelihood problem

Parameter Estimation PBnB finds the optimal solution (2,5) in seven iterations with 285 sample points

Allocation of Portable Ultrasound Machine to Orthopedic Clinics Simulate costs: (MRI expensive, ultrasound relatively low cost) and benefits (less travel time, less wait time) Diagnosis quality: Experience and training on using portable ultrasound machines may influence the diagnosis quality MRI accurate Ultrasound as accurate if well-trained

Allocation of Portable Ultrasound Machine to Orthopedic Clinics Simulate costs (MRI expensive, ultrasound relatively low cost) and benefits (less travel time, less wait time) Diagnosis quality: MRI accurate Uncertain accuracy of portable ultrasound Current Specialty Centers/Radiology Current Major Primary Care Clinics (Potential locations for orthopedic care)

Simulation-Optimization for Medical Imaging Resources Multiple objectives Tradeoff health utility and cost Efficient frontier [Huang, et al., 2015]

Results with 0.9 Accuracy Four Pareto optimal designs Solutions ranged from 11 to 5 portable ultrasound machines Varying amounts of reserved MRI capacity Burien and Northshore in all four designs

Hepatitis C Screening and Treatment Policies Chronic hepatitis C virus (HCV) infection is the most common blood-borne infection in the United States (~ 2.7 million are HCV infected) Policy decisions for HCV care: Determine the optimal HCV birth-cohort screening and treatment allocation strategies under spending budget constraints Analyze the impact of screening and treatment allocation strategies for each cohort group by decision time periods

Markov Chain of Health Status Three population groups: A: HCV Unknown B: HCV Positive C: HCV Negative Severity of disease Healthy: H Fibrosis: F0, F1, F2, F3, F4 Untreatable: UT Recovered: R1, R2, R3 Dead: M Group A: HCV unknown Group B: HCV+ Group C: HCV-

Optimization Model Maximize the discounted quality-adjusted life years (QALYs) Dynamic constraints based on the Markov model Budget constraint for each year: Cost for screening Cost for treatment Percentage of the budget for screening

Analysis and Policy Decisions  

Analysis Approach Use grid search to identify time periods with obvious dominant strategy [Li, Huang, Zabinsky, and Liu, accepted in Medical Decision Making (MDM) Policy & Practice, Dec. 2016]

PBnB Analysis First look at low budget (LB) scenarios Maximize discounted QALYs Subject to yearly budget constraint Decision variables – first two time periods Identify a set of solutions in the top 10 percent of QALYs

Approximated 0.1 Level Set for 50-59, Low Budget pruned maintained   undecided

Approximated 0.1 Level Set for 50-59, High Budget pruned maintained   undecided

Conclusions Many applications are non-linear, include uncertainty, and have multiple objectives Probabilistic Branch and Bound provides a set of solutions and enables trade-offs to be made Many possible simulation models On-going research into extensions of PBnB 22

Noisy Objective Function Idea is that we want a higher degree of confidence on our estimate of f(x) near the optimum. The range distribution reflects the level set of top delta percentile. Delta = \nu( level set S(y(\delta,S))) / \nu(S). For example, find the students in the top 10% of the class. S

Noisy Objective Function

Probabilistic Branch-and-Bound (PBnB)

Sample N* Uniform Random Points with R* Replications

Use Order Statistics to Assess Range Distribution

Prune, if Statistically Confident

Subdivide & Sample Additional Points

Reassess Range Distribution

If No Pruning, Then Continue … S