1. The role of simulation and modelling in health care Martin Utley, Clinical Operational Research Unit University College London

3. ZEUS PhD 1996 "A first study of the structure of the virtual photon at HERA“ under Professor David Saxon at the University of Glasgow. This work concerned the contribution of resolved photon processes to the dijet cross-section in photoproduction events with virtual and quasi-real photons. My HEP credentials My lack of HEP credentials

5. My understanding of MC simulation methods in HEP

6. Measured / postulated distributions Theoretical underpinning Immutable laws of nature (current best guess) Alternative models for certain physical processes Universally accepted models for other physical processes Unscrambling the impact on results of how measurements are made

7. ASIDE Gallivan S, Stark J, Pagel C, Williams G, Williams WG, Dead reckoning: can we trust estimates of mortality rates in clinical databases? Eur J Cardiothorac Surg. 2008 Mar;33(3):334-40 In 6 / 1198 reported deaths, the patient lived. In 4 / 724 reported survivals, the patient died. Cooper H, Findlay G, Martin IC, Mason DG, Mason M, Utley M, National Confidential Enquiry into Patient Outcome and Death (2008).

8. Measured / postulated distributions Theoretical underpinning Immutable laws of nature (current best guess) Alternative models for certain physical processes Universally accepted models for other physical processes Unscrambling the impact on results of how measurements are made Entire endeavour is concerned with testing, refining and verifying models

9. image courtesy of Dr Sally Barrington There are extensive parallels between detector physics and medical imaging. This is not what I do.

10. UCL Department of Mathematics University College London Gower Street London WC1E 6BT Tel: +44 (0)20 7679 4508 Fax: +44 (0)20 7813 2814 E-Mail: m.utley@ucl.ac.ukm.utley@ucl.ac.uk Web: www.ucl.ac.uk/operational-research Dr Martin Utley Director Clinical Operational Research Unit (CORU)

11. What counts for evidence in health care Combined analysis of many randomised controlled trials Single randomised controlled trial Epidemiological studies Follow up studies Anecdote modelling ?

12. prediction of risk estimating benefits of treatment decision support calculation of health insurance premiums The scope of simulation and modelling in health care capacity planning identifying bottlenecks scheduling staff rosters operation of emergency departments identifying what services to offer evaluating national policy design of screening programmes structure of services deciding whether to buy new drugs emergency planning

14. UK National Health Service Free for all at point of access Funded via general taxation Under political control Third largest organisation in the world

15. The different roles of modelling illustrating a point generating insight informing decisions making decisions?

16. Booked admissions policy Before 2001, short notice cancellations of elective operations were frequent. Government put in place policy whereby patients were given a firm commitment to date of surgery. Little thought was given to implications.

17. Full attendance No emergency admissions How many beds are needed to honour all commitments? Simple model

18. Number of beds required 16151413121110987654321 Probability.25.20.15.10.05 0 Expectation Making a firm commitment to patients requires reserve bed capacity Gallivan et al BMJ 2002;324:280-2.

19. Hospital 1 TC Hospital 2 New hospitals to separate routine elective cases from complex & emergency cases Introduction of Treatment Centres to UK

20. How could a TC affect efficiency? Reducing variability in length of stay through patient selection Economies of Scale Gains in efficiency for whole system? Structure and organisation of service Genuinely reducing length of stay Genuinely reducing variability in length of stay Management of patients Note: our work was limited to study effects associated with organisation of services

21. Hospital 1 TC Hospital 2 Compare capacity requirements We evaluated a large number of hypothetical scenarios......to identify circumstances in which a TC might be an efficient use of capacity

22. Worse Marginally better Better Much better Level of emergencies 0%10%20% Efficiency of system with TC

23. Findings In many circumstances, there is no theoretical benefit associated with treatment centres*. Theoretical benefits exist if there is successful identification of shorter-stay patients and a number of participating non-TC hospitals. *in terms of the efficient use of capacity – there are a whole host of other considerations Benefits rely on cooperation between providers. Meanwhile, other policies foster competition. Utley et al, Health Care Management Science, Sept 2008

24. Short term forecasting of PICU bed demand to assist managers Current occupancy ? Variable demand for beds ? Future demand for beds ?

25. Chance that demand for paediatric intensive care unit exceeds 9 beds or 10 beds Probability Consider staffing an extra bed in 3 days ? Pagel and Utley, ORAHS proceedings, forthcoming

26. Front page of British Medical Journal last week Mathematical modelling study £500 M decision concerning national vaccination programme Decision based on cost-per-QALY Jit et al, BMJ, Aug 9 2008

27. Woodman et al. The natural history of cervical HPV infection: unresolved issues. Nature Reviews Cancer, 2007; 7:11. HPV modelling Natural history of infection

28. HPV modelling Natural history of infection – model structure 1 SusceptibleHPV infected CIN1 CIN2 CIN3Cancer Can calibrate transition rates for this model to be consistent with empirical “cross-sectional” data.

29. HPV modelling Natural history of infection – model structure 2 SusceptibleHPV infected CIN1 CIN2 CIN3Cancer Can calibrate transition rates for this model to be consistent with empirical “cross-sectional” data.

30. HPV modelling Natural history of infection – model structure 3 SusceptibleHPV infected CIN1 CIN2 CIN3Cancer and this model.

31. HPV modelling Natural history of infection – model structure 4 SusceptibleHPV infected CIN1 CIN2 CIN3Cancer Resistant and this one.

32. HPV modelling Natural history of infection – model structure 5 SusceptibleHPV infected CIN1 CIN2 CIN3Cancer Resistant you get the idea..

33. HPV modelling MC approach to account for a myriad of uncertainties Economic model Cost of screening Cost of cancer treatment Cost of warts treatment Vaccine cost QALY loss due to screening QALY loss due to cancer QALY loss due to warts Cancer mortality rate Screening accuracy 250,000 different analytical models

34. Different tools used in health care modelling Discrete event simulation Monte Carlo simulation System dynamics Queueing theory Game theory Decision analysis Stochastic analysis Optimisation techniques Mathematical programming Hybrid methods

35. Restructuring services for common mental health problems

36. Snapshot of 1 year Simulation of traditional care Animated simulations facilitate engagement with clinicians and managers origins in simulation of shop floor / industrial processes

37. Simulation of proposed care Permits modelling of complex decision rules...... and queues and feedback

38. Number of People Traditional Model Stepped Care Model Seen by GPs 99989966 Referred by GPs 19772037 Successfully treated 402759 Unsuccessfully treated 11081 Not attending 203496 Not completing treatment 139500 In a queue 2308411 Average queuing time* 231 days46 days Different allocation of same resources can give better system performance *for patients completing treatment Example of output from a model WARNING For illustration purposes only

39. Pitfalls to simulation in health care Model development almost too easy - insufficient thought given to purpose of model detail added solely because it can be added modellers can start to believe their models. If you think 19 free parameters is untidy, you should see some of the models developed in healthcare

40. Client is scientific communityClient is alien Random means randomRandom means uncertain Immutable physical lawsHuman responses Laboratory conditionsUncontrolled environment Modelling in HEP Modelling in NHS

41. Clients Decisions are made by politicians, health care managers and clinicians...... reason vies with political dogma, professional rivalries & financial incentives*. * oh, and management consultants

42. A fundamental difference Decays and scatters are truly random processes. When can uncertain processes be modelled as random? Consider a sexually transmitted disease Random process or determined by characteristics of the individual? Important when considering multiple interactions.

43. Time dependence of models...patterns of sexual mixing among the young are subject to change. Physical laws are either static or time dependence is an intrinsic part of the model...

44. A B World before intervention of interest Intervention of interest Validating* models in health care model vs. reality * as opposed to calibration or tuning

45. A B C Intervention of interest + a whole lot more Validating models in health care model reality vs.

46. George Box said... “All models are wrong......some are useful”

47. All models are wrong...some are useful A modeller’s checklist ? What counts as useful?

48. One response but decisions will get made, with or without our input.

49. Are you a bright, financially secure, analytical thinker with exemplary people skills and the belief that you can improve the NHS? I hate you.

50. END

51. Uninfected individual Infected individual Uninfected individual Infected individual

52. The role of simulation and modelling in health care Martin Utley Director, Clinical Operational Research Unit University College London

58. prediction of risk estimating benefits of treatment

59. A hospital environment with unlimited capacity Variable demand for beds Variable length of stay Variable admissions ? ? ?

60. Chance that demand for paediatric intensive care unit exceeds 9 beds or 10 beds

65. Length of stay in intensive care Percentage of patients 151051 100 80 60 40 20 0

66. VARIABILITY

68. Compare capacity requirements for same case load Hospital 1 TC Hospital 2 Modelling to evaluate national policy initiative

69. A B World before intervention of interest Intervention of interest Validating models in health care Compare set of outcomes anticipated with the outcomes observed?

70. The role of simulation and modelling in health care Martin Utley, Clinical Operational Research Unit University College London

71. Population not homogeneous in terms of sexual activity Brownian motion models of sexual interaction do not necessarily apply Data are scarce and unreliable ?