1. Health Economics II – 2010 Health Economic Evaluations Part VI Lecture 4 Decision analytic modelling Presentation and use of economic evaluations Nils-Olov Stålhammar

2. Measurement vs. decision analysis Measurement Focus on estimating and testing hypotheses relating to particular parameters Concentration on relatively few parameters Focus on uncertainty in parameters Randomized trials as a vehicle for measurement Decision analysis Focus on identifying an appropriate course of action Informing decisions Identification of a preferred option based on expected values of the alternatives Explicit acceptance that there always will be uncertainty

3. The need for comparison of all relevant options - modelling when relevant clinical trials do not exist Synthesizing head-to-head comparisons –Clinical trials sometimes compare new compound to placebo or – at least – omit some relevant active comparators Informing decisions in the absence of hard data –Clinical trials may not be feasible because of Ethical considerations - for instance, when standard care is compared to less aggressive treatment Sample size requirements - screening Long follow-up – vaccination

4. Modelling in economic evaluations alongside clinical trials Combining several data sources to reflect all appropriate evidence –Resource use, unit costs, utilities etc. Extrapolating from intermediate clinical endpoints to final outcomes –From, for instance, clinical events avoided to life years gained (or, preferably, QALYs gained) Extrapolating beyond the period observed in a trial

5. Extrapolating beyond the clinical trial time period Same time horizon as in clinical trial –“Stop and drop”; this within trial measure will be an underestimate If longer time horizon; should treatment be assumed to continue? –If yes, should we assume same effect as in clinical trial or reduced effect? –If no, should we assume same rate of death for everyone after trial, or, higher rate of death in patients who received intervention, or, that treatment gives a continued benefit resulting in lower rate of death? Can the shapes of the survival curves within the trial give information? The biology of the intervention? Important to run alternative scenarios Low compliance in the long run, discounting and whish to minimise number of assumptions may to some extent justify assumption about stop of treatment

6. Inappropriate use of clinical data * efficacy rather than effectiveness * not representative of appropriate setting and/or country * biased selection Uncertainty in extrapolations based on assumptions Sometimes a 'black box‘ – lack of transparency Concerns about modelling

7. Keep it simple Transparent presentation Be concerned about data quality Explore uncertainty through sensitivity analysis Try to validate the model –Let someone else build a copy of the model in another programming language –Validate (parts of) output against other analyses/data Recommendations about modelling

8. Modelling

9. Tools for modelling Decision Trees - a diagram which illustrates all alternative courses of action in response to a specific problem Markov Models - describes several discrete states between which a person may move as time passes

10. Decision tree comparing omeprazole and ranitidine in reflux oesophagitis (Based on Lindberg & Jönsson, Läkartidningen 1992;89:2530-3) Probability Drug cost Effect Expected P C E Cost Effect P x C P x E 0.14 2,280 0 319 0.00 0.17 2,280 1 388 0.17 0.12 1,192 1 143 0.12 0.57 596 1 340 0.57 0.38 772 1 293 0.38 0.33 1,860 1 614 0.33 0.29 1,860 0 539 0.00 1.00 1,190 0.86 1.00 1,446 0.71 Sum RO Ome 20 mg Unhealed; Ome 20 mg Unhealed; Ome 40 mg Unhealed Healed 0.43 0.72 0.46 0.57 0.28 0.54 Healed Unhealed; Ome 40 mg Unhealed 0.38 0.54 0.62 0.46 Ran 150 mg x 2 Week04812

11. Decision tree comparing omeprazole and ranitidine in reflux oesophagitis (Based on Lindberg & Jönsson, Läkartidningen 1992;89:2530-3) Probability Drug cost Effect Expected P C E Cost Effect P x C P x E 0.14 2,280 0 319 0.00 0.17 2,280 1 388 0.17 0.12 1,192 1 143 0.12 0.57 596 1 340 0.57 0.38 772 1 293 0.38 0.33 1,860 1 614 0.33 0.29 1,860 0 539 0.00 1.00 1,190 0.86 1.00 1,446 0.71 Sum RO Ome 20 mg Unhealed; Ome 20 mg Unhealed; Ome 40 mg Unhealed Healed 0.43 0.72 0.46 0.57 0.28 0.54 Healed Unhealed; Ome 40 mg Unhealed 0.38 0.54 0.62 0.46 Ran 150 mg x 2 Week04812 Decision node Chance node Branch probability A pathway Pathway probabilities Pathway ’values’ Expected values

12. Markov processes A convenient way of modelling problems with repetitive events Well Sick Dead

13. Markov processes The patient is always in one of a finite number of states - Markov states The time horizon is divided into equal increments of time - Markov cycles; the length of each cycle depends on the problem studied During each cycle the patient may make a transition from one state to another - this is how events are modelled States can be transient, temporary (just once) or absorbing Transition probabilities can be constant or time- dependent (no. of cycles) –When the transition probability is constant the Markov models are referred to as Markov chains

14. Markov processes Each state is assigned a cost and an effect, the contribution to the overall result depends on the time spent in the state The Markovian assumption (a key limitation): –The transition probabilities depend only on the current health state – disease history is not taken into account Can be circumvented by more health states

15. Markov processes If probability of recurrence and/or death changes after first occurrence of illness Well 1 Sick 1 Dead Well 2 Sick 2 A 2nd ‘Well’ state to allow for a different set of probabilities

16. Markov cohort simulation - an example Well Sick Dead P=0.2 P=0.3 P=0.2 P=0.1 QALY-weight of the Well state = 1 QALY-weight of the Sick state = 0.7

17. Markov cohort simulation The Markov trace CycleWellSickDeadCycleTotalSum Start100-- 10.700.200.100.840.84 20.550.240.210.721.56 30.460.230.310.622.18 40.390.210.410.532.71 50.330.180.480.463.17 60.290.160.550.403.57...... The Markov model is typically run until all of the cohort are in the dead state – at least when life threatening diseases are studied

18. Markov cohort simulation The Markov trace CycleWellSickDeadCycleTotalSum Start100-- 10.700.200.100.840.84 20.550.240.210.721.56 30.460.230.310.622.18 40.390.210.410.532.71 50.330.180.480.463.17 60.290.160.550.403.57...... The Markov model is typically run until all of the cohort are in the dead state – at least when life threatening diseases are studied 70% of those who are well remain well 30% of those who are sick get well 0.70*0.70+0.30*0.20=0.49+0.06=0.55 50% of those who are sick remain sick 20% of those who are well get sick 0.50*0.20+0.20*0.70=0.10+0.14=0.25

19. Half-cycle correction in Markov models It’s assumed that all transitions occur at the end of each cycle – in reality, most transitions occur gradually (on average, half-way through) Rewards will be overestimated since a transitioning individual will have received a full cycle’s worth at the beginning of the cycle Most correct would be to implement a half cycle correction as a toll at each transition Instead, as an approximation, a) subtracting a half – reward at the beginning of the process (cycle 0), b) members remaining in the state when the process terminates, are given back the half-reward

20. Half-cycle correction in Markov models Two states with utility 1 and 0.5 respectively; 10 cycles; all individuals start in state 1 The utility for an individual transitioning in the middle of cycle 4 will be overestimated ; 7.5 (=5*1 + 5*0.5) vs. 7.25 (=4.5*1 +5.5*0.5) Correct solution would be to assign only a half-reward at the start of each cycle, and then at each transition a half-reward corresponding to the jump state (the state the individual will be in during the next cycle) The approximation: a) subtract a half –reward at the beginning of the process (cycle 0), b) give back a half-reward to individuals remaining in the state at the end of the process (Do it for all states!) 1 0.5 Cycles 0.51111 0.25 =7.25

21. Micro Simulation / 1 st order Monte Carlo Simulation / Random walk / Random trial One individual at a time transition through the model For each transition a random number is drawn from a Uniform [0,1] distribution – for the first transition: –If in range [0, 0.1], move to Death –If in range [0.1, 0.3], move to Sick –If in range [0.3, 1], stay in Well With a large number of individuals, the means will be the same as for Markov cohort simulation The main advantage is the ability to capture clinical history –A binary variable can be defined to distinguish between first occurrence of a disease and recurrences –Transition probabilities can depend on the value of the binary variable

22. Monte Carlo simulation CycleWellSickDead Start 1 2 3 4 5 6 X X X X X X X Binary variable for history of illness would be set to 1

23. 1 st and 2 nd order uncertainty Overall variability between patients –1 st order uncertainty –Reflected in standard deviations associated with a mean value –Micro simulation (one individual at a time with ‘known’ transition probabilities) will illustrate this uncertainty Parameter uncertainty –2 nd order uncertainty –Uncertainty in mean parameter values –Reflected in standard error of the mean –Probabilistic Sensitivity Analysis (PSA) increasingly used to illustrate this uncertainty (clear advantages compared to simple sensitivity analysis)

24. Probabilistic sensitivity analysis (PSA) A probability distribution is assigned to uncertain parameters –In practice only a relatively small number of distributions are relevant to consider –Parameters are sampled from assigned distributions – N samples For each sample the model can be evaluated with –Markov Cohort simulation (Expected value calculation) or –Micro simulation/1 st order Monte Carlo Simulation/Random walk/Random trial; X patients are sent through the model and means are calculated The distribution of the results from the N samples reflects the uncertainty Present as –Confidence interval around ICER or incremental net benefit –Scatter-plot in the C-E plane –CEAC

25. How often are patient-level simulation models used? Griffin S et al. Value in Health 2006;9:244-252.

26. Handling uncertainty in modelling based analyses Methodological –Reference case, sensitivity analysis Parameter uncertainty –Probabilistic sensitivity analysis Modelling uncertainty –Sensitivity analysis Generalizability –Sensitivity analysis

27. QALY league table Intervention £/QALY at 1990 prices Cholesterol testing and diet therapy (all adults aged 40–69) 220 Neurosurgical intervention for head injury 240 GP advice to stop smoking 270 Neurosurgical intervention for subarachnoid haemorrhage 490 Antihypertensive treatment to prevent stroke (ages 45–64) 940 Pacemaker implantation 1,100 Hip replacement 1,180 Valve replacement for aortic stenosis 1,410 Cholesterol testing and treatment (all adults aged 40–69) 1,480 Docetaxel (as opposed to paclitaxel) in treatment of recurrent metastatic breast cancer 1,890 CABG (left main-vessel disease, severe angina) 2,090 Kidney transplantation 4,710 Breast cancer screening 5,780 Heart transplantation 7,840 Cholesterol testing and treatment incrementally (all adults aged 25–39) 14,150 Home haemodialysis 17,260 CABG (one-vessel disease, moderate angina) 18,830 Hospital haemodialysis 21,970 Erythropoietin treatment for anaemia in dialysis patients (assuming 10% reduction in mortality)54,380 Addition of interferon-α2b to conventional treatment in newly diagnosed multiple myeloma 55,060 Neurosurgical intervention for malignant intracranial tumours 107,780 Erythropoietin treatment for anaemia in dialysis patients (assuming no increase in survival)126,290 Adapted from Hutton J et al. PharmacoEconomics 1996; 9(Suppl 2): 8–22.; Maynard A. The Economic J 1991; 101: 1277–86.; Nord E, et al. PharmacoEconomics 1997; 12: 89–103.

28. Caveats re. QALY league table Is the methodology of the studies sound and homogenous? –Rate of discount –The method for estimating health state preferences –Range of costs and consequences considered –The setting –Choice of comparison programme

29. Caveats re. cost-effectiveness threshold Society’s WTP for an additional QALY will depend on the size of the program –Would like to know the true opportunity cost Ethical reasons may influence society’s WTP Evidence that people are concerned with the distribution: –Unwilling to discriminate between patients on the grounds of the size of the QALY benefit –Tendency to distribute resources to equalise outcomes – prefer to treat severely ill patients even if the benefit is low –Health benefits to younger people are valued higher than similar benefits to older people –Many whish to give lower priority to treatment of illness caused by life style –Whish to discriminate in favour of those with dependants

30. Cost per QALY gained thresholds  Acceptable additional cost per QALY gained by using a more effective treatment strategy  No official thresholds, but… –UK 1 : £ 30,000(≈ € 45,000) –US 2 : USD 50,000 -100,000(≈ € 40,000 – 80,000) –Sweden 3 : SEK 500,000(≈ € 55,000) 1.Raftery J. NICE: faster access to modern treatments? Analysis of guidance on health technologies. British Medical Journal 2001;323:1300-3. 2.Ubel P, Hirth R, Chernew M, Fendrick M. What is the price of life and why doesn’t it increase at the rate of inflation? Arch Intern Med 2003:163;1637-41. 3.Socialstyrelsens riktlinjer för hjärtsjukvård. Artikelnummer 2004-102-2. Available from: http://www.sos.se. http://www.sos.se

31. 0 50 100 150 200 250 PBAC Decisions: Some Correlation With Cost-Effectiveness Cost/life year gained (AUS$) Recommended at Price Recommended at Lower Price Reject Source: George et al, 1999

32. 0 50 100 150 200 250 Decisions Are Not Driven Only By Cost-Effectiveness Cost/life year gained (AUS$) Recommended at Price Recommended at Lower Price Reject Similar cost- effectiveness but different outcome Source: George et al, 1999

33. Probability of rejection by NICE From: Devlin N et al. Health Economics 2004; 13:437-52 Model 1 – ICER Model 2 – ICER, UNCERTAINTY Model 3 – ICER, UNCERTAINTY, BURDEN Model 4 – ICER, UNCERTAINTY, BURDEN, OTHER THERAPY

34. Transferability of results from economic evaluations affected by Basic demography and epidemiology Availability of health care reosurces Treatment traditions Incentives to health care givers Relative prices Population values for WTP and/or utilities

35. Old exam question Assume that you are planning for a modelling analysis of a choice between two treatments. a) You consider to set up a Markov model but you realise that the ‘Markov assumption’ is too restrictive, i.e. you would like to build in memory allowing for an individuals disease history to affect transition probabilities. Describe (conceptually) two ways in which this can be done. b) The main source of uncertainty is the uncertainty around the estimates of the treatment effects. Describe the main elements of a probabilistic sensitivity analysis (PSA) when it is used to illustrate this uncertainty in the modelling analysis. c) The result from a PSA can be presented as a scatter plot in the so called cost-effectiveness plane. Illustrate how a cost effectiveness acceptability curve is derived from such a scatter of points.

36. Old exam question Regarding the transferability of economic evaluation results from one country to another. Describe at least four factors/aspects which usually differ between countries and which are likely to affect the cost-effectiveness of various interventions, thereby limiting the transferability of economic evaluation results from one country to another.

37. Old exam question a) Regarding assessment of uncertainty. a) What are the main drawbacks of a conventional sensitivity analysis? b) What is a threshold analysis? c) Describe the main elements of a probabilistic sensitivity analysis (PSA) when used to illustrate parameter uncertainty in a model. d) What is a cost-effectiveness acceptability curve (CEAC) and how can it be derived from a CE-scatter (i.e., a scatter of points on the so called cost-effectiveness plane)?

38. Old exam question A mechanistic use of a cost-effectiveness threshold to allocate health care resources would imply that all QALYs have the same value regardless of who the gainer is. But several studies have shown that the general public are concerned with the distribution of health gains and that there is an unwillingness to discriminate between patients on the grounds of the size of the QALY benefit only. Describe a situation (a patient group) where the general public – according to studies – consider the social value of a QAYL-gain to be higher than what otherwise would be regarded as normal. Similarly, describe a situation (a patient group) where the general public – according to studies – consider the social value of a QAYL-gain to be lower than what otherwise would be regarded as normal.

39. Old exam question A decision tree is presented below. The following terms represent different parts of the decision tree: Pathway Probabilities, Decision Node, Expected Values, Branch Probability, Chance Node, Pathway, Pathway Values. Associate each term with a number 1-7, as shown in the second figure, to indicate which part each term represents.