1. Economic evaluation of health programmes Department of Epidemiology, Biostatistics and Occupational Health Class no. 24: Nov 26, 2008

2. Plan of class  Transferability to other settings: Examples  Birch and Gafni’s critique of NICE  PBMA

3. Example:Expected cost of 3 more months of misoprostol prophylaxis (Drummond et al. 92)  To prevent gastric ulcers in patients on NSAIDs with abdominal pain  US trial (Graham et al. 88): Patients with OA, 400 micrograms daily for 3 months: 5.6% endoscopically determined lesions vs 21.7% with placebo  800 micrograms: 1.7%  Fewer lesions: lower expected HC costs  Evaluate CE in US but also UK, France and Belgium

4. Decision tree assumptions  No misoprostol: ulcer rate as in placebo arm, less 40% to account for silent ulcers  Tx arm: non-compliers assigned trial placebo ulcer rate  Diagnostic workup and ambulatory care patterns in each country ascertained by local expert panels  Hospital admission rates from epidemiological surveys  Surgical rates and LOS from routine hospital statistics  Free-standing surveys of costs in some countries

5. Source: Drummond et al. 2005, p. 337

6. Results by country Misoprostol more expensive in US yet more cost- effective also: Why?

7. Source: Wikipedia, http://en.wikipedia.org/wiki/Acute_coronary_syndrome, accessed Nov 25 2008http://en.wikipedia.org/wiki/Acute_coronary_syndrome

8. Other example: Cost- effectiveness of GPAs in the UK (Palmer et al. 05)  Glycoprotein 11b/111a antagonists (GPAs) in management of non-ST-elevation acute coronary syndromes (ACS) – to prevent platelet aggregation  Several clinical trials, mostly not in UK  Need to consider:  Use of GPAs in trials vs use in clinical practice in UK (need to address relevant questions)  Baseline cardiac event rates in trials same as in UK?  Relative risk reductions related to baseline risks?

9. Relevant use strategies in UK context  Evidence base:  GPAs vs non-GPAs – medical management  GPAs as adjunct to PCI  4 strategies of interest:  1: Initial medical management  2: Prelude to PCI  3: Accompaniment to PCI  4: Use other blood thinners instead (aspirin, heparin)

10. Relative treatment effects: Data sources StrategyWhere did estimates of relative treatment effects come from? 17 medical management trials; 30,280 patients 21 trial; 1,265 patients 310 trials; 15,951 patients The authors needed to make various adjustments to trial results to make the estimated relative treatment effects as relevant as possible to the UK context

11. Adjusting baseline event rates  When study done, PCIs less available, and less commonly done in UK:  Implies higher rates of death and MI in UK than found in non-UK trials  Clinicians likely to select ACS patients for acute PCI differently – this too affects PCI rates in UK vs other countries  Find baseline rates based on UK experience:  Prospective Registry of Acute Ischaemic Syndromes in the UK (PRAIS-UK), plus  audit of unstable angina patients at large UK cardiac centre

12. Relative risk reductions related to baseline risks?  Once baseline risks adjusted, can relative risk reductions (treatment effect) be assumed to be the same in UK as elsewhere?  Test using meta-regression: is log relative risk related to baseline risk?

13. “Having constructed a model with UK-specific data on baseline probabilities of clinical events, it is necessary to address the question of whether the relative risks associated with GPAs, which have been estimated in the trials, should be adjusted to reflect differences in UK practice. To inform this decision, meta-regression analysis was undertaken to establish whether, across published trials and taking each strategy separately, the relative risk in a trial was related to the absolute baseline risk in that study. No statistically significant association was found, which may reflect the small number of trials in the analysis. For this reason, the relative risks from the trials have been incorporated into the model without adjustment, which is equivalent to assuming that relative risks are transportable across health care systems whilst the baseline risks in those studies are not.” Source: http://www.nice.org.uk/nicemedia/pdf/glycoproteinmodelreport.pdf

15. Additional adjustments  Sensitivity analysis: use of clopidogrel as a fifth strategy  Use of long-term Markov model

20. Birch and Gafni’s critique of NICE

21. NICE: National Institute of Clinical Excellence (now National Institute of Health and Clinical Excellence)  Agency serving England and Wales  Inspiration for Québec’s proposed “Institut National d’Excellence en Santé”  “One of Britain’s greatest cultural exports, along with Shakespeare, Newtonian physics, the Beatles, Harry Potter and the Teletubbies” (Smith 2004, cited in Birch and Gafni 2007)

22. Background: NICE 1 Original NICE guidelines (2001): Failed to address NHS objectives Implicit threshold (HoC SC, 2002, WHO 2003) Implicit (invalid) assumptions (Birch & Gafni 2002) £836m additional costs in 2005 (Bryan & Gold 2007) ‘biggest bang for the bucks’ or ‘biggest bucks for a bang’? (Birch & Gafni 2006) Underlying assumptions dismissed (Johannesson & Weinstein (1993) Concerns with guidelines criticised (Culyer et al. no date) Source: This and following slides with blue background from presentation by Stephen Birch, 2008

23. APPLYING THE THRESHOLD RULE 1 NICE approach: Choose A Effects = 285 QALYs Maximise health gain: Choose B & C Effects = 326 QALYs Programme that satisfies ICER threshold does not represent efficient use of resources Programme ICER A. Topical Fluoride 49,120 B. Pap Smears 52,080 C. Early child dev. 59,700 D. Diabetes preven. 117,650 ICER threshold: £50K/QALY $M QALYs 14 285 10 192 8 134 2 17 Budget: £18 million

24. NICE 2 Policy context: NICE not involved in setting NHS budget Maximisation of health gain from NHS resources Approach: Reference case: Cost Effectiveness Analysis “establish whether differences in costs between options can be justified in terms of changes in health effects” Measure: Incremental Cost Effectiveness Ratio (ICER) Affordability: beyond remit – net impact on NHS resources aimed only at “effective planning”

25. NICE 2 ICER Threshold: ‘Inappropriate’ Opportunity cost of programmes displaced by new technology or shadow price of budget (Weinstein and Zeckhauser 1973, Claxton et al. 2006) Requires information on all possible programmes Dynamic: NHS budget & programmes change ICERs of current programmes ‘legitimate reference’ Past decisions – informed estimates of health forgone in other NHS activities (Claxton 2007) Implementation not considered

26. NICE 2 guidelines: Explicit cost-effectiveness thresholds £20,000/QALY £30,000/QALY Adopt Don’t adopt May or may not adopt

27. NICE 2: Multiple thresholds Judgements about “effective use of NHS resources” No link to ‘constrained maximisation’ objectives “In the absence of any information about opportunity cost, (decision-makers) cannot attempt to achieve the efficient use of resources” (Cookson et al 2001) Assumes unlimited additional resources at constant marginal opportunity cost (Birch & Gafni 1993, Sendi & Briggs 2001) Affordability not efficiency (Rawlins & Culyer 2004) If affordability could be separated from efficiency – no need for economics (Williams 2004)

28. Implications of NICE approach CEA associated with substantial increase in expenditures without evidence of net health gain Increased expenditures = Increased incomes (Evans 1984) Pervasive influence of ‘manufacturers’ “As economic models involve numerous assumptions that are subjective and often subtle, they are particularly susceptible to bias driven by conflicts of interest” (Morgan et al 2000) “A higher ICER threshold makes NICE more popular with sellers” (Williams 2004) “NICE has effectively become an advocacy mechanism by which... The pharmaceutical industry extract more public money from the NHS. Instead of challeging the pharmaceutical industry to show value for money, NICE has become their ‘golden goose’ (Cookson et al 2001)

29. NICE and conflict of interest Morgan et al (2000) Manufactures influence what questions are asked and how Increasing prevalence of companies (with high profile academic researchers as principals or sitting on Boards) specialising in serving evaluation needs of manufacturers Abraham (2002) Publicly defensible assessments uncompromisingly in the interests of public health Post Thalidamide: expert committees ‘entirely independent’ ABPI – who fund OHE – refused to support proposal

30. NICE: Defending the indefensible? Inappropriate allegations about the motives and integrity of those involved in developing NICE methods (Culyer et al. n.d.) Innuendos preventing establishment of prevalence of COI (Bryan and Gold 2007) Public documentation of relationship with pharma: NICE 1 steering committee: 3 of 14, not including chair of the Policy Board of the Office of Health Economics, an organisation funded by Association of British pharmaceutical Industry NICE 2: Methodology working party: None

31. NICE: Searching for a threshold WTP per QALY Current UK research (Donaldson et al.) “for use as an appropriate threshold for NHS decision- makers” 1.Resources cannot be invested in health care to ‘purchase’ individual QALYs 2.Perfect divisibility and constant returns to scale underlay derivation of WTP/QALY from surveys 3.Application of a WTP/QALY to programmes with different QALYs assumes perfectly elastic demands for health (and other goods) WTP per QALY not relevant to NICE (Culyer et al 2007)

32. Alternative approaches - PBMA Redeployment of available resources between uses Determine if redeployment increases benefits through ‘marginal analysis’: QALY league tables are seen as particularly useful in making comparisons across programmes (Mooney 2003) Marginal analysis based on ‘greater benefit per pound spent’ (Mitton and Donadson 2004) ICER remains the basis of comparisons

33. An economic approach to constrained maximisation Integer programming (Birch and Donaldson 1989) Consider all possible portfolios of programmes Incorporates constrained resources, indivisibilities, non- linearities and other policy constraints (e.g. equity goals) Data hungry (but so are RCTs!) Increasing efficiency: Practical approach (Birch and Gafni 1992) Identify strategy to fund new programme A (e.g., less of B & C) Estimate outcomes forgone if strategy used, E B+C If E B+C < E A unambiguous improvement in efficiency Adapted to deal with uncertainty (Sendi et al. 2002) and sample size calculations (Gafni et al. 2005)

34. Two minute summary N on-comparable options being compared I ncreases in expenditures unplanned and without any evidence of net health gain C annot determine the outcome maximizing threshold E xcludes important information on available budget and costs and effects of other interventions

35. Additional health benefits Vaccination OHOH H1H1 Resources

36. Additional health benefits Vaccination OvOv H1H1 O tf Additional health benefits Topical fluoride ICER H 1 = (CD) -1 D C

37. Additional health benefits Vaccination OvOv H1H1 O tf Additional health benefits Topical fluoride ICER H 2 = (EF) -1 < ICER H 1 = (CD) -1 D C H2H2 F E