1. The role of NICE, and of the NICE committee, in the evaluation of health technologies
Marta Soares, Senior Research Fellow, Centre for Health Economics
Kuopio, Finland, August 2018
(some of the slides developed by Susan Griffin)

2. Outline About NICE Technology Appraisal NICE Committee
Value judgements
Decisions
Only in Research
Impact of NICE

3. About NICE

4. Role of NICE in England and Wales
Established in 1999 to:
Reduce unwarranted variation
Set quality standards
Accelerate uptake of good value innovation (mandatory funding)
Ensure additional resources are spent efficiently

5. “Probably not, but it’s worth a bloody good try.”
Will NICE work?
“Probably not, but it’s worth a bloody good try.”
Frank Dobson, Health Secretary, who established NICE in 1999, when asked whether he thought it would work.

6. Role of NICE in England and Wales
Established in 1999 to:
Reduce unwarranted variation
Set quality standards
Accelerate uptake of good value innovation (mandatory funding)
Ensure additional resources are spent efficiently
Considerations
Comparative clinical effectiveness
Cost-effectiveness - £/QALY (£20-30k threshold)
Social values and equity impact
Issues
Only select technologies appraised
No direct role in price setting or negotiation (indirect influence via patient access schemes)
Multiple stakeholders/highly political

7. NICE….what is it now ?
The National Institute for Health and Care Excellence (NICE) provides national guidance and advice to improve health and social care
It does this by:
Producing evidence-based guidance and advice for health, public health and social care practitioners.
Developing quality standards and performance metrics for those providing and commissioning health, public health and social care services;
Providing a range of information services for commissioners, practitioners and managers across the spectrum of health and social care.

8. Centres & support services
Clinical Practice
Clinical Guidelines
Medicines and Prescribing Centre
Safe staffing guidelines
Health Technology Evaluation
Technology Appraisals
Surgical interventions
Devices and Diagnostics
PASL
Scientific Advice
R&D
Health &
Social Care
Quality Standards
NICE pathways
Prevention
Accreditation
Public involvement
Implementation
Business, Planning
& resources
NICE International
The MPC (formerly the NPC) will take over responsibility for the BNF contract.
NHS Evidence is a service that enables access to authoritative clinical and non-clinical evidence and best practice through a web-based portal. It helps people from across the NHS, public health and social care sectors to make better decisions as a result.
Evidence Resources
Communication
Evidence Resources
Communication 8

9. Incorporating consideration of relevant social value judgements
Drug development
Under controlled conditions and compared to placebo:
Is the drug safe?
Does the drug do more good than harm?
Regulatory approval
In routine clinical practice and compared with existing treatments:
Do the additional clinical benefits justify the expected additional cost?
NICE/HTA
Use in
healthcare system
Incorporating consideration of relevant social value judgements

10. Technology Appraisals

11. NICE Technology Appraisals
Technology appraisal one branch of NICE
Entails assessment of clinical and cost effectiveness of health technologies
Medicines, surgical procedures, medical devices, diagnostic techniques, health promotion activities
NICE decision carries legal mandate
Ensure all patients have access to cost-effective treatments

12. Topic selection Only issue guidance for selected technologies
Topics identified by DH based on:
Health benefit across all patients for whom it is indicated
Impact on NHS resources if given to all eligible patients
Impact on other outcomes of interest, e.g. health inequalities
Inappropriate national variation in use
Institute able to add value by issuing national guidance

13. Defining the scope Sets out the decision problem
The disease area, patients, technologies covered by the appraisal and the questions it aims to answer
Intervention and comparators
Must be licensed for use in UK
Ideally include current NHS practice
Indication(s)
Scope may be narrower than license
Identifies consultees
Clinical and patient groups, manufacturers and comparator manufacturers

14. Evaluation of health technologies
Single Technology Appraisal (STA)
Evaluation undertaken by company
Critiqued by independent academic centre (Evidence Review Group)
Multiple Technology Appraisal (MTA)
Evaluation by independent academic centre (Assessment Group)
Additional evaluations invited from manufacturers
Fast Track Appraisal (FTA)
Less resource-intensive, NHS funding within 30 calendar days of guidance
Company base case ICER<£10,000 and most plausible ICER<£20,000
Simple cost comparison if ≥ health benefits & ≤ cost compared to existing NICE recommended technologies for same indication
NICE and ERG determine eligibility with consultee input
NICE technical team provide report to Committee

15. Reference Case
“Because the methodology of technology appraisal continues to develop, there remain areas of controversy and uncertainty, particularly in relation to the methods of cost-effectiveness analysis. However, it is important that the methods used to inform the Appraisal Committee's decision-making are consistent. For this reason, the Institute has adopted the approach of using a 'reference case' for cost-effectiveness analysis; this was chosen as most appropriate for the Appraisal Committee's purpose”
Describes preferred methods
Supported by Decision Support Unit Technical Support Documents

16. Reference Case (Cost-Effectiveness)
Use of the QALY central
Health service perspective for costs
Reference case is prescriptive and generic
No intention to limit methods development of innovative techniques
Reference case ≠ standardisation

17. NICE Appraisal Process
Independent
Assessment reports
Patient organisation submissions
Manufacturer submissions
Appraisal committee
Expert witnesses
Professional submissions
Patient witnesses 17

18. Appraisal committee

19. Technology Appraisal Committee
Independent advisory committee drawn from:
Academia, NHS, patient and carer organisations, manufacturers
Reflect spread of interests and expertise
Tasks
Consider evidence
Come to conclusions about cost effectiveness
Come to conclusions about impact on other outcomes of interest (e.g. health inequalities)
Follow NICE’s principles on appropriate threshold (end of life, uncertainty)
Issue guidance

20. Committees in action Four TAC, meet once per month
Discuss up to six topics in one day
Most appear over at least two meetings
Documents sent out in advance
Seen early by NICE, chair, vice chair, lead team
Prepare pre-meeting briefing (PMB)
Documents include final scope, PMB, submissions from company(s), submission from independent academic centre, consultee comments

21. Public gallery for Part 1
Committee meeting
Public gallery for Part 1
12-24 Committee members
NICE staff
Clinical and patient experts (not ATA)
Manufacturer
Independent academic centre

22. Evidence considered Prior to the meeting At the meeting
Submissions from company(s) and independent academic group
Consultees comments
At the meeting
Opportunity to put questions to experts, manufacturers and independent academic centre
Clinical and patient expert testimony

23. Role of Committee
Consider generalisability to NHS in England and Wales
Come to conclusions about most appropriate value judgements, analytical choices, assumptions
Request further evidence
Lead team to ERG/AG in advance of meeting
Questions during meeting
Specify further analyses during committee meetings
Form guidance

24. Value judgements
subjective statement of opinion rather than a fact that can be tested by looking at the available evidence. Normative statements are subjective statements – i.e. they carry value judgments

25. Value judgements
When conducting economic evaluation we are required to make a series of value judgements
Scope sets out decision problem but may not define population for CEA
NICE Reference Case (methods guides) sets out preferred approach but is not exhaustive
Further, evaluation may depart from scope and/or methods guide

26. Who should make value judgements?
The results can be sensitive to these
What evidence is ‘relevant’
How does evidence relate to setting (generalisability)
How should we combine the evidence using decision model (structural uncertainty)
How should we analyse the evidence (model uncertainty)
Committee must form a view about most appropriate approach and on what impact on the ICER will be if approach inadequate

27. Considerations EVIDENCE SCOPE What subgroups are relevant?
Are presented subgroups valid?
Are comparators represented?
Patients
Comparators
EVIDENCE
Has appropriate evidence been characterised in economic evaluation?
How does evidence relate to UK?
Baseline risk
Clinical effectiveness
Health related quality of life
Resource use
METHODS
Has Ref Case been followed?
Is model structure appropriate?
Have parameter values been estimated appropriately?
Statistical analysis
Decision model

28. Value judgements - scope
Why might evaluation depart from scope?
Clinical trials for interventions include only subset of population
Company targets high risk subgroup
Comparators not used in certain patients
Characterisation of comparator
E.g. best supportive care, current practice, watchful waiting
Dose/schedule differs from that used in practice

29. Value judgements - evidence
Clinical trials conducted outside UK
Clinical trial does not match license
Which trials to include
Other evidence from different settings
Evidence out of date

30. Value judgement - methods
Departure from Reference Case should only occur if unavailable
Reference Case cannot cover all options
Often no ‘right’ answer
E.g. Survival analysis for extrapolation
Scientific approach but no statistical test
Structure of decision model

31. Decisions

32. Conclusions What is most plausible ICER?
Across whole population
In subgroup(s)
Is current evidence sufficient?
What is appropriate threshold?
Range £20-£30,000
Do End of Life criteria apply?
Survival extension > 3 months for life expectancy < 24 months
Threshold up to £50k

33. Decisions Decision reached by consensus MTA process
Anonymous vote only if consensus not reached
MTA process
Evaluate set of mutually exclusive alternatives
Yes to one implies no to others
Single Technology Appraisal and ATA
Evaluate single product
Recommended as an option
No decisions about cost-effectiveness of other comparators

34. Types of decisions Recommend Optimise Only in research
In line with marketing authorisation
NHS legally obliged to fund and resource within 3 months as standard
Optimise
Recommended only in subgroup of licensed population
Only in research
Recommended for use within CDF
Not recommended

35. Only in Research Requires judgement that
Current evidence insufficient
Reasonable prospect of technology being CE
On balance benefits outweigh cost of research
Complex considerations at end of assessment of cost effectiveness
Committee not routinely provided with supporting evidence
Value of information
Feasibility and costs of further research

36. NICE vs. research funders
Secretary of State for Health
Funding
Topic selection
National Institute for Health Research (NIHR)
Health Technology Assessment Programme
Commissioned
academic team
Assessment
NICE
Secretariat
Technology
Appraisal Committee
Appraisal
Topic referral
Topics
Technology Assessment Report
Guidance to the NHS and implementation initiatives
The Health Technology Assessment (HTA) programme, set up in 1993, is part of the National Institute for Health Research (NIHR) and is the largest single national research programme for the NHS. It funds the production of independent research information about the effectiveness, costs and broader impact of healthcare treatments and tests for those who plan, provide or receive care in the NHS. Technology Assessment reports are commissioned by the HTA programme on behalf of NICE to inform its national clinical guidance to the NHS. Assessment reports are prepared by on-contract academic or specialist centres within the UK.
HTA spend by the NIHR in 2010/11 = £49.7 million
Adapted from Walley, T. (2007) MJA; Overview of Health technology assessment in England: assessment and appraisal187: 283–285 36

37. Cancer Drugs Fund Requires judgement that
Current evidence insufficient
Reasonable prospect of technology being CE
Managed Access Scheme agreed between company and NHS England
Data Collection Arrangement to detail evidence required
CDF Commercial Agreement to determine price
Drug becomes available immediately in CDF while further evidence is collected
Further data collection not necessarily via CDF
Review in approximately 2 years

38. Consultation
YES – Recommended and within license requires no consultation
In all other cases Appraisal Consultation Document issued for consultation
Consider responses and further analyses and/or evidence at second meeting
Ultimately issue guidance
Opportunity for appeal
Failed to act fairly or exceeded powers
Decision is perverse in light of evidence submitted

39. Impact of NICE

40. NICE decisions 1 March 2000 to 31 March 2018 Recommendation categories
Single Technology Appraisal
Multiple Technology Appraisal
Total
Recommended
165 (50%)
270 (61%)
435 (56%)
Optimised
88 (27%)
88 (20%)
176 (23%)
CDF
13 (4%) -
13 (2%)
Only in Research
5 (2%)
23 (5%)
28 (4%)
Not Recommended
58 (17%)
60 (14%)
118 (15%)
329 (100%)
441 (100%)
770 (100%)
Up to 31 March 2018, EoL was considered in 115 pieces of guidance, of which 59 resulted in positive recommendations.
142 Patient Access Schemes in place 40

41. Impact of NICE Independent body Rationing made explicit and public
Hitting the headlines
“Betrayal of 20,000 cancer patients: Rationing body rejects ten drugs (allowed in Europe) that could have extended lives”
Opportunity for public to take part
Consultation
View Committee meeting

42. Impact of NICE on health
In principle approve technology only if generate more health than displaced
Evidence suggests this is not the case and ICERs>£30k not rejected
Dakin et al. Health Economics 2014;
Further, empirical threshold may be lower than £20-£30,000 range
Claxton et al. Health Economics 2015;24(4):1-7

44. Some references and websites
BBC Documentary ‘The Price of Life’
Claxton K, et al. Informing a decision framework for when NICE should recommend the use of health technologies only in the context of an appropriately designed programme of evidence development. Health Technol Assess 2012;16(46)
Collins M, Latimer N. NICE’s end of life decision making scheme: impact on population health. BMJ 2013;346:f1363
Dakin H, et al The influence of cost-effectiveness and other factors on NICE decisions. DOI: /hec.3086', Health Economics
Claxton K, et al. Causes for concern: is nice failing to uphold its responsibilities to all nhs patients? Health Econ 2015;24:1-7

45. The role of NICE, and of the NICE committee, in the evaluation of health technologies
Marta Soares, Senior Research Fellow, Centre for Health Economics
Kuopio, Finland, August 2018