1. Health Technology Assessments and Systematic Reviews
Marcia Reinhart, DPhil CMPP
Principal, Tantalus Medical Communications

2. Health Technology Assessments
Many definitions
“Technology assessment in health care is a multidisciplinary field of policy analysis. It studies the medical, social, ethical, and economic implications of development, diffusion, and use of health technology.”
– International Association of HTA (INAHTA)
“A form of policy research that examines short- and long-term consequences of technology. . . safety, efficacy, patient-reported outcomes, real-world effectiveness, cost, and cost-effectiveness as well as social, legal, the application of a health-care ethical, and political impacts.”
– International Society for Pharmacoeconomics and Outcomes Research (ISPOR)

3. Regardless of the technology assessed, HTAs include similar elements
Clinical efficacy
Safety
Cost-effectiveness

4. HTAs are used to support many health care decisions
Clinicians and patients
Prescribing decisions
Practice guidelines
Public and private payers
Drug plan formularies
Level of coverage
Hospitals
Technology acquisition
Hospital formularies

5. What is 'Market Access'?
Governments/private insurers determine the price and availability of drugs via formulary designation
Figure: Eichler GH et al. Nat Rev Drug Discov 2010;9:

6. Market Access around the world
Just as health care systems vary worldwide, the payer assessment process differs from country to country

7. Who conducts and uses HTAs?
National/regional government-based agencies
Independent academic or consulting groups

8. How do HTAs fit into the decision-making process?
HTAs as part of the market access continuum
Manufacturer
Conducts literature review, economic analysis, etc.
Submits HTA-like document based on agency template
submission
HTA agency
Assesses manufacturer submission
May conduct independent review or rely on HTAs from 3rd parties
Submits recommendation to payer
report
Payer
Makes final decision on reimbursement and availability of drug or device   
Payer often agrees with HTA agency, although may be influenced by additional factors     

9. Systematic review – an essential step of the HTA process
Cochrane Collaboration definition:
“… attempts to identify, appraise and synthesize all the empirical evidence that meets pre- specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making.”

10. Systematic review: process
Define research question
Develop study inclusion/exclusion criteria
Patients
Intervention(s)
Comparator(s)
Outcomes
Study type
Develop literature search strategy
Typically conducted across multiple databases
Search strategies should be broad enough to capture all relevant publications, but narrow enough to avoid excessive irrelevant information
Screen studies for inclusion and collect relevant data
PICOS

11. Example – acupuncture for the treatment of fibromyalgia
Define research question

12. Example – acupuncture for the treatment of fibromyalgia
Define research question
Develop study inclusion/exclusion criteria
Patients
Diagnostic criteria?
Age?
Disease severity?
Comorbidities?
Ethnicity?

13. Example – acupuncture for the treatment of fibromyalgia
Define research question
Develop study inclusion/exclusion criteria
Patients
Intervention(s)
Frequency?
Definition of acupuncture?
Adjunct therapies?

14. Example – acupuncture for the treatment of fibromyalgia
Define research question
Develop study inclusion/exclusion criteria
Patients
Intervention
Comparator(s)
Placebo?
No treatment?
Drugs?
Other TCM techniques?
Physical therapy?

15. Example – acupuncture for the treatment of fibromyalgia
Define research question
Develop study inclusion/exclusion criteria
Patients
Intervention(s)
Comparator(s)
Outcomes
Pain reduction?
Physical function?
Quality of life?
Days off work?
Adverse events?
Fatigue?

16. Example – acupuncture for the treatment of fibromyalgia
Define research question
Develop study inclusion/exclusion criteria
Patients
Intervention(s)
Comparator(s)
Outcomes
Study type
RCTs?
Real-world evidence?
Non-RCTs?
Economic evaluations?
Quasi-randomized?

17. Small group exercise – develop PICOS criteria

18. Example – acupuncture for the treatment of fibromyalgia
Develop literature search strategy
Use both indexing terms (e.g. MeSH) and plain text terms
Draft searches to identify studies in categories (e.g. disease terms, intervention terms), combined at end
Disease terms

19. Example – acupuncture for the treatment of fibromyalgia
Develop literature search strategy
Combined
Therapy terms

20. Example – acupuncture for the treatment of fibromyalgia
Develop literature search strategy
Medline
Embase
Cochrane
congress abstracts
ALL COMBINED (duplicates removed)
Other sources?

21. Example – acupuncture for the treatment of fibromyalgia
Screen studies for inclusion

22. Data extraction and synthesis: The meta-analysis
Multiple studies are combined statistically to reveal the overall effect of an intervention
A summary (pooled) effect estimate is calculated as a weighted average of the effects estimated in the individual studies
Individual studies
Box represents estimated effect
Lines represent 95% CI
Size of box represents weighting
Summary intervention effect

23. Example – acupuncture for the treatment of fibromyalgia
Meta-analysis for mean difference in pain

24. Risk of bias
A bias is a systematic error, or deviation from the truth, in results or inferences
Selection, performance, detection, attrition, reporting
Several tools have been developed to assess bias

25. Role of medical writers in HTAs
NICE single technology appraisal (STA)
Section A – Decision problem
Description of technology under assessment
Context
Equality
Innovation
Statement of decision problem
May include >1 research question

26. Role of medical writers in HTAs
Systematic review
Meta-analysis
Non-RCT evidence
Safety data
NICE single technology appraisal (STA)
Section B – Clinical and cost effectiveness
Clinical evidence
Cost effectiveness
Section C – Implementation
Assessment of factors relevant to NHS/other parties
References
Appendices
Related procedures for evidence submission
Systematic review for published analyses
Report and interpret results of own economic model(s)

27. Example – new class of drug to treat rheumatoid arthritis
Section B – Clinical and cost effectiveness
Clinical evidence

28. Example – new class of drug to treat rheumatoid arthritis
Section B – Clinical and cost effectiveness
Clinical evidence
Results from the systematic review:
Phase III clinical trial of new drug (B) vs placebo (A)
Phase II clinical trial of new drug (B) vs placebo (A)
3 RCTs of competitor (C) vs placebo (A)
2 RCTs of competitor (C) vs other competitor (D)
3 single-arm non-RCTs
How do we use this information to compare the new drug (B) to competitors C and D??

29. Network meta-analyses/indirect treatment comparison
Network meta-analysis can be used to infer the comparative effectiveness of treatments of interest that have not been directly compared in trials

30. Example – new class of drug to treat rheumatoid arthritis
Section B – Clinical and cost effectiveness
Clinical evidence
Safety data
AEs from RCTs and non-RCTs
Post-marketing surveillance data (if available)
Other sources of real-world evidence

31. Example – new class of drug to treat rheumatoid arthritis
Section B – Clinical and cost effectiveness
Clinical evidence
Cost effectiveness

32. The cost-effectiveness plane
A new treatment can be compared with an existing therapy based on its effect on cost and effect differences
Cost difference (+)
Cost difference (−)
Effect difference (+)
Effect difference (−)
Dominated
Dominant
Trade-off SC
SC = standard care

33. Key concept: QALYs
In cost-utility analyses, effectiveness is measured in quality-adjusted life years (QALYs)
QALYs incorporate both quality and quantity of life gained from an intervention
Improvement of quality of life with new treatment
Estimated using utility values
Improvement of quantity of life with new treatment
QALYs without new treatment
(standard care)
Time
QoL
Figure: adapted from McCabe C. Hayward Group Ltd. 2009

34. Key concept: ICER = ICER Incremental cost-effectiveness ratio
“costs per outcome” (cost-effectiveness analysis) or “costs per QALY” (cost-utility analysis)
CostsTreatment – CostsStandard care
QALYsTreatment – QALYsStandard care
= ICER

35. Life-years gained (LYG) cost, Treatment vs Std. Care
Key concept: ICER
CostsTreatment – CostsStandard care
QALYsTreatment – QALYsStandard care
= ICER
Total costs
Life-years gained (LYG)
Utility for LYG
QALYs
Treatment
$20,000 7
0.5
3.5
Standard Care
$10,000 5
0.6
3.0
Incremental
cost, Treatment vs Std. Care
life-years
cost/LYG
QALYs
ICER
cost/QALY
$10,000 2
$5000/year
0.5
$20,000/
QALY

36. Optional group exercise – calculate the ICER of our new arthritis drug

37. Economic analysis: “willingness to pay” threshold
Some countries/health care systems use an official or unofficial threshold of acceptable ICERs for new technology assessments
Cost difference (+)
Cost difference (−)
Effect difference (+)
Effect difference (−)
Reject
Accept
WTP threshold SC

38. Example – new class of drug to treat rheumatoid arthritis
An economic model determines that the ICER for the new drug is $40,000/QALY
If the WTP was $50,000/QALY, this drug would fall into the range of acceptable cost/QALY
Cost difference (+)
Cost difference (−)
Effect difference (+)
Effect difference (−)
Reject
Accept
WTP threshold SC X

39. Cost-utility: sensitivity analysis
Calculating costs per QALY requires estimation of several parameters
utility values
cost of treatment
duration of treatment/patient lifespan (“time horizon”)
Base case = best estimate of all parameters
Sensitivity analyses test alternative parameter estimates to assess the range of possible results

40. One-way sensitivity analysis
Each parameter varied one-at-a-time over plausible range
base case
Tornado
diagram
ICER ($thousand/QALY)
Figure: Reynolds MR et al. Circ-Arrhythmia Elec 2009;2:362–69.

41. Example – new class of drug to treat rheumatoid arthritis
Sensitivity analysis shows that the new drug may rise above the WTP threshold in certain circumstances
Cost difference (+)
Cost difference (−)
Effect difference (+)
Effect difference (−)
Reject
Accept
WTP threshold SC X X X X X X X

42. Other sensitivity analyses used in HTA
Scenario analysis
Similar to a one-way sensitivity analysis, however, assessments are made by varying multiple parameters at the same time
Example = best-case and worst-case analyses
Probabilistic sensitivity analysis
“Drug X has a 86% probability of falling within the WTP threshold of $50,000/QALY”
More statistically complex; assesses distributions of data for multiple inputs over the course of multiple simulations

43. Cost-utility is not the only economic analysis used in HTAs
Cost-consequences analysis
Estimates cost and value of interventions, but leaves it to the reader to draw conclusions
Cost-minimization analysis
Compares input costs, but assumes outcomes are equivalent (e.g., bioequivalent drug comparisons)
Cost-effectiveness analysis
Measures costs in dollars and reports outcomes in natural health units (e.g., mmHg reduction) or ratios (differences in cost/difference in outcomes)
Cost-utility analysis
Measures outcomes based on years of life and quality of life obtained with treatment
Cost-benefit analysis
Enumerates and compares costs and benefits achieved in monetary terms
Budget impact analysis
Estimates effect of intervention on overall cost to organization or health plan

44. Desirable qualities in HTA medical writers
Well-versed in literature searching
Experience with multiple databases and multiple interfaces (e.g. PubMed vs Ovid)
Attention to detail – screening and data extraction
Understanding of evidence grading systems
Knowledge of statistical methods (e.g. meta- analysis, NMA, sensitivity analyses)
Understanding of health economic models
Excellent writing skills

45. Thank you