HERU is supported by the Chief Scientist Office of the Scottish Executive Health Department. The author accepts full responsibility for this talk. Economic Evaluations – What they are and what they are not Moira Napper Health Economics Research Unit University of Aberdeen

Methodological standards Quality of reporting Value-added information sources Formulation of search strategies Issues Relating to the Health Economic Evaluation Literature

Purpose of Economic Evaluation Studies To ensure that the benefits from health care programmes implemented are greater than the the opportunity cost of such programmes by addressing questions of allocative efficiency or technical efficiency

Economic Evaluation Studies A FULL economic evaluation compares BOTH the costs AND consequences of two or more interventions Requires –identification –measurement –valuation of both costs and benefits

Economic Evaluation Methodologies Cost-benefit analysis (CBA) Cost-utility analysis (CUA) Cost-effectiveness analysis (CEA) Cost-minimisation analysis (CMA)

Economic Evaluation Methodologies

Partial Economic Evaluation Studies cost comparison/cost analysis study cost of illness study cost outcome description cost description

Mislabelling of Economic Studies Zarnke, KB; Levine, MAH; O’Brien, BJ. Cost-benefit analyses in the health-care literature: don’t judge a study by its label. Journal of Clinical Epidemiology 1997;50: Of a set of economic studies labelled CBA: 68% did not use defined CBA methodology 53% were cost comparisons only

Quality of Health Economic Evaluation S tudies Poor study design –poor data sources –failure to define clearly the economic evaluation method used –use of an inappropriate method if economic evaluation to address the research question –omission of sensitivity analysis to test robustness of modelling

Quality of Health Economic Evaluation Studies Inadequate reporting –lack of transparency by authors –journal editorial policy surrounding economic submissions Publication bias –publication bias in the clinical literature –motivation for conducting an economic evaluation

Guidelines for Economic Evaluation Conduct –Pharmaceutical Reimbursement e.g. Australian “fourth hurdle” –Methodology e.g. NICE, Washington Panel* Reporting –Drummond MF; Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 1996;313: Appraisal –NHS Economic Evaluations Database (NHS EED) guidelines for structured abstracts† –BMJ Guidelines *Weinstein MC. et al. Recommendations of the Panel on Cost-effectiveness in Health and Medicine. JAMA 1996;276: †Improving access to cost-effectiveness information for health care decision-making: the NHS Economic Evaluation Database. NHSCRD Report no.6 (2nd ed). York. NHS Centre for Reviews

Searching Bibliographic Databases for Economic Evaluations Indexing policy –Availability of controlled vocabulary –Specificity of indexing terms Indexing accuracy –Clarity of description –Accurate labelling –consistency

Indexing - MeSH Economics Costs and Cost Analysis Cost Allocation Cost-Benefit analysis Cost Control (+1) Cost of Illness Cost Sharing (+2) Health Care Costs Direct Service Costs Drug Costs Employer Health Costs Hospital Costs Health expenditures (+1)

Indexing - EMTREE Health Economics Economic Evaluation Cost Benefit Analysis Cost Control Cost Effectiveness Analysis Cost Minimization Analysis Cost of Illness Cost Utility Analysis Fee (+5) Health Care Cost Drug Cost Health Care Financing Hospital Cost (+3) Health Insurance (+8) Pharmacoeconomics(+5)

Value-added Bibliographic Sources of Health Economic Evaluations Databases NHS Economic Evaluation Database (NHS EED) Health Economic Evaluations Database (HEED) EURONHEED CEA Registry (formerly Harvard CUA Database) Journals Evidence Based Health Care ACP Journal Club

NHS Economic Evaluation Database NHS EED NHS Centre for Reviews & Dissemination University of York, UK Economic evaluations critical appraisal detailed structured abstract Other types of health economic analysis with bibliographic details MeSH searching 1995-

Health Economic Evaluations Database HEED Office of Health Economics, UK & International Federation of Pharmaceutical Manufacturers Associations (OHE-IFPMA). Structured format –Bibliographic details –Study definitions –Analysis and results –Keywords Expert review

The CEA Registry: Database of Cost- Utility Analysis Studies Harvard Center for Risk Analysis Reference list of studies that have used costs per QALY to measure health benefits Comprehensive league table of Cost-QALY Ratios League table of cost-QALY ratios which adhere to the Washington Panel Catalogue of Preference Scores used for QALYs Audit checklist

European Network of Health Economics Evaluation Databases (EURONHEED) Project Launched year development –Country-specific databases set to be available in regional databases covering 17 European countries –2 established viz. NHS EED (UK) and CODECS (France) –5 to be created viz. Germany, The Netherlands, Italy,Spain and Sweden

Health Economic Evaluation Studies Summary Points Strict definition criteria Quality variation Methodology Reporting Differences in bibliographic database indexing Specificity and Scope Use of value-added sources

National Information Center on Health Services Research & Health Care Technology (NLM/NIH) Etext of Health Technology Assessment (HTA) Information Resources (Chapter 11) Health Economics Information Resources: a self-study course