Setting Standards for Global Health Costing Anna Vassall, LSHTM, on behalf the GHCC consortium

Why set standards in costing? Improving the nature and extent of use of cost data: Systematic reviews of costs suggest a wide variety of costing methods and metrics Poor quality may lead to poor decisions - but how to judge poor quality Improving the production of cost data Inefficiency – what is important? Limited use of current guidelines Limited capacity in costing

"Data that is comprehensive" would have been a good additional option. What do you value most in cost data? Overall Rank Item Score Total Respondents 1 Data that is transparent 180 35 2 Data that is accurate 166 32 3 Data that is relevant 145 34 4 Data that is reliable 142 31 5 Data that is timely 123 6 Data that is robust 101 29 7 Data that is precise 69 27 It is artificial to have to order them, as I value and require all of the above "Data that is comprehensive" would have been a good additional option.

The approach What is a ‘reference case’? The aim of a reference case is to provide guidance on a set of standardised set of principles, methods and reporting standards US panels on Cost-Effectiveness in Health and Medicine iDSi reference case for economic evaluations Country specific reference cases Tool for both users and producers

Reference Case content Set of ‘acceptable’ principles Methodological guidance on how to achieve those principles (theory and evidence based) Reporting standards Standardisation for specific interventions with additional guidance where available => Reference case compatible guidelines/tools

I don't think one can or should be developed What would be the greatest benefit of adopting a Reference Case for collecting cost data in LMIC? I don't think one can or should be developed To ensure consistency of methods and comparability of results across different studies/ settings/ time as well as to improve the quality of costing studies. Ensuring cost data was transparent so that users could apply it correctly to decision makers Increase understandability and comparability of data. Encourage efficiency of costing.

What would be the greatest risk of adopting a Reference Case for collecting cost data in LMIC? Limited acceptability for future innovative approaches to cost data collection and analysis Could put additional burden on researchers (both with and without capacity/skills) and increase costs of data collection. It would become quickly outdated. If too general, it is not useful, and if too specific it may too long and overwhelming for non-academics. Ensuring relevancy across country contexts (e.g., reflecting differences between LICs and MICs) Hard to provide guidance to address all possible questions-- might not get used to the extent we wish

What would be your top suggestions for a reference case to make cost data useful to decision makers at the country level? Cannot be seen as a "one-off" that is primarily an exercise carried out by outsiders; in order to ensure that data are used, local authorities must "buy in" and own the process and results. I am unclear at the moment if a 'reference case' logically makes sense. Decision makers need to understand that how their program operates will drive costs, so costs will logically vary across time and space (countries). Determining what added value the reference case would have in relation to existing guidance/tools Providing clear guidance on interpreting results

Don't think a reference case will make data cost useful What would be your top suggestions for a reference case to make cost data useful to decision makers at the country level? Have a general section that is applicable across countries/diseases, then allow for (and suggest) country/disease-specific supplemental modules Clearly specify methods (and if possible have an adaptable MS Excel spreadsheet to aid collection) Specify minimum reporting standards (and if possible use a Drummond et al-like checklist) Don't think a reference case will make data cost useful

What do we want to achieve? What is a ‘good enough’ standard in costing? Costing is a process of estimation Example characteristics of a good estimate: Precision Accuracy But how accurate and precise is good enough? Cost of getting it right compared to the cost of getting it wrong

Other ‘desirable’ characteristics Generalisability Can we apply the cost to other settings? More important to be relevant to context? Comparability and standardisation Are cost estimates comparable with on another? Innovation? Reliablilty

Our aim To improve the relevance, use and quality of cost estimates by: Ensuring that the process of cost estimation is transparent, so that those using the data can apply estimates widely and appropriately Framework for producers of cost data to consider how their methodological choices influence the quality and relevance of their estimates, and present data in way that maximises the extent of use

Principles for purpose Many purposes Economic evaluation and priority setting Medium term planning Budgeting Price setting Efficiency analyses Study design and valuation methods differ Tolerance for uncertainty may differ

Please indicate the areas below for which you (or your organization) use cost data? (tick all that apply) Value Percent Count Budgeting 45.5% 15 Medium / long-term resource needs estimates 60.6% 20 Priority setting 63.6% 21 Investment case 51.5% 17 Sustainability planning 48.5% 16 Technical efficiency analysis Health Technology Assessment 30.3% 10 Economic Evaluation (eg. cost-effectiveness analysis) 93.9% 31 Equity and poverty analyses 27.3% 9 Other 6.1% 2 Other Count HIV allocative efficiency and scenario analysis mathematical modelling 1 price negotiation Total 2

For what purpose(s) are you currently producing cost data For what purpose(s) are you currently producing cost data? (tick all that apply) Value Percent Count Budgeting 51.6% 16 Medium / long-term resource needs estimates 54.8% 17 Priority setting 61.3% 19 Investment case 41.9% 13 Sustainability planning 45.2% 14 Technical efficiency analysis Health Technology Assessment 16.1% 5 Economic Evaluation (eg. cost-effectiveness analysis) 83.9% 26 Equity and poverty analyses 25.8% 8 Other (please describe) Other (please describe) Count Insurance 1 Mathematical modelling to improve HIV and TB allocative efficiency Performance improvement decision analysis price negotiation Total 5

Scope of the reference case ‘Unit costs’ and/or cost functions estimated using ‘research’ approaches, but also can be used to think about strengths and weaknesses of routine cost data Does not include guidance on cost analysis, such as estimating investment case costs Currently, focuses on provision costs (service and above service), not access (time and transport)

Terminology and definitions Intervention, episode, components and service/output units, activities and inputs An m-health intervention to reduce default amongst patients with TB Cost per person successfully treated Cost per patient receiving the intervention Cost for first line treatment Cost per person followed up with SMS’s Cost per visit

Other challenges Gross or micro costing Top down or bottom up Activity based costing Indirect costs Resource use vs Service use Do we just select one, where currently used differently?

Questions How can the reference case best meet the needs of users and producers of cost data? What would be its main benefits? What are the main risks?

Questions Is the current scope sufficient? Could the form be improved? Process for tomorrow

Study Design 1 Cost estimates should be communicated transparently to enable the decision-maker(s) to interpret and use the results.   2 The type of the cost estimated should be defined in terms of economic vs financial, gross vs micro, real world vs per protocol, and incremental vs full cost, and justified relevant to purpose. 3 The population, intervention, perspective and scope (extent of the resource use of the intervention captured) of the cost estimation should be stated and justified relevant to purpose. 4 The time horizon should be of sufficient length to capture all costs relevant to purpose, and consideration should be given to disaggregating costs into separate time periods where they vary over time. 5 The ‘units’ in the unit costs for interventions, episodes of care and service use should be relevant for the costing purpose and generalizable.

Resource use measurement 6 The selection of the data source for estimating service use should be described, with potential biases reported in the study limitations 7 The use of ‘top-down’ or ‘bottom-up’ resource measurement methods should be stated by input   8 The sampling strategy should be determined by the precision demanded by the costing purpose and designed to minimise bias. 9 Consideration should be given to the timing of data collection to minimise recall bias and, where relevant the impact of seasonality and other differences over time. 10 Above site costs should be considered, and where excluded potential biases reported in the study limitations 11 Methods for capturing human resource use should be reported, and where relevant, limitations and biases reported in the study limitations 12 Research costs should be reported separately and excluded from intervention costs, where relevant to the purpose of the costing.

Valuation and pricing   13 The sources for price data should be listed by input, and clear delineation should be made between local and international price data sources 14 Capital costs should be appropriately annuitized or depreciated to reflect the expected life of capital inputs 15 Where relevant an appropriate discount rate, inflation and exchange rates should be used, and clearly stated. 16 The use and source of shadow prices, for goods and for the opportunity cost of time, should be reported.

Reporting results 17   The total costs, number of units and unit costs of the intervention should be reported, including where relevant any component (eg. service) unit costs. 18 All total and unit costs should be disaggregated, by input category (e.g., capital/ recurrent, personnel, supplies), tradable/ non-tradable inputs, and where relevant by activity and site 19 The cost of the intervention for sub-populations, and other areas of heterogeneity should be explored 20 The uncertainty associated with cost estimates should be appropriately characterised.

Possible additions Guidance around cost functions Quality adjusted units Societal costs Unrelated costs

Methodological specifications The population, intervention, perspective and scope (extent of the resource use incurred by the intervention) of the cost estimation should be stated and justified as is relevant for purpose. The full production process of an intervention should be considered in the first instance. Exclusions to scope can be made based on purpose, type of cost and the costs of data collection. The direction and likely extent of any bias should be transparent

Reporting standards Cost estimates should be communicated transparently to enable the decision-maker(s) to interpret and use the results The purpose should be stated, clearly identifying: The relevance for health practice and policy decisions The aim of any cost analysis where relevant The intended user (s) of the cost estimate   The intervention and context of the intervention being costed should be clearly outlined, describing: Main activities/technologies involved Target population Coverage level or phase (pilot, implementation, post scale up) Delivery mechanism (health system level/ facility types/community/ownership/ where relevant integration with other services) Epidemiological context (incidence/prevalence of the illness being addressed)

Specific reporting standards Intervention Sub-intervention Population Unit cost TB Treatment - intensive phase   New Retreatment drugs MDR Pre –XDR XDR Other Children Community Contacts General Prisons Facility attendees High risk Cost per person (or person month) treated in intensive phase” Intervention Service use units (and unit costs) TB Treatment - intensive phase   Inpatient bed-days Outpatient visits Community observation visits Drugs regimen Microscopy TB Treatment - continuation phase

The approach Set of ‘acceptable’ principles Methodological guidance on how to achieve those principles (theory and evidence based) Standardisation for specific interventions with additional guidance where available Reporting standards

Process

Scope of the reference case ‘Unit costs’ estimated using ‘research’ approaches, but also can be used to think about strengths and weaknesses of routine cost data Does not include guidance on cost analysis, such as estimating investment case costs Focuses on provider costs

What do we want to achieve? Costing is a process of estimation Two characteristics of a good estimate: Precision Accuracy But how accurate and precise is good enough? Depends on the decision to be made using the cost

Other characteristics Generalisability Can we apply the cost to other settings? More important to be relevant to context? Comparability Are cost estimates comparable with on another? Standards or standardisation?

Cost functions vs unit costs

Limited empricial validation of measurement tools/ approaches

First challenge – cost for purpose a cost is not a cost First a caveat, principles for purpose Economic evaluation and priority setting Medium term planning Budgeting Price setting Efficiency analyses Study design and valuation methods differ

Common terminology and definitions Intervention, episode and service/output units – activities and inputs An m-health intervention to reduce default amongst patients with TB Cost per patient receiving the intervention Cost for first line treatment Cost per person followed up with SMS’s Cost per visit

Standardisation of interventions/ inputs Intervention, episode and service/output units – activities and inputs An m-health intervention to reduce default amongst patients with TB Cost per patient receiving the intervention Cost for first line treatment Cost per person followed up with SMS’s Cost per visit

Other challenges Gross or micro costing Top down or bottom up Activity based costing Indirect costs Resource use vs Service use

Scope of costs

Suv

Study Design 1 Cost estimates should be communicated transparently to enable the decision-maker(s) to interpret and use the results.   2 The type of the cost estimated should be defined in terms of economic vs financial, gross vs micro, real world vs per protocol, and incremental vs full cost, and justified relevant to purpose. 3 The population, intervention, perspective and scope (extent of the resource use of the intervention captured) of the cost estimation should be stated and justified relevant to purpose. 4 The time horizon should be of sufficient length to capture all costs relevant to purpose, and consideration should be given to disaggregating costs into separate time periods where they vary over time. 5 The ‘units’ in the unit costs for interventions, episodes of care and service use should be relevant for the costing purpose and generalizable.

Resource use measurement 6 The selection of the data source for estimating service use should be described, with potential biases reported in the study limitations 7 The use of ‘top-down’ or ‘bottom-up’ resource measurement methods should be stated by input   8 The sampling strategy should be determined by the precision demanded by the costing purpose and designed to minimise bias. 9 Consideration should be given to the timing of data collection to minimise recall bias and, where relevant the impact of seasonality and other differences over time. 10 Above site costs should be considered, and where excluded potential biases reported in the study limitations 11 Methods for capturing human resource use should be reported, and where relevant, limitations and biases reported in the study limitations 12 Research costs should be reported separately and excluded from intervention costs, where relevant to the purpose of the costing.

Valuation and pricing   13 The sources for price data should be listed by input, and clear delineation should be made between local and international price data sources 14 Capital costs should be appropriately annuitized or depreciated to reflect the expected life of capital inputs 15 Where relevant an appropriate discount rate, inflation and exchange rates should be used, and clearly stated. 16 The use and source of shadow prices, for goods and for the opportunity cost of time, should be reported.

Reporting results 17   The total costs, number of units and unit costs of the intervention should be reported, including where relevant any component (eg. service) unit costs. 18 All total and unit costs should be disaggregated, by input category (e.g., capital/ recurrent, personnel, supplies), tradable/ non-tradable inputs, and where relevant by activity and site 19 The cost of the intervention for sub-populations, and other areas of heterogeneity should be explored 20 The uncertainty associated with cost estimates should be appropriately characterised.

Reporting standards Cost estimates should be communicated transparently to enable the decision-maker(s) to interpret and use the results The purpose should be stated, clearly identifying: The relevance for health practice and policy decisions The aim of any cost analysis where relevant The intended user (s) of the cost estimate   The intervention and context of the intervention being costed should be clearly outlined, describing: Main activities/technologies involved Target population Coverage level or phase (pilot, implementation, post scale up) Delivery mechanism (health system level/ facility types/community/ownership/ where relevant integration with other services) Epidemiological context (incidence/prevalence of the illness being addressed)

Additional and complementary activities? Support the TB Taskforce and develop principles and methods around access costs, including income measurement, sampling methods, inclusion and exclusion TB provider costing study in 5 countries will enable us to pilot and apply methods iDSi work to hold workshop to link reference case (s) to modelling work TB MAC work on unit costs and cost functions in resource allocation models (linking to analytics)

Going forward - Questions for TAG Reference case How can we engage funders to encourage adoption? Are there important constituencies we should include beyond those coming this week? How do we best link to other efforts? Are there other methods areas? How can GHCC stimulate methods work to advance the reference case in other areas?

Next steps