Elicitation Some introductory remarks by Tony O’Hagan

Welcome! Welcome to the third CHEBS dissemination workshop This forms part of our Focus Fortnight on “Elicitation” Our format allows plenty of time for discussion of the issues raised in each talk, so please feel free to join in!

Two strands The talks today address elicitation in two quite different contexts ›Elicitation of expert knowledge in the form of probability distributions of unknown parameters ›Elicitation of people’s preferences for health states, in order to formulate utilities A key objective of this FF is to bring these two strands together, to see what each can learn from the other

Eliciting probabilities Bayesian statistics concerns the updating of beliefs from prior to posterior In principle, specification of prior beliefs is therefore a key component of any analysis In practice it is rarely done seriously ›Given sufficient data, it doesn’t matter much ›It is difficult and time consuming to do properly ›It doesn’t lend itself to nice papers in leading statistics journals

Not necessarily ‘prior’ Eliciting expert opinion probabilistically does not have to be thought of as ‘prior’ to some data Expert information is a crucial input to risk analyses, etc, where there is no prospect of getting more data This is the area where elicitation methodology has been developed most

Elicitation in C-E analysis Elicitation of prior beliefs for Bayesian analysis of clinical trial and other data ›Often, sample sizes are not very large and prior information is appreciable Elicitation of expert knowledge of inputs to economic models ›Needed for sensitivity or value of information analysis Fully Bayesian synthesis of information ›Use posterior distributions for model inputs where possible

Challenges Regulatory context ›Elicitation will need to be robust, defensible Correlations ›How to elicit dependence? ›Or how to structure/model the parameters so that the structure induces the right dependence?

Eliciting preferences What is the ‘effectiveness’ for which ‘we’ are willing to pay? We can only compare costs across the whole health care spectrum, and allocate budgets rationally, if effectiveness is on a common scale Ultimately, we are willing to pay for health improvements that yield ›Longer life and/or ›Better quality of life

QALYs The Quality-Adjusted Life Year seems to be the gold standard ›Health providers’ willingness to pay can be reduced to how much they would pay for one QALY Elicitation then involves asking people how they value different states of health ›This is in order to know how to ‘adjust’ a life year Various challenges seem to arise, both theoretical and practical

Questioning the QALY Is two years of one person’s life worth one year of each of two people’s? If two years of poor health is worth one year of perfect health, are four years of poor health worth two of perfect health? Whose preferences/values? ›Individuals vary wildly ›Systematic differences between sectors of population ›What does society or the health provider think?

Practical challenges Choice of instrument: SG versus TTO Health state is massively multidimensional ›Can’t value every possible combination ›Relevant dimensions depend on medical condition How to quantify uncertainty in valuations Deferred benefits valued differently from immediate benefits ›Does that make sense from a societal perspective?

Some commonalities Different instruments yield different answers ›SG is also used to elicit individual probabilities ›But probabilistic elicitation is more about distributions A multidisciplinary perspective is needed ›There is a lot of psychology involved Questions of whose judgements are needed ›Individual versus community Results are subject to uncertainty ›Always need to extrapolate from limited elicited data Fundamental duality