Value of information Marko Tainio Decision analysis and Risk Management course in Kuopio

Outline of lecture Aim – To give an overview on what is VOI and in what situation it is useful. Content – What is Value of Information (VOI) analysis? – How to calculate VOI? – How and in what situation you can use VOI?

Different definitions Value of information (VOI) is the amount a decision maker would be willing to pay for information prior to making a decision. – Definition in Wikipedia ( Expected value of perfect information (EVPI) is the price that one would be willing to pay in order to gain access to perfect information – 2nd definition in Wikipedia VOI is„a decision analytic technique that explicitly evaluates the benefits of collecting additional information to reduce or eliminate uncertainty” – Yokota and Thompson, 2004

Key elements of VOI Decision maker – VOI analysis is a decision analysis tool aimed to help in decision making Information that contains uncertainty – Originally VOI analysis is used in situation where there is 2 or more available decision options and their outcomes are uncertain For example: Should we vaccinate population or not Price for the information – VOI also assumes that gathering of more information is possible and that this information reduces or eliminates uncertainty!

Calculation of VOI (expected value of perfect information, EVPI)

Equation to calculate EVPI In the equation, s is the uncertain input, and f(s) represents the probability distribution representing prior belief about the likelihood of s. Yokota and Thompson (2004) Expected monetary value (EMV) Expected value given perfect information (EV|PI) EVPI = EV|PI - EMV

Example 1

Example with point values Example from wiki: Suppose you were going to make an investment into only one of three investment vehicles: stock, mutual fund, or certificate of deposit (CD). Further suppose, that the market has a 50% chance of increasing, a 30% chance of staying even, and a 20% chance of decreasing. – If the market increases the stock investment will earn $1500 and the mutual fund will earn $900. – If the market stays even the stock investment will earn $300 and the mutual fund will earn $600. – If the market decreases the stock investment will lose $800 and the mutual fund will lose $200. – The certificate of deposit will earn $500 independent of the market's fluctuation.

Decision tree Stock Mutual fund Certificate $500 (+) 50% (-) 20% (+/-) 30% $1500 $900 $300 $600 -$800 -$200 What are the expectations for each vehicle?

Continuation of example Expectation for each vehicle: – Exp stock = 0.5 * * * ( − 800) = 680 – Exp mutualfund = 0.5 * * * ( − 200) = 590 – Exp certificateofdeposit = 500 The maximum of these expectations is the stock vehicle. Not knowing which direction the market will go (only knowing the probability of the directions), we expect to make the most money with the stock vehicle. Expected monetary value (EMV) = 680

Continuation of example (3/3) On the other hand, consider if we did know ahead of time which way the market would turn. Given the knowledge of the direction of the market we would (potentially) make a different investment vehicle decision. Expectation for maximizing profit given the state of the market: – EV | PI = 0.5 * * * (500) = 1030 That is, given each market direction, we choose the investment vehicle that maximizes the profit. Hence = EVPI = EV|PI –EMV = 1030 – 680 = $350

Example 2

Decision situation Lets assume that you can spend time either in Helsinki or in Kuopio; Only thing you care is temperature (you want to be in warmer place all the time); Lets also assume that every morning you could decide in which city you are (you have magic); What is the value of information for you to know the exact temperature of every day vs. knowing average temperature?

Decision before new information At the moment you know that average temperatures for these cities are: – Helsinki: -15 Celsius (range to -19.3) – Kuopio: -16 Celsius (range to -19.9) Which city you would choose if you care only from temperature and you have this data? In previous equation, this is EMV (Expected monetary value) In next phase we add more data for the decision situation!

Temperature data for January Difference in average temperature is 1 Celsius degree so based on that knowledge you would prefer Helsinki However, Helsinki is not warmer in every day! And since you have magic, you can choose every morning in which city you are! Therefore you might want to calculate in which days Helsinki is warmer and in which days Kuopio is warmer to optimise your decision every morning.

Temperature data for January 9 out of 31 days Kuopio was warmer than Helsinki. If you take the higher temperature for each day and calculate the average temperature, you find out that average temperature was Celsius degree Now, what is the value of information? Before exact data you would have stayed all the time in Helsinki and enjoyed average temperature of -15. With the exact data you can change the city each day to warmer one and experience average temperature of Therefore you gained 0.6 Celsius degree! Thus, the value of information is: 0.6!

How to calculate VOI in Monte Carlo? Calculation of EVPI in Monte Carlo model is done similarly as in previous example with following modifications: - We assume that each iteration is an individual data point; - We consider each iteration as a separate decision and then we maximize the benefit similarly as in temperature example; - The calculation is simple and can be done e.g. with Excel (when the result sample is known).

Different variations of VOI

Expected value of perfect information Expected value of perfect information assumes that uncertainty is reduced to zero Two analyses: – The expected value of perfect information (EVPI) – Expected value of perfect X information (EVPXI) EVPI consider whole model while EVPXI calculates VOI for individual parameter – E.g. the EVPXI can be calculated separately for dose-response function, exposure estimates etc.

Example 1: Which decision option is better, A or B Costs Probability density AB How certain you are that A is better than B?

Example 1: Which decision option is better, A or B Costs Probability density AB How certain you are that A is better than B?

Imperfect information Expected value of imperfect information (EVSI) Expected value of imperfect X information (EVPXI) – Imperfect information assumes that uncertainty can be reduced but not eliminated (example 1 vs. Example 2) Imperfect information is more realistic assumption than perfect information!

Example 2: Which decision option is better, A or B Costs Probability density AB How certain you are that A is better than B?

Example 2: Which decision option is better, A or B Costs Probability density AB How certain you are that A is better than B?

How to use VOI in risk assessment & management?

Requirements for VOI analysis To be able to perform a VOI analysis a modeller needs information on the – (i) available decision options; – (ii) the consequences of each options; – (iii) uncertainties and reliability of the data. In addition to these, both gains and losses of the actions must be qualified with common metrics (monetary or non-monetary).

(i) Decision options Decision options depend on the purpose of the assessment In environmental health field decision options could be e.g.: – Choice between different decision options to reduce emissions of pollutants Who defines the decision options depends on the case – Authorities that ordered assessment, modeller himself, stakeholders etc.

(ii) Consequences of each options This one is the assessment model that you have defined during the assessment We will not go to detailes in this lecture

(iii) uncertainties and reliability of the data In VOI analysis we reduce uncertainty so the assessment should have uncertain parameters Identifying and assessing uncertainties have been considered in other lectures.

Two possible ways of using VOI in assessment 1/2 Guide the information gathering and model building – The decisions can be made based on available information or wait and collect more information – VOI analysis can be used to inform decision maker on the possible benefits of collecting additional information – However, in the field of environmental health and risk assessment, situations, where decision maker is known, the decision maker has possibility to allocate more funding for additional research, and more data can be collected, are rare and this kind of exploitation of VOI analysis is more an exception than rule

Two possible ways of using VOI in assessment 2/2 Guide the process of model building – the decision maker is the modeller him/herself or the research team who makes the decisions of the modelling work – Thus, VOI analysis can be used like sensitivity analysis – The decisions that can be addressed are e.g. (i) should the model define uncertainties (ii) what are the key input parameters or assumptions in the model (iii) which parts of the model should be specified more detailed

Further reading Morgan M.G. and Henrion M. (1998). Uncertainty: A guide to dealing with uncertainty in quantitative risk and policy analyses. Cambridge University Press. 332 pp. Yokota F. and Thompson K.M. (2004). Value of information literature analysis: A review of applications in health risk management. Medical Decision Making, 24 (3), pp Yokota F. and Thompson K.M. (2004) Value of information analysis in environmental health risk management decisions: Past, present, and future. Risk Analysis, 24 (3), pp