Estimating Outcomes in Decision Analysis Miriam Kuppermann, PhD, MPH Associate Professor Departments of Obstetrics, Gynecology & Reproductive Sciences.

Estimating Outcomes in Decision Analysis Miriam Kuppermann, PhD, MPH Associate Professor Departments of Obstetrics, Gynecology & Reproductive Sciences and Epidemiology & Biostatistics University of California, San Francisco

Review—Last Lecture Formulate an explicit question Formulate an explicit question Make a decision tree Make a decision tree Estimate probabilities Estimate probabilities Estimate utilities Estimate utilities Compute expected utility of each branch Compute expected utility of each branch Perform sensitivity analysis Perform sensitivity analysis

Preview—Where We Are Going Recall Ms. Brooks and her aneurysm. We want to: Determine her utility: clip, and not clip Determine her utility: clip, and not clip Compare incremental utility and cost Compare incremental utility and cost Compare cost-per-unit of utility across private and public uses of funds. Compare cost-per-unit of utility across private and public uses of funds.

Overview – Today’s Lecture Outcomes Outcomes Measuring Utilities Measuring Utilities Quality Adjusted Life Years (QALYs) Quality Adjusted Life Years (QALYs) Discounting Discounting Prenatal testing example Prenatal testing example

Quantifying Health Outcomes Mortality Mortality Life years Life years Number of expected years of life Number of expected years of life Significant morbidity Significant morbidity Paralysis, loss of sight Paralysis, loss of sight Quality-Adjusted Life Years (QALYs) Quality-Adjusted Life Years (QALYs) Expected life years adjusted for the valuation of the possible states in each year Expected life years adjusted for the valuation of the possible states in each year Financial valuation of outcomes Financial valuation of outcomes Costs to patient, payor, or society Costs to patient, payor, or society Willingness to pay to avoid outcomes, obtain treatment Willingness to pay to avoid outcomes, obtain treatment

Health Outcomes – Mortality Mortality Mortality Death from disease/accident/procedure e.g. If Ms. Brooks undergoes surgery, one of the possible outcomes is mortality Life Years Life Years Calculate an expected value of life years using a probabilistically weighted average of expected life e.g. If Ms. Brooks does not undergo surgery, her life expectancy is less than if she did not have aneurysm, these outcomes are measured in expected life years

Health Outcomes – Morbidity Morbidity Morbidity Some health state that is less than perfect e.g. disability from stroke, chronic pain Comparison of morbidities Comparison of morbidities Difficult – apples and oranges problem e.g. which is worse: Blind vs. deaf Deaf vs. paraplegia Paraplegia vs. blind

Outcomes - Utility Utility is the currency of outcomes Scaled from 0 to 1 Scaled from 0 to 1 Commonly Death = 0 Commonly Death = 0 Perfect Health = 1 Perfect Health = 1 Sounds good, but how can this be measured? Sounds good, but how can this be measured?

Utility Measurement Utilities are commonly estimated using comparisons to the 0 and 1 anchors Methods are: Methods are: Visual Analog Scale Visual Analog Scale Standard Gamble Standard Gamble Time Trade-off Time Trade-off

Utility Measurement Clinical Scenario: Patient in the hospital has infection of the leg Patient in the hospital has infection of the leg Two options: BKA v. Medical Management Two options: BKA v. Medical Management BKA – below the knee amputation (1% mortality risk) BKA – below the knee amputation (1% mortality risk) Medical – 20% chance of infection worsening Medical – 20% chance of infection worsening If worse, AKA – above the knee amputation (10% mortality risk) If worse, AKA – above the knee amputation (10% mortality risk) How can we value these outcomes? How can we value these outcomes?

Utility Measurement Visual Analog Scale Linear Scale from 0 to 1 Linear Scale from 0 to 1 (death 0-----------------------------1.0 cure) Where is AKA? (death 0-----------------------------1.0 cure) Where is BKA?

Utility Measurement Visual Analog Scale Advantages: quantitative, easy to understand, visual Advantages: quantitative, easy to understand, visual Disadvantages: may bias values to the middle, seems disconnected from medical reality Disadvantages: may bias values to the middle, seems disconnected from medical reality

Utility Measurement-- Standard Gamble Method relies on patients choosing between: 1) a certain outcome 1) a certain outcome 2) a gamble between a better outcome and a worse outcome 2) a gamble between a better outcome and a worse outcome How it works: Choice 1: You live with a BKA Choice 1: You live with a BKA Choice 2: The gamble – you might have a cure; you might die. Choice 2: The gamble – you might have a cure; you might die. Goal of method is to find the break-even point. What probability of death would you accept to avoid living with the BKA?

Utility Measurement – Standard Gamble

Standard gamble measurement involves questioning patients to determine the p at which the two outcomes are equivalent Standard gamble measurement involves questioning patients to determine the p at which the two outcomes are equivalent Using expected utilities, the value of p [p = P(cure)] gives the utility: Using expected utilities, the value of p [p = P(cure)] gives the utility: Utility (BKA) x Prob (BKA) = Utility (BKA) x Prob (BKA) = Utility(cure) x (p) + Utility(death) x (1-p) Utility(cure) x (p) + Utility(death) x (1-p) [Note that here, P(BKA) = 1.] [Note that here, P(BKA) = 1.] You can demonstrate that the utility of BKA = p: Utility (BKA) = [Utility(cure) x (p) + Utility(death) x (1-p)] = [1.0 * p + 0 * (1-p)] = p

Utility Measurement – Standard Gamble In utility estimation utilizing the standard gamble measurement, using the simple comparison described, the value of p, the probability of perfect health needed to make the patient indifferent between the two choices is the estimated utility of the outcome. In utility estimation utilizing the standard gamble measurement, using the simple comparison described, the value of p, the probability of perfect health needed to make the patient indifferent between the two choices is the estimated utility of the outcome. Advantages: the incorporation of risk into the model, comparison or choice between different outcomes. Advantages: the incorporation of risk into the model, comparison or choice between different outcomes. Disadvantages: possible nonrealistic choices patients may be asked to make, the difficulty of understanding the question (especially for non-gamblers), and the difficulty for some in understanding a math equation.

Utility Measurement – Time Trade-off Time Trade-off involves patients choosing between: quality of life vs. length of time alive quality of life vs. length of time alive We want to determine when patients are equivocal between choice: Time A * Utility A = Time B * Utility B Time A * Utility A = Time B * Utility B e.g. -- If you have a life expectancy of 30 years with a BKA, how much time would you give up to live in your current state? Would you give up 5 years? 3 years? 1 year? 30 years * Utility (BKA) = (30-x) years * 1.0 If you’re willing to give up 3 years, that means the utility of BKA is: 30 years * Utility (BKA) = (30-x) years * 1.0 If you’re willing to give up 3 years, that means the utility of BKA is: = [(30-3)*1/ 30] = 27/30 = 0.9 = [(30-3)*1/ 30] = 27/30 = 0.9

Utility Measurement – Time Trade-off Time trade off can be used to measure the utility of an outcome in the fashion described. Like standard gamble, patients need to understand subtlety of questions being asked. Advantages: the simplicity of the choice between different outcomes, consideration of long-term outcomes. Advantages: the simplicity of the choice between different outcomes, consideration of long-term outcomes. Disadvantages: fails to incorporate risk, lack of clarity of when time traded occurs, different valuation of time during life, and the theoretical lack of realism of the choice. Disadvantages: fails to incorporate risk, lack of clarity of when time traded occurs, different valuation of time during life, and the theoretical lack of realism of the choice.

Utility Measurement – Health State Excluding Clinical Problem 0 to 1 utility scaling is simple and useful for determining the utility of different outcomes for a single patient. 0 to 1 utility scaling is simple and useful for determining the utility of different outcomes for a single patient. However, comparisons across patients or across programs raises the issue of ‘Is a utility of 1 really available?’ However, comparisons across patients or across programs raises the issue of ‘Is a utility of 1 really available?’ Consider: (a) polio vaccinations for children, and (b) hip replacement surgery for 85-year old patients. Consider: (a) polio vaccinations for children, and (b) hip replacement surgery for 85-year old patients. Typically (on average) the health state of a polio-free 5-year old will be very close to 1, whereas the health state of an 85-year old even with a perfectly functioning hip will typically be less than one. Typically (on average) the health state of a polio-free 5-year old will be very close to 1, whereas the health state of an 85-year old even with a perfectly functioning hip will typically be less than one.

Utility Measurement – Chaining The utility of the health state of the 85-year old can be determined by “chaining”. The utility of the health state of the 85-year old can be determined by “chaining”. Consider first the clinical situation under review: what is the utility, on a 0 to 1 scale, of a hip with reduced function and some pain? Let us say utility =.8. Consider first the clinical situation under review: what is the utility, on a 0 to 1 scale, of a hip with reduced function and some pain? Let us say utility =.8. Consider next the utility of an 85-year old with no hip problems, but some other reduced function or chronic pain. Here, perhaps overall utility =.9. Consider next the utility of an 85-year old with no hip problems, but some other reduced function or chronic pain. Here, perhaps overall utility =.9. Total utility for the bad-hip case =.8 *.9 =.72 Total utility for the bad-hip case =.8 *.9 =.72 Here, the perfect-hip case utility = 1 *.9 =.9 Here, the perfect-hip case utility = 1 *.9 =.9

Utility Measurement – Additional Information Multi-Attribute Health Status Classification System Multi-Attribute Health Status Classification System Developed by Health Utilities, Inc. Developed by Health Utilities, Inc. Available at: http://www.healthutilities.com/overview.htm Available at: http://www.healthutilities.com/overview.htm

Utilities in Decision Analysis Now that we have methods to estimate utilities, these can be used in the DA Now that we have methods to estimate utilities, these can be used in the DA However, our outcomes often include both mortality and morbidity However, our outcomes often include both mortality and morbidity Want a way to add in life expectancy Want a way to add in life expectancy Quality Adjusted Life-Years (QALYs) Quality Adjusted Life-Years (QALYs)

QALYs QALYs are generally considered the standard unit of comparison for outcomes QALYs are generally considered the standard unit of comparison for outcomes QALYs = time (years) x quality (utils) QALYs = time (years) x quality (utils) e.g. 30 years after BKA, util(BKA) = 0.9 e.g. 30 years after BKA, util(BKA) = 0.9 = 30 x 0.9 = 27 QALYs = 30 x 0.9 = 27 QALYs

QALYs Aneurysm Example We said life expectancy is reduced by 2/3, so instead of 35, it is = 35 *.333 = 11.67 We said life expectancy is reduced by 2/3, so instead of 35, it is = 35 *.333 = 11.67 Here, we have assigned a utility of.5 to surgery-induced disability, so QALYs = Here, we have assigned a utility of.5 to surgery-induced disability, so QALYs = years * utils = 11.67 *.5 = 5.8 years * utils = 11.67 *.5 = 5.8

Outcomes - Discounting However, are all years considered equal? However, are all years considered equal? Consider: Favorite Meal Consider: Favorite Meal Extreme Pain Lifetime Income

Outcomes - Discounting Generally, present > future Generally, present > future One common way to value the different times is discounting One common way to value the different times is discounting Essentially this year is worth δ more than next year Essentially this year is worth δ more than next year δ is commonly set at 0.03 or 3% δ is commonly set at 0.03 or 3% In order to compare values of all future times, a calculation, net present value, is often used In order to compare values of all future times, a calculation, net present value, is often used NPV = 1 / (1 + δ) t Where t is number of years in the future NPV = 1 / (1 + δ) t Where t is number of years in the future

Outcomes - Discounting Aneurysm Example Aneurysm Example If utility is 0.5 and life expectancy is 3 years If utility is 0.5 and life expectancy is 3 years NPV would be:  Utility / (1 + δ) t NPV would be:  Utility / (1 + δ) t However, when is year 1? Often, since events in year one occur on average half way through, we use 0.5 for year 1 However, when is year 1? Often, since events in year one occur on average half way through, we use 0.5 for year 1 NPV = 0.5 / (1.03) 0.5 + 0.5 / (1.03) 1.5 + 0.5 / (1.03) 2.5 NPV = 0.5 / (1.03) 0.5 + 0.5 / (1.03) 1.5 + 0.5 / (1.03) 2.5 NPV = 0.5 * {(1.03) -0.5 + (1.03) -1.5 + (1.03) -2.5 } NPV = 0.5 * {(1.03) -0.5 + (1.03) -1.5 + (1.03) -2.5 }

Outcomes - Discounting

Another issue is partial years Another issue is partial years Can use similar adjustment: Can use similar adjustment: Exponent is half way through time period Exponent is half way through time period Numerator is multiplied by fraction of year Numerator is multiplied by fraction of year e.g. disability for 1.6 years, utility of 0.5 e.g. disability for 1.6 years, utility of 0.5 NPV =  Utility / (1 + δ) t NPV =  Utility / (1 + δ) t NPV = 0.5 / (1.03) 0.5 + 0.5*0.6 / (1.03) 1.3 NPV = 0.5 / (1.03) 0.5 + 0.5*0.6 / (1.03) 1.3

Discounting – Special Topic Issues with exponential discounting: Issues with exponential discounting: Relatively easy to manipulate Relatively easy to manipulate However, assumes same difference between any two time periods is the same value difference However, assumes same difference between any two time periods is the same value difference Consider Consider today : vs. tomorrow and 10 yrs vs. 10 yrs + 1 day Essentially, “today” versus all other time periods is valued higher for many outcomes Essentially, “today” versus all other time periods is valued higher for many outcomes

Discounting – Special Topic Importance of discount rate chosen: Importance of discount rate chosen: Imagine a benefit of $1,000,000 to be received in 30 years Imagine a benefit of $1,000,000 to be received in 30 years Present value of this benefit at different discount rates: Present value of this benefit at different discount rates: 1% -- $741,923 3% -- $411,987 8% -- $99,377 17% -- $9,004

Overall Review Outcomes OutcomesMortalityMorbidity Measuring Utilities Measuring Utilities Visual Analog Standard Gamble Time Trade-off Quality Adjusted Life Years (QALYs) Quality Adjusted Life Years (QALYs) QALYs = time (years) x quality (utils) QALYs = time (years) x quality (utils) Discounting Discounting NPV =  Utility / (1 + δ) t

Informed Decision-Making in Prenatal Testing Miriam Kuppermann, PhD, MPH Associate Professor Decision and Cost-Effectiveness Analysis EPI 213 January 20, 2005 Department of Obstetrics, Gynecology, & Reproductive Sciences

Prenatal Tests for Chromosomal Disorders Screening Tests (non-invasive) Expanded AFP Nuchal translucency Diagnostic Tests (invasive) Amniocentesis Chorionic villus sampling (CVS)

Guidelines For Prenatal Testing Are Dichotomized By Maternal Age Women < 35 Screening first Invasive testing only if “positive” results Women > 35 Invasive testing offered Screening as an option

Rationale for Guidelines Need to limit access to invasive testing Inherent risk of procedure Limited availability of providers, laboratories Age 35 selected as the threshold Increasing risk with advancing maternal age Threshold set where risks equal Cost/benefit considerations Kuppermann, Nease, Goldberg, Washington. Who should be offered prenatal diagnosis? The 35-year-old question. Am J Public Health 1999; 89:160-3

Equal Risk Threshold Risk of Miscarriage = Risk of Down Syndrome Implicit assumption: equally burdensome outcomes Procedure-related miscarriage Down-syndrome affected infant

First Challenge to Guidelines Do women find procedure-related miscarriage and Down-syndrome- affected birth to be equally burdensome?

How Do Women Value Testing Outcomes?   1082 socioeconomically and age- diverse women   English-, Spanish- or Chinese- speaking   Interviewed <20 weeks pregnant   Preferences for testing outcomes   Demographic/attitudinal questions   Testing behavior

Simplified Decision Tree for Prenatal Testing

Preference Score “Utility” Ranges from 0 to 1 1 = ideal outcome, 0=death ” Measured using “time tradeoff”

Time Tradeoff Preference Elicitation Choice AChoice B Which do you prefer? 40 years of life remaining with affected child 40 years of life remaining with unaffected child (give up 0 years of life)

Time Tradeoff Preference Elicitation 40 years of life remaining with affected child 30 years of life remaining with unaffected child (give up 10 years of life) Which do you prefer? Choice AChoice B

Calculation of Time Tradeoff Scores reduced life expectancy with unaffected child (30 years) U TTO = __________________________________________ full life expectancy with affected child (40 years) = 0.75

Median utility for procedure-related miscarriage = 0.86 Median utility for Down-syndrome affected infant = 0.73 On average, women do not equally weight the outcomes of procedure-related miscarriage and Down syndrome-affected birth P<0.001 by Wilcox sign rank test Kuppermann, Nease, Learman, Gates, Blumberg, Washington. Procedure- related miscarriages and Down syndrome-affected births: implications for prenatal testing based on women’s preferences. Obstet Gynecol 2000; 96:511-6.

Preference Difference Score One way to look at the relative value women assign to procedure-related miscarriage and DS-affected birth Pref score misc – Pref score DS Higher score = greater preference for miscarriage over DS

0 25 50 75 100 125 150 175 200 Number -.75-.5-.25 0.25.5.751 Preferences Vary Substantially Utility misc - Utility DS

First Conclusion Current guidelines do not adequately reflect the preferences of pregnant women.

Second Challenge to Guideline Old paradigm: COST BENEFIT Benefits (in $$ terms) of program should exceed costs. Costs of offering testing should be offset by savings accrued by averting the birth of Down-syndrome-affected infants New paradigm: COST EFFECTIVENESS No $$ value assigned to outcomes. Cost of offering testing should be “worth” the gain in quantity and quality of life.

How is Cost Effectiveness Measured? CΔ CCΔ C EΔ EEΔ E Δ Costs Δ QALY = QALY = Quality–Adjusted Life Year

Cost Effectiveness of Prenatal Diagnosis QALYs Lifetime cost Cost-utility ratio Age 20 Amniocentesis24·16$54,080$14,200 No testing24·08$52,940 Age 35 Amniocentesis20·39$61,490$12,600 No testing20·30$60,360 Age 44 Amniocentesis17·08$59,020$11,300 No testing16·98$57,890 Harris, Washington, Nease, Kuppermann. Cost utility of prenatal diagnosis and the risk-based threshold. Lancet 2004; 363:276-82.

Second Conclusion Offering invasive testing to women of all ages and risk levels can be cost effective.

Implications Guidelines should be changed to enable all women to make informed choices about which prenatal tests, if any, to undergo. Tools that help women and partners make informed choices should be implemented.

Estimating Outcomes in Decision Analysis Miriam Kuppermann, PhD, MPH Associate Professor Departments of Obstetrics, Gynecology & Reproductive Sciences.

Similar presentations

Presentation on theme: "Estimating Outcomes in Decision Analysis Miriam Kuppermann, PhD, MPH Associate Professor Departments of Obstetrics, Gynecology & Reproductive Sciences."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Estimating Outcomes in Decision Analysis Miriam Kuppermann, PhD, MPH Associate Professor Departments of Obstetrics, Gynecology & Reproductive Sciences.

Similar presentations

Presentation on theme: "Estimating Outcomes in Decision Analysis Miriam Kuppermann, PhD, MPH Associate Professor Departments of Obstetrics, Gynecology & Reproductive Sciences."— Presentation transcript:

Similar presentations

About project

Feedback